Nginx, my new favorite front end for mongrel cluster.

Posted by ezmobius Wed, 23 Aug 2006 20:27:00 GMT

I have been on the lookout for a better front end for mongrel_clusters. I wanted one that had the flexibility of pound when it comes to proxying requests. But I also want something to serve static and cached files fast. Apache2.2/mod_proxy_balancer does a good job of this but it is way overkill if you don’t need apache for other sites with php or whatever. Lighttpd has always been one of my favorites but the current release has a broken mod_proxy that will not re-enable backends without a restart if they go down and back up again. Lighty will have a much better mod_proxy_core when 1.5 is released but thats a ways off it looks like and is not currently usable.

So I am very happy that I recently found a lightweight webserver that can serve static files very fast and that also has very complete proxy and rewrite modules. Its called nginx(which I have been pronouncing in my head as ‘engine x ;). I am very impressed with its performance and features. It is also super lightweight, taking up only a few Mb of ram when fully loaded while benchmarking. Nginx also has mod_fcgi. So it will work for externally spawned dispatch.fcgi’s or php-fcgi scripts. It does balancing and failover for fcgi as well. I haven’t tested the fcgi support yet but will do so in the near future. The only drawback I have found is that it is developed in Russia and while there are some english translations of the docs, they are not complete and don’t cover the proxy and rewrite modules yet.

But don’t fret, I have done most of the hard work for you and figured out most of the configuration needed to use nginx as a great front end for mongrel clusters. Nginx is my new favorite front end for my rails apps and has been very stable during my testing and abuse of it. It is actually faster then lighty for static files and the proxy and rewrite modules are way more complete. In my benchmarks on my macbookpro, nginx serves static files at 13+k requests/sec. And it performs better for proxied requests then pen, pound or apache2.2. It also handles ssl and vhosts like pound does, including regex conditionals and other niceties.

Click throuhg for the full details.

So far I have a fairly complete config file that is working great for me. All static files get served fast by nginx. Except I am still having a few issues with coming up with the incantations to get rails cached pages served by the front end as well. So all your images, css, js and other static files in public are served by nginx. Rails cached pages are still served my mongrel though(although mongrel is no slouch clocking in at ~900 req/sec for these static cached pages.). I will post an update once I get the exact config needed to support page caching. I wanted to get this post out there so people can play with nginx and I can get some more eyes on it to figure out that last config issues.

Its pretty easy to compile with these steps here:

curl -O http://sysoev.ru/nginx/nginx-0.3.60.tar.gz
tar -xzvf nginx-0.3.60.tar.gz
cd nginx-0.3.60
./configure --sbin-path=/usr/local/sbin --with-http_ssl_module
make
sudo make install
sudo chmod +x /usr/local/sbin/nginx

That will put the executable at /usr/local/sbin/nginx . ANd it will put the log files and config files in /usr/local/nginx.

And here is my config file so far. Of course replace the paths with your own values:
user  ez ez;
worker_processes  2;

error_log  logs/error.log notice;
pid        logs/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       conf/mime.types;
    default_type  application/octet-stream;

    # no sendfile on OSX uncomment 
    #this if your on linux or bsd
    #sendfile        on;
    tcp_nopush     on;

    keepalive_timeout  65;
    tcp_nodelay        on;

    upstream mongrel {
        server 127.0.0.1:5001;
        server 127.0.0.1:5002;
        server 127.0.0.1:5003;
    }

    gzip on;
    gzip_min_length  1100;
    gzip_buffers     4 8k;
    gzip_types       text/plain;

    server {
        listen       80;
        #server_name  localhost;
        root /Users/ez/nginx/public;

        access_log  off;
        rewrite_log on;

        location ~ ^/$ {
          if (-f /index.html){
            rewrite (.*) /index.html last;
          }
           proxy_pass  http://mongrel;
        }

        location / {
          if (!-f $request_filename.html) {
            proxy_pass  http://mongrel;
          }
          rewrite (.*) $1.html last;
        }

        location ~ .html {
           root /Users/ez/nginx/public;
        }

           location ~* ^.+\.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js|mov)$ {
          root /Users/ez/nginx/public;
        }

        location / {
            proxy_pass  http://mongrel;
            proxy_redirect     off;
            proxy_set_header   Host             $host;
            proxy_set_header   X-Real-IP        $remote_addr;
            proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        }

    }
}


I will continue to refine this config file and post my experiences here as I figure things out more. So far though. nginx is poised to be my new favorite light webserver and proxy front for rails apps.

Tags , , , ,  | 48 comments

Comments

  1. Luis Lavena said about 2 hours later:
    Hey Ezra, nice gem! ;-) I'll add this to my tool box too (on the *nix side, of course) -- shame no option like this exist for win32.
  2. Zack said about 2 hours later:
    Interesting... Apache does seem like an overkill for many sites (especially on a VPS where RAM is scarce). BTW - Are you still working on a Rails deployment book?
  3. Ezra said about 2 hours later:
    Yup I am still working on the book. It will have a btea release in September and the printed version will be out in November. http://pragmaticprogrammer.com/titles/fr_deploy/index.html
  4. Yan said about 3 hours later:
    Is there anyone known to be running nginx in production? The stats look great but does anyone know about longterm stability, memory leaks, etc? Not trying to spread fud here, I'm really interested in nginx just wondering if it has been tested in real life...
  5. Ezra said about 3 hours later:
    Yan- It is a Russian project and from what I can find out it seems like there are quite a few places using it for large scale stuff in Russia. I cannot personally attest to its long term stability yet. But from what I have seen so far from abusing it as hard as I can for the last few days, it has been rock solid. It is of course up to your own judgement whether you want to use it in production or not yet. But it is a relatively new project compared with some other options out there. So I can only say that I will be conducting more experiments with it in some production environments I am setting up soon that will really test the load(addservers). I will report my findings as I go. So far this is a very nice project that I feel pretty confident about.
  6. dominic said about 3 hours later:
    http://blog.kovyrin.net/2006/08/22/high-performance-rails-nginx-lighttpd-mongrel/ is another post about mongrel and nginx
  7. Alexey Kovyrin said about 14 hours later:
    Nginx is really mature and stable software. It is being used on some really large sites in Russia (actually, it was created for Rambler - one of the largest search portals in .ru). I'm using it on some really loaded sites - it is pretty stable. 2Ezra: According to your questions on my blog - you can contact me with any nginx-related questions directly by email/im.
  8. Meekish said about 20 hours later:
    Hey Ezra, this sounds super cool, but being english speaking and *nix anemic, I'm a little worried about diving into a Russian server project. What's your second favorit mongrel-cluster front at the moment?
  9. Adrian Madrid said about 23 hours later:
    Thanks for the config info. I was hoping to find a lightweight proxy to serve mongrel and static files but when I found nginx I couldn't really understand how to get it going. With this example I can give it a try. Thanks! Adrian Madrid
  10. Alex said 1 day later:
    Hi, cool blog ;-) I don't know when you was on http://nginx.net/ but there is a doc at the end of the page http://nginx.net/ngx_http_proxy_module.txt I can't speak russian therefore this translation is with some *maschines* out there :-( If you know a native speaker which want to help I'am open for help ;-)) Alex
  11. Chris Carter said 1 day later:
    Do you know how well it fares with php? I would love to convert my VPS to it, but right now I run 2 rails sites, one static site and 1 PHP site. I think I will mess with it at wok today and see how it handles PHP. We have a couple old P3 servers that this would be better suited for anyways. Thanks for the post and I look forward to more info about nginx
  12. anon said 3 days later:
    I can smell a fight brewing between Apache+mod proxy+mongrel, nginx+mongrel and litespeed. Looking forward to reading your thoughts on the litespeed setup.
  13. Joe said 6 days later:
    Why is Apache overkill? Memory isn't a problem for me at the moment. I've got multiple vhosts to serve and need to do SSL. Is nginx significantly faster than Apache2 at proxying to mongrel?
  14. Stoyan Zhekov said 6 days later:
    Hi Ezra. Did you found something for sticky sessions (ip, url and cookie based) + expiring them. I didn't find anything on the question in the docs, ML etc. There was just a small discussion without results on the ML.
  15. Dominic Damian said 8 days later:
    Hey Ezra, I just posted a tutorial on getting mongrel_clusters setup with nginx has a front end on OSX. It's mostly a refreshing of what you've said, but perhaps you might see something I didn't. Thanks again for the digging and work to get this started. http://blog.integralimpressions.com/articles/2006/08/30/nginx-the-front-end-solution-for-rails-deployment
  16. Amr said 21 days later:
    Litespeed isn't so bad from a configuration and admin perspective. It seems pretty fast but I don't know if anyone is running high traffic sites on it yet.

    Btw, George Wang has this nGnx+mongrel vs Litespeed+LSAPI comparison (he uses ab and I have read at ZedShaw's blog that ab is not reliable .. not sure why tho)

    here is the link to the detailed article, maybe someone can independently corroborate this?

    my-rails-benchmark-results-lsws-lsapi-vs-nginx-fcgi

  17. beenhero said 35 days later:
    非常期待部署简易,性能优越的web server出现。 Ezra, thanks for the infor
  18. Michael D'Auria said 37 days later:
    Amr: Wordpress uses LiteSpeed as does my site: trebleNation.com. I don't have a complaint yet :)
  19. zeca40 said 151 days later:
    can nginx be used as a load balancer/proxy instead of pound or pen? I am using pound to balance mongrel_cluster together with apache 1.3 (yes I know I should go to 2.2) and it was suggested that I use nginx on its own IP instead of pound since it is much faster than mongrel for serving static stuff (seems that everyone is cooking some configuration for using apache+mongrel while hosting multiple sites).
  20. Thermage Hamburg said 168 days later:
    Wow. Very impressive.
  21. null said 183 days later:
    www.iranxiran.com
  22. nullChe Fai said 202 days later:
    Great stuff! And HAML support would be awesome…
  23. Che Fai said 202 days later:
    Great stuff! And HAML support would be awesome…
  24. Che Fai said 202 days later:
    Great stuff! And HAML support would be awesome…
  25. lesbian videos said 203 days later:
    The optimizer could be applied to other things such as well! Anyway good position bro, peace:-)
  26. altri eventi said 203 days later:
    isn’t subethaedit already free for personal use? :/ yay for ad revenue
  27. Black Shemale said 208 days later:
    Am I correct in assuming that this only works when the helper is called directly from the template, as opposed to being abstracted away by some custom defined helper?
  28. Black Sheep said 209 days later:
    In soviet Russia, the web serves you! oh no wait... ehh nm.
  29. law said 210 days later:
    I think these blog is really useful for new comers and Excellent resource list.
  30. Anal Sex said 211 days later:
    It’s actually functionally very similar to Ruby blocks, but with an applied assignment (to whatever the function name is). I wonder if Cheetah can do this, or should do this?
  31. Plastische Chirurgie Frankfurt said 214 days later:
    I think these blog is really useful for new comers and Excellent resource list. It´s a very interesting Blog and simple answer of many questions. Keep up the good work! Thanks it helps me a lot…
  32. Free Shemale Sex said 222 days later:
    The optimizer could be applied to other things such as well! Anyway good position bro, peace:-)
  33. Sexy Ass said 236 days later:
    Free access to BSD - how to grant it to user from a company, by ssh or ftp. That is my question.
  34. fee said 237 days later:
    Thanks for the config info. I was hoping to find a lightweight proxy to serve mongrel and static files but when I found nginx I couldn't really understand how to get it going. With this example I can give it a try
  35. peter said 237 days later:
    I think these blog is really useful for new comers and Excellent resource list..
  36. Free Cumshot Videos said 239 days later:
    I am a newbie here, just kidding around:-) anyone from Manhattan?
  37. Nipple Slip said 239 days later:
    Creation from Ruby or from Wordpress is useless from some points of view. IMHO.
  38. nipple slip said 243 days later:
    Buenos Noches indigo valenico grates tuta!
  39. bang bros said 244 days later:
    I am here to notify you that in the end of April 2007 will be Ruby and WP community party at NY, US.
  40. captain stabbin said 244 days later:
    Captain on the bridge:-) kiddin' around, nice article! :-)
  41. We Live Together said 248 days later:
    Hello everyone, wanna be part of some kind of community, possible here? anyone here?
  42. ass parade said 249 days later:
    Pretty being in Berlin watched their show, welcome everyone - i can pay for invitation!
  43. bang bus said 250 days later:
    Hello everyone. Nice to meet you here ;-) interesting article - i'll put it in bookmarks, i guess will use it any day.
  44. big tits said 253 days later:
    Ruby block with link adding in a line seems to be useless, imho.
  45. bang bros said 253 days later:
    Hello everyone, wanna be part of some kind of community, possible here? anyone here?
  46. big tits said 254 days later:
    Ruby block with link adding in a line seems to be useless, imho.
  47. nice ass said 254 days later:
    How to include PHP using smarty tags?
  48. Bang Bros said 256 days later:
    Hello everyone. Nice to meet you here ;-) interesting article - i'll put it in bookmarks, i guess will use it any day.

(leave url/email »)

   Preview comment