At long last my Deploying Rails Applications book is done!

Posted by ezmobius Thu, 21 Feb 2008 23:32:00 GMT

So after being in the works for almost 2 years now and after going through many rewrites as Rails deployment tech has changed, my book is finally done! I just have to thank all of my coauthors who put in tons of work to get this thing as polished as it is now. Especially Bruce Tate and Clinton Begin, thanks guys!

New in this release is the chapter on Apache and Nginx scaling as well as a performance chapter. ALso included is setting up your own Xen installation and setting up Mysql Master → Slave and Master → Master replicated databases.

If you were holding off on getting this book until the advanced content was in there then now is the time to snap it up. This is the final beta release and the book will be going to print fairly soon. I really like the way the book has come together and I think it is a wealth of knowledge about deploying Rails applications.

\m/ Yay! Writing a book was a lot more work then I thought it would be and I could not have done it without my wonderful co-authors and all of the great feedback from the early beta readers.

Thanks to everyone who provided feedback. You can grab your copy of the book here:

Deploying Rails Applications

Tags ,  | 18 comments

A Fair Proxy Balancer for Nginx and Mongrel

Posted by ezmobius Fri, 09 Nov 2007 22:50:00 GMT

Nginx is my favorite webserver as you astute reader all probably know. It’s very fast, light and rock solid in production. My only complaint was that the proxy modue was round robin only. This means that if you had 3 mongrels and one of them performed and action that took say 20 seconds, for that 20 seconds every 3rd request would get stuck in line behind the mongrel doing the long blocking request. This is because nginx was merrily doing round robin and didn’t care if a mongrel was busy or not.

Now we all know that you should not have actions that take this long in our apps right? Of course we do, but that doesn’t stop it from happening quite a bit in a lot of apps I’ve seen.

So EngineYard.com put out a bounty on getting a fair proxy balancer that would keep track of when a mongrel is busy and not send requests to it until it has finished and is ready to take requests. This means that now we can effectively queue inside nginx very efficiently and only send requests to non busy mongrels. Thereby avoiding the waiting in the wrong line for service problem described above.

Grzegorz Nosek has stepped up to claim the bounty and has done a wonderful job of making this work very nicely. this is not a production release but I wanted to get this out there to get some testing from interested parties.

You can grab a tarball of the git repo here:

http://git.localdomain.pl/?p=nginx.git;a=tree;hb=upstream_fair-0.6

click the snapshot link to download a tarball. Then you can compile nginx with the new module like this:

$ cd nginx.git $ ./configure—with-http_ssl_module $ make $ sudo make install

The only difference in configuration will be to add the “fair;” directive to your upstream block. For example:

  upstream mongrel {
    fair;
    server 127.0.0.1:5000;
    server 127.0.0.1:5001;
    server 127.0.0.1:5002;
  }

Here is an easy way to try this out and verify it’s working for you. Create a rails or merb app with one action that does a “sleep 15” and another action that returns some text immediately. Fire up a cluster of 3 mongrels behind your newly compiled nginx with the fair directive commented out.

Now open up 2 browser windows and hit the sleepy action in one window and repeatedly hit the other action in the second window. with the fair directive commented out every 3rd request for the fast action will hang until the sleeping action is done. This is because mongrel is queuing requests and nginx is just using round robin.

‘Now uncomment the “fair;” line and restart nginx and do the experiment again. This time you will notice that nginx avoids sending any requests to the sleeping mongrel and only balances between the free ones until the sleep finishes, then it is added back to the available pool.

Please give this a shot and beat it up some folks. Report any problems you have here in the comments. This is going to be huge for any rails apps running under nginx and I;’d like to ferret out any bugs before we have an official release.

Tags , ,  | 23 comments

Deploying Rails Applications is Released as a Beta!

Posted by ezmobius Thu, 16 Aug 2007 16:13:00 GMT

Just a quick note to say that after 22 months and many many rewrites my Book “Deploying Rails Applications” Has gone Beta! This is a super early beta and we will be shooting to add a chapter every 3 weeks until the print book is done.

I have to give a huge thanks my coauthors Bruce Tate, Geoffrey Grosenbach and Brian Hogan. Thanks for making it happen guys!

Now this is an early beta so expect rough edges and markup. But we wanted to get this out there before deployments changes again and caused another rewrite ;)

Deploying Rails Applications( A step by step guide)

Tags , , ,  | 7 comments

Slides from my Mongrel/Merb talk

Posted by ezmobius Mon, 23 Apr 2007 15:35:00 GMT

Here are the slides form my talk on Custom Mongrel Hanlders and Merb at the SDForum Rubyconf this past weekend. There was video shot that I hope will be posted soon. The slides don’t show the entire story as I did a bunch of live benchmarks and some hacking and code exploring, but you can get the idea.

Mongrel: Learning how to walk the dog

Tags , , ,  | 5 comments

Building Custom Mongrel Handlers for Speed and Concurrency

Posted by ezmobius Thu, 05 Apr 2007 18:59:00 GMT

I will be giving a talk at the SDForum Ruby Conference April 21&22. I speak sometime on Sunday. Here is my talk’s abstract. I will post slides and hopefully audio after my talk.

All the cool kids are using Mongrel to run their Rails applications these days. And sometimes Mongrel itself gets so tightly coupled with Rails in peoples thoughts that they don’t realize that Mongrel is something they can use outside of Rails to build their own fast, custom HTTP servlets.

Zed Shaw has built an awesome HTTP server library that is easily extensible and very high performance. In this talk we will explore how to make your own hybrid Application Server with a custom Mongrel handler and some Ruby code.

The web is moving towards a world of resources and services. Rails is the tool of choice for building these web2.0 applications. But sometimes you have a critical section of your application that Rails is just too slow at serving, or you need to be able to support many concurrent users in portions of your site. Learning how to build custom Mongrel handlers will help you scale these performance critical areas of your application.

We will build a simple dynamic javascript server from scratch. This type of app can be used for web stat tracking or javascript widget serving. We will end with some benchmarks showing off how fast a Ruby web service can be when you get closer to the metal.

Tags ,  | 3 comments

EngineYard.com gets some love

Posted by ezmobius Sat, 27 Jan 2007 21:16:00 GMT

We have been extremely busy working on Engine Yard. I couldn’t be happier with our progress. We are already running more then 130 slices and growing rapidly.

But I think our website didn’t do a very good job of explaining what kind of service we realy offer. Also it wasn’t clear what an EngineYard slice really gets you.

Well Damien Tanner from new-bamboo.co.uk and Lance form EY have been working very hard on revamping our site to better showcase our offerings. So here is our new updated website:

www.engineyard.com

And the what you get page is much better now.

what’s a slice really mean?

Tags , ,  | 3 comments

Merb 0.1.0

Posted by ezmobius Fri, 19 Jan 2007 05:02:00 GMT

This is mainly a big fix release. The 0.9.0 release had a bad app skeleton. I have fixed this problem.

I also added support for freezing the merb framework into your app for portability. Just deploy the entire app and no need to install the merb gem on your server.

A few refactors and updates go in as well. Gem install merb in a few hours when the gem propogates.

Tags , , ,  | 4 comments

Merb 0.0.9 released

Posted by ezmobius Mon, 15 Jan 2007 01:54:00 GMT

I have just released version 0.0..9 of Merb to rubyforge. Within a few hours you should be able to gem install merb and get the new version.

There have been a ton of updates since the last release. It’s now easier to get started with merb. There is an app generator as well as mrblog, a lightweight blogging engine build on merb(not finished but good for examples)

Merb is running in production on more then 12 apps that I know of. Some of these apps are handling insane amounts of file uploads every day. And a few javascript tracker style apps are running on merb now for speed.

Here is an excerpt from the changelog. I know I missed a ton of stuff because merb has grown twice as much code since the last release and is working sweet now.

* Added merb app generator  $ merb -g appname
* added super snazzy error pages and stack trace. install the coderay gem to get syntax highlighting of error code pages.
* many fixes for efficeincy of overall framework. 
* extensiuve refactoring since last version
* form helpers
* template caching
* production test and development environments.
* upload progress built in now. no need for mup gem.
* added helper system.
* Updated merb rails session parasite mode.
* vendored Paginator gem
* added memory sessions
* added transactional yaml store.
* before and after filters, with very powerfult filter chain halting and redirection
* refactor merb_server to be in the framework and not in bin/
* catch_content and throw_content
* updated routing tobe more precise
* added status codes module
* added Request object
* added Merb::Mailer frameowkr
* fixed up exception handling 
* added basic auth mixin
* added drb service provider
* refactored merb halnder for speed
* added Enumerable#injecting
* too many more tweaks to list

You can get your own merb app started in a few steps with the new app generator:

$ sudo gem install merb
$ merb -g myapp

And you can look at mrblog to see a merb app with a decent amount of functionality to see how merb works a little bit easier then the old sample app:

mrblog svn

And the rdoc has been updated quite a bit as well:

Merb Rdoc

Stay tuned for some detailed tutorials on making a secure file upload/download server with merb and integrating it with your rails app.

Tags , , ,  | 5 comments

New Nginx.conf with optimizations

Posted by ezmobius Wed, 03 Jan 2007 22:28:00 GMT

I have been working on optimizing my nginx.conf file as I use it on more and more sites. Thanks to folks on the nginx and mongrel mailing lists for some of these fixes. Main improvements is in gzip of dynamic and other content as well as setting the proxy buffers to 0 and a few improvemets in the proxy header settings. This conf also includes a second vhost for ssl that points to the same mongrel cluster so you can hanlde ssl and non ssl with the same cluster and rails request.ssl? helper will work correctly.

Another big improvement is with static file handling. The credit on this one goes to Zed Shaw. He notcies that the rewrites and regex tests for rails cached pages were getting run for every request including static images. So a request for /images/foo.jpg woudl get served in this manor:

/images/foo.jpg/index.html  # no file found continue
/images/foo.jpg.html  # no file found continue
/images/foo.jpg # good send it!

This is obviously a performance hit for static files as it has two extra regexes and two extra file stats for every static image. Suck, I wish I found this sooner. To fix this all we need to do is add this test right at the top before the other rewrite tests:

      # If the file exists as a static file serve it directly without
      # running all the other rewite tests on it
      if (-f $request_filename) { 
        break; 
      }
In my benchmarks this does increase the speed of static files about 8%. Also allowing gzip on HTTP 1.0 requests made sure that the proper static assets are gzipped when possible. Also we force all proxied requests to be gzipped if they are the right content type. I also added more gziped content types and corrected a wrong one for javascript.

This is an important lesson about your production environment folks. With regards to what you accept as gospel from someone like me. “Trust but verify”. Zed was trying to track down how to do a custom rewrite for alternate cache directories and so he at my behest turned on the debug logging and consequently saw the rewrites happening on every static file. I haven’t been in the rewrite log for a while since I thought my config was good. So when you get config files from people like me off the net, please trust but verify on your own systems.

This new config is running close to 100 instances of nginx at Engine Yard. Nginx has proven to be an excellent choice for a front end proxy balancer for mongrel. It beats apache in all my tests now as far as non gzipped proxy requests, static file requests and it ties with apache for gzipped proxy requests. And its about 30 times more lightweight then apache.

If you were holding off to try nginx until it was proven out by others then let me tell you it has proven itself to me time and time again already as a very easy to manage, high performance options for rails deployment on mongrel. I don’t personally use apache2.2 right now for anything other than mod_svn for subversion over HTTP. But there is stirring of upcoming support for nginx and svn over webdav so stay tuned. When that happens I will happily drop apache from our rotation entirely. Of course take this with a grain of salt as always, I don’t have to support any php apps or legacy stuff right now, just rails. I haven’t personally used php with nginx yet but I see others getting fantastic results so I am not worried for when I may need it.

All in all this new config file performs significantly better then my old one. Somewhere on the order of 20% total improvement overall over the last config file. I have updated the link to poitn from the old article to the right confi file and I am linking it here again for your enjoyment:

New Nginx.conf

Tags , , , ,  | 17 comments

Merb 0.0.8 released

Posted by ezmobius Sat, 16 Dec 2006 18:49:00 GMT

I am happy to announce the release of Merb 0.0.8. This release is a huge improvement over the last release. There are too many changes to list. So here are some highlights:

* Performance optimization of mime parsing.
* Refactoring of entire codebase. 
* reorganize layout of source files.
* Added GlobaleHelper that all view can use. 
* Helpers named after the current controller will be available in the view as well as global helper.
* Added support for returning an IO stream from your controller.
* Merb::SimpleModel is a transactional yaml storage for smaller projects.
* Cleaned up the app_skeleton, it is now up to date.
* Added Merb::Const module to incapsulate constants
* Added HTTP status codes module
* Optimized and improved the ViewContext object
* Added rails session parasite mode
* Added new improved ActivceRecord session store. much faster
* Added Request object to encapsulate all helpers that use the headers to grab info
* Added merb_upload_hanlder for upload progress.
* Added merb_upload_progress class. if only one merb is being run then it will just load the upload prgress class in memory. When more then one merb is run, merb has a drb server that gets started and holds the state for the upload progress meter.
* Added merb_exceptions for custom, exceptions
* Added Kernel#aquire, like require on steroids with glob
* Huge improvements in the before/after filter code
* Added after filter capability to Merb::Controller.

I am working on a tutorial that I will post this weekend as a quick primer to getting started with Merb. The rdoc is improved though now and has some good information. The new implementation of befopre and after filters is really sweet too.

sudo gem install merb

rdoc: http://merb.rubyforge.org

Tags , , ,  | no comments

Older posts: 1 2