Deferred requests with merb, ebb and thin

Posted by ezmobius Fri, 18 Apr 2008 06:20:00 GMT

There is a classic tradeoff between threaded servers and event driven servers. Event driven servers tend to be much faster than threaded servers when all the requests are fairly fast. But the event model falls down if you have long requests like file uploads or reporting actions. This is because the long action blocks the event loop, effectively keeping other requests from running.

There are two new event driven ruby webservers at your disposal these days, Thin and Ebb. Both of these servers support the Rack web server interface that merb uses. Until now both of these servers were not the best choice for file uploads or long blocking actions but that’s all changing.

Both ebb and thin have added a deferred?(env) method to their rack adapter interface. Both webservers will call this method on your Rack @app object before they call your call(env) method. This allows your rack adapter to determine if the request should be run in its own thread if it’s slow.

I’ve just committed support for this deferred_actions construct in merb-core. If you want to run on thin or ebb but you have say a file upload action and some slow reporting actions you could add this to your init.rb in your app:

Merb::Config[:deferred_actions] = ["/uploads/create", "/reports/longaction"]

What this means is that all of your actions will run in fast event driven mode except for requests to /uploads/create and /reports/longaction. Any request for either of these urls will be served from a newly spawned thread, all other requests will be served by the main event loop.

This allows us to have the best of both world. Combining threaded and event driven styles to use the strengths of both makes a lot of sense here.

Cheers to the authors of ebb and thin for making the same interface. All of this requires the HEAD versions of ebb or thin so if you want to play along at home you will need to build thin or ebb from source on github: ebb and thin

Tags , ,  | 18 comments

wiki.merbivore.com lives!

Posted by ezmobius Fri, 11 Apr 2008 06:59:19 GMT

Jed Hurt and Lance Carlson made a nice wiki engine in merb that we’ve thrown up at http://wiki.merbivore.com. Thanks Guys!

You can get the source to the wiki app on github: Collective Wiki Engine

There are the starts of some good information published already, please feel free to add to it!

1 comment

Fork You!

Posted by ezmobius Thu, 10 Apr 2008 19:46:00 GMT

GitHub has officially launched! No more invites required to sign up.

In preparation for their launch we set them up on a shiny new Engine Yard cluster. With all the adoption of git/github as well as Rails itself about to switch to github we wanted to make sure everyones favorite git repo hosting can scale as far as folks want to take it.

So go Sign Up already and fork your favorite projects to your hearts desire!

Tags  | no comments

Videos of my Merb keynote posted by Confreaks

Posted by ezmobius Sat, 29 Mar 2008 21:38:33 GMT

Confreaks has done a great job of recording Mountain West Rubyconf and getting the video online quickly. You can go see my talk here:

Strengthening the Ruby Ecosystem: Merb

9 comments

Mountain West Rubyconf Merb Slides

Posted by ezmobius Sat, 29 Mar 2008 00:09:00 GMT

Here are my slides from my keynote at Mountain West Rubyconf this morning:

http://www.slideshare.net/ezmobius/merb-core/

Tags  | 2 comments

Merb-0.9.1 Developer release

Posted by ezmobius Wed, 05 Mar 2008 06:17:00 GMT

We’ve released the 0.9.1 version of merb to the merbivore.com gem server. This release has a lot of polish and is getting very close to being stable api wise after the big 0.5.3 → 0.9.x refactoring.

You can see the big changelog here.

I’m pretty happy with how the codebase is shaping up, this was a major refactoring of merb and we’ve come out with a very clean system. Performance is improved quite a bit from older 0.5 versions of merb.

We’ve split the code base into multiple parts, merb-core, merb-more and merb-plugins. merb-core is the heart of the system, it has the rack abstraction along with the dispatcher, router, controller and view layers. You can make very fast, small footprint services and apps with just merb-core.

merb-more has a bunch of add-ons for core. More consists of:
merb-action-args
merb-assets
merb-builder
merb-gen
merb-haml
merb-mailer
merb-parts
merb-test

And merb-plugins consists of:

merb_activerecord
merb_datamapper
merb_helpers
merb_param_protection
merb_sequel
merb_stories
merb_test_unit

Wow that’s a lot of gems! Merb is built in a modular way that allows you to cherry pick features so you never have to load code you aren’t going to use. This helps keep the memory footprint down when building service style apps. But still allows for all the advanced features you want.

You see, Merb is built on rubygems and Merb plugins are just rubygems so plugins have the same standing as built-ins code loading wise. Merb is just begging you to peek under the hood to see how it ticks ;)

To make things easier to get started we still have a merb gem that will install all of merb-core and merb-more for you. You should uninstall all of your old merb gems before you install the new version.

$ sudo gem install merb -y --source http://merbivore.com

Merb development has moved to GitHub for source control and Lighthouse for ticketing.

The best place to get merb questions answered is still #merb on irc.freenode.net. We will have a wiki and a google group up shortly.

Let us know what you think, kick the tires and all that. After it settles for a few days we will push it to rubyforge, hopefully by this weekend.

6 comments

At long last my Deploying Rails Applications book is done!

Posted by ezmobius Thu, 21 Feb 2008 23:32:00 GMT

So after being in the works for almost 2 years now and after going through many rewrites as Rails deployment tech has changed, my book is finally done! I just have to thank all of my coauthors who put in tons of work to get this thing as polished as it is now. Especially Bruce Tate and Clinton Begin, thanks guys!

New in this release is the chapter on Apache and Nginx scaling as well as a performance chapter. ALso included is setting up your own Xen installation and setting up Mysql Master → Slave and Master → Master replicated databases.

If you were holding off on getting this book until the advanced content was in there then now is the time to snap it up. This is the final beta release and the book will be going to print fairly soon. I really like the way the book has come together and I think it is a wealth of knowledge about deploying Rails applications.

\m/ Yay! Writing a book was a lot more work then I thought it would be and I could not have done it without my wonderful co-authors and all of the great feedback from the early beta readers.

Thanks to everyone who provided feedback. You can grab your copy of the book here:

Deploying Rails Applications

Tags ,  | 18 comments

So merb-core is built on rack you say? Why should I care?

Posted by ezmobius Sat, 16 Feb 2008 00:11:00 GMT

Rack is an awesome webserver abstraction that distills the idea of a ruby web app down to an object or Proc that has a call method. The call method takes the rack environment, which is all of the cgi style headers for the current request, and returns an array of [status, headers, body]. The status is a number like 200 or 404, the headers is a hash of header key value pairs for the response and the body is the output from your application, The body must respond to each and yield lines or chunks of strings to be flushed down the socket to the client.

Here is the most basic example of a rack app using the rackup builder DSL for mounting and running rack apps:

rack_app = Proc.new do |env|
  # env here has all the headers you would expect
  [200, {"Content-Type"=>"text/html"}, "hello world!"]
end  

run rack_app

This app will return “hello world!” to the client. Pretty simple eh?

Now that merb-core is all based on rack, there are some very interesting things you can do with this knowledge. In the config/ directory of a freshly generated merb-core app you will see a rack.rb file. By default the file just contains this:

  
run Merb::Rack::Application.new

This will run the main merb rack app class that handles the standard dispatching of requests through the whole merb framework. Now merb-core is small and fast, but what if you have some certain requests that don’t really need the router or controllers/views of a full merb stack.

Say we need to handle a ton of concurrent file uploads, and we don’t want to invoke the full merb stack just for these uploads. The solution is to use the config/rack.rb file in your merb app along with the Rack::Cascade middleware to mount a simple Proc object to handle the uploads instead of merb proper. Here is what a rack.rb file for this would look like:

uploader = Proc.new do |env|
  request = Merb::Request.new(env)
  if request.path =~ /\/images\/upload/
    #file uploads can get the params from request.params and do whatever 
    # you want with it, this allows for multiple concurrent uploads
    # with very minimal overhead, doesn't go through the merb 
    # framework at all
    params = request.params
    FileUtils.mv params[:file][:tempfile].path,
                 Merb.root / 'public' / 'uploads' / params[:file][:filename]

    headers = {"Content-Type"=>"text/html",
              "Location" => "/images"}
    [302, headers, "You are being redirected"]
  else
    [404, {}, "No Upload here, move along"]
  end    
end  

merb = Merb::Rack::Application.new

run Rack::Cascade.new([uploader, merb])

Rack::Cascade works by trying to call each app you specified in order and actually use the results from the first one that does not return a 404 error. So what our new uploader Proc/app does is creates a Merb::Request object and checks if the request.path matches /images/upload. If it does match then you can use code in here to handle the file upload and place it wherever you want. Once you’ve done whatever you need to the file upload you can redirect by setting the Location header like in the above example, or you can render out a page or return JSON or really anything you like.

Having the power of merb-core and also the power of raw rack access conveniently in the same app is very powerful. I think this is one very compelling thing merb-core has going for it now that we are all rack based.

Hopefully you enjoyed this installment of what is new and cool in merb. Hope to write some more articles shortly about special features in merb-core that go undernoticed.

Tags , ,  | 9 comments

What do you want to see in mod_rubinius?

Posted by ezmobius Tue, 12 Feb 2008 00:25:00 GMT

So we are just getting started on mod_rubinius here at EY. We’ve hired Eero Saynatkari ( rue in the #rubinius irc channel) full time to work on the project.

The architecture for mod_rubinius is still up in the air at this point. We do know that it will be rack based so the interface from mod_rubinis into ruby apps will be via rack. Other then that we don’t yet know what the best way to architect the platform will be. It could be an embeded rubinius VM inside the apache processes. Or it could be a process manager that manages separate rubinius VM’s, or a combination of both of these approaches.

We would like to hear from you folks.. What would you like to see in a mod_rubinius? How is deployment of ruby apps painful for you and how would you like it to work in a perfect world?

We’d appreciate some feedback so please leave feature requests and ideas in the comments.

Tags  | 31 comments

Engine Yard Is Hiring Application Support Engineers

Posted by ezmobius Wed, 06 Feb 2008 20:25:00 GMT

Want to work for the best and with the best? Are you interested in merb, rails, rubinius, and other ruby projects? Are you passionate about learning what you don’t know?

This position requires direct interaction with customers through the phone, ticketing system, irc, and email. You will be supporting some of the largest rails deployments on the internet. Can you handle it?

Desired: Experience with memcached, merb, monit, postgres, swiftiply.

Required: Solid ability to speak and write English. Good Attitude. Great work ethic. Experience with capistrano, mongrel, mysql, nginx, rails, subversion. Basic networking knowledge.

This position is right someone who isn’t afraid of hard work, loves facing new challenges each day, and wants to contribute to Engine Yard’s projects.

We offer comprehensive dental and medical insurance as well as the ability to work from home (or anywhere). Stock options are around the corner too.

Drop us a line at info@engineyard.com and tell us why you would be a great addition to our team. If you are the right person—we will send you an offer immediately. It’s a bonus if you are on the West Coast, maintain open source projects, or have a great community presence.

Older posts: 1 2 3 4 5 6 ... 15