Hey Rails, nice Rack!

Posted by ezmobius Fri, 25 Apr 2008 00:34:00 GMT

So i’ve spent this week hacking on Rails, specifically going spelunking in ActionPack and porting Merb’s rack machinery to rails. I figure that merb is a very nice experimentation ground and decided it was time to give some love back to the framework that inspired merb.

While still not complete, I have made significant headway on racking up rails in my github fork of Rails. I’ve added rack adapters for mongrel, eventedmongrel, thin, ebb and webrick. All of this is controlled via ./script/rackup in a rails app. So to start a cluster of 5 thin servers you would run this command:

./script/rackup -a thin -c 5 -e production

I do have to say that parts of ActionPack haven’t been touched in a long time and had accumulated some cruft, with the rails rack adapter that thin and ebb use there was a dogpile of wrappers going on. A web request was wrapped in many layers like this(the → means ‘wrapped in’)

raw request -> rack env -> Rack::Request -> CGIWrapper -> CgiRequest

That was far too many wrappers, it also turned out that some of these wrapper were making duplicates of all the CGI headers, meaning quit a bit of wasted memory on every request. I’ve slimmed it down to this now:

raw request -> rack env -> ActionController::RackRequest

With no more duplication of CGI headers, win!

I’ve also changed the giant mutex lock to be much smaller, it used to lock around the dispatcher callbacks, route recognition, controller instantiation, filters and action dispatch. Now it only locks around filters and action dispatch, Dispatcher callbacks, route recognition and controller instantiation all happen outside of the lock. This makes for a nice little speed boost with standard mongrel under concurrent load.

Of course this doesn’t work so hot in development mode when you have concurrent requests, it falls apart due to route set reloading and class reloading done in dev mode. But if you are just using your browser to test the app in dev mode you won;t ever notice since you won;t be making concurrent requests. In production mode the route recognition appears to be thread safe, more extensive testing and stressing will be needed to be 100% certain though.

All in all I feel like these are some big wins for Rails. And I’m not done yet, I plan on beating up ActionPack quite a bit more until it submits ;)

Tags  | 40 comments

Comments

Leave a response

  1. Maxime Guilbot said 37 minutes later:
    Super cool! I am amazed by those changes and expecting to try them soon.
  2. Ian Ownbey said about 1 hour later:
    This has been a long time coming. I approve.
  3. Brian Dainton said about 1 hour later:
    Amazing stuff, Ezra. I'm anxious to give your changes a whirl.
  4. Stoyan Zhekov said about 2 hours later:
    Is it possible to use the rack middleware (like use ..., map ...)? And if yes, how to use/where to put the .ru file? For example with thin something like: 
    thin start .... -R rails.ru
    btw, great job :)
  5. Ezra said about 2 hours later:
    @Stoyan- Yeah if you create a config/rackup.rb in your rails app you can use middleware and all that fun stuff.
  6. Rodrigo Kochenburger said about 2 hours later:
    Sweet! :)
  7. Daniel Fischer said about 4 hours later:
    Rock on Ezra, this looks like some very valuable changes. I appreciate your donation of time and skill!
  8. Trevor Turk said about 4 hours later:
    go! go! go!
  9. AkitaOnRails said about 4 hours later:
    Awesome! Kudos again for Ezra. I was looking Merb growing up nicely and I was wondering if anyone would (even myself when I manage to have more time) to backport some of the Merb's niceties back into Rails itself. It is a breadth of fresh air to see you himself doing just that. Kudos++ for this. Hope you can bring even more of your experience to the Rails core (and I hope the Core Team is smart enough to pull your changes back into trunk).
  10. Antonio Cangiano said about 6 hours later:
    Ezra, the force is strong in you. :)
  11. Kurt Schrader said about 7 hours later:
    Awesome. Hopefully even more of what you've learned while writing Merb can be applied to Rails too. Now that Rails is finally maturing there certainly are a number of dark, dusty corners in the code base that can be cleaned up.
  12. Damien Le Berrigaud said about 7 hours later:
    Simply great :)
  13. Matthijs Langenberg said about 8 hours later:
    Rack support for Rails! I love it! By the way, how's mod_rack / mod_rubinius going on?
  14. nap said about 12 hours later:
    Very cool. I really do hope these changes get pulled into the master repo. It seems a bit silly at this point that Rails isn't (already) using Rack under the hood and would place even further emphasis on the need for a real mod_rack :). == everybody wins.
  15. Fred Lee said about 13 hours later:
    Awesome Ezra! This probably a stupid question. But, how do I start using/testing this fork? Do I git clone it into my vendor directory?
  16. macournoyer said about 14 hours later:
    omg, nice work! especially on the removing of the CGI wrapper, I tried that but it was too much work :/ And moving the mutex deeper is equally delicious!
  17. johan said about 17 hours later:
    I'd hit that
  18. Phil said about 19 hours later:
    Awesome sauce. Hope this makes it into 2.1.
  19. Bruno Michel said about 19 hours later:
    It rocks. Thanks for doing it :)
  20. Lucas Húngaro said about 20 hours later:
    Congratulations! Keep up the awesome work!
  21. Drogomir said about 22 hours later:
    Great job! :) Keep going. Rails definetely needs some more love.
  22. meekish said about 22 hours later:
    Mad props
  23. Tobias Lütke said about 22 hours later:
    Awesome ezra. Thank you so much.
  24. Pratik said about 23 hours later:
    Awesome stuff :-) !
  25. Chad Fowler said 1 day later:
    Nice job Ezra!
  26. Maximilian Schulz said 1 day later:
    Wow… this sounds amazing! Can't wait to check it out.
  27. Stefan Saasen said 1 day later:
    That is great news! Works like a charm.
  28. zerohalo said 1 day later:
    Awesome! Thanks, Ez. Will these changes at some point be committed to the main Rails trunk?
  29. Finanzamt said 1 day later:
    Helpful post! Thank you!
  30. Anil Wadghule said 2 days later:
    This is really great! :)
  31. Adam said 3 days later:
    Awesome work man! You are my personal hero! I have the same question that zerohalo has and that's are there talks about this being added to the "official" branch or will this be something that sits on the side? If it is the latter, what's the strategy for continuing support of the new rails code? In addition to that, this is an open question to all: Has anyone tried to see if these changes still work with mod_rails? If no one has, I suppose I can look into seeing if this works later.
  32. Ezra said 3 days later:
    yeah the idea is for this to be merged into rails proper once it is complete and tested, the core team knows of my work and says it will make it in once it is done. As far as mod_rails goes... I think they made a huge mistake by not using rack as their interface, so I'd hope they will realize this and add rack support. mod_rubinius is all rack based and will be able to run any ruby framework or simple rack handlers you can throw at it.
  33. MIke Costa said 3 days later:
    Thanks for your work Ezra. I was hoping you would port Merb thread-safety to Rails one day. You're a winner!
  34. Alan Brown said 3 days later:
    "just using your browser to test the app in dev mode you won;t ever notice since you won;t be making concurrent requests" Can't concurrent requests happen alot, even in test mode, especially if your app uses AJAX? Appreciate the work on this stuff. There's a Moore's Law for Rails and you're making it happen.
  35. DaveS said 5 days later:
    Whoa... thank you Ezra!!!! PS: Why isn't Ezra part of rails core???? He's amazing!
  36. Kevin Stewart said 6 days later:
    Great work, Ezra! Of course, this now raises an interesting question. Merb was created to address technical issues with Action Pack. If the approaches pioneered in Merb are now pushed back into Rails, why should one continue to use Merb? Or, more importantly should the Merb efforts be directed towards making Rails better? It's sort of like you forked and are now bringing the forks back together. I'm currently using Merb (quite happily, BTW) but if Merb development stalls to "fix" Rails I would be a little concerned. Not that I have anything against Rails, I just like Merb's philosophy more. Is it the philosophy that will keep Merb and Rails separate, or will they evolve and converge toward esch other?
  37. Ezra Zygmuntowicz said 6 days later:
    @Kevin- At this point merb is not going anywhere. If the rails core team wanted to get radical and decide to build rails 3.0 on top of merb-core technology then I might consider merging the two projects. But I think that would take a lot of politics and I'm not sure if they would agree to it. So don't fret, merb is here to stay unless the rails-core has a drastic change of heart. And the code base for merb is *way* more simple and scalable for the future since it has no baggage. So expect merb to stick around and innovate for a long time to come. I just figured that there is no reason for everyone using Rails to suffer when I could spend a few weeks and improve a bunch of stuff for folks.
  38. Kevin Stewart said 6 days later:
    @Ezra: Whew! That's good to hear. Now, how about releasing 0.93? Or better yet, 1.0? :-) Sorry, I couldn't resist. I'll get back to working on my Merb app now...
  39. Yuri Leikind said 8 days later:
    Been doing Rails development professionally (i.e. getting paid for this :) for some time now. Watched your MountainWest RubyConf presentation, got impressed and finally realized that Merb is not a toy or a bastardized version Rails :) Your work on the crufty guts of Rails is promising, really looking forward to see this included in Rails. Keep it up, you're a coding Jedi.
  40. random8r said 23 days later:
    I'll notice making concurrent requests, because in one of my apps, I'm doing concurrent requests. How I've done it in the past has been to run two copies on different ports, and change the parts of code the do concurrent requests to request on the different port. KLUDG-O-RAMA. I love you Ez. -Random8r

RSS feed for this post trackback uri

(leave url/email »)

   Preview comment