Event Driven Mongrel and Swiftiply Proxy

Posted by ezmobius Sat, 12 May 2007 08:49:00 GMT

Kirk Haines of the IOWA project has finally released his swiftiply package. I’m really excited about this project. The project has a few parts. There is a monkey patch to Mongrel that rips out ruby’s threading and Socket classes and replaces them with an eventmachine event loop. I have been testing this evented mongrel extensively for the past few weeks and it is notably faster and higher throughput then normal multi threaded mongrel.

It’s a drop in compatible with rails and merb and should work with any custom mongrel handlers you have. Once you install the swiftiply package you can start mongrel_rails with the new evented mongrel like this:

$ EVENT=1 mongrel_rails start

I am getting a huge speedup in merb from evented mongrel. And rails apps run very stable and do not start to slow down and bloat under high concurrent load like threaded mongrel does.

This is big news people. This thing makes shit faster and more stable under load

Now that it the easy way to get started with evented mongrel and buy yourself some performance. But the swiftiply proxy is also very very cool.

Swiftiply works differently from a traditional proxy. In Swiftiply, the backend processes are clients of the Swiftiply server—they make persistent socket connections to Swiftiply. One of the major advantages to this architecture is that it allows one to start or stop backend processes at will, with no configuration of the proxy. The obvious disadvantage is that this is not behavior that backends typically expect.

Luckily there is a swiftiplied version of mongrel in this package as well. I’ll make another post about how to use swiftiply later.

A week or so ago I was able to speed merb up from 299req/sec fro hello world to 620req/sec. Well dropping in evented mongrel gets merb’s hello world to a blazing 987req/sec!! no bull, see this pastie:

http://pastie.caboo.se/57881

So go grab swiftiply and play around a bit. I am using this in production for more then a week now with no adverse affects and the app that runs it has much better response times and memory usage then it did before.

http://swiftiply.swiftcore.org/index.html

Tags , ,  | 31 comments

Comments

  1. jherber said about 5 hours later:
    "And rails apps run very stable and do not start to slow down and bloat **under high concurrent load** like threaded mongrel does." under high concurrent load, mongrel should be running under mongrel cluster as a mongrel instance queues requests beyond the one in the giant lock being processed by the unthreadsafe-rails. my question is this - does a cluster of evented mongrels outperform a cluster of non-evented mongrels by a similar amount? i know async state machines are the most performant designs, but is your experience being positively skewed by the architecture and test data?
  2. Jeremy said about 7 hours later:
    The switfiply site has benchmarks for single as well as multiple mongrels proxied through swiftiply that seem to indate performance is better both in single mongrel instance as well as proxied through swiftiply. There is also a comparison against HAProxy. This is very good news. I'm sure they'll be a lot of chatter about it on the Mongrel list, but I need to start testing this out. Ezra, I know you'll keep us posted on your results longer term.
  3. Jeremy said about 7 hours later:
    The switfiply site has benchmarks for single as well as multiple mongrels proxied through swiftiply that seem to indate performance is better both in single mongrel instance as well as proxied through swiftiply. There is also a comparison against HAProxy. This is very good news. I'm sure they'll be a lot of chatter about it on the Mongrel list, but I need to start testing this out. Ezra, I know you'll keep us posted on your results longer term.
  4. Matt said about 11 hours later:
    How long til this makes it to Engine Yard as the standard implementation?
  5. Ezra said about 12 hours later:
    @ jherber- Of course you need a cluster of mongrels to handle concurrent load with Rails. But when you get traffic spikes, mongrels in a cluster will still get overloaded when too many requests queue up in the threaded model. Evented mongrel does not exhibit this behavior. Its faster for a single mongrel and a cluster of mongrels and behaves much better when you abuse it.
  6. Luis Lavena said about 15 hours later:
    Ez, this excellent news means my secret project is busted! Whatever, is great to see some progress on event-driven designs. Added to my list of "Things to Check if they will work on Windows or require too much hacking". Thanks Kirk for the work, and you for the news!.
  7. Kirk Haines said about 19 hours later:
    Luis, it should work on Windows, but I admit that I haven't tested it on Windows, yet. If I am not to busy with family stuff tomorrow, I'll check it and see if I run into any issues.
  8. Scott Fleckenstein said about 22 hours later:
    Exciting Stuff! Free speed boosts for all! Using the simplest of hello world mongrel handlers, I was getting 3000+ requests per second, nearly 3x what I get for mongrel alone. Granted, that is imaginary world... Weird issue i'm getting though, the server seems to fail when load is sustained: ab -v -c 1 -n 55000 http://localhost:9998/ nets about 30000 failed requests whereas pure mongrel just chugs along. This is really exciting though, i'm looking forward to your next post on swiftiply.
  9. Lourens Naude said 1 day later:
    Excellent find. However, on startup it doesn't seem to queue requests the way stock Mongrel does and have to do a 2 or 3 second 'sleep' in my Capistrano restart task to avoid 500's being raised. Anyone else experience the same?
  10. Kirk Haines said 1 day later:
    Scott, would you email me details about what you are doing so that I can try to reproduce it. I've not seen anything like that in my hammering and abusing of this code. Same with you, Lourens. Could you email me the details of what you are doing? Thanks, Kirk Haines
  11. Jeremy said 1 day later:
    It seems as though the evented_mongrel does not handle invalid http format gracefully, at last for me: /usr/lib/ruby/gems/1.8/gems/swiftiply-0.5.0/src/swiftcore/evented_mongrel.rb:33:in `execute': Invalid HTTP format, parsing fails. (Mongrel::HttpParserError) from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/configurator.rb:293:in `join' from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/configurator.rb:293:in `join' from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/configurator.rb:293:in `each' from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/configurator.rb:293:in `join' from /usr/lib/ruby/gems/1.8/gems/swiftiply-0.5.0/bin/mongrel_rails:145:in `run' from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/command.rb:211:in `run' from /usr/lib/ruby/gems/1.8/gems/swiftiply-0.5.0/bin/mongrel_rails:252 from /usr/bin/mongrel_rails:18:in `load' from /usr/bin/mongrel_rails:18
  12. Kirk Haines said 1 day later:
    Jeremy, thanks. I will address that ASAP.
  13. Luis Lavena said 1 day later:
    Thanks Kirk, doing some testing on event-machine alone, but tests fails (and segmentation fault), will investigate further later.
  14. Kirk Haines said 1 day later:
    Luis, are you on OSX, too?
    Scott sent me his little test script, and on my systems I can't get a single failure.
    I tested it with up to 100k requests at concurrencies from 1 to 100. No failures at all. They all look pretty much like this:

    Concurrency Level: 100
    Time taken for tests: 19.542144 seconds
    Complete requests: 100000
    Failed requests: 0
    Write errors: 0
    Total transferred: 252200000 bytes
    HTML transferred: 240000000 bytes
    Requests per second: 5117.15 [#/sec] (mean)
    Time per request: 19.542 [ms] (mean)< br /> Time per request: 0.195 [ms] (mean, across all concurrent requests)
    Transfer rate: 12602.97 [Kbytes/sec] received

    I do not have an OSX machine to test with, though, so that might be the common thread in these failure reports.
  15. Luis Lavena said 1 day later:
    Kirk, no, I'm on Windows, but event-machine tests failed on me, they are broken or Windows it is ;-) Didn't jump to swiftiply yet, will do this week and check stability of a few applications. Anyway, mail you later about Win32 issues (if they show up).
  16. Kirk Haines said 1 day later:
    Ah! An updated EM gem was going to be uploaded, but Francis C. was travelling all week, so that may not be done yet. I bet that is where the trouble lies.
  17. Kirk Haines said 1 day later:
    Just FYI. I'll be releasing a 0.5.1 that fixes a few bugs, tomorrow.
  18. Paul said 2 days later:
    What about a comparison to Apache/mod_proxy_balance?
  19. Kirk Haines said 3 days later:
    Just FYI, but a 0.5.1 has been released that fixes several reported bugs.
  20. Nick said 6 days later:
    Wonderful change. I'm not a fan of the lighthttpd/fastcgi set up and this let's me use mongrel with the possibility of one day doing swiftiply/mongrel.
  21. Eliot said 7 days later:
    Does Zed seem interested in integrating the EventMachine stuff or does this amount to a fork of Mongrel?
  22. Kirk Haines said 9 days later:
    Eliot, I don't think either evented_mongrel or swiftiplied_mongrel is a fork of Mongrel, right now. I intentionally did not package up a complete mongrel source with my patches applied. Instead, right now, it is more of an extension on Mongrel that is intended to be transparent to whatever Mongrel handlers are being used; they shouldn't know that they are not running on the standard Mongrel. Kirk Haines
  23. Nick (nickjost@yahoo.com) said 9 days later:
    Actually I have the same question and it becomes a concern, not right now, but in the future. Are there talks going on? If its a patch its a fork. If talks are going on then a future patch of Mongrel becomes much safer to me/
  24. Jeremy said 9 days later:
    I asked Zed this at Railsconf. From what I gathered, there's no interest in pulling this into Mongrel directly, because of some licensing issues with EventMachine. He expressed that if the evented_mongrel worked well for you and you trusted the code, more power to you. Zed also previously wrote an Ruby/Event when he was working on SCGI (pre-mongrel), so there's a possibility he might pick that back up in the future.
  25. Kirk Haines said 10 days later:
    EventMachine is currently released under the GPL. If it makes sense to turn evented_mongrel/swiftiplied_mongrel into a full fledged fork, then so be it. I'll get a mailing list started up for swiftiply discussions. This is something that can be explored better in that medium, I think.
  26. W. Andrew Loe III said 10 days later:

    For those curious about Mongrel vs Evented Mongrel on some generic hardware (Celeron 2.4, 512MB RAM running Debian). http://journal.andrewloe.com/2007/5/22/mongrel-vs-evented-mongrel

    I did 3 tests of 5000 requests with a concurrency of 1, both Mongrel and Evented Mongrel achieved about 270 req/sec. When the concurrency was bumped to 10, Mongrel fell to 120 req/sec whereas Evented Mongrel was able to maintain pace at 270 req/sec.

  27. Agres said 11 days later:
    So, how to make all of this play nice with nginx? :) Switching from Nginx->Mongrel to Nginx->Swiftiply->Evented Mongrel? Or its better to go just Swiftiply->Evented Mongrel?
  28. Kirk Haines said 11 days later:

    Agres,

    Good question. I'll write up more information on this whole topic for the website. But, in short:

    If you are running nginx right now, the simplest thing is to just use evented_mongrel.

    swiftiplied_mongrel is event based, as well. The reason evented_mongrel exists is mostly because I got it for free while creating swiftiplied_mongrel -- swifiplied_mongrel does everything evented_mongrel does, plus a little bit.

    If you use Swiftiply with Mongrel, you would do it with swiftiplied_mongrel.

    If you used it behind nginx, you would have nginx proxy to Swiftiply in the same way you would if you were using haproxy, pen, or something else in that spot.

  29. Jim Kane said 11 days later:
    For those of you who choose to step onto the bleeding edge, note that the docs are slightly out of date for 0.5.1. The env var for turning on swiftiplied mode is now SWIFT instead of SWIFTIPLY.
  30. Kirk Haines said 12 days later:
    Argh! That was a brain burp when I wrote the docs. It's always been EVENT and SWIFT. Fixing that right now.
  31. Saimon Moore said 18 days later:
    "swifiplied_mongrel does everything evented_mongrel does, plus a little bit." So when using with nginx is there any reason to uses evented mongrel? In order to migrate my current nginx > mongrel setup should I get nginx to proxy to swiftiply (on a custom port) and then startup either evented or swiftiplied mongrels?

(leave url/email »)

   Preview comment