Searching...
What do you want to see in mod_rubinius?
Posted by ezmobius Tue, 12 Feb 2008 00:25:00 GMT
So we are just getting started on mod_rubinius here at EY. We’ve hired Eero Saynatkari ( rue in the #rubinius irc channel) full time to work on the project.
The architecture for mod_rubinius is still up in the air at this point. We do know that it will be rack based so the interface from mod_rubinis into ruby apps will be via rack. Other then that we don’t yet know what the best way to architect the platform will be. It could be an embeded rubinius VM inside the apache processes. Or it could be a process manager that manages separate rubinius VM’s, or a combination of both of these approaches.
We would like to hear from you folks.. What would you like to see in a mod_rubinius? How is deployment of ruby apps painful for you and how would you like it to work in a perfect world?
We’d appreciate some feedback so please leave feature requests and ideas in the comments.
I'm migrating all of my servers from Apache to Nginx. I remember you mentioned on the Ruby on Rails Podcast that mod_rubinius may be offered as an Nginx module. Is this still a possibility?
(*sigh* why would ppl leave coments on reddit on not here?) It would be nice to deploy with a simply rsync like PHP but I would hate to give up migrations or other rake tasks when I do an upgrade (or a downgrade). Capistrano is pretty nice, much nicer than most other deployment systems.
I would like to see bells and whistles, bells and whistles!!! Actually I wouldn't have a clue.
In all seriousness, looking at what made PHP so successful (it wasn't the damn language design that's for sure), was the ability for people to get moving with the existing metaphor "create a file -> upload via ftp -> check my_app.php in the web browser". So despite how it'd play with Merb, Rails, and the like, if you could upload a tiny 3 line ruby file and get some results that might foster adoption. The ability to get feedback quickly is a huge factor in people learning and adopting junk. It's the reason why I was doing PHP instead of Java 3 years ago. PHP I was getting feedback within 5 minutes. Java I couldn't figure out how to get a "Hello World" let alone a bloody WAR container set up.
I don't really think PHP is an appropriate analogy here. Having deployed a ton of Rails apps in the past 2 years, here are some issues that come to mind: * Mongrels are a pain to start/stop/manage. I have used every trick in the book, and now, finally it's almost working. * Mongrels are not dynamically allocated. Meaning that we start 3 or 4, and heavy load or not that's all we have. * Each Rails instance takes up a ton of memory.... This might not necessarily relate directly to mod_rubinus but I had heard of possibly using a trick to load a Rails instance once and have all the members of the cluster use it. * I think having sitting in an Apache/Nginx process is great for light weight apps, but the option of having separate VMs with load-balancing/failover is crucial. * There needs to be a graceful restart mode. A lot of the problems, especially when it comes to Rails deployment, need to be addressed both at the vm level as well as the framework level.
One thing that I used to do in my php days was to use .htaccess to run all requests through a single php file. Something like that would be nice to have. I would think about the Rack based approach. I love it to death for frameworks but if you are trying to do mod_ruby done right you need to have /xyz.rhtml just work. AEM
Installation (linux) - RPM - Standardized paths - Clean un-install Security/Policy Skin - Load only signed rbx - Opening of classes - Do not allow eval(some string) - No system/exec calls - etc. Dual Mode Of Operation a.) In-process b.) Out-of-process with a standard protocol to communicate with a app-server
If given the choice I'd prefer an Nginx module rather than an Apache one -- haven't had a need to setup an Apache based web server in years. I used to be heavily into mod_perl 1 & 2 and one of the things I really liked about it was that I had the ability to step into any of the request phases. I could alter nearly anything from how the request was parsed, to access/auth/authz, response handling, logging and more. I could even affect how the server configuration was parsed at startup. Keep in mind that most of the mod_perl handlers I wrote were response handlers, but it was nice to have the power when I needed it. I also think modelling it after mod_php is probably a mistake. Make it easy enough for someone to build mod_erb or mod_haml on top of mod_rubinius, but I'd hate to see it limited to that.
One of the main problem w/ Ruby right now IMO is the amount of memory it uses. Plus having to run multiple backend server each running a full Ruby VM doesn't help. I've heard Rubinius can run multiple VM in the same process, Maybe that could help w/ memory usage. And of course it should output random mom jokes in the log files.
I think that being able to script "hello world" and drop the file on the server is crucial for the long-term adoption of the platform. The framework should be entirely optional. Configuring an extra container like Mongrel makes things much more complex than they need to be, the removal of this step is a BIG win. Threading. The only we need clustered Mongrels is because of threading issues.
> Or it could be a process manager that manages separate rubinius VM’s I'd prefer this option. There's something to be said for separation; the moment it's all in a single process you're opening Pandora's box. Don't forget that Apache has worker and event MPMs. Let's say Rubinius becomes threadsafe, what about any app dependencies? Since PHP is being mentioned a lot here, apparently mod_php stuffed itself with the above problem. I also concur with most of what Mazdak Rezvani says.
Yeah I am personally leaning towards the separate process arch. This way the modules would be small and easy to port to nginx as well as apache. So the front server(apache or nginx) would start up the rubinius process manager that would in turn start and stop more VM's and/or processes as needed. This would allow for us to exploit the fork and cow friendly GC rubinius has to preload rails and an app into a master process and then fork children off of it as necessary in order to share the bulky framework code in memory. The rack based interface would just cover access to the request ENV from apache, there could easily be a built in rack handler that can make mod_rubinius act like php for .rb files if that is wanted. The flexibility of rack based apps would allow for any framework or no framework to run easily.
Making Ruby more like PHP is moving backwards not forwards. Some of the features that make Ruby great also add to its deployment difficulty. Migrations, production mode, etc. etc. If you want a simple one page app you should stick to PHP. Pick the right tool for the job. Ruby fills the nice void inbetween PHP and Java and I think it should stay there. In terms of mod_rubinius, the first focus should be on getting ruby to stop using so much memory. It will never be used in shared hosting environments when it has 80mb mongrels running rails. The second thing would be to make good packages. I always install ruby from source but I shouldn't have to.
Ezra, I'm quite excited about this initiative :) Here are my 2 cents on what the sweet spot is - to get a broader use, ruby needs the same kind of deploy (upload) and forget ease that made php what it is today (no complicated setup, no extra processes to manage beyond your web server). - make it easy for webhosts to provide a solid ruby deployment for moderate traffic apps in as reliable fashion as they can do it for php (one click install for ruby apps, endless possibilities...) - don't try to address high traffic apps. For those, it's fine to keep the higher complexity deployments that require a lot more involvement (same goes for php, actually) You may find this post (http://izumi.plan99.net/blog/index.php/2008/01/27/rails-deployment-wouldnt-it-be-great-if-it-worked-like-this/), and the ensuing discussion interesting.
Hi Ezra - what are your goals with mod_rubinius? Is it to be the "PHP of ruby", such that a "hello world" file can be dropped into a directory and will run instantly? That seems to be what a lot of people are clamoring for. I'll personally stick with rails but php-style ruby would certainly lower the barrier to entry.
What I would like for a mod_rubinius is the ability to add a server entry to my nginx configuration file and say 'server app.domain.com', and configure a path to a dispatcher. Then mod_rubinius should be able to spawn (and drop) processes as needed without becomming a resource hog.
One Rubinius for one Rails/Merb instance loaded in memory and several apps is the goal. php-like deployment is a step backwards.
I second all who talked about memory. And Rubinius being fork friendly is good step in that direction. I am all for easing deployment too. I am not sure if I would want a PHP like deployment though. Something like Tomcat's 'webapps' directory or something but without all the complexities.
Hmm, again it seems some of the "Ruby" community is getting confused that Ruby != Rails. Although I know it's been stated that Rails will be the primary platform for this, it would still be nice to see this really take form as mod_ruby done right. For instance, one of the more interesting aspects of mod_perl was the ability to write generic request handlers. I know this is adding a lot to the scope of mod_rubinius and that it's doubtful it would be an easy thing to implement, but it would really push forward a lot of the capabilities of the Ruby server-side environment.
Thomas is dead on. I'd love to be see mod_rubinius have the same ability to reach down into Apache (and/or nginx) that mod_perl has.
I would say that everyone has made good points by looking what they like about mod_(php,perl). While I do agree, I would just like to make the point that the mod_rubinius team has a chance to go beyond. I would really suggest you look at mod_(php,perl, and even jsp) to see what did each of these do right. What was a real pain in the ass about each. How did the solution fit the need and how is that need similar to the Ruby community's. Then look beyond, such as the suggestion for adaptive vm spawning with load increase. That analysis is beyond my humble little post, just hope to spawn some creative thinking with in the community (^_^).
If you're moving the VM outside of the Apache process context, how exactly will this be a win over, say, FastCGI? The whole reason that mod_perl, mod_python, and even (*gasp!*) mod_ruby rock is the fact that they are tightly coupled with the Apache runtime, which is for all its failings a pretty well battle-hardened codebase. If you're going to create mod_rubinius, I think it should follow the same model: wrap the Apache internal APIs using the oh-so-slick Rubunius FFI infrastructure, with the additional advantage of being one of the only fully thread-safe Apache dynamic handler implementations.
@Thomas, @pate: Have you guys actually used mod_ruby? I mean, it exposes pretty much the exact same API as mod_perl, and most definitely does allow you to define dynamic handlers at any point in the request processing chain. The fact that Rails does basically depends on a bunch of global variables to accomplish its magic (and therefore doesn't allow one process to easily switch between Rails applications) doesn't mean that mod_ruby is somehow broken.
I've been wishing for this for years!! Mongrel is great, Rails is great. But it's certainly very useful to simply be able to upload some ruby code and have it working on the server except for the fact that CGI is sloow. I use it here : http://labs.parkerfox.co.uk/ruby.search It would have been a pain to create a whole rails/merb app etc to do something so simple.
@lennon: yes, that's why I'm so hopeful we'll see the same it show up here. The promise of the multi-vm support that Rubinius has combined with that capability are what I'd really like to see.
Yap, more bells and whistles are a pretty good idea! :D
There are two concerns: 1. Webserver configuration: should be as easy as plugging in the rubinius_mod.so file 2. Developer deployment: should be as easy as dragging up a shotty file via ftp. I reckon mod_rubinius oughta be rack-pluggable obviously, but why can't we un-f*ck the way java works? Compile your stuff, then create a a "shotty" file, and upload them. It then auto-senses the change, and gracefully kills all existing processes, and relaunches new ones with the new shotty file. The crap thing about Rails is that it doesn't share memory. If it did share memory between processes, there'd be less memory taken per whole-machine. (one set of the memory for each core, I'd say). Making ruby deployment more like PHP is BAD, Adam? I can't imagine why that'd be bad. You seem to not know the difference between rails and ruby. Migrations aren't Ruby-specific, they're rails-specific. If you're worried about Memory use, use Merb, and Thin. These issues would automatically be solved with mod_rubnius, I'd wager... ANYWAY.
Hi, This is a great question. Here is what I recommend. Put the whole VM inside Apache. Avoid creating any long running processes. Don't reinvent the slower, but extremely solid wheel that is Apache. Avoid sharing memory between processes. Just make the system *solid*. Then improve performance. Here is a blog post on this as related to PHP: http://www.xxeo.com/archives/2007/03/16/thoughts-on-php.html
I want't to see mod_rubinius on Google Summer Code.
Do you have any support within the Apache project yet? If you'd like some exposure there, let me know. I'll see what I can do to drum up some interest.
I'd like to see it being able to run in a similar mode to mod_php (creates a process for each request, quickly handles it and dies). This would be a good solution for lots of small apps/shared hosting where there are memory constraints.