Nginx Upload Module

Posted by ezmobius Sun, 20 Jul 2008 20:32:00 GMT

Valery Kholodkov has written a very cool nginx module for handling uploads.

The way this works is that you specify a location block to handle the uploads. So if you are using the standard nginx.conf for rails apps then you would add this in your server block right above your “location /” block:

    # Upload form should be submitted to this location
    location /upload {
      # Pass altered request body to this location
      upload_pass   /internalupload;

      # Store files to this location
      upload_store /tmp;

      # Set specified fields in request body
      upload_set_form_field $upload_field_name.name "$upload_file_name";
      upload_set_form_field $upload_field_name.content_type "$upload_content_type";
      upload_set_form_field $upload_field_name.path "$upload_tmp_path";
    }

    # Pass altered request body to a proxy
    location /internalupload {
        proxy_pass   http://mongrel;
    }

Then make a simple upload form that does a multipart POST to /upload. Now you can have your Rails or Merb app on the backend with a route called /upload. In the action of your app that responds to the /upload route you will get a set of params that look like this(assume the name of your upload fields is called ‘file1’ and ‘file2’):

{"file2.path"=>"/tmp/0000123459", "file1.path"=>"/tmp/0000123458",
"file2.content_type"=>"image/png",  "submit"=>"Upload",
 "file2.name"=>"Picture 2.png", "action"=>"index", 
"file1.name"=>"Picture 1.png",  "controller"=>"test", 
"file1.content_type"=>"image/png", "test"=>"value"}

What this is doing if parsing the multi-part mime boundaries in C in the nginx plugin, putting the parsed files into files in /tmp and then stipping the multipart stuff out of the POST body and replacing it with the info you need to get the name and location of the file on disk.

This means that by the time the request hits your application, the expensive mime parsing is already done and you simply move the file to it’s final resting place. This is a huge win since now the hard work is done in C in nginx before your app ever gets involved.

Of course this is a fresh new module so do your own testing and deciding whether or not this is a fit for your needs. But I think this is a great plugin and have verified it works as advertised.

Tags ,  | 10 comments

So merb-core is built on rack you say? Why should I care?

Posted by ezmobius Sat, 16 Feb 2008 00:11:00 GMT

Rack is an awesome webserver abstraction that distills the idea of a ruby web app down to an object or Proc that has a call method. The call method takes the rack environment, which is all of the cgi style headers for the current request, and returns an array of [status, headers, body]. The status is a number like 200 or 404, the headers is a hash of header key value pairs for the response and the body is the output from your application, The body must respond to each and yield lines or chunks of strings to be flushed down the socket to the client.

Here is the most basic example of a rack app using the rackup builder DSL for mounting and running rack apps:

rack_app = Proc.new do |env|
  # env here has all the headers you would expect
  [200, {"Content-Type"=>"text/html"}, "hello world!"]
end  

run rack_app

This app will return “hello world!” to the client. Pretty simple eh?

Now that merb-core is all based on rack, there are some very interesting things you can do with this knowledge. In the config/ directory of a freshly generated merb-core app you will see a rack.rb file. By default the file just contains this:

  
run Merb::Rack::Application.new

This will run the main merb rack app class that handles the standard dispatching of requests through the whole merb framework. Now merb-core is small and fast, but what if you have some certain requests that don’t really need the router or controllers/views of a full merb stack.

Say we need to handle a ton of concurrent file uploads, and we don’t want to invoke the full merb stack just for these uploads. The solution is to use the config/rack.rb file in your merb app along with the Rack::Cascade middleware to mount a simple Proc object to handle the uploads instead of merb proper. Here is what a rack.rb file for this would look like:

uploader = Proc.new do |env|
  request = Merb::Request.new(env)
  if request.path =~ /\/images\/upload/
    #file uploads can get the params from request.params and do whatever 
    # you want with it, this allows for multiple concurrent uploads
    # with very minimal overhead, doesn't go through the merb 
    # framework at all
    params = request.params
    FileUtils.mv params[:file][:tempfile].path,
                 Merb.root / 'public' / 'uploads' / params[:file][:filename]

    headers = {"Content-Type"=>"text/html",
              "Location" => "/images"}
    [302, headers, "You are being redirected"]
  else
    [404, {}, "No Upload here, move along"]
  end    
end  

merb = Merb::Rack::Application.new

run Rack::Cascade.new([uploader, merb])

Rack::Cascade works by trying to call each app you specified in order and actually use the results from the first one that does not return a 404 error. So what our new uploader Proc/app does is creates a Merb::Request object and checks if the request.path matches /images/upload. If it does match then you can use code in here to handle the file upload and place it wherever you want. Once you’ve done whatever you need to the file upload you can redirect by setting the Location header like in the above example, or you can render out a page or return JSON or really anything you like.

Having the power of merb-core and also the power of raw rack access conveniently in the same app is very powerful. I think this is one very compelling thing merb-core has going for it now that we are all rack based.

Hopefully you enjoyed this installment of what is new and cool in merb. Hope to write some more articles shortly about special features in merb-core that go undernoticed.

Tags , ,  | 9 comments