Nginx Upload Module

Posted by ezmobius Sun, 20 Jul 2008 20:32:00 GMT

Valery Kholodkov has written a very cool nginx module for handling uploads.

The way this works is that you specify a location block to handle the uploads. So if you are using the standard nginx.conf for rails apps then you would add this in your server block right above your “location /” block:

    # Upload form should be submitted to this location
    location /upload {
      # Pass altered request body to this location
      upload_pass   /internalupload;

      # Store files to this location
      upload_store /tmp;

      # Set specified fields in request body
      upload_set_form_field $upload_field_name.name "$upload_file_name";
      upload_set_form_field $upload_field_name.content_type "$upload_content_type";
      upload_set_form_field $upload_field_name.path "$upload_tmp_path";
    }

    # Pass altered request body to a proxy
    location /internalupload {
        proxy_pass   http://mongrel;
    }

Then make a simple upload form that does a multipart POST to /upload. Now you can have your Rails or Merb app on the backend with a route called /upload. In the action of your app that responds to the /upload route you will get a set of params that look like this(assume the name of your upload fields is called ‘file1’ and ‘file2’):

{"file2.path"=>"/tmp/0000123459", "file1.path"=>"/tmp/0000123458",
"file2.content_type"=>"image/png",  "submit"=>"Upload",
 "file2.name"=>"Picture 2.png", "action"=>"index", 
"file1.name"=>"Picture 1.png",  "controller"=>"test", 
"file1.content_type"=>"image/png", "test"=>"value"}

What this is doing if parsing the multi-part mime boundaries in C in the nginx plugin, putting the parsed files into files in /tmp and then stipping the multipart stuff out of the POST body and replacing it with the info you need to get the name and location of the file on disk.

This means that by the time the request hits your application, the expensive mime parsing is already done and you simply move the file to it’s final resting place. This is a huge win since now the hard work is done in C in nginx before your app ever gets involved.

Of course this is a fresh new module so do your own testing and deciding whether or not this is a fit for your needs. But I think this is a great plugin and have verified it works as advertised.

Tags ,  | 10 comments

Comments

  1. jack dempsey said about 17 hours later:
    Very cool stuff!
  2. Michael Koziarski said about 17 hours later:
    This is definitely an interesting module. I'd been mulling something similar for apache + passenger for a while, but have neither the time nor the parsing-chops to give it a go. I assume it protects the upload action to prevent me from getting access to local files with params like file1.path="/etc/shadow"? Does it rewrite the multipart post into a regular urlencoded one? Nice find.
  3. Dylan said 1 day later:
    Michael, yeah looks like it does some security centric things, ie: $upload_file_name -- the original name of the file being uploaded with leading path elements in DOS and UNIX notation stripped. I.e. "D:\Documents And Settings\My Dcouments\My Pictures\Picture.jpg" will be converted to "Picture.jpg" and "/etc/passwd" will be converted to "passwd".
  4. Vamsee said 2 days later:
    @Michael, this option has worked out very well for me: http://tinyurl.com/62cysp
  5. Nahum Wild said 11 days later:
    Awesome, I've been looking out for an excuse to switch from apache2 to nginx. Other than geeky reasons of course :-)
  6. fred said 21 days later:
    This is awesome, but i didn't understand how to make it work. Could anyone make a sample rails app with this implemented? and the nginx.conf please? :) Thanks a lot.
  7. Nahum Wild said 22 days later:
    I've subsequently been playing with it a bit and can't get it to consistently behave. Also getting problems with the rails forgery protection.
  8. Ray said 24 days later:
    I just got a requirement to do some uploading with a progress indicator. I've been scouring the net looking for a good example or documentation on how to use MERB to do this sort of thing. So far I've found nothing. Everyone even the author is raving about how easy and elegant using MERB is for uploading. But no examples! Can someone please point me in the right direction? Thanks
  9. Ivor said 25 days later:
    Hi. Would you be able to upload to a seperate subdomain and have the nginx upload module specific config in the "location /" of that subdomain? I have tried setting up 2 servers - one at uploads.mydomain.com and one for www.mydomain.com. The uploads has the funky upload config in "location /" but that does not seem to work. Any ideas?
  10. Ivor said 26 days later:
    Just to confirm. You can have a seperate subdomain instead of a seperate route, which allows your app to function without having special routes. Just add the upload_set_form_field lines to your location /

(leave url/email »)

   Preview comment