ShellScriptBuilder for Capistrano

Posted by ezmobius Mon, 22 Jan 2007 02:31:00 GMT

This is a first release of ShellScriptBuilder. This release is to gather some feedback about this dsl and what it still needs to be completely useful.

ShellScriptBuilder is a small ruby dsl for buidling bash shell scripts in ruby. It will insulate you from the need to remember all of the flags a shell if statement can take. And it also insulates you from needing to remember all the complex escape rules that the shell has as well as how capistrano itself escapes commands.

Here is a small example:
script = shell do |sh|
  sh.sudo.ln_nsf "foo/bar", "bar/bar" 

  sh.echo "some string" => "/path/to/log.txt" 

  sh.if :directory? => "some/dir" do |sub|
    sub.rm_rf 'some/dir'
    sub.ln_nsf "shared/foo", "some/dir" 
    sub.if_not :file? => 'foo' do |ssub|
      ssub.mkdir_p "some/foo" 
    end  
  end

  sh.unless :file? => 'foo/bario' do |sub|
    sub.touch 'foo/bario'
  end

  sh.if :writable? => "some/file.txt" do |sub|
    sub.echo "#{Time.now}" => 'some/file.txt'
  end  
  sh.mkdir_p 'foo/bar'
  sh.rm_rf 'foo/bar'
end

puts script

OUTPUT:

#!/bin/sh
sudo ln -nsf foo/bar bar/bar
echo "some string" >> /path/to/log.txt
if [ -d some/dir ]
  then
    rm -rf some/dir
    ln -nsf shared/foo some/dir
    if [ ! -a foo ]
      then
        mkdir -p some/foo
    fi
fi
if [ ! -a foo/bario ]
  then
    touch foo/bario
fi
if [ -w some/file.txt ]
  then
    echo "Sun Jan 21 19:18:32 -0800 2007  " >> some/file.txt
fi
mkdir -p foo/bar
rm -rf foo/bar

Right now this is a standalone library with no external dependencies. But I plan on making it a capistrano plugin here shortly and rewriting all the standard recipes to use this dsl. Then it’s possible Jamis will put this in core capistrano.

I would love some feedback on the ‘feel’ of the dsl as well as anything else you feel it needs to support.

This builder uses a cool method_missing hack to allow for commands like this: “ln_nfs” to become “ln -nfs”. Let’s take a quick look at how that works.

Here is the method_missing declaration: 
  def method_missing(sym,*args,&blk)
    if sym.id2name =~ /(.*)_(.*)/
      __send__($1,'-'+$2,*args)
    else
      @cmdbuff << "#{@nesting}#{sym} #{args.join(' ')}\n" 
    end    
  end

This is a double dispatch example. So when we call a method like:

script = ShellScriptBuilder.new
script.ln_nfs "some/dir", "some/other/dir" 
puts script
# => "ln -nfs some/dir some/other/dir\n"

So what happens is that method_missing gets called the first time like this:

method_missing(:ln_nfs, "some/dir", "some/other/dir")
Since the missing method has an underscore it will match the regex in the first if statement here: 
    if sym.id2name =~ /(.*)_(.*)/
      __send__($1,'-'+$2,*args)
What this does is munge the method name and args and then send’s that new method signature. Now method_missing gets called like this: 
method_missing(:ln, '-nfs', "some/dir", "some/other/dir")

So there is still no ‘ln’ method defined in the code. Somethod missing gets called again, this time there is no _ in the method name so it goes into the else part of the if statement. All the else part does is create the new command and place it in the @cmdbuff:

    else
      @cmdbuff << "#{@nesting}#{sym} #{args.join(' ')}\n"

@nesting is just a way for the output to get indented when inside if statements to make the outputted shell script look nice and clean, @nesting is an empty string to represent no indentation when you are in a top level call like this. The output of the else part of the method_missing is just the name of the symbol(command) and all the arguments join’d together with spaces.

This little method missing hack makes it so we can use any arbitrary command line program with or without command line arguments in our new DSL. This allows for you to specify any command line program, you just use _ instead of a space and a – to represent the args. So somecmd_foo becomes somecmd -foo

The output of the method call we just disected is:

ln -nfs some/dir some/other/dir

Fun stuff. Anyway, I would appreciate it if people played with this a little bit and let me know what you think it still needs to be as useable as possible. The plan is to have this buidl shell scripts and then for Capistrano to upload the script to the server into a tmpfile and then execute the script like a normal shell script.

You can get a tarball here. I am still waiting for a rubyforge project so no svn yet. There is a full test suite in the tarball. You can build a gem by unpacking the tarball and cd’ing into the root dir and running:

$ rake package
$ sudo gem install pkg/shell-script-builder-1.0.0.gem
<pre>

Tags ,  | 16 comments

Comments

Leave a response

  1. jperkins said about 2 hours later:
    This is very nice, Ezra. Did you catch the post a couple of days ago to the capistrano mailing list for the use of umask and mkdir with the -p swtich? That would be the sort of thing that would be very nice to include as part of this, no?
  2. Blake Watters said about 4 hours later:

    Couldn't you use instance_eval to eliminate all those sh. explicit calls? Seems to me that since the block is explicitly going to be generating shell scripts, it's just unnecessary extra typing.

    Something else that might be interesting would be to map the long form of options to a hash...

    Constants defined inside the block exported as environment variables might also be nifty.

    Just some ideas off the top of my head. I'll come back after I get some play time in :-)

  3. Michael Daines said about 5 hours later:
    Blake: I'd thought the same thing about removing the "sh.", but what about "if" statements?
  4. Dr Nic said about 9 hours later:
    Awesome.
  5. Neil Wilson said about 10 hours later:
    How would it handle temporary environment variable settings, eg. TZ=UTZ touch -t 01211200 release_file NeilW
  6. Stoffe said about 10 hours later:
    This looks promising, I always have to look up syntax of this and that when doing shell scripts, so it may come in handy. :)

    One thing though: if it specifically builds bash shell scripts, and therefore uses bash extensions, then you should use #!/bin/bash instead of plain #!/bin/sh or the scripts will fail on systems that has another default shell. For instance, recent Debians and Ubuntus use dash as the default instead.
  7. DSLr said about 13 hours later:
    Hey, this is great, even though you've got to wonder why you have to use a scripting language to create a domain specifice language to generate shell commands when shell is also a DSL, the domain being systems administration. :)

    Not knocking your effort Ezra, but from a newbies perspective, there isn't much difference between ln -nsf and ln_nsf .. Really, it is still blackboxy and for added frustration, now you've got to look through the gem documentation to find out what the "ln_nsf" "command" means???

    I mean, if people are too stupid to do systmes admin type work in ruby or shell itself, then creating this extra level of insulation will only confuse them even more and just be more headaches for you when they start claiming that certain "commands" no longer work in the ruby shell script builder.

  8. mathie said about 14 hours later:
    Nice idea. How about taking a step further though, and thinking about the high level tasks that you might want do in the shell and providing methods to make them work in a platform-independent way. For example, `sh.soft_link 'foo/bar', 'foo/baz'` which might expand to a different shell syntax depending upon whether you were on Linux or Solaris?
  9. Ezra said about 17 hours later:
    This is not meant to be a replacement for shell scripts. It is meant to insulate you from all the weird shell escaping and remembering the right if flags for bash. It also shields you from the shell escaping and from how capistrano escapes shell commands which can be tricky.

    @Blake - of course I considered using instance_eval. In fact that was what I used until it came time for if statements. So the extra sh. is a necessary evil so we can use if.

    @DSLr- of course if you already know shell scripting and all the proper flags and escape sequences then by all means you don't need to use this, knock it all you want ;) But I can never remember the right flags to bash if statements. And also dealing with shell escasping at the same time you deal with ruby's and capistrano string escaping can be complicated to remember how to escape things properly.

    Also one thing I forgot to mention in the post is that I have left a way to get raw commands to intermix. Just like RJS in fact, you can use << to push any text onto the command stack. 
    script = shell do |sh|

   sh.ln_nfs "foo", "bar"

   sh << "SOME_CONST=foobar mongrel_rails start -C some/config.conf"

end

puts script


#!/bin/sh

ln -nfs foo bar
SOME_CONST=foobar mongrel_rails start -C some/config.conf
    This is just a few days old so I will be incorporating more features in this lib soon.
  10. Ezra said about 18 hours later:
    So I just added support for setting ENV variables per command. I think it looks pretty good. What do you think about this syntax?
    shell do |sh|
  sh.RAILS_ENV('production').rake "db:migrate"
  sh.TZ('UTZ').touch_t '01211200 release_file' 
end

# outputs:
#!/bin/bash

RAILS_ENV=production rake db:migrate
TZ=UTZ touch -t 01211200 release_file
  11. np said 1 day later:
    This is hot Ezra. Bash scripting is such a pain sometime. Integration with Capistrano sound great from here. And your CONSTANT declarations do look good to me :)
  12. DSLr said 3 days later:
    I actually would like to second mathie's call for a more platform/shell independent way of doing things like softlinking file.

    I totally agree that the weird escaping and flag placement rituals can get confusing, so any effort in that regard is much needed (and appreciated!) :)

    personally I'm not a shell guru, so ln_nsf is just as opaque to me as is "ln -nsf"

    what is ln_nsf btw? what I also find confusing at times (regarding "ln") is, which file do you place first? the one being softlinked to or the softlink being created? I always forget, and always have to trudge through the man pages. I guess I'm having trouble seeing how ln_nsf will save me from a trip to the man page (yes, I _am_ one of those forgetful bumbleheads! :)

    now if there was a DSL'ish way to do that, I think this could become real hot.. thanks.
  13. Splash said 13 days later:
    I second that. Maybe add a 'concept' interface as well (aka 'sh.symlink')
  14. gnufied said 15 days later:
    lol, I have done the exact same thing over here at my end to deploy remotely applications other than rails at my end.
  15. anon said 50 days later:
    I'm curious, why not stick with ruby for the scripting instead of generating bash?
  16. Beck Ham said 105 days later:
    why it’s always writing NULL in empty fields?

RSS feed for this post trackback uri

(leave url/email »)

   Preview comment