Merb master/worker monit control setup

Posted by ezmobius Sun, 07 Dec 2008 19:55:00 GMT

Going into 1.0, Merb has changed the way it handles options and processes. This tends to break people’s existing monit configs. So today I’m releasing a custom Engine Yard monit script for controlling merb master/worker processes.

You can get the script from here: monit_merb_mpc. Put this somewhere in your $PATH, or make it live here: /engineyard/bin/monit_merb_mpc and make sure it is executable.

In order to make proper use of this script it assumes certain paths are setup. In this example we will assume you have an app called ‘iggy’ that lives at /data/iggy/current, where /data/iggy is the main app dir that contains the capistrano releases and current directory. The script assumes these paths so if you want top use different paths you will need to hand edit the script as desired.

This script also assumes a logging directory of /var/log/engineyard/$appname, you can edit the script if you want to change this as well.

You need to have one master configured per app, as well as monit configuration for each worker.

Master Config

check process merb_iggy_master
  with pidfile /var/log/engineyard/iggy/iggy-production-merb.main.pid
  start program = "/engineyard/bin/monit_merb_mpc iggy start_master -c3 -n5000" 
  stop program = "/engineyard/bin/monit_merb_mpc iggy stop_master" 
  #if totalmem is greater than 80.0 MB for 2 cycles then restart       # eating up memory?
  group merb_iggy

Note that the start program needs to be adjusted if you need more/less instances or want to run the workers on different ports /engineyard/bin/monit_merb_mpc iggy start_master -c -n . Also, memory monitoring is turned off for this process because the memory includes all processes under its control, including the workers and they are already monitored.

Worker Config (one for each port)

check process merb_iggy_5000
  with pidfile /var/log/engineyard/iggy/iggy-production-merb.5000.pid
  start program = "/engineyard/bin/monit_merb_mpc iggy register_worker 5000" 
  stop program = "/engineyard/bin/monit_merb_mpc iggy restart_worker 5000" 
  if totalmem is greater than 80.0 MB for 2 cycles then restart       # eating up memory?
  group merb_iggy

How it works

Since Merb 1.0 (actually version 0.9.8 and above) uses a master process and a spawner process in addition to the worker processes, the monit_merb_mpc (multi-process-control) script needs to control the master process separately from the worker processes. To accomplish this, there is a fair amount of trickery to make monit treat the merb processes in a way monit was never intended to be able to handle. When the master process is started, it will create all the worker processes, so when monit starts the worker processes, they are infact already started – so the start command for the worker simply “registers” the worker – in other words, it just ensures that the pid files contains the worker pid. Also since the master process will automatically restart the worker processes, the stop action for the workers is actually a restart.

Notes:

Although this looks like it should use the monit dependency directive, the scripts are actually written to not need it, and making the worker processes dependent on the master process, it will cause problems during shutdown. since the master specified the number of worker processes using the -c option, you can’t simply shutdown a single worker process by stopping it with monit – it will simply restart by itself and operate unmonitored, thus making it a possible runaway process.

Thanks to all the EY support guys who worked on this one to make it smooth. Hopefully this clears up how to deploy merb with the master/worker server cluster setup under monit.

7 comments

Comments

  1. benburkert said about 3 hours later:
    Thanks for releasing this, I was starting to feel stuck between a rock and a hard place trying to run merb w/ monit. Looks like this should make things easier.
  2. Marek Nowak said about 13 hours later:
    Thank you for updating this. I'm going to check it:-)
  3. max said 3 days later:
    Very good and interested post. Thanks!!!
  4. marcel said 3 days later:
    Your resources were very helpful.
  5. yingg said 5 days later:
    wow power leveling The eyes of the international Counter-Strike and World of Warcraft communities will be focused north of the 49th parallel buy wow gold — World of Warcraft: EM Montreal Viewers Guide this weekend as the city of Montreal will be play host to the third ESL Extreme Masters Global Challenge.cheapest power leveling, This marks the first time that a Canadian city will host an international Counter-Strike and World of Warcraft event.wow power leveling The Extreme Masters Global Challenge will feature top international wow gold — World of Warcraft: EM Montreal Viewers Guide Counter-Strike teams such as SK Gaming, Fnatic and Made in Brazil. They will be joining several Canadian and American teams that include Gravitas Gaming (formerly X3O), Evil Geniuses USA and Evil Geniuses Canada. power leveling cheap Below you will find a full viewers guide to help make sure you don’t miss out on any of the action from either of the Extreme Masters Montreal Challenges.cheapest power leveling, wow gold GotFrag will once again be live on site with a crew of staff members to bring you exclusive content, interviews, HLTV, demos,12 power leveling . Gamesense Statistics,wow gold galleries, wow gold — World of Warcraft: EM Montreal Viewers Guide blogs and much much more. So make sure to keep your web browser’s locked on GotFrag.com throughout the three day event.
  6. bluza said 6 days later:
    good thinking!
  7. null said 21 days later:
    wow power leveling The eyes of the international Counter-Strike and World of Warcraft communities will be focused north of the 49th parallel buy wow gold — World of Warcraft: EM Montreal Viewers Guide this weekend as the city of Montreal will be play host to the third ESL Extreme Masters Global Challenge.cheapest power leveling, This marks the first time that a Canadian city will host an international Counter-Strike and World of Warcraft event.wow power leveling The Extreme Masters Global Challenge will feature top international wow gold — World of Warcraft: EM Montreal Viewers Guide Counter-Strike teams such as SK Gaming, Fnatic and Made in Brazil. They will be joining several Canadian and American teams that include Gravitas Gaming (formerly X3O), Evil Geniuses USA and Evil Geniuses Canada. power leveling cheap Below you will find a full viewers guide to help make sure you don’t miss out on any of the action from either of the Extreme Masters Montreal Challenges.cheapest power leveling, wow gold GotFrag will once again be live on site with a crew of staff members to bring you exclusive content, interviews, HLTV, demos,12 power leveling . Gamesense Statistics,wow gold galleries, wow gold — World of Warcraft: EM Montreal Viewers Guide blogs and much much more. So make sure to keep your web browser’s locked on GotFrag.com throughout the three day event.

(leave url/email »)

   Preview comment