Thursday, January 29, 2009

Monitoring thins with monit...

Short story:
Download and install the latest code from http://mmonit.org/monit/download/ (configure, make, make install.. done! )

edit /etc/monitrc

# ========================================================
# Monit global settings:
# ========================================================
set daemon 60
set logfile syslog
set mailserver localhost
set mail-format { from: monit@cfandersen.com }
# ========================================================
# Monit http server settings:
# ========================================================
set httpd port 2812
allow localhost
INCLUDE "/etc/monit/thins.monitrc"

/etc/monit/thins.monitrc


check process thin-8000 with pidfile /app/shared/pids/thin.8000.pid
group rails
start program = "/usr/local/bin/thin -C /app/current/config/thin_cluster.yml start --only 8000"
stop program = "/usr/local/bin/thin -C /app/current/config/thin_cluster.yml stop --only 8000"
if failed host localhost port 8000 protocol HTTP
request "/token/monit" with timeout 60 seconds then restart
if cpu is greater than 60% for 2 cycles then alert
if cpu > 80% for 5 cycles then restart
if totalmem > 100.0 MB for 5 cycles then restart
if loadavg(5min) greater than 10 for 8 cycles then stop
if 5 restarts within 5 cycles then timeout
depends on thin_bin

check file thin_bin with path /usr/local/bin/thin
group rails
if failed checksum then unmonitor
if failed permission 755 then unmonitor
if failed uid root then unmonitor
if failed gid root then unmonitor

Thats the setup now the ISSUE!

Monit would NOT start thin, and the error I was looking at in the thin log was:

/usr/local/lib/ruby/gems/1.8/gems/rails-2.1.0/lib/rails_generator/lookup.rb:35:in `expand_path': couldn't find HOME environment -- expanding `~' (ArgumentError)
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.0/lib/rails_generator/lookup.rb:35:in `user_home'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.0/lib/rails_generator/lookup.rb:114:in `use_component_sources!'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.0/lib/rails_generator/lookup.rb:55:in `included'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.0/lib/rails_generator.rb:38:in `include'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.0/lib/rails_generator.rb:38:in `send'
from /usr/local/lib/ruby/gems/1.8/gems/rails-2.1.0/lib/rails_generator.rb:38
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
... 26 levels...
from /usr/local/lib/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/runner.rb:139:in `run!'
from /usr/local/lib/ruby/gems/1.8/gems/thin-1.0.0/bin/thin:6
from /usr/local/bin/thin:20:in `load'
from /usr/local/bin/thin:20

I did not want to have a "startup" script hanging around SO I added this in my /usr/local/bin/thin file:

ENV["HOME"] = "/tmp"

Problem solved!