One big challenge is making sure our services are up and running at all times, that we have enough free space on our drives for running smoothly, restarting services when new versions are deployed etc.
There are many ways for pulling these tasks, ranging from hand-rolled scripted solutions to full-fledged uptime management systems, however in most cases a simple yet effective tool which is easy to deploy and configure is desired.
Process Monitoring Systems
In this post we’re going to look at Monit. By all means I am not trying to say that it is superior to any other system, however I personally find to be extremely easy to install and configure to do exactly what I need to.
Installation & Configuration
$> sudo yum install monit
Next thing we want to do is configure Monit so that it runs as a daemon (service), wakes up on our preferred intervals, and make sure we have its HTTP management interface open.
Edit the config file:
$> nano /etc/monit.conf
At the very bottom of the file make sure to have these:
# set daemon mode timeout to 30 seconds (or other preferred interval) set daemon 30 # enable the http interface (so for example we can check its status using ‘monit status’) set httpd port 2812 use address localhost allow localhost
Save the file and exit. We’re now ready to run monit – simply type monit. voila!
Monit Control Files
Keep A Service Running
check process custom_bidder matching "com.codebyz.adexchange.custombidder" start program = "/sbin/service custom_bidder start" with timeout 30 seconds stop program = "/sbin/service custom_bidder stop"
Check Filesystem Usage
check filesystem fs1 with path /dev/sdb1 if space usage > 70% then exec "/sbin/service fs_cleaner cleanup"
check filesystem fs2 with path /dev/hda1 if space usage > 80% then exec "/sbin/service fs_cleaner cleanup"
Recycle Processes On File Changes
In the example below we’re checking apache’s main configuration file for changes, and gracefully recycle the web server if the file has been touched:
check file apache_conf with path /etc/apache/httpd.conf if changed checksum then exec "/usr/bin/apachectl graceful"
The identifiers you give to each must be unique (as in check process p1 with pidfile /var/run/p1.pid — p1 must be unique across your control files).
Few Useful Monit Commands
monit -t – check our control files for errors / issues. Always run this after you’ve created or edited a monit control file.
monit reload – recycle the monit daemon and re-read configuration, be sure to run this command each time you update or create new control files or your changes will not take effect
monit status – get a status report of all our control checks