Developing back-end services is today part of every serious developer’s work, and there are unlimited options for developing them – using different programming languages, paradigms tools and frameworks.

One big challenge is making sure our services are up and running at all times, that we have enough free space on our drives for running smoothly, restarting services when new versions are deployed etc.

There are many ways for pulling these tasks, ranging from hand-rolled scripted solutions to full-fledged uptime management systems, however in most cases a simple yet effective tool which is easy to deploy and configure is desired.

 

As stated above, there is a wealth of available systems/tools which we just need to choose from, and just to name a few we have Nagios, Monit, Supervisor, God and daemontools.

In this post we’re going to look at Monit. By all means I am not trying to say that it is superior to any other system, however I personally find to be extremely easy to install and configure to do exactly what I need to.

 

To install monit choose your preferred/available package manager (yum, rpm, apt) per your linux distribution. Using yum for example:

    $> sudo yum install monit

Next thing we want to do is configure Monit so that it runs as a daemon (service), wakes up on our preferred intervals, and make sure we have its HTTP management interface open.

Edit the config file:

    $> nano /etc/monit.conf

At the very bottom of the file make sure to have these:

    # set daemon mode timeout to 30 seconds (or other preferred interval)
     set daemon 30
     # enable the http interface (so for example we can check its status using ‘monit status’)
     set httpd port 2812
     use address localhost
     allow localhost

Save the file and exit. We’re now ready to run monit – simply type monit. voila!

 

Control files are simple free format text files you create under /etc/monit.d/, which define tasks for monit to perform. There are many options at your disposal and here I’m going to show few specific use-cases.

 

Ensure a process or service are running by checking their lock/pidfile or using a regex to check the processes list for a particular process, as shown in the following example:

check process custom_bidder
     matching "com.codebyz.adexchange.custombidder"
     start program = "/sbin/service custom_bidder start" with timeout 30 seconds
     stop program  = "/sbin/service custom_bidder stop"

 

Nobody likes discovering services were down for a while because a drive was full. Here is a short monit control file which watches space on two mounted volumes and runs a cleanup when a set watermark level is reached:

check filesystem fs1 with path /dev/sdb1
     if space usage > 70% then exec "/sbin/service fs_cleaner cleanup"

check filesystem fs2 with path /dev/hda1
     if space usage > 80% then exec "/sbin/service fs_cleaner cleanup"

 

Auto-recycle a process once a file has been updated with a new version, file contents have changed and even look for specific file content changes using regular expressions.

In the example below we’re checking apache’s main configuration file for changes, and gracefully recycle the web server if the file has been touched:

check file apache_conf with path /etc/apache/httpd.conf
      if changed checksum then exec "/usr/bin/apachectl graceful"

 

Once you’ve installed (or updated) a script, always run monit -t to have monit check your control file syntax and warn you of any potential mistakes/errors.

The identifiers you give to each must be unique (as in check process p1 with pidfile /var/run/p1.pid — p1 must be unique across your control files).

 

monit – wakes up the monit daemon instantly and run all our control files (instead of waiting for the next run interval)

monit -t  – check our control files for errors / issues. Always run this after you’ve created or edited a monit control file.

monit reload – recycle the monit daemon and re-read configuration, be sure to run this command each time you update or create new control files or your changes will not take effect

monit status  – get a status report of all our control checks

Thanks for reading – share and prosper!

– Zacky