Using PERL to Monitor Website / Server Status

Many webmasters, website resellers and web hosts that run from home, like myself, struggle with knowing if their servers are up or down, if their network is operational or if their primary website is up.

Paranoia has website owners checking their website every couple of hours to see if it is up. Then they get a call from a client that says the website was inaccessable earlier and they have no way to verify what the problem may have been. Was it the customer just typing in a misspelled name? Or was the server down? Was someone else on the server using too much bandwidth, or too many connections?

Frantic calls to the isp generally yeild no information, since they never admit their mistakes or tell you the server is overloaded.

So what is a webmaster to do to have a piece of mind?

Hooray! For The Simple Website Monitor in PERL. A simple program with 2 parts. A script that runs in your website(s) and a script to run at a remote site, usually your home or office.

By using a script to access the website at a regular interval, usually about every 15 minutes, you can log remotely the servers activity. If you do this from your home or office you can use a browser window on refresh to show the current status of each of your servers or websites.

You can have the script e-mail your cell phone if a website is down, giving you immediate notice of any problems. You can expand the script to run a trace route so you know exactly where the connection failed.

The biggest advantage of running your own monitoring system is that you can customize it as needed. If you run it from your home or office, it wont cost you anything and you can monitor an unlimited number of websites or servers. At least as unlimited as your connection can handle.

The web based script runs at the website level. The advantage to this over basic server monitoring is that the webserver must be working to deliver the script. So even if the server is running and a traceroute or ping confirm it is responding, that does not mean the httpd daemon in working.

By using a script at the http level, we insure the server is working in the maner we need. It also tells us that the server is not overloaded or out of memory if the script is running properly.

Each script can easily be modified to meet your needs. Of course the server response script can feed you back all sorts of details like the servers uptime, load averages and even website stats. But for the sake of demonstration, we are keeping these simple.

I am going to provide a few different examples and setups ready to use. If you are looking for a plug and play solution, just download sample #3 and install it. If you want to understand how it works and the theory behind it, read samples 1 & 2 for a logical outline. Samples 1 and 2 also offer a solution for anyone that is not able to modify the /etc/crontab file to run the program.

Sample Website Monitor #1 Simple 2 script basic visual comfirmation, script executes from browser window refresh.

Sample Website Monitor #2 Script with logging and server uptime, execute using meta refresh with any browser.

Sample Website Monitor #3 The full package. Remote server script with e-mail notification, logging. Seperate display script with main monitor program executed by crontab.

HINT: My best advice in using this technique is to run an old pc from your home or office or both. You won't need much. Even an old 300mhz PII is fine and will handle the task.

If you are into websites, you no doubt have a few old PC's laying around (since none of us can bear to through them out no matter how useless they are) or can at least get one for under $50. Still way cheaper than paying for website monitoring or for a remote site to monitor from. It will also be less invasive to your websites and you wont pay for additional bandwidth as you will running the script from a remotely hosted website.

The biggest issue with getting a remote website to monitor your other websites is cost, bandwidth and having root access so you can schedule the cron files to run the script at regular intervals. If you run your own server from home, those problems go away.

Websites VS Servers: The documentation provided here constantly refers to servers / websites. Of course you only need one website on a server to host the response script mystatus1.cgi, unless you are running a server with multiple http daemons.

If you are hosting websites on multiple servers at multiple hosting companies, you can post a script in each website and check each website.

If you host websites but do not run your own you can set up a default virtual host file for one of the servers ip addresses. Or if you know hosting, you can name your servers ww1.yourdomain.com, ww2.yourdomain.com and so on. Set up a virtual host for that host name and you have a good format for installing your web monitor scripts.

The visual display function: In an ideal world, everyone will set up their own server at home to monitor their remote servers. Run apache on that server and use the browser to show the results.

However, the nice part about the internet is portability. You can install the scripts in one remote website and access the display page using any computer with a browser from any location. If you travel, this is a great way to check up on your websites. Using one website to show you the status of all of your websites at any given time.