Webmaster Mike

I regularly monitor my network and CPU usage. After running into performance problems with my Apache web server I decided to update the network statistics process I was using to try and catch performance problems before they happen.

Back in December I noticed problems with timeouts while trying to connect to our website. I could secure shell into the server with no problem. I ran top and noticed no processes running amok and CPU usage was low. There were some httpd Apache server processes running. I could also login to the Plesk control panel. Everything looked ok.

I opened a ticket with our host provider and found out that I needed to learn how to tune my Apache web server.

Now that the new VPS has stabilized I wanted to see what my network and CPU stats are like. I lost my old scripts when the old hard disk failed. I'll post something about my new and improved backup process soon. However, because I blogged about them, and I was backing up my Movable Type database, so I had copies of my Perl scripts.

Yesterday we started having problems with our VPS. It started out simply enough, it just looked as if the MySQL server process had failed. But by early this morning our VPS provider was saying the hard disk had failed and they were moving us to a new server.

But the real problem started when they said our VPS backups were also corrupt.

I had one outstanding problem with my Plesk 8.1.1 upgrade. After the upgrade I started getting errors in Logrotate, and syslog starting doing weird things with my log files.

I finally got around to doing some simple network stats. Our VPS is on a 1.5Mbps connection, and there are times I wonder if we're pushing the limit. I had tried to estimate this from by our http log files. But using /proc/net/dev instead is much simpler and much more accurate.

I upgraded to Plesk 8.1.1 awhile back. I was a bit nervous about the whole thing. But the actual upgrade was quick, unfortunately it wasn't painless.

I just re-ran my CPU monitoring for a typical weekday. Not much different from last time. But I thought I'd include how I'm generating these charts.

I revised my perl program to capture the output of vmstat. Now VMSTAT is left running and the output is captured by my perl program and saved in MySQL.

I thought I'd do a little CPU monitoring to determine if our VPS is still meeting our needs. As with most things it was more complicated than I thought.

I just upgraded to MT 3.34 from MT3.33. A totally painless experience.

After getting my CentOS server up and running it's time to load up my Apache access log files. To do this I needed a perl script to load the data. And some SQL queries to answer some questions like who's hotlinking our images or hogging our bandwidth.

I finally decided to make some use out of an old 400 mHz Celeron PC that I had lying around gathering dust. I thought I'd install Linux and create a little system for analysing web logs and perhaps testing out some scripts and other programs before using them on our main server.

My initial research into hotlinking on our web site has produced mixed results. A lot of the hotlinking is back to our logo or banner images, so we want to keep that. But a couple of web sites are posting our text content and hotlinking our images.

I also checked out bc2.region5.ang.af.mil. Their DNS wasn't screwed up, but why is Bentwaters Royal Air Force Base a top ten web site by usage?

I noticed in my webalizer statistics that tm.net.my has been in the top 10 websites by usage for many months. And that's pretty odd, since those spots are usually held by search engines (usually Google) or proxy caches (usually AOL). Anything not a search engine has to be a little suspicous. Why would an ordinary user hit our site thousands of times in a month, or transfer tens of megabytes of data?