When it comes to setting up Web servers, very few administrators ask about Web server log configurations. Instead, they often simply go with the default setting. This applies to both in-house and ISP Web server administrators. The result for those of us who actually look at the logs is a lack of extremely valuable information.
So let’s start at the beginning. In the early days of the Web, people liked reporting one number only. And that number was how many hits a site received. Even today many executives want to know “How many hits did we get today?” even when they don’t know what a hit is. The more sophisticated person might ask how many pages were generated and what were the most popular pages.
Yet the information in the access logs goes far beyond this base information. So turn on all the possibilities of what a log can capture and let your favourite Web log analyzer have a go at it. You’ll be surprised at what you can find out.
Once you have this information you can use it to your advantage. Are you tired of having to build Web pages that work equally well in Netscape 4.7 and IE 4? Have your logs capture the user’s browser and versions. It’s easy to drop support for something when numbers support it. Recently I had to show why there was no need to test for Mac users. How did I do it? I showed from the client’s logs that less then 0.2 per cent of their traffic was coming from those using Macs.
There are other benefits to be had. Perhaps your marketing department wants to know how people are responding to the site. Simply turn on the referrer information and you’ll know which sites are sending you traffic. When it comes to search engines, the referrer log (when properly configured) can even tell you which search terms they entered to find you.
Also, take a look at the error log. It’s a great way to find problems with your site before someone else tells you. Got a 404 error for a file not found? When combined with the referrer information, you can even find out which page made the call. This makes it a lot easier to find and fix the problem.
Logs can also help you audit bandwidth costs. Does your hosting company charge you based on how much bandwidth you use? Then extract a bandwidth report from your logs to see exactly how much bandwidth you’ve been consuming.
Like any valuable ore, to extract true value you need to dig and possibly dig deep. If you’re new to looking or managing Web logs, roll up your sleeves and open your log in a text editor and start reading and interrupting the information. Once you’ve done this for an hour or two, you’ll appreciate log analysis software even more.
Just remember that this software summarizes things so you may still need to role up those sleeves once in a while to find that little gem.
K’necht is a speaker and president of K’nechtology Inc., a technology, business strategy and Web development company. He can be reached atalan@knechtology.com.