Create your blog and photo album with postbit
Create your blog and photo album

Create new post


Upload a picture:
Tags (keywords separated by comma)

Save Cancel
webtools:   Followers: 9 ; Following: 2

Explore webtools's photo albums:
Photos from posts (2)
Photos from posts (2)

Custom access_log format and awstats

When using Apache web server, I like to have a customized access log format because I have many distinct domains in a single IP using virtualhosts configuration. It's a variation from the NSCA extended common log format with virtual host.

In my virtualhosts.conf, I usually redefine access_log file format to include more useful information. This is what I write in the beginning of the virtual hosts conf file:

LogFormat "%V %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vcombined
CustomLog logs/access_log vcombined
Then inside the tags <VirtualHost *:80> ... </VirtualHost>, this custom log format can be included like this:
CustomLog logs/ vcombined
So it will use the "vcombined" format specified in the beginning of the file above.
Example of the resulting log file (real sample from one of my logs): - - [27/Aug/2014:18:03:23 -0300] "GET / HTTP/1.1" 200 14281 "" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36 OPR/23.0.1522.77"
Description of the custom log format using this example:
%V = (host name)
%h = (user's IP number)
%l = - (remote logname from identd)
%u = - (remote user from auth)
%t = [27/Aug/2014:18:03:23 -0300] (date, time and gmt)
\"%r\" = "GET / HTTP/1.1" (Method URL with GET or POST command)
%>s = 200 (status result, 200 OK in this case, could be 404, 500, 301, etc.)
%b = 14281 (size of the answer, number of bytes)
\"%{Referer}i\" = "" (Referer page)
\"%{User-Agent}i\" = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36 OPR/23.0.1522.77" (user agent browser)

Web Analytics:
How to run AwStats to process web log files

When I need to process the access log with an anlyser tool (e.g. AWStats), then I need to change the log format in the default configuration file:
AwStats configuration file path: ./awstats-7.4/wwwroot/cgi-bin/
The log file format I need to configure to match my Apache log and use awstats is this:
LogFormat = "%host %host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"
Then run awstats with command:
perl -config=my-domain
Warning: If you get the error "Never updated (See 'Build/Update' on awstats_setup.html page)", then check if you put only the name of your configuration file in the "-config=my-domain" (correct). Do NOT run with parameter "" (wrong). This will correct the problem of having zero in all values (awstats all zero, showing only 0, empty stats).
The results stats can be seen using a local web server:
(start this webserver in wwwroot directory, that is cgi-bin previous directory)
Then see the webstats results in your browser:

For some reason the images (icons) were broken and to correct I had to change in the conf file:
DirIcons="/awstatsicons" changed to DirIcons="/icon"

To see geo stats you need to instsall Geo::IPfree perl module and uncomment this line in the conf file:
(see more about GeoIP)

Then you need to run again (rebuild) the stats. Just remove the file "cgi-bin/" (change the date and filename to your file) and rerun awstats again (-update parameter):

perl -config=my-domain -update

AwStats hints:

* If you have multiple log files (a log file split in many parts) you can run awstats with parameter "-LogFile=x" to override the access log path defined in configuration file.
perl -config=my-domain -LogFile=/var/log/httpd/my-domain.access_log

* If you need to consider 301 redirect as a valid requests, change conf file to include "301" (warning: if the redirect is to a page in your own domain, then 1 access will count as 2 pages viewed):

# ValidHTTPCodes="200 304"
ValidHTTPCodes="200 304 301"


Post by webtools (2014-08-27 18:09)

Post your comment:

Name: Email: Site:

| Explore users | New posts | Create your blog | Create your photo album |
| About Postbit | Our blog | Terms of use | Contact Postbit |

Copyright © 2018 -