Create your blog and photo album with postbit
Create your blog and photo album

Create new post

Content:

Upload a picture:
Tags (keywords separated by comma)

Save Cancel
webtools:   Followers: 10 ; Following: 2

Explore webtools's photo albums:
Photos from posts (2)
Photos from posts (2)

Custom access_log format and awstats


When using Apache web server, I like to have a customized access log format because I have many distinct domains in a single IP using virtualhosts configuration. It's a variation from the NSCA extended common log format with virtual host.

In my virtualhosts.conf, I usually redefine access_log file format to include more useful information. This is what I write in the beginning of the virtual hosts conf file:

LogFormat "%V %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vcombined
CustomLog logs/access_log vcombined
 
Then inside the tags <VirtualHost *:80> ... </VirtualHost>, this custom log format can be included like this:
CustomLog logs/my-domain-example.com-access_log vcombined
 
So it will use the "vcombined" format specified in the beginning of the file above.
 
Example of the resulting log file (real sample from one of my logs):
postbit.com 177.148.210.156 - - [27/Aug/2014:18:03:23 -0300] "GET / HTTP/1.1" 200 14281 "http://dig.do/postbit.com" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36 OPR/23.0.1522.77"
 
Description of the custom log format using this example:
%V = postbit.com (host name)
%h = 177.148.1.1 (user's IP number)
%l = - (remote logname from identd)
%u = - (remote user from auth)
%t = [27/Aug/2014:18:03:23 -0300] (date, time and gmt)
\"%r\" = "GET / HTTP/1.1" (Method URL with GET or POST command)
%>s = 200 (status result, 200 OK in this case, could be 404, 500, 301, etc.)
%b = 14281 (size of the answer, number of bytes)
\"%{Referer}i\" = "http://dig.do/postbit.com" (Referer page)
\"%{User-Agent}i\" = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36 OPR/23.0.1522.77" (user agent browser)
 

Web Analytics:
How to run AwStats to process web log files


When I need to process the access log with an anlyser tool (e.g. AWStats), then I need to change the log format in the default configuration file:
AwStats configuration file path: ./awstats-7.4/wwwroot/cgi-bin/awstats.example.com.conf
 
The log file format I need to configure to match my Apache log and use awstats is this:
LogFormat = "%host %host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"
 
Then run awstats with command:
perl awstats.pl -config=my-domain
 
Warning: If you get the error "Never updated (See 'Build/Update' on awstats_setup.html page)", then check if you put only the name of your configuration file in the "-config=my-domain" (correct). Do NOT run with parameter "-config=awstats.my-domain.conf" (wrong). This will correct the problem of having zero in all values (awstats all zero, showing only 0, empty stats).
 
The results stats can be seen using a local web server:
(start this webserver in wwwroot directory, that is cgi-bin previous directory)
 
Then see the webstats results in your browser:
http://localhost:8000/cgi-bin/awstats.pl?config=my-domain
 

For some reason the images (icons) were broken and to correct I had to change in the conf file:
DirIcons="/awstatsicons" changed to DirIcons="/icon"

To see geo stats you need to instsall Geo::IPfree perl module and uncomment this line in the conf file:
LoadPlugin="geoipfree"
(see more about GeoIP)

Then you need to run again (rebuild) the stats. Just remove the file "cgi-bin/awstats082014.my-domain.txt" (change the date and filename to your file) and rerun awstats again (-update parameter):

perl awstats.pl -config=my-domain -update

AwStats hints:

* If you have multiple log files (a log file split in many parts) you can run awstats with parameter "-LogFile=x" to override the access log path defined in configuration file.
Example:
perl awstats.pl -config=my-domain -LogFile=/var/log/httpd/my-domain.access_log
 

* If you need to consider 301 redirect as a valid requests, change conf file to include "301" (warning: if the redirect is to a page in your own domain, then 1 access will count as 2 pages viewed):

# ValidHTTPCodes="200 304"
ValidHTTPCodes="200 304 301"
 

 

Post by webtools (2014-08-27 18:09)

Post your comment:

Name: Email: Site:




| Explore users | New posts | Create your blog | Create your photo album |
| About Postbit | Our blog | Terms of use | Contact Postbit |


Copyright © 2018 - postbit.com