SQUID (Caching Proxy Server) Logs

SQUID is a caching proxy server for WebServers, FTP Servers etc. It reduces bandwidth and improves response times by caching and re-using frequently accessed webpages. Squid runs on almost all Operating Systems and is licensed under GNU GPL.

Squid defaults to TCP port 3128. You can use “squidclient” command to test the speed with which the cache is service the page:

Before testing with squidclient, you have to start squid as follows:

service squid start

[root@linuxgenius ~]# squidclient https://kernelcraft.wordpress.com

The above command dumps the entire HTML page on the screen.

You can check for various option for squidclient with –help option

Some of the important options are
-g -> count
-h -> hostname if not the localhost
-p -> port if not the default 3128
-v -> Verbose Output

You can now test the speed with which the cache is serving the page using -g option

[root@linuxgenius ~]# squidclient -g 0 https://kernelcraft.wordpress.com

‘0’ (Zero) specifies infinite times.

[root@linuxgenius ~]# squidclient -g 5 https://kernelcraft.wordpress.com
2011-12-06 21:26:49 [1]: 2.113 secs, 27.498344 KB/s
2011-12-06 21:26:51 [2]: 1.982 secs, 29.537841 KB/s
2011-12-06 21:26:54 [3]: 2.254 secs, 26.076752 KB/s
2011-12-06 21:26:54 [4]: 0.001 secs, 58783.000000 KB/s
2011-12-06 21:26:55 [5]: 0.001 secs, 58783.000000 KB/s
5 requests, round-trip (secs) min/avg/max = 0.001/1.270/2.254

As you can see the time taken is less on each count as the squid serves the same page from it’s cache. Sometimes you might see a bit of variations because of dynamic images loaded on the sites.

[root@linuxgenius ~]# squidclient -g 5 -h centos.sandbox.internal https://kernelcraft.wordpress.com

The above command is same as previous one, but with a particular squid cache hostname specified.

Log files in SQUID are as follows:

1. /var/log/squid/cache.log -> Displays the RAM, CPU, Virtual Memory, Networking information etc.
2. /var/log/squid/squid.out -> Displays the basic system information
3. /var/log/squid/access.log -> Main user log file

Access.log file has the following fields (using native squid log format):

1323187011.878 1982 TCP_MISS/200 58544 GET https://kernelcraft.wordpress.com/ – DIRECT/ text/html

* Request_Time (date +%s) -> 1323187011.878 (measured in UNIX Epoch milliseconds)
* Elapsed_Time -> 1982 (milliseconds of page/object delivery)
* Remote Host IP Address ->
* Code/Status (TCP_MISS/200 or TCP/HIT/200)
{TCP_MISS (New page or Page accessed which is not from cache)}
{TCP_HIT (Page accessed from cache)}
* Bytes Delivered to Client -> 58544
* Method (GET/POST/CONNECT) CONNECT is secure connection using SSL or TLS
* URL -> https://kernelcraft.wordpress.com/
* – -> IDENT Identification (Not reliable, if configured user names)
* Hierarchy -> DIRECT/ (DIRECT/IP)
* Type -> MIME

Note:- Squid also support CLF (Common Log Format) which other applications like webalizer understands

To enable CLF, go to Squid Configuration file,
vi /etc/squid/squid.conf
{Search for ’emulate’}
emulate_httpd_log on (By default this line is commented and in off status. Uncomment and turn it on)

If you check the Squid access.log, you will see directives like TCP_HIT and TCP_MISS.
When you access a new page via squid proxy, the squid logs will show it as TCP_MISS, which means it’s not a page from proxy cache, instead the page was pulled directly from the originating server. If this page is again invoked, then you will see a TCP_HIT meaning the page is pulled from proxy cache and not from Internet.

Webalizer (Proxy Log):

In order for Webalizer to access the squid log files, enable ’emulate_httpd_log’ as mentioned above so that native squid logs could be converted to CLF format which Webalizer recognizes.

vi /etc/webalizer.conf
LogFile /var/log/httpd/access_log (This is the default line which has to be changed)
LogFile /var/log/squid/access.log (Change the path and filename as follows)

{Now, search for ‘HostName’. Update the HostName to your Squid server name}.

HostName centos.sandbox.internal

This is not mandatory, but it’s useful for future analysis.

Webalizer will write it’s output to /var/www/usage/

In order to get the output to this directory, you have to run webalizer with ‘c’ option followed by Webalizer configuration file as follows:

webalizer -c /etc/webalizer.conf

This will process squid.conf. Now, if you check /var/www/usage/ you will see all the files. You have to start apache and now you will be able to access the webalizer usage report as follows:



2 responses to “SQUID (Caching Proxy Server) Logs

  1. Pingback: SQUID ACLs | Kernel Craft

  2. Pingback: Transparent SQUID Proxy | Kernel Craft

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s