
			log_format rt_cache '$remote_addr - $upstream_cache_status [$time_local]  '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent"';

Then in server block:

Either replace lines like:


			access_log   /var/log/nginx/example.com.access.log;

With line:


			access_log   /var/log/nginx/example.com.access.log rt_cache;

OR you can use multiple access logs like below:


			access_log   /var/log/nginx/example.com.access.log;
access_log   /var/log/nginx/example.com.cache.log rt_cache;

Please note when using multiple logs, use separate file names.

Once you make above changes, reload nginx config and wait for log files to fill up!

Analysing cache efficiency:

HIT vs MISS vs etc

Run a command like below on your access.log file:


			awk '{print $3}' access.log  | sort | uniq -c | sort -r

Sample output:


			800 HIT
    779 -
    392 BYPASS
     19 EXPIRED
     14 MISS

Note: dash (“-“) means request never reached to upstream module. Most likely it means return 403, return 444, and so on.

MISS Request URLs


			awk '($3 ~ /MISS/)'  access.log | awk '{print $7}' | sort | uniq -c | sort -r

BYPASS Requests URLs


			awk '($3 ~ /BYPASS/)'  access.log | awk '{print $7}' | sort | uniq -c | sort -r

MISS v/s BYPASS

MISS occurs when a pattern is configured to cache but at the time of request was not cached. In correct configuration, subsequent requests will be served from cache based on caching duration other parameters.

BYPASS occurs when a pattern was explicitly configured NOT to use cache. e.g. skipping cache for logged in user. Subsequent requests will also be bypassed.

Recommendations:

If MISS are frequent, consider increasing cache size and/or duration.
For BYPASS, analyse patterns. Many times we set requests with query strings to BYPASS cache. On high traffic site, it will be wise to cache output of certain requests with query strings, even that query_string alters outcome of pages. Be careful to not cache wp_nonce and auth_tokens. I will post more about this soon!

Must Read: Useful Commands to parse access.log

Tweaking fastcgi-buffers

PREVIOUS

Using Pagespeed

NEXT

Comments

David Rahrer

December 28, 2013

I’ve set up a test site following your instructions on enabling fastcgi_cache and the new phpopcache (using Nginx 1.41 and PHP 5.5.7). When I check the nginx-cache folder in /var/run it is accumulating files. I can purge them from the Nginx Helper plugin button and they go away, and they come back with more browsing of the site. However, when I set up this cache log there is nothing there. Also, when I check the headers after setting up the “add_header rt-Fastcgi-Cache $upstream_cache_status;” line, the header isn’t there at all. I’ve checked and double checked, but can’t find anything wrong. Can you think what would cause the header to not appear even though it seems to be caching? Also, the Nginx Helper plugin is adding it’s line to the HTML source saying it is working, however it does this even when I’m logged into the site and there should be no caching. I would appreciate any help greatly, as I’m stuck. Thanks.
1. Rahul Bansal
  
  December 30, 2013
  
  To sum up – I guess you are not seeing rt-Fastcgi-Cache header in HTTP response.
  
  Did you add line add_header rt-Fastcgi-Cache $upstream_cache_status; inside /etc/nginx/nginx.conf and also reloaded nginx config (service nginx reload) ?
  
  If you are still facing issue, please send a URL to test site on your server here (if its publicly accessible).
David Rahrer

December 30, 2013

Thank you for your reply and please forgive me for not coming back to update this. It was very late and I had bee working too long on the same things — I made a stupid mistake. In fact, it did work fine once I started updating the correct server, lol. I had FileZilla open to one server and terminal open to another. I was saving my attempts to correctly configure on one server, but I was restarting the service on another, so nothing changed when I viewed it in the browser. Like I said, stupid mistake and I did that for a couple of hours! I’ll try to make sure any future questions are better founded. I tend to binge learn and I don’t know when to quit and get some rest. Thanks again, you have a great resource here and thank you so much for the Nginx Helper plugin.
1. Rahul Bansal
  
  December 31, 2013
  
  No probs David. Sometimes it happens with us also.
  
  My colleague added some codes to each server so we get different text colors in terminal for every.
panda

January 22, 2014

Hi, Rahul Bansal. I’ve tested my site and found that there were so many “miss”, I followed your suggestion by increasing “keys_zone=wordpress” from 10m to 100m, “max_size” from 60m to 200m, “inactive” from 60m to 460m.

But I still have too many “miss”:

905 - 4 updateing 363 BYPASS 1799 MISS 12911 HIT 1416 EXPIRED

Any suggestion?

BTW: I got 12911 hits + 1799 MISS + 1416 EXPIRED in the report but Google Analytics told me I have 7k pageviews. 16k – 7k = 9k, confused, the 9k traffics was an attack?
1. Rahul Bansal
  
  January 22, 2014
  
  Try adding fastcgi_ignore_header line. Ref – https://rtcamp.com/support/topic/fastcgi_cache-problem/
  
  Google Analytics and server logs record different kind of hits so ignore any relation between them.
  1. panda
    
    January 24, 2014
    
    Saily, still have so many “miss” and “Expired” after added below line above server { … } section in site-available.
    
    fastcgi_ignore_headers Cache-Control Expires;
    
    or
    
    fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
    1. Rahul Bansal
      
      January 24, 2014
      
      You need to wait sometime before checking stats. Check stats next day or flush previous log.
      
      Also, can you make sure if your caching is indeed working fine?
      
      Ref – https://rtcamp.com/tutorials/nginx/upstream-cache-status-in-response-header/
      1. panda
        
        January 25, 2014
        
        I was doing that as you said. Added fastcgi_ignore_header line and checked the stats one day later.
        
        I’ve post my conf files at https://rtcamp.com/support/topic/php5-fpm-high-cpu-usage-and-111-connection-refused/, and I think the caching is working.
      2. Rahul Bansal
        
        January 27, 2014
        
        Why “think” when you can confirm by simply running curl -I ? 🙂
        
        Please confirm.

Features

Useful Links

Product

Other Products

GoDAM

Subscribe for curated product updates, tips, and announcements.

"*" indicates required fields

This field is for validation purposes and should be left unchanged.

Email*

SIGN ME UP