Customize Pound's Logging with Cronolog 2 comments

Posted by robon July 16, 2006

By default Pound (a software load balancer) logs to syslog. This is probably not where you want your web application’s access logs, especially if your site gets a lot of traffic. Using a very cool log rotation utility called Cronolog, and with a little reconfiguration of Pound, you can have Pound log to a directory of your choosing, just as you’re used to with Apache or Lighttpd.

Installing cronolog

To get this working, first download and install cronolog. On Debian based systems, this is as easy as:


$ apt-get update
$ apt-get install cronolog

Alternatively, you can download the source and build it yourself with:


$ wget http://cronolog.org/download/cronolog-1.6.2.tar.gz
$ tar xvzf cronolog-1.6.2.tar.gz
$ cd cronolog-1.6.2
$ ./configure --prefix=/usr/local
$ make
$ sudo make install

Once you’ve got cronolog installed, you can test it by sending it some output from the command-line with echo. For example:


$ echo "Arrg, this be a test..." | /usr/bin/cronolog \
> /var/log/www/%Y/access.%m-%d-%y.log

Running this command demonstrates how cronlog accepts input and creates log files based on a specified template string consisting of the current date and time. In this case, cronlog receives the output of the echo command and creates a directory named “2006” under /var/log/www containing a file called “access.07-17-06.log.”


$ cat /var/log/www/2006/access.07-17-06.log
Arrg, this be a test...

The date template format options are the same as those of the Unix date command (which is in turn the same as your systems C library’s implementation of the strftime). See the cronolog man page for a full listing of format options.

Using cronolog with Pound

The idea behind using Pound with cronolog is basically the same as the echo command above. You want to pipe the output of Pound directly to cronlolog. In order to get at Pound’s logs, you have to disable its built-in logging behavior that sends all of its output to syslog. To do this you reconfigure Pound, having it log to stderr instead. Then you can pipe Pound’s output directly to cronolog.

The option to change the default logging behavior of Pound must be done compile-time. You need to pass the —disable-log to configure when building pound. For example:


$ tar xvzf Pound-2.0.9.tgz
$ cd Pound-2.0.9
$ ./configure --disable-log
$ make
$ sudo make install

The final step is to pipe Pound’s output to cronolog. On a Debian system you can do this by modifying Pound’s init script a little. Basically, anywhere in this script where pound is started, you add an additional pipe string to the cronolog command. Here’s my Pound init script:

/etc/init.d/pound:

#! /bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/local/sbin/poun
CRONOLOG='/usr/bin/cronolog /var/log/www/pound/%Y/access.%m-%d-%y.log'
NAME=pound
DESC=pound
PID=/var/run/$NAME.pid

test -f $DAEMON || exit 0

set -e

# check if pound is configured or not
if [ -f "/etc/default/pound" ]
then
  . /etc/default/pound
  if [ "$startup" != "1" ]
  then
    echo "pound won't start unconfigured. configure & set startup=1 in /etc/default/pound" 
    exit 0  
  fi
else
  echo "/etc/default/pound not found" 
  exit 0  
fi

case "$1" in 
  start)  
    echo -n "Starting $DESC: " 
    start-stop-daemon --start --quiet --exec $DAEMON | $CRONOLOG &
    echo "$NAME." 
    ;;
  stop)
    echo -n "Stopping $DESC: " 
    start-stop-daemon --oknodo --pidfile $PID --stop --quiet --exec $DAEMON 
    echo "$NAME." 
    ;;
  restart|force-reload)
    echo -n "Restarting $DESC: " 
    start-stop-daemon --pidfile $PID --stop --quiet --exec $DAEMON 
    sleep 1 
    start-stop-daemon --start --quiet --exec $DAEMON | $CRONOLOG &
    echo "$NAME." 
    ;;
  *)
    N=/etc/init.d/$NAME
    # echo "Usage: $N {start|stop|restart|reload|force-reload}" >&2
    echo "Usage: $N {start|stop|restart|force-reload}" >&2
    exit 1  
    ;;
esac

exit 0

To avoid some repetition, I store the call to cronolog in a Bash variable named “CRONOLOG.” Then, in each place where pound is called, I append: ”| $CRONOLOG &” (a pipe, the output of the CRONOLOG variable, and an ampersand to put the process into the background).

After starting Pound with the init script with


$ sudo /etc/init.d/pound start

Pound logs its Apache style logs (Pound loglevel 3) to the following file:

/var/log/www/pound/2006/access.07-17-06.log:

blog.tupleshop.com 24.60.34.25 - - [11/Jul/2006:10:51:15 -0700] "GET /favicon.ico 
    HTTP/1.1" 200 1406 "" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; 
    rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4" 
blog.tupleshop.com 67.121.136.191 - - [11/Jul/2006:10:55:12 -0700] 
    "GET /images/figures/pound-deploy.png HTTP/1.1" 200 45041 "" "Mozilla/5.0 
    (Macintosh; U; Intel Mac OS X; en) AppleWebKit/418.8 (KHTML, like Gecko) 
    Safari/419.3" 
blog.tupleshop.com 68.142.33.136 - - [11/Jul/2006:10:55:50 -0700] 
    "GET /images/figures/pound-deploy.png HTTP/1.1" 200 45041 
    "http://www.oreillynet.com/ruby/blog/" "Mozilla/5.0 (Macintosh; U; PPC 
    Mac OS X; en) AppleWebKit/418 (KHTML, like Gecko) NetNewsWire/2.1" 

So, that’s it. If you think of anything I missed or got wrong, please leave it in the comments.

Thank you,
Rob


Comments

Leave a response

  1. Ezra ZygmuntowiczJuly 17, 2006 @ 03:59 PM

    Nice one for sure. Thanks.

  2. Andrew BennettJuly 25, 2006 @ 01:58 PM

    Here’s how to do the same thing for FreeBSD:

    !/bin/sh

    #

    1. $FreeBSD: ports/www/pound/files/pound.sh.in,v 1.2 2006/02/20 20:47:49 dougb Exp $ # #
    2. PROVIDE: pound
    3. REQUIRE: NETWORKING SERVERS
    4. BEFORE: DAEMON
    5. KEYWORD: shutdown #
    6. Add the following line to /etc/rc.conf to enable `pound’: #
    7. pound_enable=”YES”
    8. # optional if compiled with—disable-log
    9. pound_log_enable=”NO”
    10. pound_log_cmd=”/usr/local/sbin/cronolog”
    11. pound_log_args=”/var/log/www/pound/%Y/access.%Y-%m-%d.log” #
    12. and be sure to configure /usr/local/etc/pound.cfg

    . ”/etc/rc.subr”

    1. Set some defaults pound_enable=${pound_enable:-”NO”} pound_log_enable=${pound_log_enable:-”NO”} pound_log_cmd=${pound_log_cmd:-”/usr/local/sbin/cronolog”} pound_log_args=${pound_log_args:”/var/log/www/pound/%Y/access.%Y%m-%d.log”}

    name=”pound” rcvar=`set_rcvar`

    command=”/usr/local/sbin/$name” pidfile=”/var/run/$name.pid” required_files=”/usr/local/etc/$name.cfg”

    1. read configuration and set defaults load_rc_config $name : ${pound_enable=”NO”}

    if checkyesno pound_log_enable then echo “Logging enabled.” command_args=”| ${pound_log_cmd} ${pound_log_args} &” fi

    run_rc_command ”$1”