[ Top | Up |
Prev | Next |
Map | Index ]
Coping with low memory
This section describes how to run analog with lower amounts of memory. For
a normal logfile this will make analog run a bit slower. But if your computer
is running out of memory when running analog, it will go very slowly indeed:
so for large logfiles, this can make analog run much faster, or even make an
analysis possible that wouldn't otherwise be possible.
Recall what happens to an item when it has been read in. First it is
aliased. Secondly, it is checked to see whether
it is included or excluded. Then finally, if all
the items are wanted, one request is added to its score.
Normally the name of the item is saved before the aliasing takes place. This
avoids analog having to do the aliasing again next time the same item is
encountered. But this can take up more memory than necessary. So there is a
family of LOWMEM commands provided, which tell analog to record the
name at a later stage, or even not at all. If you use these commands, analog
will have to do a bit more work than normal, but it will use less memory.
On most sites, the hosts take up most of the memory, so I'll use the
HOSTLOWMEM command as an example.
The command
HOSTLOWMEM 0
represents the normal case, when the hostname is recorded before being aliased.
If you specify
HOSTLOWMEM 1
instead, then the hostname is not recorded until after the aliasing. If you
specify
HOSTLOWMEM 2
then the name is not recorded until after the inclusion and exclusion lookup
has been done as well. And finally, if you give the command
HOSTLOWMEM 3
then the hostname is not saved at all, and the Host Report will not be
constructed, even if you've asked for it. (The Domain Report can still be
constructed though.) The analogous commands for the other items are
FILELOWMEM, BROWLOWMEM, REFLOWMEM,
USERLOWMEM and VHOSTLOWMEM.
So what should you do if analog runs out of memory? First, look in your
logfile to see which items are taking up all the memory. If you have lots
of different filenames, ones you generate on the fly for example, you would
want to use the FILELOWMEM commands. Maybe you could combine all
the similar filenames into one with a FILEALIAS command, and use
FILELOWMEM 1. (If you have lots of different filenames caused by
different search arguments, then using
ARGSEXCLUDE might solve your
problem
without any need to use LOWMEM at all). But for most users, it
is the hostnames which cause the problem. If you only want to analyse requests
from certain hosts, then you could use HOSTLOWMEM 2 to exclude
the others before recording those that are left. If you don't want to exclude
any hosts, and you haven't got enough memory to record all the different
hostnames, then HOSTLOWMEM 3 would be appropriate.
Stephen Turner
Need help with analog? Subscribe to the analog-help
mailing list
[ Top | Up |
Prev | Next |
Map | Index ]