Posts: 3,631
Threads: 899
Joined: Dec 2003
Reputation:
0
hi all,
i'm looking for some guidance as to the best way to rotate my apache logs, they are now large....
i want to retain the data as much as possible,
any advice ?
Code: -rw-r--r-- 1 root root 0 Jan 10 21:34 access_bad.log
-rw-r--r-- 1 root root 434839497 Jan 15 12:00 access_log
-rw-r--r-- 1 root root 130760500 Jan 15 11:59 agent_log
-rw-r--r-- 1 root root 11575178 Jan 15 11:59 error_log
-rw-r--r-- 1 root root 6 Jan 13 21:37 httpd.pid
-rw-r--r-- 1 root root 149330258 Jan 15 11:59 referer_log
cheers
anyweb
Posts: 1,229
Threads: 45
Joined: Mar 2005
Reputation:
0
2006-01-15, 06:22 PM
(This post was last modified: 2006-01-15, 08:48 PM by znx.)
First off, there are replacements to the basic syslog (or syslog-ng) that have builtin logrotating skills but anyway... its easy to do.
First install "logrotate", this is the utility for rotating logs.
Example /etc/logrotate.conf
Code: # rotate log files weekly
weekly
# keep 4 rotations (4 weeks in this case)
rotate 4
# create new log files
create
# compress the files after rotation
compress
# packages can have their own personal files (i like this)
include /etc/logrotate.d
# dont rotate if empty, dont mail, dont move rotated into another dir
notifempty
nomail
noolddir
Ok so thats the default for packages, now lets create a file for apache /etc/logrotate.d/apache
Code: /var/log/apache/*log {
missingok
notifempty
sharedscripts
postrotate
/etc/init.d/apache reload > /dev/null 2>&1 || true
endscript
}
Then make a cron entry /etc/cron.daily/logrotate.cron (in daily in case you switch to daily rotation and forget the cron like i do ;))
Code: #! /bin/sh
/usr/sbin/logrotate /etc/logrotate.conf
You will need to touch up on the paths for your system but should be it. :) weee
Posts: 3,631
Threads: 899
Joined: Dec 2003
Reputation:
0
ok thanks znx
done,
now what i'm seeing is this
Quote:[anyweb@www ~]$ ls -al /usr/local/apache/logs/total 762196
drwxr-xr-x 2 anyweb anyweb 4096 Jan 22 04:26 .
drwxr-xr-x 15 anyweb anyweb 4096 Apr 3 2005 ..
-rw-r--r-- 1 root root 0 Jan 10 21:34 access_bad.log
-rw-r--r-- 1 root root 0 Jan 22 04:26 access_log
-rw-r--r-- 1 root root 487006197 Jan 25 15:05 access_log.1
-rw-r--r-- 1 root root 0 Jan 22 04:26 agent_log
-rw-r--r-- 1 root root 130760500 Jan 15 11:59 agent_log.1
-rw-r--r-- 1 root root 0 Jan 22 04:26 error_log
-rw-r--r-- 1 root root 12588341 Jan 25 15:04 error_log.1
-rw-r--r-- 1 root root 6 Jan 13 21:37 httpd.pid
-rw-r--r-- 1 root root 0 Jan 22 04:26 referer_log
-rw-r--r-- 1 root root 149330258 Jan 15 11:59 referer_log.1
does that look correct to you ?
cheers
anyweb
Posts: 1,229
Threads: 45
Joined: Mar 2005
Reputation:
0
2006-01-25, 02:21 PM
(This post was last modified: 2006-01-25, 02:22 PM by znx.)
Er yes .. but only if it was instantly after the rotation, if not it looks like your apache isn't logging again, this is an issue with
1) your httpd.conf either moving the logs elsewhere or not logging at all
2) you failed to reload apache on the log rotation
Also I note that you chose not to compress (which is possibly a good idea, as those logs will take a year to compress ;))
If 1) mod your apache conf and restart it.
If 2) restart apache manually to get it logging again and look at the line just after "postrotate".
Hope this helps
Posts: 3,631
Threads: 899
Joined: Dec 2003
Reputation:
0
ok
heres the log part of httpd.conf
look ok ?
cheers
anyweb
Quote:# Possible values include: debug, info, notice, warn, error, crit,# alert, emerg.
#
LogLevel warn
#
# The following directives define some format nicknames for use with
# a CustomLog directive (see below).
#
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
#
# The location and format of the access logfile (Common Logfile Format).
# If you do not define any access logfiles within a <VirtualHost>
# container, they will be logged here. Contrariwise, if you *do*
# define per-<VirtualHost> access logfiles, transactions will be
# logged therein and *not* in this file.
#
#CustomLog /usr/local/apache/logs/access_log common
#
# If you would like to have agent and referer logfiles, uncomment the
# following directives.
#
#CustomLog /usr/local/apache/logs/referer_log referer
#CustomLog /usr/local/apache/logs/agent_log agent
#
# If you prefer a single logfile with access, agent, and referer information# (Combined Logfile Format) you can use the following directive.
#
#CustomLog /usr/local/apache/logs/access_log combined
CustomLog /usr/local/apache/logs/access_log combined env=!BAD
CustomLog /usr/local/apache/logs/access_bad.log combined env=BAD
after restarting apache i now get
Quote:-rw-r--r-- 1 root root 17368 Jan 25 15:31 access_log-rw-r--r-- 1 root root 487170458 Jan 25 15:31 access_log.1
-rw-r--r-- 1 root root 0 Jan 22 04:26 agent_log
-rw-r--r-- 1 root root 130760500 Jan 15 11:59 agent_log.1
-rw-r--r-- 1 root root 161 Jan 25 15:31 error_log
-rw-r--r-- 1 root root 12589458 Jan 25 15:31 error_log.1
-rw-r--r-- 1 root root 6 Jan 25 15:31 httpd.pid
-rw-r--r-- 1 root root 0 Jan 22 04:26 referer_log
i think that means its working now,
but why didnt it work before ?
Posts: 1,229
Threads: 45
Joined: Mar 2005
Reputation:
0
Quote:i think that means its working now,
but why didnt it work before ?
That means that the postrotate failed, what did you use for the postrotate?
For instance, if you never start apache with an initscript, what I have above wouldn't work, if you use apachectl then thats the line you want.
Basically you need to reload (or restart if you wish but its not needed) apache once you "steal" its log files with logrotate.
Ensure that the postrotate line actually works :)
Posts: 3,631
Threads: 899
Joined: Dec 2003
Reputation:
0
ok znx i'll check it
heres the code
Quote:/usr/local/apache/logs/*log { missingok
notifempty
sharedscripts
postrotate
/usr/local/apache/bin/apachectl reload > /dev/null 2>&1 || true
endscript
}
testing...
Code: [root@www longhorn]# /usr/local/apache/bin/apachectl reload > /dev/null 2>&1 || true
cheers
anyweb
Posts: 1,229
Threads: 45
Joined: Mar 2005
Reputation:
0
2006-01-25, 08:02 PM
(This post was last modified: 2006-01-25, 08:03 PM by znx.)
Via some live IRC stuff :P
snip'd btw
Code: <anyweb> [root@www longhorn]# /usr/local/apache/bin/apachectl reload > /dev/null 2>&1 || true
<znx> /usr/local/apache/bin/apachectl reload try just that
<anyweb> [root@www longhorn]# /usr/local/apache/bin/apachectl reload
<anyweb> usage: /usr/local/apache/bin/apachectl (start|stop|restart|fullstatus|status|graceful|configtest|help)
<anyweb> lol
<znx> so its not working :) hehe
<anyweb> okies
<anyweb> /usr/local/apache/bin/apachectl restart > /dev/null 2>&1 || true
<anyweb> got it
<znx> yup
ain't irc great :)
Thus:
Code: /usr/local/apache/logs/*log {
missingok
notifempty
sharedscripts
postrotate
/usr/local/apache/bin/apachectl restart > /dev/null 2>&1 || true
endscript
}
Posts: 3,631
Threads: 899
Joined: Dec 2003
Reputation:
0
sorted thanks again znx :)
Posts: 3,631
Threads: 899
Joined: Dec 2003
Reputation:
0
2006-11-01, 06:34 AM
(This post was last modified: 2006-11-01, 06:53 AM by anyweb.)
yet another problem
after upgrading to fcr5 on the webserver, and adding logrotate again
i tried to manually run webalizer today and get 'no valid records found'
Code: [root@www ~]# /usr/bin/webalizer
No valid records found!
[root@www ~]# cd /usr/local/apache/logs/
here's the contents of the logs dir
Quote:[root@www logs]# ls -altotal 164380
drwxr-xr-x 2 root root 4096 Oct 29 04:05 .
drwxr-xr-x 14 root root 4096 Jan 1 1970 ..
-rw-r--r-- 1 root root 0 Oct 29 04:05 access_log
-rw-r--r-- 1 root root 98791114 Nov 1 07:30 access_log.1
-rw-r--r-- 1 root root 0 Oct 29 04:05 agent_log
-rw-r--r-- 1 root root 33517582 Nov 1 07:30 agent_log.1
-rw-r--r-- 1 root root 0 Oct 29 04:05 error_log
-rw-r--r-- 1 root root 1396758 Nov 1 07:30 error_log.1
-rw-r--r-- 1 root root 5 Oct 27 09:44 httpd.pid
-rw-r--r-- 1 root root 0 Oct 29 04:05 referer_log
-rw-r--r-- 1 root root 34414268 Nov 1 07:30 referer_log.1
as i can see it's rotating the logs alright but it's not looking right at all, hence the webalizer failure
the webalizer.conf file seems to be looking at one file only so how do i get it to work with rotated log files
Quote:# LogFile defines the web server log file to use. If not specified# here or on on the command line, input will default to STDIN. If
# the log filename ends in '.gz' (ie: a gzip compressed file), it will
# be decompressed on the fly as it is being read.
LogFile /usr/local/apache/logs/access_log
any ideas ?
cheers
anyweb
|