Jump to content

how do i rotate my apache logs


Recommended Posts

hi all,

 

i'm looking for some guidance as to the best way to rotate my apache logs, they are now large....

 

i want to retain the data as much as possible,

 

any advice ?

 

-rw-r--r--   1 root   root		   0 Jan 10 21:34 access_bad.log
-rw-r--r--   1 root   root   434839497 Jan 15 12:00 access_log
-rw-r--r--   1 root   root   130760500 Jan 15 11:59 agent_log
-rw-r--r--   1 root   root	11575178 Jan 15 11:59 error_log
-rw-r--r--   1 root   root		   6 Jan 13 21:37 httpd.pid
-rw-r--r--   1 root   root   149330258 Jan 15 11:59 referer_log

 

cheers

anyweb

  • Like 2
Link to post
Share on other sites

First off, there are replacements to the basic syslog (or syslog-ng) that have builtin logrotating skills but anyway... its easy to do.

 

First install "logrotate", this is the utility for rotating logs.

 

Example /etc/logrotate.conf

# rotate log files weekly
weekly

# keep 4 rotations (4 weeks in this case)
rotate 4

# create new log files
create

# compress the files after rotation
compress

# packages can have their own personal files (i like this)
include /etc/logrotate.d

# dont rotate if empty, dont mail, dont move rotated into another dir
notifempty
nomail
noolddir

 

Ok so thats the default for packages, now lets create a file for apache /etc/logrotate.d/apache

/var/log/apache/*log {
 missingok
 notifempty
 sharedscripts
 postrotate
 /etc/init.d/apache reload > /dev/null 2>&1 || true
 endscript
}

 

Then make a cron entry /etc/cron.daily/logrotate.cron (in daily in case you switch to daily rotation and forget the cron like i do ;))

#! /bin/sh

/usr/sbin/logrotate /etc/logrotate.conf

 

You will need to touch up on the paths for your system but should be it. ^_^ weee

Link to post
Share on other sites
  • 2 weeks later...

ok thanks znx

 

done,

 

now what i'm seeing is this

 

[anyweb@www ~]$ ls -al /usr/local/apache/logs/

total 762196

drwxr-xr-x 2 anyweb anyweb 4096 Jan 22 04:26 .

drwxr-xr-x 15 anyweb anyweb 4096 Apr 3 2005 ..

-rw-r--r-- 1 root root 0 Jan 10 21:34 access_bad.log

-rw-r--r-- 1 root root 0 Jan 22 04:26 access_log

-rw-r--r-- 1 root root 487006197 Jan 25 15:05 access_log.1

-rw-r--r-- 1 root root 0 Jan 22 04:26 agent_log

-rw-r--r-- 1 root root 130760500 Jan 15 11:59 agent_log.1

-rw-r--r-- 1 root root 0 Jan 22 04:26 error_log

-rw-r--r-- 1 root root 12588341 Jan 25 15:04 error_log.1

-rw-r--r-- 1 root root 6 Jan 13 21:37 httpd.pid

-rw-r--r-- 1 root root 0 Jan 22 04:26 referer_log

-rw-r--r-- 1 root root 149330258 Jan 15 11:59 referer_log.1

 

does that look correct to you ?

 

cheers

 

anyweb

Link to post
Share on other sites

Er yes .. but only if it was instantly after the rotation, if not it looks like your apache isn't logging again, this is an issue with

 

1) your httpd.conf either moving the logs elsewhere or not logging at all

2) you failed to reload apache on the log rotation

 

Also I note that you chose not to compress (which is possibly a good idea, as those logs will take a year to compress ;))

 

If 1) mod your apache conf and restart it.

If 2) restart apache manually to get it logging again and look at the line just after "postrotate".

 

Hope this helps

Link to post
Share on other sites

ok

 

heres the log part of httpd.conf

 

look ok ?

 

cheers

 

anyweb

 

# Possible values include: debug, info, notice, warn, error, crit,

# alert, emerg.

#

LogLevel warn

 

#

# The following directives define some format nicknames for use with

# a CustomLog directive (see below).

#

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

LogFormat "%h %l %u %t \"%r\" %>s %b" common

LogFormat "%{Referer}i -> %U" referer

LogFormat "%{User-agent}i" agent

 

#

# The location and format of the access logfile (Common Logfile Format).

# If you do not define any access logfiles within a <VirtualHost>

# container, they will be logged here. Contrariwise, if you *do*

# define per-<VirtualHost> access logfiles, transactions will be

# logged therein and *not* in this file.

#

#CustomLog /usr/local/apache/logs/access_log common

 

#

# If you would like to have agent and referer logfiles, uncomment the

# following directives.

#

#CustomLog /usr/local/apache/logs/referer_log referer

#CustomLog /usr/local/apache/logs/agent_log agent

 

#

# If you prefer a single logfile with access, agent, and referer information# (Combined Logfile Format) you can use the following directive.

#

 

#CustomLog /usr/local/apache/logs/access_log combined

CustomLog /usr/local/apache/logs/access_log combined env=!BAD

CustomLog /usr/local/apache/logs/access_bad.log combined env=BAD

after restarting apache i now get

 

-rw-r--r-- 1 root root 17368 Jan 25 15:31 access_log

-rw-r--r-- 1 root root 487170458 Jan 25 15:31 access_log.1

-rw-r--r-- 1 root root 0 Jan 22 04:26 agent_log

-rw-r--r-- 1 root root 130760500 Jan 15 11:59 agent_log.1

-rw-r--r-- 1 root root 161 Jan 25 15:31 error_log

-rw-r--r-- 1 root root 12589458 Jan 25 15:31 error_log.1

-rw-r--r-- 1 root root 6 Jan 25 15:31 httpd.pid

-rw-r--r-- 1 root root 0 Jan 22 04:26 referer_log

 

i think that means its working now,

 

but why didnt it work before ?

Link to post
Share on other sites
i think that means its working now,

 

but why didnt it work before ?

 

That means that the postrotate failed, what did you use for the postrotate?

 

For instance, if you never start apache with an initscript, what I have above wouldn't work, if you use apachectl then thats the line you want.

 

Basically you need to reload (or restart if you wish but its not needed) apache once you "steal" its log files with logrotate.

 

Ensure that the postrotate line actually works :)

Link to post
Share on other sites

ok znx i'll check it

 

heres the code

 

/usr/local/apache/logs/*log {

missingok

notifempty

sharedscripts

postrotate

/usr/local/apache/bin/apachectl reload > /dev/null 2>&1 || true

endscript

}

 

testing...

 

[root@www longhorn]#  /usr/local/apache/bin/apachectl reload > /dev/null 2>&1 || true

 

cheers

anyweb

Link to post
Share on other sites

Via some live IRC stuff :P

 

snip'd btw

<anyweb> [root@www longhorn]#  /usr/local/apache/bin/apachectl reload > /dev/null 2>&1 || true
<znx> /usr/local/apache/bin/apachectl reload	 try just that
<anyweb> [root@www longhorn]# /usr/local/apache/bin/apachectl reload
<anyweb> usage: /usr/local/apache/bin/apachectl (start|stop|restart|fullstatus|status|graceful|configtest|help)
<anyweb> lol
<znx> so its not working :) hehe
<anyweb> okies
<anyweb> /usr/local/apache/bin/apachectl restart > /dev/null 2>&1 || true
<anyweb> got it
<znx> yup

 

ain't irc great ^_^

 

Thus:

/usr/local/apache/logs/*log {
 missingok
 notifempty
 sharedscripts
 postrotate
/usr/local/apache/bin/apachectl restart > /dev/null 2>&1 || true
 endscript
}

Link to post
Share on other sites
  • 9 months later...

yet another problem

 

after upgrading to fcr5 on the webserver, and adding logrotate again

 

i tried to manually run webalizer today and get 'no valid records found'

 

[root@www ~]# /usr/bin/webalizer 
No valid records found!

 

 

[root@www ~]# cd /usr/local/apache/logs/

here's the contents of the logs dir

 

[root@www logs]# ls -al

total 164380

drwxr-xr-x 2 root root 4096 Oct 29 04:05 .

drwxr-xr-x 14 root root 4096 Jan 1 1970 ..

-rw-r--r-- 1 root root 0 Oct 29 04:05 access_log

-rw-r--r-- 1 root root 98791114 Nov 1 07:30 access_log.1

-rw-r--r-- 1 root root 0 Oct 29 04:05 agent_log

-rw-r--r-- 1 root root 33517582 Nov 1 07:30 agent_log.1

-rw-r--r-- 1 root root 0 Oct 29 04:05 error_log

-rw-r--r-- 1 root root 1396758 Nov 1 07:30 error_log.1

-rw-r--r-- 1 root root 5 Oct 27 09:44 httpd.pid

-rw-r--r-- 1 root root 0 Oct 29 04:05 referer_log

-rw-r--r-- 1 root root 34414268 Nov 1 07:30 referer_log.1

 

as i can see it's rotating the logs alright but it's not looking right at all, hence the webalizer failure

 

the webalizer.conf file seems to be looking at one file only so how do i get it to work with rotated log files

 

# LogFile defines the web server log file to use. If not specified

# here or on on the command line, input will default to STDIN. If

# the log filename ends in '.gz' (ie: a gzip compressed file), it will

# be decompressed on the fly as it is being read.

 

LogFile /usr/local/apache/logs/access_log

any ideas ?

 

cheers

anyweb

Link to post
Share on other sites
# LogFile defines the web server log file to use. If not specified

# here or on on the command line, input will default to STDIN. If

# the log filename ends in '.gz' (ie: a gzip compressed file), it will

# be decompressed on the fly as it is being read.

LogFile /usr/local/apache/logs/access_log

 

Well attempting to specify it in the file will plainly fail, however it gives us a clue of a way around it:

 

cat /usr/local/apache/logs/access_log* | webalizer

 

Using stdin we can simply pipe the information into webalizer. Of course if the logs get compressed then we would have a slightly different command.

 

Anyway that should capture all the logs and rotated logs, I guess that order is unimportant as the log file actually has that information on every line.

Link to post
Share on other sites
Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...