34

We have a huge logfile being written by a vendor's application. Let's assume the vendor won't do anything that we ask. Is there any way of rotating that logfile? We're looking at about 300 MB an hour being written - I'd much rather chunk that into 10 MB pieces, and let anything older than a day or over 1000 files fall off a cliff.

(I know I know, possible duplicate of How do you rotate apache logs on windows without interrupting service?)

Aha - the Chomp log was dead, but searching for "chomp logrotate" brought me to its new site. I'll give it a try tomorrow and reply if I like it. I'd still like to hear about software anyone else is using that works for this.

Community
  • 1
mfinni
  • 35,711
  • 3
  • 50
  • 86
  • Is periodically restarting your service going to cause problems? Can the application send its output to a program, or a network socket? – Zoredache Feb 08 '12 at 23:35
  • It can log to stdout, or to a named file. Possibility to restart - possible, not certain yet. – mfinni Feb 09 '12 at 02:24
  • There are hundreds of scripts for log rotations. For example, search Google for "powershell log rotation". – Jim B Apr 04 '12 at 19:29
  • That one specifically handles old (stale) files that are no longer being written to, which is a fairly easy case. I was looking for something to handle a single file. – mfinni Apr 04 '12 at 20:33
  • Searching that forum for "logrotate" has 0 results. – mfinni Apr 04 '12 at 20:34
  • Why would you search for logrotate? Do a search for rotate log files in windows. The "normal" (eg the one I've seen the most frequently) method is forfiles.exe -p "C:\logsfolder" -s -m *.log -d -31 -c "cmd /c del @path". Typically however moving in use logfiles fails since the file is in use, so a script has to be written to also close the file handle. typically text logfiles don't get rotated that often since they compress down often to a 10:1 ratio – Jim B Apr 05 '12 at 13:41
  • Jim - I would look for the string "logrotate" because that's a Unix tool that has the functionality I'm looking for - rotating old entries out of a single file. The things you're describing are indeed simple to find, and specifically *not* what I'm looking for. – mfinni Apr 05 '12 at 13:43
  • @mfinni OK I thought you wanted to chop the log up into 10 meg pieces and dump any over 24 hours old or with a filecount greater then 1000. I didn;t realize you specifically wanted logrotate since you said you wanted equivalents – Jim B Apr 06 '12 at 01:04
  • Without something like logrotate, I don't know how to get that first step - the chopping of a single large logfile (being written to) into pieces. – mfinni Apr 06 '12 at 13:59

3 Answers3

14

I haven't used it yet, but LogRotateWin is a native implementation of logrotate for Windows that looks promising. At least it doesn't require Cygwin.

bmaupin
  • 306
  • 2
  • 13
  • 6
    I have used this on windows 2012 R2. Works perfectly. `choco install logrotate` then edit c:\program files (x86\LogRotate\Content\logrotate.conf – spuder May 18 '16 at 21:24
9

As much as I wince at the suggestion, installing Cygwin is one of the very few options that you have available to you. From there, you can use logrotate.

Wesley
  • 32,320
  • 9
  • 80
  • 116
  • 2
    Not scared of Cygwin if it includes a good solution, although I'm always mindful of not using a cannon to swat a fly. – mfinni Apr 04 '12 at 13:26
  • 1
    Sorry, gotta pull back my checkmark. I don't see logrotate in the catalog for the current Cygwin. It might build properly from source, but the MAKEFILE doesn't include anything for CYG or WIN. – mfinni Apr 04 '12 at 13:42
  • @mfinni Alas, it's not in the package, although others have used it in Cygwin. =/ – Wesley Apr 04 '12 at 15:32
  • Sounds good to me – mfinni Apr 04 '12 at 17:23
  • @WesleyDavid Assuming the process keeps the log file open for writing continuously, would a tool such as logrotate even work on Windows? – Evgeniy Berezovsky Sep 26 '12 at 04:44
  • @myself Just discovered [dwShareMode](http://msdn.microsoft.com/en-us/library/aa363858%28VS.85%29.aspx), so it might be possible after all, assuming the app that writes the log opens the file in the appropriate dwShareMode) – Evgeniy Berezovsky Sep 26 '12 at 05:16
  • 2
    I confirm the ability to move the log file of Cygwin's stock `syslog-ng`. – ivan_pozdeev Feb 20 '15 at 08:07
1

Try LogRotateWin

I've tried LogRotateWin over the last few days.

The projects seems somewhat abandoned and I've found a few bugs in just the little testing I've done. (E.g. some command line parameters that "--help" shows are not actually implemented. E.g. state tracking via logrotate.status statefile does not work for relative paths in logrotate.conf. Instead they ALWAYS get rotated. Not just when their time has come. So you can only use absolute paths.)

But the exe is just 40KB and it does have a few nice features. And I like that fact their mission statement is "The goal is to use the same command line parameters and files as the Linux version." I like it.

Usable example starter-batch and example conf file below.

Logging logrotate's output

Something of a meta-problem: how do you log the activities of logrotate itself? I wrapped the call to logrotate.exe in a batch file. It logs everything to a tempfile, then calls logrotate, then appends everything from the tempfile to a final logfile, then deletes that tempfile. And then that final logfile is then taken care of by logrotate.exe itself in the next run. I then run that starter-batch-file via Windows Task Scheduler.

Mega-dodgy but does the job.

Do-LogRotate.bat:

setlocal

REM Quick and dirty. Wild mixture of Batch and PowerShell. No error checking.

REM Change current directory to where our script is.
cd /D "%~dp0"

set "TEMPOUTPUTFILE=logrotate.temp-log"
set "FINALOUTPUTFILE=logrotate.log"

echo. 1>> %TEMPOUTPUTFILE% 2>&1
powershell -command "& {Write-Output((get-date -format o) + ' Logrotate script START.')}" 1>> %TEMPOUTPUTFILE% 2>&1
logrotate.exe -s logrotate.status logrotate.conf --verbose 1>> %TEMPOUTPUTFILE% 2>&1
powershell -command "& {Write-Output((get-date -format o) + ' Logrotate script END.')}" 1>> %TEMPOUTPUTFILE% 2>&1

REM Somewhat of a hack. I can't have logrotate handle its own logs on
REM Windows it seems.
REM -- I guess file options for shared access are not set in the right way when 
REM you use the ">>" redirect operator. Neither "move", nor "copytruncate" 
REM logrotate directives work.
REM So instead append the contents to the other outputfile. -- Not sure this 
REM this really works. But oh, well. The logrotate logs themselves are not 
REM really that important. So it's not too terrible if we lose a few of them.
TYPE %TEMPOUTPUTFILE% >> %FINALOUTPUTFILE%
DEL %TEMPOUTPUTFILE%

logrotate.conf:

C:\winscp\WinSCP.log {
    daily
    minsize 1M
    rotate 10
}

# Unfortunately relative paths don't work.
C:\dev\logrotatetests\logrotate.log {
    monthly
    rotate 5
}

Related reading: I've struggled with Apache Tomcat logs on Windows. This is what made me look into logrotatewin. https://stackoverflow.com/questions/19787279/where-to-configure-internal-tomcat7-stdout-stderr-log-files/54423803#54423803

StackzOfZtuff
  • 1,754
  • 12
  • 21