IIS7 logging in Central Location for web farms
Ok so it appears IIS7 has thought of almost everything this time, and IIS7 is perfect for web Farms now they have the ability to share the configuration file.
While we are in the mists of setting up a load balanced environment with a IIS7 farm, we have everything working wonders. But when it comes to log files it seems Microsoft missed this. they really should have added in the functionality to combine log files to a central location, while we can prove a UNC path and also log to this, but only one server can do this.
the problems that you are faced with in a web farm is that:
If you have shared you ApplicationHost.config which in a web farm, (I highly recommend you do to make your life that little be easier) you will find that y our log file location will be the same on all servers. which could be a problem if:
you have used a UNC path, all servers in the farm will try and log to this location and only one can win, as the HTTP.SYS for that winning server will keep this open and therefore other servers will not be able to log.
If you have specified a local path, which really lest face it is your only option, you MUST make 100% sure that location or that drive exists on all of your servers in the web farm.
Now if you don’t care so much for your log files you will be happy to have them on each server and not collate them. But for the people like us we need them in a central location for a: We like to find our data quickly and be able to find what is going on, on our servers b; we have webstats that we need to be able to provide to our clients or internal staff and c; for a backup / management purpose its just easier to have them centralised.
So our requirement is to be able to copy all of our log files to one location, keeping of course the web log file structure i.e. W3SVC1, W3SVC2 etc…
there is one other problem I should tell you, all of the log file names are the same name, so we can just schedule a move / copy script because they will fail or overwrite the other log files, you MUST rename the files before moving. In hind sight we have do this but its a good idea anyway, because this will allow us to easily track down a problem on a server via the logs.
So we have come up with a script that can be scheduled on a daily basis that will rename all files to the server name_<orginalfilename>.log (this will look in all sub folders i.e. W3SVC1, W3SVC2 etc…) once the name change is done it will then MOVE the files to the central location.
A Big thanks to Alan Lee for this script, as he wrote the find and replace part (hats off)
set logs=C:\folder\folderagain
cd /d %logs%
dir /b %logs% > tmp.txt
for /D %%I in ("%logs%") do For /F "tokens=*" %%J in (tmp.txt) do cd /d %logs%\%%J && for %%i in (u*.log) do move %%i %computername%_%%i
del %logs%\tmp.txt
robocopy %logs% \\servername\FolderName /mov /E /minage:0 /R:3 /W:5 /LOG:%logs%\logs.log
Of course the script uses robocopy so you will need to include the .exe for this where ever you are executing this .bat from.
With 2008 Server you can also schedule emails with attachments, might not be a bad idea to attach the log file so you can archive it away.
January 15th, 2009 at 3:08 pm
Shouldn’t this be using PowerShell?
January 16th, 2009 at 9:47 am
will be converted soon
April 9th, 2009 at 10:53 pm
Reading this I had the idea that you could map a network drive say Z:\ to point to different subdirectories (\server1,\server2,\server3) in the UNC folder you want to merge. Not 100% sure this would work, but to be honest I like your solution better because if for some reason the UNC path can’t be reached at some point, log entries won’t go missing.
May 29th, 2009 at 12:51 am
Excellent post, keep it up
January 25th, 2010 at 4:26 am
thanks for share
April 23rd, 2010 at 2:30 pm
I Will have to come back again when my class load lets up – nonetheless I am taking your RSS feed so I can read your site offline. Thanks.
May 17th, 2010 at 3:10 am
this like… thanks for share
June 1st, 2010 at 11:12 pm
Logging to
\\server.domain.com\sharename\%COMPUTERNAME%\
works great in a shared configuration
September 23rd, 2010 at 2:14 am
Me read http://bradmarsh.net/index.php/2009/01/15/iis7-logging-in-central-location-for-web-farms everyday, good information and practice for English. Thank You as is pleasure read your site.
November 9th, 2010 at 9:43 am
Thought provoking info!
December 13th, 2010 at 6:32 am
I got what you intend, saved to fav, very decent site.
April 28th, 2011 at 1:44 am
Thank you for this article….
January 8th, 2022 at 11:31 am
2verified…
…