Lets face it, there’s nothing really exciting about log file management. But like it or not its a pretty common task for DevOps to deal with log files. We’d all like our applications to use a Logging Service like Splunk or Loggly, but until that time comes we need to deal with log files on servers. I developed this script based off the following requirements:
– Log files older than a certain number of days need to be removed from servers.
– Log files need to be archived for a certain number of days before being removed from the servers.
– Archived log files need to be compressed to minimize storage use.
– Archived log files need to be removed after a certain number of days.
– An email notification needs to be sent to administrators with an activity report.
For this script I made use of the Simple Powershell Central Config that I’ve written about before, take a look at that article for how it works. I call the Get-EnvServers function from that script to get the servers to target. I’m also using a helper script that performs zipping of log files, see further down for the details of that script.
The “Script Configuration” Section contains variables that contain the number of days to keep for both log files and log archives. Mail server names, file paths, and a log file name filter are also defined here.
The primary logic of the script searches for files with a LastWriteTime less than the date we set in the config section. If the file name matches the name in the filter, the file gets added to the zip file and also an array of files to be deleted. The zip file is then saved and the files to be deleted are then deleted.
An array of objects is used to track the log files that are archived. The ConvertTo-HTML cmdlet is used later to format the array as an HTML table. I’m then manipulating the HTML string to place CSS IDs on alternating rows. I did this to allow the CSS to function in Outlook and other email clients. The Send-MailMessage cmdlet is then used to email out the message.
Powershell Log Archival Script
param (
[Parameter(Mandatory=$true)]
[String]$EnvTarget
)
Write-Host " "
Write-Host " " -BackgroundColor DarkCyan
Write-Host " Log Archive Script " -BackgroundColor DarkCyan
Write-Host " Environment: $EnvTarget " -BackgroundColor DarkCyan
Write-Host " "
### Include Scripts ###
Write-Host "Loading Included Scripts"
if (!(Test-Path "\\atl0fs01\platform\scripts\include-configuration.ps1")){Write-Error "Include Script Not Found: \\atl0fs01\platform\scripts\include-configuration.ps1"; exit}
. "\\atl0fs01\platform\scripts\include-configuration.ps1"
if (!(Test-Path "\\atl0fs01\platform\scripts\Include-DotNetZip-Helper.ps1")){Write-Error "Include Script Not Found: \\atl0fs01\platform\scripts\Include-DotNetZip-Helper.ps1"; exit}
. "\\atl0fs01\platform\scripts\Include-DotNetZip-Helper.ps1"
##########################
# Script Configuration
$ArchivePath = "\\atl0fs01\LogArchive"
$daysToKeep = -2
$daysToKeepArchives = -45
$smtpServer = "mail.tekmortar.com"
$LogPath = "Logs"
$LogNameFilter = "My-App-Log"
# Script Working Variables
$DeleteFiles = @()
$TargetServers = Get-EnvServers $EnvTarget
$counter = 0
$date = Get-Date
$date = $date.AddDays($daysToKeep)
$arrayLogs = @()
$arrayZips = @()
$xcounter = "1"
$zipfile = new-object Ionic.Zip.ZipFile
Write-Host " "
foreach ($s in $TargetServers)
{
$p = "\\$s\e$"
$p = Join-Path $p $LogPath
Write-Host "Adding $p"
$logs = gci -Path $p | Select-Object Name, FullName, LastWriteTime | Where-Object { $_.LastWriteTime -lt $date }
if ($logs){
foreach ($l in $logs){
$lpath = $l.FullName
$lname = $l.Name
$lwrite = $l.LastWriteTime
if ($lname.Contains($LogNameFilter))
{
Write-Host "Archiving $lname - $lwrite"
$e = $zipfile.AddFile($lpath)
$counter += 1
$DeleteFiles += $lpath
$obj = New-Object psobject -Property @{
X = $xcounter
File = $lname
TimeStamp = $lwrite
Server = $s
}
if ($xcounter -eq "2"){
$xcounter = "1"
}elseif($xcounter -eq "1"){
$xcounter = "2"
}
$arrayLogs += $obj
}
}
}
Write-Host " "
}
if ($counter -gt 0){
$datestr = Get-Date -format M-d-yyyy-hh-mm-ss
$ArchiveFile = $ArchivePath + "\Logs.$EnvTarget.$datestr.zip"
$zipfile.Save($ArchiveFile)
$zipfile.Dispose()
foreach ($f in $DeleteFiles)
{
Remove-Item $f -Force
Write-Host "Deleted $f" -ForegroundColor Yellow
}
}
Write-Host " "
# Purge Zip Files from the Log Archive
Write-Host "Purging Old Archive Files from $ArchivePath"
$date = Get-Date
$date = $date.AddDays($daysToKeep)
$archives = gci -Path $ArchivePath | Select-Object Name, FullName, LastWriteTime | Where-Object { $_.LastWriteTime -lt $date }
if ($archives){
foreach ($arch in $archives){
$lpath = $arch.FullName
$lname = $arch.Name
$lwrite = $arch.LastWriteTime
$obj = New-Object psobject -Property @{
File = $lname
TimeStamp = $lwrite
Server = "atl0fs01"
}
$arrayZips += $obj
Write-Host "Purging $lname - $lwrite"
Remove-Item $lpath
}
}
Write-Host " "
Write-Host "Sending Email"
$a = "<style>"
$a = $a + "BODY{background-color:white;}"
$a = $a + "TABLE{border-width: 1px;border-style: solid;border-color: #2c3e50;border-collapse: collapse;}"
$a = $a + "TH{border-width: 1px;padding: 5px;border-style: solid;background-color:#34495e;color: #ecf0f1;}"
$a = $a + "TD{border-width: 1px;padding: 5px;border-style: solid;}"
$a = $a + ".EVEN{background-color: #2980b9}"
$a = $a + ".ODD{background-color: #ecf0f1}"
$a = $a + "</style>"
$body = "<b>Performance Log Archive Report</b><p>The following Log Files were archived to $ArchivePath and removed from the Application Server.</p>"
$errstring = $arrayLogs | Select-Object X, Server, File, TimeStamp | ConvertTo-HTML -Head $a | Out-String
$errstring = $errstring.Replace("<tr><td>1</td>", "<tr class='ODD'>")
$errstring = $errstring.Replace("<tr><td>2</td>", "<tr class='EVEN'>")
$errstring = $errstring.Replace("<th>X</th>", "")
$body = $body + $errstring
#Sending email
send-MailMessage -SmtpServer $smtpServer -to "notifications@tekmortar.com" -From "alerts@tekmortar.com" -Subject "Performance Log Archive Report" -Body $body -BodyAsHtml -Priority high
Zip Helper Script
The following script is included to perform file compression of the log files. It uses Ionic.Zip to perform file compression. The script copies the dll locally if it doesn’t exist and then loads the .NET Assembly into memory for use.
$LocalDllPath = "C:\Temp\Ionic.Zip.dll"
$DllPath = "\\atl0fs01\platform\Base\Shared\Ionic.Zip.dll"
### Load SharpSVN ###
if (!(Test-Path $DllPath)){
Write-Error "Could not locate $DllPath. Check if present and rerun"
exit
}
if (!(Test-Path $LocalDllPath)){
Copy-Item $DllPath $LocalDllPath -Force
}
$currentScriptDirectory = Get-Location
[System.IO.Directory]::SetCurrentDirectory($currentScriptDirectory)
$a = [Reflection.Assembly]::LoadFile($LocalDllPath)
