PowerShell to the rescue; solving the PI Server Archive Management shortcomings

Blog Post created by RJKSolutions on Jul 23, 2013

Yes the title is correct, despite the brilliant work that has gone into PI Server there are still numerous management issues. Some of those issues are related to the PI Server archives and I want to cover a couple of them in this blog post which follows on from my previous blog post on connecting to the PI System via PowerShell.


Archive Time Span

Let me start with a simple scenario that I wanted to track automatically; I wanted to continuously know what the complete lifespan of all on-line archives is. I am working with space limited PI Systems where there is a requirement to only have a small time span of data available - for example the last 12 months of data. To achieve this from the PI Server perspective you need to set the tuning parameter "Archive_OverwriteDataOnAutoShiftFailure" to "1" to allow old archive files to be overwritten. The main reason for that overwrite of data to occur is typically low disk space, which in my scenario is perfect (and it works fine).


However, what I cannot tell automatically via Performance Counters is what is the time span of all on-line archive. You can't even deduce that from the existing performance counters. So how am I supposed to monitor that my rolling archives are indeed keeping at least 12 months of archive data on-line? The answer is to use the PowerShell Tools for the PI System with a couple of lines of code, it is so simple that I encourage everyone to start exploring the PowerShell Tools for the PI System to find other "quick wins" for your PI System management.


At the moment I am dealing purely with a single PI Server and running the PowerShell scripts on the PI server - I will tackle PI Collective and Collective monitoring in my next blog post.


We need to connect to the PI server, which you should remember from my last blog post:

if ((Get-PSSnapin "OSIsoft.PowerShell" -ErrorAction SilentlyContinue) -eq $null) { Add-PSSnapin "OSIsoft.PowerShell" }

[OSIsoft.SMT.PIServer]$PI = Get-PIServer -Name "vCampusDemo" | Connect-PIServer



Okay, so now we are connected.
In order to retrieve all the registered archives from the connected PI server OSIsoft have kindly built us the "Get-PIArchive" CmdLet. Without specifying an archive name the CmdLet will return an Array of all the registered archives. 

[Array]$ARCHIVES = Get-PIArchive -PIServer $PI



Simple, right? 
However, I'm not interested in empty archives that currently have no data, and I want to order the archives in a chronological descending order so that I can take a short cut to get the time span of all archives. So I'm going to pipe the archive list to a "where" filter to eliminate the empty archives and then sort the remaining archives:

[Array]$ARCHIVES = Get-PIArchive -PIServer $PI | where { $_.StartTime -ne $null } | sort -Property StartTime -Descending



 A PI Server archive without a start time is an empty archive waiting to be used/shifted to. They've now gone, great. I also sorted the archive list because I want to use the first and last element of the array to provide a simple time span result of all archives. Before you start shouting at the screen, I know there could be archive gaps in between...I am coming to that next. For now here is the code:

$Now = [DateTime]::Now
$TimeSpan = New-TimeSpan -Seconds ($Now.Subtract($ARCHIVES.GetValue($ARCHIVES.Length - 1).StartTime).TotalSeconds)
Write-Host "(Simple) Total archive online timespan = " $TimeSpan.ToString()



Something to note with this is that the first Archive object in the array will be the primary archive, which has no end time. The end time is the current time so for simplicity I substitute the current time of the PI Server as that archive's end time. I then subtract the last Archive object's start time to get the time span (inclusive of archive gaps) of on-line archive data. Now with this time span I could do what I like with the value, most likely send to a PI tag (via the "Add-PIValue" CmdLet - subject to a later blog) and notify on its value.


As I alluded to earlier in this blog post I am not taking into account archive gaps. So the result I just got is not accurate enough for me, I want to know the time span of on-line archive and if there are archive gaps. This got me thinking as to the best way to achieve this with as little code as possible, after all I don't necessarily want a a "PI Professional" to maintain these PowerShell script, they should be as obvious as possible for others to maintain. I had to do some more research into some PowerShell CmdLets and better use of piping between CmdLets...some time later...I decided on how I would do it for now and was pleasantly surprised on how I ended up doing it.


Each PI Server archive has a LifeTime property, which is a TimeSpan object. So I could filter out empty archives and now select the LifeTime property of each archive. I then discovered the "Measure-Object" CmdLet that will provide you with statistics depending on what you want, and one of the statistics available is "Sum". Perfect, but you cannot sum TimeSpans. Instead I had to use the "ExpandProperty" parameter for the Select CmdLet so that I could sum up the "TotalSeconds" property of each TimeSpan. Now it is perfect.

$sum = Get-PIArchive -PIServer $PI | where { $_.StartTime -ne $null } | select -ExpandProperty LifeTime | Measure-Object TotalSeconds -Sum



Now I can compare the time span of Primary Archive -> Oldest Archive Time Span with the LifeTime TotalSeconds Time Span to detect if there are any archive gaps.

$ArchiveGaps = ((New-TimeSpan -Seconds $sum.Sum).ToString() -eq $TimeSpan.ToString()) 



Job done.


Archive names for rolling archives


Following on from the above scenario I found out to my horror that when the PI Server overwrites an old archive as per the tuning parameters configuration it doesn't rename the archive file! This means that the first time the archive file is created it is given a name based on the tuning parameters "Archive_AutoArchiveFileRoot" and "Archive_AutoArchiveFileFormat" then that archive name will remain forever no matter how many times the archive is overwritten. If like me you have some form of self-confessed OCD then the archive names not matching the configuration after an archive shift overwrite was keeping me up at night.  


Anyway, after already tackling the archive on-line data time span issue I had done a lot of the work for getting at the archives. What I need to do now was check the name of each archive and make sure it matched the tuning parameter configuration. The first check of the Powershell Tools for the PI System yielded the CmdLet I needed; Get-PITuningParameter. This was straightforward to get what I needed for my checks, having already connected to the PI Server:



$AutoArchiveFileRoot = (Get-PITuningParameter -Name "Archive_AutoArchiveFileRoot" -PIServer $PI).Value
$AutoArchiveFileFormat = (Get-PITuningParameter -Name "Archive_AutoArchiveFileFormat" -PIServer $PI).Value



The archive file format is one of three possibilities:


0: [root]_D_Mon_YYYY_H_M_S[.ext]
1: [root]_YYYY-MM-DD_HH-MM-SS[.ext]
2: [root]_UTCSECONDS[.ext]


Okay so the format is really just a date time format for the archive's start time, so we'll use the same logic for our check and, if required, rename.


Let's get the archives in a chronological order and assign what we think the name of the archive "should be":



$AutoArchiveFileExt = (Get-PITuningParameter -Name "Archive_AutoArchiveFileExt" -PIServer $PI).Value
$AutoArchiveFileRoot = (Get-PITuningParameter -Name "Archive_AutoArchiveFileRoot" -PIServer $PI).Value
$AutoArchiveFileFormat = (Get-PITuningParameter -Name "Archive_AutoArchiveFileFormat" -PIServer $PI).Value

[Array]$ARCHIVES = Get-PIArchive -PIServer $PI | where { $_.StartTime -ne $null } | sort -Property StartTime -Descending
foreach ($ARCHIVE in $ARCHIVES)
     $ARCHIVE_NAME = ""
     switch ($AutoArchiveFileFormat)
          0 {$ARCHIVE_NAME = '{0:d_Mon_yyyy_H_M_s}' -f $ARCHIVE.StartTime}
          1 {$ARCHIVE_NAME = '{0:yyyy-MM-dd_HH-mm-ss}' -f $ARCHIVE.StartTime}
          2 {$ARCHIVE_NAME = $ARCHIVE.StartTime.ToFileTimeUtc()}
     $ARCHIVE_NAME = [String]::Concat($AutoArchiveFileRoot, $ARCHIVE_NAME)


 We can simply format the Archive StartTime property to the required archive name format defined in the tuning parameters, then join that with the file root tuning parameter.


Next the comparison...



if ($ARCHIVE.Name -eq ($ARCHIVE_NAME))
          Write-Host "Archive name '" $ARCHIVE.Name "' is correct. No action."
          Write-Host "Archive name '" $ARCHIVE.Name "' is incorrect, should be '$ARCHIVE_NAME'."



You can use this as a sanity check that the logic for comparison is accurate. On "normal" usage of the PI Server you'll likely have all archives named correctly, unless you changed the tuning parameters after the first archive was created. For my scenario there was no telling how many of the archives would have been rolled over so I had to do this check periodically based on my data rates to the PI Server.


What do I do now that I know there are archives named incorrectly? Well unregister them and rename them of course. Couple more interesting CmdLets that OSIsoft have provided to us, "Unregister-PIArchive" and "Register-PIArchive". This is becoming easier than I first thought.


Okay then, I am going to unregister each non-conforming archive, rename it, then register it as the new name. However, I am not going to do anything with the primary archive...I'll wait for the next archive shift before renaming that one. I can live with having the primary archive named incorrectly (although OSIsoft should fix this whole issue in PI Server 2013).



          if ($ARCHIVE.EndTime -ne $null)
               Unregister-PIArchive -Name $ARCHIVE.Name -PIServer $PI 
               Move-Item $ARCHIVE.Name $ARCHIVE_NAME
               Move-Item ($ARCHIVE.Name + ".ann") ($ARCHIVE_NAME + ".ann")
               Register-PIArchive -Name ($ARCHIVE_NAME) -PIServer $PI 
               Write-Host "Primary archive '" $ARCHIVE.Name "' will not be renamed until after the next archive shift."



Yep, it really is that easy. All those sleepless nights wiped out in a few lines of code. You could have all kinds of fun with these OSIsoft CmdLets.


Job done.






For a couple of issues that historically have been complicated to achieve using regular command file scripts they have been solved with very little PowerShell code. Big thanks to OSIsoft for providing the CmdLets to simplify these real-world issues with very little effort.


Obviously any system management automation carries potential for unexpected exceptions, irregularities in PI Server setup/configuration ... so you should have some detailed knowledge of your PI System before going in with all guns blazing. I've omitted any detailed exception handling for simplicity of this blog post - make sure you handle exceptions and check your environment first!




What's next?


I want to look at some PI Module Database management for PI Interfaces. Whilst we don't have AF based PI Interfaces some of us still have to get our hands dirty with the PI Module Database. I didn't want to get my hands dirty, I like my hands, so I automated a whole bunch of PI Interface & PI Module Database work. This is coming up next.