16 Replies Latest reply on Mar 20, 2019 1:34 PM by Rick Davin

    Are there SDK calls that can retrieve the Archive Properties?


      Are there SDK calls that can get to any of this information?


      Arc property.jpg

        • Re: Are there SDK calls that can retrieve the Archive Properties?
          Rick Davin

          No.  Those properties are only available to a PI Admin or someone with appropriate level of security.


          When I was a customer, I dabbled with a half-baked idea of using the .NET System.Diagnostics.Process to run issue an arlist call behind-the-scenes, and quietly parse the captured console input to variables to show to end-users.  Again, the security stopped me dead in my tracks.  As regular users could not see this privileged info, I abandoned the project.  That was a few years ago.


          Recently, I tried to revive this idea by once again using the Process class, but this time instead of calling arlist to call the PowerShell cmdlets instead.  Once again, the problem was security on the archive files.  If I was a PI Admin, I can see them.  If I am not a PI Admin, I can't.  So yet again I abandoned the project.

          2 of 2 people found this helpful
          • Re: Are there SDK calls that can retrieve the Archive Properties?

            The idea behind the SDK call would be to be able to calculate the amount of storage space per minute/hour/day generated by a system.

            Currently, 'Primary Archive % used' and 'Time to Archive Shift' performance counters are available but that is not enough to do the calculation(s), because it is missing the size of the arc-file. Technically if it is a set size it would not be an issue but if the archives are set by time to shift (like every week) then it is variable. Also getting the data through the PI Perfmon means it need to run on every system (Calculations for a fleet of servers).

            So the idea is to make an SDK call and get the 'size' of the primary archive, the '% used' and the 'current duration' of the archive.

            To generate the 'size' you need 'current duration' devided by '% used' times 100.

            2000 MB needs 75 hours / 38 % * 100 = 197.4 Hours or 10.13 MB per hour.

            If I can make this call per server in the fleet, I simply have to add each result to know how much data is created per hour over the entire fleet at the moment of the call.

            Why I would want to know this number ... I can give you many reasons but without it... it is just another dream..

            Maybe someone will get the inception ...

              • Re: Are there SDK calls that can retrieve the Archive Properties?
                Rick Davin

                If you are the intended audience for your custom app, then PowerShell cmdlets may have some of that information.  Or you would be able to use your privileged access to run on the PI Data Archive itself, where the OS would have access to the file system.

                • Re: Are there SDK calls that can retrieve the Archive Properties?
                  Eugene Lee

                  The Powershell cmdlet Get-PIArchiveFileInfo might have just what you need.


                  2 of 2 people found this helpful
                    • Re: Are there SDK calls that can retrieve the Archive Properties?
                      Rick Davin

                      UPDATE: What follows below is one way, but slightly roundabout.  There's got to be a better way.  Rather than dumping a single PS command to the console window, which forces everything to string, you should be able to capture the object the Get-PIArchiveFileInfo cmdlet sends.  Somewhere inside that cmdlet is the actual object type (UInt32, bool, etc.). 


                      Following up on Eugene Lee 's post, here's a Proof-of-Concept code snippet to show how to invoke the PowerShell cmdlet in a C# application.  As the code comments state, there are a LOT OF ASSUMPTIONS to make this code work.  The output will be a List in archive index order.  So this reads all archives, with each one being a separate list item.  The contents of each list item is a case-insensitive dictionary where the Key is the trimmed string to the left of the first colon, and the Value is the trimmed text to the right of the first colon.  Note that date & time strings will contain additional colons.  Remember we are parsing console output, and that output is TEXT STRINGS.  If you want to parse into type-specific objects, that exercise is left to you.



                      // The index of the list corresponds to the archive index.
                      public static IList<IDictionary<string, string>> GetPIArchiveFileInfo(string piDataArchiveName)
                          // This assumes A LOT and is offered as Proof-of-Concept and/or Learning Example.
                          //    * OSIsoft PowerShell cmdlet's have been loaded 
                          //    * powershell.exe resides on the PATH
                          //    * executing user has PIAdmin permissions
                          // It is left to the reader to test and debug further and to add better error checking.
                          // It is also left to the reader to assign type-specific values, e.g. string "False"
                          // to a Boolean false.
                          string script = $"Get-PIArchiveFileInfo -Connection (Connect-PIDataArchive -PIDataArchiveMachineName {piDataArchiveName})";
                          string[] lines = null;
                          using (Process process = new Process())
                              process.StartInfo = new ProcessStartInfo()
                                  UseShellExecute = false,
                                  RedirectStandardOutput = true,
                                  WindowStyle = ProcessWindowStyle.Hidden,
                                  CreateNoWindow = false,
                                  FileName = "powershell.exe",
                                  Arguments = " /C " + script
                              lines = process.StandardOutput
                                              .Split(new string[] { Environment.NewLine }, StringSplitOptions.None);
                          List<IDictionary<string, string>> list = new List<IDictionary<string, string>>();
                          Dictionary <string, string> dict = new Dictionary<string, string>(StringComparer.InvariantCultureIgnoreCase);
                          foreach (string line in lines)
                              // Blank lines separate archive entries.
                              // Be sure to write previous dictionary to the output list.
                              if (string.IsNullOrWhiteSpace(line))
                                  if (dict.Count > 0) // flush to list
                                      dict = new Dictionary<string, string>(StringComparer.InvariantCultureIgnoreCase);
                              string[] tokens = line.Split(new char[] { ':' }, count: 2);
                              // Both the Key and Value are strings and need to be trimmed.
                              // It is left to reader to parse values into type-specific objects.
                              dict[tokens[0].Trim()] = (tokens.Length == 2) ? tokens[1].Trim() : null;
                          if (dict.Count > 0) // flush last archive dict to output list
                          return list;



                      Other than peeking at the output list in my Locals windows, I have not attempted, nor will I attempt, even a tiny bit of testing or debugging.  So USE AT YOUR OWN RISK.


                      If you do attempt to parse into type-specific objects, keep in mind a few cautions:

                      • What you think looks like an int (Int32) may very likely be a long (Int64) because some values can be greater than Int32.MaxValue.
                      • When parsing date and times, consider the DateTimeKind.  As is, it is a text string.  AFTime would parse that as Local time.  DateTime would parse it with Kind of Unspecified, which may not be what you want.


                      My recommendation is that you should perform most of the processing in a PowerShell script.  Not everything has to be a .NET executable.  It is possible to run the PS script as a scheduled task, if that's something you would need.

                      1 of 1 people found this helpful
                        • Re: Are there SDK calls that can retrieve the Archive Properties?
                          Rick Davin

                          What makes this difficult, and again why I consider my code gymnastics to be a half-baked idea, is because trying to sync up output from PowerShell inside a .NET application requires either (a) working from strings at the console window, or (b) even greater code gymnastics to perform interprocess communications (perhaps pipes or WCF).  All this is easier to deal with simply inside a PowerShell script.


                          With PowerShell, a curious person may check the return type from Get-PIArchiveFileInfo.  It's type is Object[], where each item in the object array is info for a different archive index.  If you then check the data type of each item, you discover it is PIArchiveFileInfo17, which is based upon OSIsoft.PI.Net.PIArchiveFileInfo16.  Note that we do not recommend you ever work directly with the OSIsoft.PI.Net.dll, which you may find difficult anyway since the methods are completely UNDOCUMENTED.


                          For example, if against our advice you attempted to reference the OSIsoft.PI.Net.dll into your C# project, and then tried to create an instance for PIArchiveFileInfo17, you will get a message that it does not take 0 arguments in a constructor.  Okay, then how many arguments does it take, and what type are they, and in what order?  Remember that UNDOCUMENTED blurb above.  Well, that class is undocumented so it essentially useless to you.


                          Since PowerShell already has the PIArchiveFileInfo17 class defined, if you did all your processing inside of PowerShell then you would be able to use the class with its type-specific properties.  That or you would have to meticulously duplicate that class in C# with the understanding that future releases of OSIsoft.PowerShell or AF SDK may include a change to class and cmdlet.

                          • Re: Are there SDK calls that can retrieve the Archive Properties?

                            Hi Rick,


                            Taking the Process class to invoke PowerShell cmdlets is an interesting approach. I admit that I only had read about using System.Management.Automation class for the purpose but your post made me curious trying it and - if you don't mind - I find it way easier. My same is way shorter than your - not only because of the missing disclaimer.


                            I am still a big fan of using the System.Diagnostics.Process class e.g. for interactions with piconfig.exe, pidiag.exe or DOS commands from within a .Net application.


                            using System;
                            using System.Management.Automation; // <== Install System.Management.Automation through NuGet
                            namespace GetPIArcInfoPS
                                class Program
                                    static void Main(string[] args)
                                        // Instantiate PowerShell
                                        PowerShell ps = PowerShell.Create();
                                        // Define a string with the script. Don't forget adding '\r\n' to each but the last line
                                        string psScript = "Import-Module -Name OSIsoft.PowerShell \r\n";  // <== This is required if you have not already imported OSIsoft.PowerShell module
                                        psScript += "$piDataArchiveConnection = Connect-PIDataArchive -PIDataArchiveMachineName piDataArchiveHostName \r\n";  // <== The PI Data Archive Connection
                                        psScript += "Get-PIArchiveFileInfo -Connection $piDataArchiveConnection";  // <== Get-PIArchiveFileInfo; see https://techsupport.osisoft.com/Documentation/PI-Powershell/PI-DataArchive/Get-PIArchiveFileInfo.html
                                        // Assign the script to our PowerShell object
                                        // Execute (Invoke method) and iterate through the resulting item(s)
                                        foreach (PSObject result in ps.Invoke())
                                            // We are interested in the Members of the PSObject
                                            // With this specific script, I have seen Members could be of type PSProperty or PSMethod
                                            foreach (var member in result.Members)
                                                // We are only interested in properties (object PSProperty)
                                                if (member.GetType() == typeof(PSProperty))
                                                    Console.WriteLine("{0,-30}: {1}", member.Name, member.Value);
                                        Console.Write("\r\nDone! Press any key to quit .. ");
                            2 of 2 people found this helpful
                              • Re: Are there SDK calls that can retrieve the Archive Properties?
                                Rick Davin

                                Gregor Beck  This is AWESOME!  Many, many thanks!


                                To others, IF you want to find a specific property name, you may always cast the PSObject to dynamic.  You could then use the property name directly.  As is the case with dynamic, Intellisense won't help, and you must be sure to spell the property name exactly in the correct casing.  Example code snippet based on Gregor's:


                                IList<PSObject> psArchives = ps.Invoke().ToList();
                                // Execute (Invoke method) and iterate through the resulting item(s) 
                                foreach (PSObject psArchive in psArchives)
                                    dynamic arc = psArchive;
                                    Console.WriteLine($"Index={arc.Index}, IsPrimary={arc.IsPrimary}");
                                    Console.WriteLine($"PercentFull={arc.PercentFull}, AddRatePerHour={arc.AddRatePerHour}");



                                SAMPLE CONSOLE OUTPUT


                                Index=0, IsPrimary=True

                                PercentFull=3, AddRatePerHour=6.573908

                                Path=C:\Program Files\PI\arc\archive_2019-02-27_08-33-00.arc


                                Index=1, IsPrimary=False

                                PercentFull=0, AddRatePerHour=0

                                Path=C:\Program Files\PI\arc\archive2.arc


                                Index=2, IsPrimary=False

                                PercentFull=0, AddRatePerHour=0

                                Path=C:\Program Files\PI\arc\archive3.arc


                                Index=3, IsPrimary=False

                                PercentFull=0, AddRatePerHour=0

                                Path=C:\Program Files\PI\arc\piarch.001


                                Done! Press any key to quit ..

                                  • Re: Are there SDK calls that can retrieve the Archive Properties?
                                    Rick Davin

                                    I said there must be a better way and Gregor Beck came through.  I mentioned the OSIsoft.PI.Net.dll library earlier.  It's UNDOCUMENTED and UNSUPPORTED.  We advise others not to use it directly, but they do use it indirectly through things like OSIsoft.PowerShell.  The data type from PowerShell's Get-PIArchiveFileInfo cmdlet is OSIsoft.PI.Net.PIArchiveFileInfo17, and I previously mentioned you cannot construct an instance, which would typically render that library useless for you.  But the PowerShell cmdlet has already created the instances for you.  All you would need to do is cast it from PSObject back to its base type.  And while we discourage you from using the OSIsoft.PI.Net library, I fail to see what it's perfectly okay to use a PSObject with an ImmediateBaseObject of PIArchiveFileInfo17, but so wrong to use PIArchiveFileInfo17 directly.


                                    Do note that if you want to do this, that it is once again UNDOCUMENTED and UNSUPPORTED, and it makes your code fragile to any changes within OSIsoft.PowerShell or OSIsoft.PI.Net.  If a version release to either includes a new PIArchiveFileInfo18 type, then your code breaks.  Consider yourself forewarned!


                                    That said, let's consider this small snippet:


                                    PSObject firstArchive = ps.Invoke().First();
                                    // Set breakpoint and view Locals window


                                    If you set a breakpoint in the debugger and check your Locals window, you should see something like:


                                    2019-03-19 17_49_05-PowerShell Process ArchiveInfo (Debugging) - Microsoft Visual Studio.png


                                    Which indicates that if we can do a cast to the immediate base object type that we can use Intellisense in our application.  Note if you were using PowerShell natively that you would be directly working with a PIArchiveFileInfo17 object and not a PSObject.


                                    To do that, you may need to add these usings in your class:


                                    using System.Management.Automation; // <== Install System.Management.Automation through NuGet 
                                    using OSIsoft.PI.Net;  // UNDOCUMENTED and UNSUPPORTED Library for data types exposed in PowerShell cmdlets
                                    using System.Reflection;


                                    To add a reference to OSIsoft.PI.Net in your Visual Studio project, try looking in this folder:




                                    and now the guts of a method could be:


                                       // Instantiate PowerShell  
                                        PowerShell ps = PowerShell.Create();
                                        // Define a string with the script. Don't forget adding Environment.NewLine to each EXCEPT the last line  
                                        var script = new StringBuilder();
                                        script.AppendLine("Import-Module -Name OSIsoft.PowerShell");  // <== This is required if you have not already imported OSIsoft.PowerShell module  
                                        script.Append($"Get-PIArchiveFileInfo -Connection (Connect-PIDataArchive -PIDataArchiveMachineName '{piDataArchiveHostName}')");
                                        // Assign the script to our PowerShell object  
                                        // A lot of magic happens in this next line.
                                        // PowerShell is invoked and objects sent back to us as PSObject.
                                        // The immediate base object type is PIArchiveFileInfo17 from OSIsoft.PI.NET.dll.
                                        // But that library is UNDOCUMENTED and NOT SUPPORTED and you should never try to create
                                        // or construct objects directly from that library.  However, the objects were created already
                                        // by PowerShell, so all we care about is using the data type in that library.
                                        IList<PIArchiveFileInfo17> archives = ps.Invoke().Select(x => x.ImmediateBaseObject).Cast<PIArchiveFileInfo17>().ToList();
                                        // You may now use specific property names and Intellisense works!  
                                        foreach (PIArchiveFileInfo17 archive in archives)
                                            Console.WriteLine($"Index={archive.Index}, IsPrimary={archive.IsPrimary}");
                                            Console.WriteLine($"PercentFull={archive.PercentFull}, AddRatePerHour={archive.AddRatePerHour}");
                                        // and you can always use reflection to list over public properties
                                        Console.WriteLine("List properties of Primary Archive");
                                        PIArchiveFileInfo17 primaryArchive = archives[0];
                                        foreach (var prop in primaryArchive.GetType().GetProperties())
                                            Console.WriteLine($"{prop.Name} = {prop.GetValue(primaryArchive, null)}");
                                        Console.WriteLine("Done! Press ENTER key to quit .. ");



                                    By the way, the properties come back in a given order.  Here is the a default for me for the first 10 properties:


                                    List properties of Primary Archive

                                    IsCorrupt = False

                                    ArchiveSet = 0

                                    TotalEvents = 636719

                                    PrimaryIndexCount = 36

                                    OverflowIndexCount = 0

                                    OverlappingPrimaryMax = 0

                                    PercentFull = 3.1

                                    AverageEventsPerRecordCount = 198.6023

                                    LastModifiedTime = 3/19/2019 8:27:02 PM

                                    AddRatePerHour = 6.598092



                                    If you want the properties sorted alphabetically by property name, just toss in your own sort order:


                                        foreach (var prop in primaryArchive.GetType().GetProperties().OrderBy(p => p.Name))



                                    Here's the first 10 properties sorted by name:


                                    List properties of Primary Archive

                                    AddRatePerHour = 6.598092

                                    AnnotationFileSize = 2064

                                    AnnotationMax = 67108864

                                    AnnotationsUsed = 1

                                    AnnotationUid = 721f0289-a9bb-4dfa-8071-abd124a6e6d2

                                    ArchiveSet = 0

                                    AverageEventsPerRecordCount = 198.6023

                                    EndTime =

                                    FreeOverflowRecords = 99229

                                    FreePrimaryRecords = 42

                                    2 of 2 people found this helpful
                                      • Re: Are there SDK calls that can retrieve the Archive Properties?
                                        Rick Davin

                                        I open a side topic of showing the property name, value, AND type.  This builds upon code in my previous reply. The tips shared here apply to .NET in general and not specifically to working with PI archive files.


                                        Some interesting things to keep in mind about types.  The property has its own type but the value could be a different type!  Sounds crazy when you first hear it, but a great example helps illustrate why:  the PropertyType could be a Nullable<DateTime> and the value's type would simply be DateTime.


                                        Also, when a value type is DateTime, I want a little extra shown - I want to see it's Kind to provide context to the displayed date and time string. Either that or I could customize the displayed value to show ISO 8601 formatting.


                                        Since I am displaying to a console window, I will use 2 different types of delimiters ("::" and "==") for my pattern:  


                                        Property Name :: Property Type == Value


                                        But if I were writing to a text file, perhaps a CSV file, I would use the tab character as a delimiter.


                                        Lastly, I want to customize the type name.  That is, I do not like type.ToString() because it is too long, nor do I like type.Name because it is sometimes too short!  Examples:


                                        For ToString"System.Double", I want simply "Double".  That's simply using the Name instead of the ToString().  So what's the big deal?  The deal is Nullable<valueTypes>, in particular for this example is Nullable<DateTime>.


                                        FullName:     "System.Nullable`1[[System.DateTime, mscorlib, Version=, Culture=neutral, PublicKeyToken=b77a5c561934e089]]" (WAY TOO LONG)

                                        Name:     "Nullable`1" (MISSING CRITICAL INFO)

                                        ToString():     "System.Nullable`1[System.DateTime]"  (TOO LONG BUT BETTER)

                                        Desired Output:     Nullable[DateTime]


                                        Code to customize the type name

                                        private static string CustomTypeName(Type type)
                                            // type.ToString() is sometimes too long, as in 'System.Double'.  I prefer 'Double'.
                                            // type.Name is sometimes too short, as in 'Nullable`1'.  I prefer 'Nullable[DateTime]'.
                                            // Goal:
                                            // Customize "System.Double" to return "Double"
                                            // Cutomzie "System.Nullable`1[System.DateTime]" to return "Nullable[DateTime]"
                                            string name = type.ToString();
                                            // Define custom replacement pairings here.
                                            var replacements = new Dictionary<string, string>();
                                            replacements.Add("System.", "");
                                            replacements.Add("Nullable`1", "Nullable");
                                            foreach (var entry in replacements)
                                                while (name.Contains(entry.Key))
                                                    name = name.Replace(entry.Key, entry.Value);
                                            return name;



                                        Code using reflection to list out the info using custom method above:

                                        Console.WriteLine("List of ORDERED Properties of Primary Archive");
                                        foreach (var prop in primaryArchive.GetType().GetProperties().OrderBy(p => p.Name))
                                            string extra = "";
                                            // Note the value type is not always exactly the same as the PropertyType!
                                            // Example: the PropertyType could be Nullable<DateTime> but the value type is stricly DateTime.
                                            object value = prop.GetValue(primaryArchive, null);
                                            if (value?.GetType() == typeof(DateTime))
                                                extra = $"(Kind={((DateTime)value).Kind})";
                                            Console.WriteLine($"{prop.Name} :: {CustomTypeName(prop.PropertyType)} == {value} {extra}");



                                        Console Output

                                        List of ORDERED Properties of Primary Archive


                                        AddRatePerHour :: Single == 6.622521

                                        AnnotationFileSize :: UInt32 == 2064

                                        AnnotationMax :: UInt32 == 67108864

                                        AnnotationsUsed :: UInt32 == 1

                                        AnnotationUid :: String == 721f0289-a9bb-4dfa-8071-abd124a6e6d2

                                        ArchiveSet :: UInt16 == 0

                                        AverageEventsPerRecordCount :: Single == 197.8879

                                        EndTime :: Nullable[DateTime] ==

                                        FreeOverflowRecords :: UInt32 == 99109

                                        FreePrimaryRecords :: UInt32 == 44

                                        Index :: UInt32 == 0

                                        IsCorrupt :: Boolean == False

                                        IsPrimary :: Boolean == True

                                        IsShiftable :: Boolean == True

                                        IsWriteable :: Boolean == True

                                        LastBackupTime :: Nullable[DateTime] ==

                                        LastModifiedTime :: Nullable[DateTime] == 3/20/2019 12:51:00 PM (Kind=Utc)

                                        MaxOverflowRecords :: UInt32 == 102399

                                        MaxPrimaryRecords :: UInt32 == 51200

                                        OverflowIndexCount :: Int32 == 12

                                        OverlappingPrimaryMax :: UInt32 == 0

                                        Path :: String == C:\Program Files\PI\arc\archive_2019-02-27_08-33-00.arc

                                        PercentFull :: Single == 3.2

                                        PrimaryIndexCount :: Int32 == 37

                                        RecordCount :: UInt32 == 102400

                                        RecordSize :: UInt32 == 1024

                                        StartTime :: Nullable[DateTime] == 2/27/2019 2:33:00 PM (Kind=Utc)

                                        State :: OSIsoft.PI.Net.PIArchiveFileInfoState == Mounted

                                        TotalEvents :: UInt64 == 658373

                                        Type :: OSIsoft.PI.Net.PIArchiveFileInfoType == Fixed

                                        Version :: UInt32 == 17


                                        Again, just one of many ways to do this general thing.

                              • Re: Are there SDK calls that can retrieve the Archive Properties?

                                Hello Francis,


                                If you intend asking an enhancement, please consider posting an enhancement request at uservoice.


                                The information you are looking for definitely belongs to the category PI System Management. For this reason, I believe a request to enhance information available through PI (Archive) Subsystem Performance Counters has higher chances than the ask to make the information available through AF SDK. By the way, one of the great things with Windows Performance Counters is that they also can be collected remotely if the user account used for the purpose is member of the local "Performance Monitor Users" group.


                                I am also a little uncertain if the way you approach the challenge is the right one. It may be nice to know the size of the primary archive but what the value of this information if a PI Data Archive has empty archives prepared to shift into or - which we hope will not be the case - is configured to overwrite the oldest historical archive at the next shift. What if someone decides to copy a larger amount of files to the drive hosting the archive files of a PI Data Archive node? Wouldn't it make more sense to monitor the available disk space (through Windows Performance Counters) and having PI Notifications set up to alert about the available disk space dropping below a certain threshold?


                                If you are looking for a programmatic approach to get detailed information about the primary archive, invoking the Get-PIArchiveFileInfo as suggested by Eugene Lee is likely the best option.