4 Replies Latest reply on Aug 3, 2016 11:40 PM by shillman

    Writing code to read all events in all PI points of a Data Archive

    DSmith1

      One issue we have when reprocessing archives is we're not sure if it has deleted data or PI points from our PI system after the reprocessing is finished.

       

      Right now we've decided to write an AFSDK program that reads all of the events for every PI tag in a given archive. My gut feeling makes me concerned about writing something like that, but can someone help me with perhaps a better way to check the creation/deletion of PI tags or PIpoint data after the completion of reprocessing archives?

        • Re: Writing code to read all events in all PI points of a Data Archive
          gregor

          Hello Daniel,

           

          You cannot read data for a deleted PI Point. The attempt to do so returns error [-11147] Point ID not found in archive

          Please see 2785OSI8 - How to recover deleted points, or how to restore deleted tags in PI Data Archive for additional information.

          Please also note that deleting a PI Point usually causes all related records being wiped out from the current primary archive. The reprocessing of a historical archive should cause records being cleared for deleted PI Points.

          If this is a concern, you may want to hide PI Points through their security settings instead of deleting them. This means changing Point Security and Data Security in a way to make points invisible to normal users. Administrative users would still be able to look at these points. To avoid interfaces still attempt to pick up those PI Points, you may want to add "_delete" to the Point Source attribute. This will also make it a lot easier to query for "deleted" points. The big advantage however is that for the PI Data Archive these PI Points remain intact but for sure will as well count against the licensed amount of Data Streams.

            • Re: Writing code to read all events in all PI points of a Data Archive
              DSmith1

              Gregor what do you think about the idea of reading all of the PI points in the archive.


              Basically if the PI point was NOT deleted, it would count all of the events before reprocessing, and then count all of them for every archive afterwards to see if any events were lost / added

                • Re: Writing code to read all events in all PI points of a Data Archive
                  lmurphy

                  Daniel,

                   

                  If you want to check the total events in an archive, and which record and point id combinations those events correspond to, you can do an archive check by running the following command from %PISERVER%\ADM

                   

                  pidiag -archk <archive path>

                   

                  This will give you how many events are in that archive, associated with each tag. At the end of the output, it will list the total events in the archive. You could run the archive check before and after reprocessing. Keep in mind, the archive has to be unregistered to run this command.

                  3 of 3 people found this helpful
              • Re: Writing code to read all events in all PI points of a Data Archive
                shillman

                It seems a safer solution would be to make sure the data does not live in a single place. For example, using a collective or using PI to PI to replicate the data. The chances of multiple things getting corrupted are very low. And, the more places replicated, the lower the chance. Then you can always compare the data between the different PI Data Archives in case anything was lost - And if data was lost, you can recover it since it is safely archived elsewhere.

                 

                In order to accurately track what happens following reprocessing... You would need to constantly look at all the data in an archive. You need to constantly check the archive while it is healthy because, if something gets corrupted and you have to reprocess, you might not be able to get accurate event count etc. from that now-corrupt archive.

                 

                I really don't see any purpose to writing an application to read all the data for all the points for a particular system, especially if you are going to store that for future reference. This would simply be duplicating your PI Data Archive. As I point out above, this functionality already exists - PI to PI Interface and PI Collectives.

                1 of 1 people found this helpful