3 Replies Latest reply on Mar 7, 2016 7:27 PM by Mike Zboray

    Need AFSDK failover solution for data redundancy


      I'm developing a fail over data collection mechanism. There will be two collectors listening to the same data source and when one goes down other will come into play.

      For collecting data, I'm using bulk update like:


      _piserver.UpdateValues(afValues, AFUpdateOption.InsertNoCompression, _afBufferOption);


      I have tried all available AFUpdateOptions in both the collectors but still I get duplicate values. From my experiments so far, it seems that there is no way to handle it using AFSDK as both the collectors act differently to the PI system.

      I might need to add some kind of signaling between the collectors which would tell me when the a particular collector should start/stop.


      Please suggest


        • Re: Need AFSDK failover solution for data redundancy
          Eugene Lee

          Hi Anil,


          Theoretically speaking, NoReplace should work for you. You shouldn't be seeing duplicates with NoReplace. This is the description from the reference:


          Add the value to the archive only if no value exists at the same time. If a value already exists for that time, the passed value is ignored.


          Which version of AF SDK are you using? You might want to set small delay in the other collector so that both of them doesn't send at the same time in order for the collector that sends later to detect that there is already a value so that the passed value will be ignored. Methods such as Thread.Sleep might do the trick.

          1 of 1 people found this helpful
            • Re: Need AFSDK failover solution for data redundancy
              Mike Zboray

              One gotcha with NoReplace is that it doesn't work quite that way at the snapshot. Since there is nothing in the archive at that time the value is appended, but once the snapshot is archived you will have a duplicate value. Analysis Service had a similar problem when backfilling. To work around it, backfilling has to check that it is not writing data at the snapshot timestamp.

              1 of 1 people found this helpful
            • Re: Need AFSDK failover solution for data redundancy

              Hello Anil,


              You will need to implement a mechanism to know about the other interface state otherwise you may be "flooding" the network with unnecessary data.  Eugene's proposition about NoReplace would prevent duplicates, however you will still have unnecessary network traffic and also an additional load on the PI Data Archive server because it will need to check the same values twice.


              To give you ideas, please check how OSIsoft interfaces are working:

              UniInt Failover

              KB01346 - UniInt Phase II Failover Scenarios


              The topic is not as trivial as single-interface data collection.  So we typically recommend that if an OSIsoft standard interface exists for your data collection you should use it, this is the faster and most straightforward way to have your system up and running with failover.  Maybe this is not possible in your scenario but it is sometimes good to tell .


              Could tell us a bit more about what type of data you are collecting?  Maybe this will give us some more ideas.


              Looking forward to know more about your work!

              1 of 1 people found this helpful