9 Replies Latest reply on Nov 12, 2015 10:09 AM by pthivierge

    A way to uniquely identify AFValue with the same datetime values

    ChewCheeLim

      Hi, there, I was able to insert 2 identical time series data (with the same timestamp) into PI.

      (before compression happened)

      Turn around, I queries these values. During this 'bogus' time, for the given time stamps:-

      2 rows returned.

       

      Then I called this method:-  (thinking that the timestamp has to be unique in this case since there is no unique key for AFValue Object)

       

      AFValue retrieve = afAttribute.Data.RecordedValue(afTime, AFRetrievalMode.Exact, null); -- which row to returned? not sure.

       

      then I updated a value to 20  --> the Yellow down arrow key = 'Substituted'

       

       

      Will PI keeping 2 values in PI Archive? if not, which one to throw away?

        • Re: A way to uniquely identify AFValue with the same datetime values
          bshang

          Both values should be kept but one will be Substituted as you saw. Which one exactly gets substituted will depend on the order of insertion into the PI Data Archive and is an implementation detail that the client code should not rely on.

           

          I believe you can play with AFRetrievalMode.AtOrBefore and AtOrAfter, passing in the target timestamp. I believe they should return different AFValues, but the actual values I believe still depends on the history on insertions/deletions from the PI Data Archive.

           

          Why is this behavior important to your application?

            • Re: A way to uniquely identify AFValue with the same datetime values
              ChewCheeLim

              Thanks Barry,

              When I inserted both values with the same timestamp and value. (none of the values were marked as substituted at that point)   Without realizing a dup, then subsequently submitted an Update command with below sdk call & changed one value. Pls note: I need a exact match on the time stamp since I am doing an data fixing on the archive data.

               

               

              retrieve = afAttribute.Data.RecordedValue(afTime, AFRetrievalMode.Exact, null);

               

               

              As you can see, I want to remove one time series & keeping one -->  either 20 or 991 from my delete. If I understand your answer correctly,  I have no way to do so from AF SDK.

              And also, what would happen in this case if compression logic ran in PI? both are kept? or one being thrown away? if so, which one will be kept?

            • Re: A way to uniquely identify AFValue with the same datetime values
              pthivierge

              Hello Chew Lee,

               

              In addition to what Barry has already said, if it is your application that is inserting data into the PI System, you may use the option  AFUpdateOption enumeration to indicate what behavior you want when having to deal with duplicated time stamps.

               

              AFUpdateOption Enumeration details

               

               

                • Re: A way to uniquely identify AFValue with the same datetime values
                  ChewCheeLim

                  Thanks Patrice,

                   

                  Since I am doing archive data fixing, hence I selected this option:- ( I don't want to create any new data but fixing exactly what I found in archive only)

                   

                   

                  <AFValue> afUpdateResult = afAttribute.Data.UpdateValues(afUpdateList, AFUpdateOption.ReplaceOnly);

                   

                  if this is not the right one, do you mind to suggest?

                   

                    • Re: A way to uniquely identify AFValue with the same datetime values
                      bshang

                      Are you trying to replace an event at a timestamp where two events exist in the archives?

                       

                      Also, in the above where the value at 6:00AM 20 is trying to be removed, what is in afUpdateList? Can you read in this value via RecordedValue and then pass it back in to UpdateValue?

                        • Re: A way to uniquely identify AFValue with the same datetime values
                          ChewCheeLim

                          Thanks Barry,

                          Yes, that's what I am trying to do. Possible?

                          Yes, the value 6:00AM 20 is being read with method .RecordedValue() by giving the time stamp. The method returns the record without problem.

                          Turn around, submit the Updatevalue() into the afUpdateList, with remove command. But nothing happened. -- the same way was tested with other test cases.. and for sure I know that this af sdk call is working. But for some reason, not this record. Any properties I should check ? where Remove command won't work?

                           

                          Will AF SDK call adding 2 values going through PI Compression process? if yes, which values will be compressed?

                           

                          Thanks much!

                            • Re: A way to uniquely identify AFValue with the same datetime values
                              bshang

                              I don't think I can reproduce this. Here is an example:

                               

                               

                               

                                          AFAttribute attr = (AFAttribute) AFObject.FindObject(@"\\server\Sandbox\Element1|Duplicate");
                              
                                          AFValue val1 = new AFValue(1, new AFTime("t"));
                                          attr.Data.UpdateValue(val1, AFUpdateOption.Insert);
                              
                                          AFValue val2 = new AFValue(2, new AFTime("t"));
                                          attr.Data.UpdateValue(val2, AFUpdateOption.Insert);
                              
                                          AFValue val3 = new AFValue(3, new AFTime("t"));
                                          attr.Data.UpdateValue(val3, AFUpdateOption.ReplaceOnly);
                              
                                          AFValue toDelete1 = attr.Data.RecordedValue(new AFTime("t"), AFRetrievalMode.AtOrAfter, null);
                                          attr.Data.UpdateValue(toDelete1, AFUpdateOption.Remove);
                              
                                          AFValue toDelete3 = attr.Data.RecordedValue(new AFTime("t"), AFRetrievalMode.Exact, null);
                                          attr.Data.UpdateValue(toDelete3, AFUpdateOption.Remove);
                              

                               

                              I add value = 1 at midnight.  I insert 2 at midnight. Replace 2 with 3 at midnight. Value = 3 now is Substituted. I grab value = 1 and delete it, then grab value = 3 and delete it. At the end, there are no more events. Can you reproduce this simple example? Can you show what is exactly in afUpdateList here?

                              • Re: A way to uniquely identify AFValue with the same datetime values
                                pthivierge

                                Hello Chew Chee,

                                 

                                The compression process applies only for in-order data.  As soon as you send values older than the snapshot the compression process does not apply.

                                 

                                What you are trying to do is definitely possible but there could be things in the way that prevent it to work properly:

                                 

                                • My first recommendation is: look at the data you have in the archive before you start the application and identify which value should be replaced ( take a screenshot) then put some logging in place in your application and output values you read as well as values you write.  Then please use this information to report here. ( I know this is a bit of work, but you will see this is very beneficial )
                                • Its always good to check the PI Data Archive message logs.  Check of there are error messages for this point ID. ( that could indicate corruption).
                                • Are you manipulating the time stamp you get when your read?  omitting milliseconds could prevent your call to work.
                                • Are you trying to write in a collective? if yes do you have SDK Buffering enabled? (this is required with a collective)

                                 

                                Hope this helps,