7 Replies Latest reply on Apr 15, 2016 4:27 PM by ekuwana

    AF SDK Update Values Timestamp Error Handling

    matt.inglis

      I am at the end of development of an large custom interface using AF SDK Update Values inserting around 200,000 values a second to our PI Server.

       

      On bulk insert, we capture the AFErrors returned, however what i am struggling to test is what if the input system has the wrong timestamp.

       

      i.e. if the input timestamp is an hour/month/year ahead of where it should be, PI will not store these values as its ahead of current time.

       

      However, PI does not throw back all the values in error as i would expect, instead i either get no errors or one or two errors.

       

      Does the AFSDK not return errors when PI cannot store values due to invalid timestamp like in this scenario?

        • Re: AF SDK Update Values Timestamp Error Handling
          bshang

          What are the call parameters to UpdateValues? Are you writing to PI Buffer System? If so, the AF SDK may not return an error (will get null back) as the buffer will not check this. If you bypass the buffer (AFBufferOption.DoNotBuffer), you should expect to see errors like  [-11046] Target Date in Future.

           

          For this amount of throughput though, you probably want to still use the buffer. I wonder if it's feasible to perform the timestamp filtering client-side.

            • Re: AF SDK Update Values Timestamp Error Handling
              matt.inglis

              Hi,

               

              I am using options AFUpdateOption.NoReplace and AFBufferOption.DoNotBuffer.

               

              I cannot use the Buffer subsystem as it is not fast enough to cope with the data rate (PI Server 2010), instead i have written my own buffer subsystem.

               

              I could handle this error before feeding into the AFSDK however, i am concerned with the lack of errors i am receiving.

               

              For instance in one packet, im sending 252 values to PI, none of which are written to the server, yet i get 1 error.

               

              I have no issues in normal operating conditions, but during negative testing this was surprising to me.

                • Re: AF SDK Update Values Timestamp Error Handling
                  bshang

                  Just to be sure, there is no compression in the PI Point configuration? Were there any errors in the PI Data Archive message logs? Are the only values not written those with timestamps in the future or are other values not written as well? What are the errors that are returned in AFErrors?

                   

                  Does your own buffer exist in a separate process and what is the data structure used (e.g. most use BlockingCollection)?

                    • Re: AF SDK Update Values Timestamp Error Handling
                      matt.inglis

                      I will need to check the Data Archive Logs.

                       

                      Only the values with Future timestamps were not stored. All compression and exception is turned off for all points.

                       

                      The buffer is completely separate and thoroughly tested, it uses a bespoke data structure. We buffer the data prior to feeding into the AFSDK, all values are decoded and checked before firing them in blocks to the UpdateValues method.

                       

                      The whole system has been point to point tested for many months, and we have 100% confidence in it.

                       

                      I find it very bizarre that we don't get an error for every point which has an invalid timestamp, as each of these inserts should by definition be in error.

                • Re: AF SDK Update Values Timestamp Error Handling
                  matt.inglis

                  No these are live values constantly changing, with the source system just set with the wrong the time.

                    • Re: AF SDK Update Values Timestamp Error Handling
                      ekuwana

                      Hi Matt,

                      - What version of AF SDK are you using ? There are some improvements on error reporting in AF 2015.

                      - Which SDK call are you making ? AFListData.UpdateValues or PIServer.UpdateValues ? Do you associate each value with a corresponding AFAttribute or PIPoint ?

                      - When you said you got 1 error which level is it ? AFErrors.PIServerErrors, or AFErrors.PISystemErrors, or AFErrors.Errors ? Could you confirm if the error is '[-11046] Target Date in Future' ?

                      - You mentioned that the values are constantly changing and not identical, is this also true in your negative testing (sending 252 values with invalid times and only getting 1 error) ? Also, the values are distinct not only within the same PIPoint, but across PIPoint, i.e. no identical values between PIPoints ? As you may already know that the errors in AFErrors are keyed on AFValue so if all values being sent are identical then only one exists in the keyed collection.