3 Replies Latest reply on Feb 1, 2017 10:02 PM by ekuwana

    AF SDK app using ReplaceValues() is too performant for Data Archive Snapshot

    acote

      Hello geeks,

       

      Before anyone ask: PI Data Archive 2016 R2, PI AF Server R2, AFSDK 2.8.6.7801...

       

      I have several C# Windows Services using AFSDK (latest version 2.8.6.7801), these apps are built around AF (act as data store) so it reads/writes AFAttributes and generate S88 batch structure in Event Frames. The apps architecture allows "recovery / backfilling" capability, so I can go back at the beginning of a specific batch and recalculate everything. Right now I'm building a new instance of these apps for a new process unit (ie Continuous Steel Casting of Billets), so I'm enjoying this feature to recalc since Jan-01-2017, imagine the service parsing > 120M events and writing 60M in a few minutes... And I noticed something that scares me crazy: the app write values so fast that they mostly go in the Snapshot "Events in Queue", which grows and grows even faster with ALL events from ALL sources (ie our 25k OPC Int tags pushed events to the Snapshot which can no longer process them to the archive in a timely fashion).

       

      I am using the following to push data to PI Data Archive

      attribute.Data.ReplaceValues(timeRange, newValues);
      

       

      But in the previous version I was using UpdateValues and it have the same effect (much slower)

      // Gather current data
      AFValues initialvalues = attribute.Data.RecordedValues(timeRange, AFBoundaryType.Inside, null, string.Empty, false);
      
      // Erase current data
      if (initialvalues.Count > 0)
           attribute.Data.UpdateValues(initialvalues, OSIsoft.AF.Data.AFUpdateOption.Remove, bufferOption);
      
       // Add new data
      if (newValues.Count > 0)
           attribute.Data.UpdateValues(newValues, OSIsoft.AF.Data.AFUpdateOption.Replace, AFBufferOption.BufferIfPossible);
      

      Note that to maximize the processing speed, this function is called as less as possible to limit the network traffic from several thread / parallel processing / async tasks... For example, one recovery function update 2 attributes with an average 8k events, about 650 times --> (10.4M events)... in approx 45 seconds

       

      I'm just starting to investigate the possible workaround and solutions... what do you think?

      - Is there a way to write directly to the archive without passing by the snapshot (therefore not performing Exception and Compression, which is bad)?

      - Is there a way to speed up the processing of Snapshot "Events in Queue"?

       

      Thanks

        • Re: AF SDK app using ReplaceValues() is too performant for Data Archive Snapshot
          Dan Fishman

          I'm am pretty sure you can do some optimization such as adjusting the threads flushing to the archive and the eventQueueSize, but you probably should speak to OSIsoft technical support before adjusting those.

           

          In general, it sounds like you are inserting a lot of out-or-order data and that puts additional stress on the archive. Also, it means more disks rights writes because compression is not applied to out-or-order data.  All PI values go through the snapshot table, into the event queue, and then eventually flushed to the archive.

           

          Over here at Exele, we hit some issues because data is stuck in the event queues during our recalcs so we provide the option to throttle recalculations if desired.  How is the data historian sized?  Are all of these events out-or-order?

          • Re: AF SDK app using ReplaceValues() is too performant for Data Archive Snapshot
            tramachandran

            It is normal for the event queue buildup to because of a large backfilling operation. Due to the large influx of incoming events, piarchss subsystem may not be able to keep up.  This should only be a temporary situation.  Once new events are no longer being added, piarchss should start catching up.

            As Dan suggested, you can try throttling the events that are being back filled.

            Another temporary measure you could also try is adjusting the Archive_SecondsBetweenFlush  and the Archive_FlushThreadCount tuning parameters to have more frequency and threads respectively. Disk I/O plays a part too, so if you want support for high values of read/write, then SSD is recommended.

            • Re: AF SDK app using ReplaceValues() is too performant for Data Archive Snapshot
              ekuwana

              Hi Alexandre,

               

              Regarding your question:

              - Is there a way to write directly to the archive without passing by the snapshot (therefore not performing Exception and Compression, which is bad)?

               

              Have you tried:

              - first delete-range.

              attribute.Data.ReplaceValues(timeRange, new AFValues());

              - then update with InsertNoCompression mode.

              attribute.Data.UpdateValues(newValues, AFUpdateOption.InsertNoCompression);

               

              Note that with this approach you will have better performance than your previous version (by avoiding UpdateValues with Remove mode). Also, you will be bypassing compression calculation.

              Caution if doing ReplaceValues for delete-range and then UpdateValues through buffering (PIBufSS) with compression:

              You need to check if the timestamp of newValues in UpdateValues are within timeRange of ReplaceValues, especially if going through buffer (PIBufSS). Otherwise, if timestamp of newValues is older than starttime of ReplaceValues timeRange, writing through PIBufSS may have some issues (data profile) due to discrepancy of snapshot/compression state between PIBufSS and PI Data Archive.