6 Replies Latest reply on Sep 3, 2015 9:02 PM by Rhys Kirk

    Where are my future events hiding for my AF Analysis?

    Rhys Kirk

      Simple scenario, 1 real time PI Point, 1 future PI Point, 1 event driven analysis that is triggered by the future PI Point and writes to the real time PI Point. The analysis works just fine, my future values are written to the real time PI Point when they fall within the current time.

      However, I cannot hunt down where the future events are hiding upon writing a future event in to the future PI Point.


      - I don't see AF Analyses Data Cache increasing (or any other statistic).

      - I don't any of the Time Series Update Queues with any pending events.


      I am interested to know whether restarting the Analysis Processor will lose the future events, or are they hiding on a PI Server Update Queue? Or is the Analysis Processor checking explicitly for any new futures values that are current?


      What happens if the PI Server is rebooted, are the time series queues persisted anywhere?

      How many future events can be queued for an Analysis?


      The Analysis Processor used to work off a 250ms cycle for collecting events for event triggered analyses, is that still the case? So within 250ms of a future event becoming current I should expect the analysis to be triggered?

        • Re: Where are my future events hiding for my AF Analysis?
          Dan Fishman

          I haven't learned these internals yet and have been wondering on this works!


          I believe this is all done by the update manager (update queue) as you suggested.  I would think queue sizes as they are for the regular updates. Yes, within 250ms of the future event becoming current you should expect the analysis to be triggered if the CalculationWaitTimeInSeconds is set to zero.  I hope someone else can comment more on how this is working under the hood.

          • Re: Where are my future events hiding for my AF Analysis?

            I think it would depend on the AFEventHorizonMode of the AFDataCache used by PI Analysis Service. AFDataCache should have events up to current time + EventHorizonOffset after a client update call (if I'm interpreting the doc correctly). However, I imagine there is also a client-side look-ahead buffer beyond the event horizon that's updated at a slower rate and maybe that is another queue for future events (i.e. AFDataPipe looks here first). We'll ask internally for details. Maybe it is a good thought experiment for the community to figure out too...

            • Re: Where are my future events hiding for my AF Analysis?

              The data source feeding the AFDataPipe/Cache for future PIPoint keeps a buffer of a few minutes of data into the future and sends this data "down the pipe" when it becomes current (according the the EventHorizonMode and EventHorizonOffset).  The buffer is a detail of the data source and not visible to the cache so it doesn't turn up in the statistics.  The data source uses a TimeSeries subscription to PI and periodic bulk recorded value calls to keep the buffer up to date.


              So restarting the Analysis Service or PI Server will have the same effect as with historical tags - the subscription will need to be reestablished (and the buffer rebuilt for future tags) but nothing is lost and recovery is automatic.

              1 of 1 people found this helpful
                • Re: Where are my future events hiding for my AF Analysis?
                  Rhys Kirk

                  Thank you David Moler (and Dan Fishman, Barry Shang).


                  It makes more sense now, especially why I couldn't find the events.

                  I can see that it seems to be every 30 seconds or so that new events are retrieved I assume via the Recorded Value calls you mention. So if the Time Series Data Cache object only maintains a couple of minutes into the future what does it do with events that come down the future time series data pipe whose timestamps are beyond the event horizon offset? Discard them because it assumes the periodic recorded value call will pick them up when they come within scope of the future time series cache?

                  Won't most of the future values be picked up via a recorded values call rather than the time series data pipe? Imagine a day ahead forecast at 30 minute intervals, those events will be updated once per day, when they are first inserted and the PI Data Archive queues them then when the Analysis Engine pulls the values down the pipe it will discard them all only to later retrieve them via a recorded values call.

                  How far into the future will the recorded values call go? EventHorizonOffset?

                    • Re: Where are my future events hiding for my AF Analysis?

                      You are correct - if the values from the pipe fall past the buffer range, they are discarded and will eventually be re-acquired via recorded values call.  The typical case is that the data is retrieved via periodic recorded values calls and the update subscription ensures this buffer doesn't go stale and out-of-order data is not missed.  Here we are attempting to strike a balance between client-side memory usage and network I/O.


                      The query will go a couple minutes beyond the EventHorizonOffset which for the Analysis Service is current time.  So yes, it will be retrieved, discarded, and later queried.  The reasoning here is that it is cheaper to re-query the data in 30 minutes (or 24 hours) than to hold it in memory the whole time.  We make use of the bulk recorded values call, so this doesn't add a lot of chatter.