AnsweredAssumed Answered

Buffered Data Detection and Reprocess of Analytics

Question asked by TimCarmichael Champion on Mar 11, 2017
Latest reply on Mar 21, 2017 by Eugene Lee

So... we have a number of analytics set up with natural scheduling on any input, and, from those, a number of analytics set up to create aggregate values periodically. An external system is querying the aggregate values and using them for reporting purposes.

If the source data is received in a timely manner, all well and good... however, since a number of the source systems are remote, periodically we have data flow interruptions - that can be caused by a loss of network communication, a loss of power at a site which results in a corrupt buffer queue, etc.


When the source data flows in, the naturally based analytics will execute... but, in the mean time, the periodic aggregate calculations have been executed with little to no data.


Given all of this, how can we:

determine the data flow has been disrupted and for how long?

I was thinking about the buffered data tags; determine what a 'normal' buffered count is, start a counter/marker when the buffer exceeds that amount and stop the counter/maker when the buffer returns to 'normal' values; since the buffered tag is from the API node, the time stamps will be based on when the event actually happened, not when it was received at the PI server. Then, create a notification to indicate the time range and reprocess the periodic values during that time


can analytic back fill be accomplished programatically or is it still a manual operation?


Has anyone else encountered this scenario?