Interruption in real-time data stream: How data loss events are handled

Discussion created by jmarriott on Aug 24, 2012
Latest reply on Aug 26, 2012 by chuck
24 Aug. 2012

During a recent pre-sale discovery process, a prospective customer, with a plant subject to power and communiction failures beyond their control, was interested in how the PI System would handle data loss events.  The following explanation was written.  This may be helpful to other members of the OSIsoft community.

The graphic below shows 3 PI ProcessBook trends.  The same data is used in all 3 trends except there is a period in the red circle where the 2nd and 3rd tags suffered data loss.  The green trace in each trend contains raw data. The blue trace calculates the maximum value for a each 1-minute period.  The yellow trace calculates the 1-minute averge.  In the second trend, there is no indication that data has been lost.  The events on either side of the data loss are connected by a straight line.  In the 3rd trend, the data loss is indicated by a value of “No Data”.


A flat line is about the worst case scenario, because calculations can be made on invalid data.  The PI System interfaces constantly verify communications with their data source.  If comm is lost, the interface writes a string to the PI tag to indicate there is data loss and to help identify the cause.  If power is lost to the interface PC and the interface does not shutdown gracefully, the interface may not be able to write the message to the PI tags.  There is an Interface Status Utility that runs on the PI Server which checks to make sure the interface is communicating with the PI Server.  If it is not, this utility can write a value, indicating data loss, to the PI tags from the failed interface.

You can see how the lost data impacts these calculations of Maximum (blue) and Averge (yellow).  Using PI DataLink in Excel would give similar results, but you can specify a minimum percentage of good data that must be present for the calculation to be valid.  A lack of good data would be indicated in place of an erroneous calculation.

The PI interfaces also have fault tolerance.  They include a buffer, so that if the PI interface can collect data from the control system, but is unable to communicate with the PI Server, the data will be buffered locally.  Upon re-connection to the PI Server, the buffer will be dumped and there would be no data loss.  Additionallly, most PI interfaces support failover to further reduce the risk of data loss.