How Data Moves through the PI System

Version 3

    This document is an excerpt from the Visualizing PI System Data Workbook v2017



    • Explain how the Interfaces filter noise and define noise.
    • Explain how the Data Archive applies compression to store only meaningful data


    You can tune your PI Points for maximum efficiency with the configurable attributes that specify compression and exception reporting. The configuration of these specifications impacts the flow of data from the interface node to the server for that point (exception reporting) and the efficiency of data storage in the archive for that point (compression testing).


    The settings of these two testing and reporting mechanisms have default values set in PI. However, since every organization is unique, your PI Administrator would need to modify these settings according to your data collection needs.


    Exception and Compression Quick Summary:



    Below is a brief description of each of these two testing and reporting mechanisms.

    Exception Reporting (filtering noise)

    In an ideal world, the interface would apply some sort of logic to data collection. This is often referred to as “Reporting by Exception”. The exception test filters all values considered noise.

    This process filters out noise, and thereby reduces the communication (I/O) burden between the Data Archive and the interface node. OSIsoft recommends that the exception deviation is set to slightly smaller than the precision of the instrument. Exception reporting is a simple linear test that occurs on the interface node.

    Noise: Insignificant changes, are defined as those below the instrument’s accuracy threshold, as set by the person creating PI Points, and identical values, such as a valve that is reading OPEN repeatedly.

    The value passing the exception reporting and sent to Data Archive is called the Snapshot value, or current value.



    Compression Testing (Storing only meaningful data)

    Compression testing is performed on the Data Archive to enhance data storage efficiency and thereby conserve disk space. The compression test uses a sophisticated algorithm, sometimes called the swinging door compression algorithm, to determine which events should be stored in the PI archives. The Data Archive needs to store only those events deemed meaningful by the compression test; it can essentially recreate other events through interpolation of surrounding events.

    The value passing the compression testing gets archived and therefore is called the Archive value.

    Note: Detailed explanations on compression algorithm can be found in KB Article “KB00699 – Compression Explained”:



    Interested in learning more? Watch “OSIsoft: Exception and Compression Full Details” :