Pablo Benvenuto

PI 101, Module 8: How is Data Processed by the PI System

Discussion created by Pablo Benvenuto Champion on Oct 2, 2012
Benefits of Efficient Data Processing

Any system that handles large amounts of data must provide efficient ways to process and store the data.  The PI System can acquire, process, store and display millions of values for very long periods of time.  Some customers have up to 30 years of historical data stored in their PI Systems.

Efficient processing of the historical data stored in your PI System gives you the following benefits:

Storage: it reduces the amount of space needed to store data, freeing up valuable space on a hard drive. Some historical data can be reduced to 90 percent and more of their original size.

Transmission Speeds: the amount of time it takes to send data sets over the network depends on the size of the transmitted data set. Reducing data sets minimizes the amount of time it takes to move PI Data in the network by a considerable margin.  This also reduces the financial cost of running a network, since less equipment and bandwidth is needed to transmit the data sets.

Archiving and Backup: reducing your data allows other processes to be faster and more efficient such as archiving and backups.  By reducing the time to backup historical data, we can perform more frequent and more comprehensive backups, reducing the potential of data loss.

PI System Performance: the ultimate result of efficient data processing is an increase in performance that allows the PI System to store more data for a given disk size, with faster data retrieval time for display and in a more secure way with backup features that prevent data loss.

How is Data Processed by the PI System

The PI System combines two processes to efficiently handle large amounts of data.  At the Interface level there is an Exception Test that filters redundant data points.  At the PI Server level the data is filtered once again with a Compression Algorithm before it is archived.  These two processes together, commonly refer to as Exception and Compression; provide a powerful and efficient way to handle all PI Data.

It is important to understand how these two processes work and the impact that they have in the data being archived and the performance of the PI System.  Every new installation of the PI System has the Exception and Compression enabled with default values.  It is important that you review these values and determine if they need to be changed and calibrated in a way that is optimal for each one of the tags being collected.  There might be situations also where you want to turn these processes off.

Review Questions


In a previous module you searched for the total number of tags in your PI System.  Using that number and assuming that all those tags are scanned every 10 seconds on the average, calculate the number of values scanned by your PI System in one year.

Go to Module 9 or back to the Workshop Outline.