I have manually uploaded values in PI for certain tags. Values are in 3 decimal places. However, when retrieved via DataLink, the retrieved values are more that 3 decimal places. What could have caused this? Thanks.

I have manually uploaded values in PI for certain tags. Values are in 3 decimal places. However, when retrieved via DataLink, the retrieved values are more that 3 decimal places. What could have caused this? Thanks.

- 1 person found this helpful
Hi Jediael,

A string or an integer will be stored exactly as entered by the user, but floating point values are stored differently, and may not exactly equal the entered value. This inaccuracy is inherent in floating point numbers.

1172OSI8 - PI Server Storage and precision of Float32 and Float64 data

When calculations and plots are generated from PI Data Archive data, OSIsoft guarantees accuracy within the range of the configured data type. Therefore, if you want a high level of accuracy you can use float64 at the expense of space.

- 2 people found this helpful
This is due to inevitable precision limitation of Floating point data type.

For example, 29979.38 would be shown as 29979.38085937 in Excel, i.e. Excel displays (and calculates with!) values with greater precision than IEEE-defined Float in some cases.

Here's some very useful Microsoft links:

Floating-point arithmetic may give inaccurate results in Excel

https://support.microsoft.com/en-us/kb/78113

How to correct rounding errors in floating-point arithmetic

https://support.microsoft.com/en-us/kb/214118

I usually go for Solution 2 from the second link:

a) Set the number format to whatever number of decimal places you need

b) Excel > Options > Advanced > scroll down to "When calculating this workbook" and check the "Set precision as displayed" option

This is due to inevitable precision limitation of Floating point data type.

For example, 29979.38 would be shown as 29979.38085937 in Excel, i.e. Excel displays (and calculates with!) values with greater precision than IEEE-defined Float in some cases.

Here's some very useful Microsoft links:

Floating-point arithmetic may give inaccurate results in Excel

https://support.microsoft.com/en-us/kb/78113

How to correct rounding errors in floating-point arithmetic

https://support.microsoft.com/en-us/kb/214118

I usually go for Solution 2 from the second link:

a) Set the number format to whatever number of decimal places you need

b) Excel > Options > Advanced > scroll down to "When calculating this workbook" and check the "Set precision as displayed" option