avanfosson

Why am I missing non-repeating values after setting compression deviation to zero?

Blog Post created by avanfosson Employee on Dec 31, 2015

Most users assume that setting compression deviation to 0 will mean that even the smallest change in a tag's value is still archived and that repeated values are discarded. This assumption is valid for the vast majority of data. However, there are two special cases where that assumption is not true. In an earlier blog post I described why a user can see repeated values in his or her archive even when compression deviation is set to 0. In this post I will address the other special case where changes in a tag's value are not archived even when the compression deviation is set to 0.

 

Let's say we have incoming data from an integer tag that looks like this:

TimestampValue
00:00:011
00:00:022
00:00:033
00:00:044
00:00:055
00:00:065

We would expect to see each value in the archive because the value is changing. What we actually see in the archive is the following:

TimestampValue
00:00:011
00:00:055
00:00:065

The values 2, 3, and 4 are compressed out because they fall exactly on the line between 1 and 5. In other words, their deviation from the slope of the line is 0, which is equal to the specified compression deviation; therefore, they do not pass compression. This is the key difference between turning compression off and setting the compression deviation to 0. With compression off, all values are archived. If compression is on and the compression deviation is set to 0, then all values whose deviation* is greater than the compression deviation are kept.

 

Note that we have not lost any information after compression. If we were to interpolate for the value at 00:00:02, then we would get a value of 2, which is exactly the value that entered the snapshot and was compressed out.

 

*that is the deviation from the line drawn between the most recent archive value and the newest value

Outcomes