13 Replies Latest reply on Jul 12, 2017 7:25 PM by Rick Davin

    PiPoint.Summary Total value precision is off

    skdesiraju

      I am trying to use PiPoint.Summary to get the total of recorded values and I found that it is not accurate.

       

      These are the recorded values for the month of Jan and total for the month should be 426829.560 but pipoint.summary is giving the value of 426829.494. We have to be accurate to the 100th place, is there is a reason why the number is off? Here are the table of values for jan month, Time Range.

        

      7842.191/3/2016 12:20
      141327.101/4/2016 2:20
      7567.531/15/2016 2:50
      110155.301/19/2016 6:30
      7839.041/27/2016 5:35
      152098.401/27/2016 22:15

       

      +time{1/1/2016 12:00:00 AM - 2/1/2016 12:00:00 AM}OSIsoft.AF.Time.AFTimeRange


        var values = point.Summary(time,   AFSummaryTypes.Total,  AFCalculationBasis.EventWeighted  , AFTimestampCalculation.Auto);

       

      Is this a bug or am I missing something here?

       

      Thanks!

        • Re: PiPoint.Summary Total value precision is off
          Rick Davin

          Your calc basis should be Time-Weighted.

           

          Also what is the last value and timestamp in December?  Or to really get to the point, what is the interpolated value at 1/1/2016 12:00:00 AM?

           

          What you are missing here is the notion that there are values outside of just the month you want to look at, and those values affect the summary.  If the tag is not stepped, which is the best fit since it's a Float32 or Single, there will be interpolated values at the time range start and end times, namely a Jan 1 and Feb 1 midnight.  If the tag is stepped, then the value previous to Jan 1 will considered to the starting value, and the last value in Jan will be duplicated again at the end time.

           

          You want the total to be time-weighted.  You will note they are not the same duration nor occur at the same time every day.  For example, the tag read 7842.19 starting at Jan 3 but lasting 26 hours.  The next value was 141327.10 but its duration was 11 days and 30 minutes.  I would expect the total should therefore be skewed closer towards 141327 than in the midpoint between that and 7842.

          1 of 1 people found this helpful
            • Re: PiPoint.Summary Total value precision is off
              skdesiraju

              Should it be time-weighted if I am trying to aggregate actual recorded values?

               

              Interpolated value at 1/1/2016 is 82045.16

               

              When I change to TimeWeighted the values are way too off.

               

              I am actually upgrading my code to PI AF SDK my old code which was the legacy PI SDK gets me the right value for the same values above.

                iPiData.Summaries2(startDate, endDate, duration,

                                          PISDK.ArchiveSummariesTypeConstants.asTotal,

                                          PISDK.CalculationBasisConstants.cbEventWeighted                             );

               

               

                • Re: PiPoint.Summary Total value precision is off
                  Rick Davin

                  Thanks for giving me the interpolated value at Jan 1.  Can you give me the recorded value at or before Jan 1 as well as the recorded value at or after Feb 1.  I will create a tag on my end with your same recorded values for tests.

                   

                  And what particular eng unit are you using?

                    • Re: PiPoint.Summary Total value precision is off
                      skdesiraju

                      Here is the data from 12/1/2015 -4/1/2016

                        

                      17784.8212/1/2015 7:00
                      62754.1112/3/2015 16:45
                      25036.9412/9/2015 23:30
                      24721.5312/22/2015 4:00
                      790712/22/2015 6:10
                      10552612/22/2015 16:20
                      7739.0612/29/2015 9:55
                      142104.512/29/2015 23:10
                      7842.191/3/2016 12:20
                      141327.11/4/2016 2:20
                      7567.531/15/2016 2:50
                      110155.31/19/2016 6:30
                      7839.041/27/2016 5:35
                      152098.41/27/2016 22:15
                      126381.53/1/2016 14:05
                      83563.13/2/2016 9:45
                      34844.293/13/2016 9:15
                      7778.063/13/2016 18:30
                      7854.263/22/2016 5:20
                      91343.953/22/2016 14:30
                      7760.453/23/2016 17:30
                        • Re: PiPoint.Summary Total value precision is off
                          Rick Davin

                          I am using AF Client 2017 SP1 and PI Data Archive 2017.  My point is a Float32 and is not stepped.  The only data for the tag is:

                           

                          2017-07-11 17_57_45-Archive Editor - PI System Management Tools (Administrator).png

                           

                          My code is:

                           

                          public static void TestTotal(string dataArchiveName, string tagName)
                          {
                              var tag = GetPIPoint(dataArchiveName, tagName);
                          
                              var timeRange = new AFTimeRange("1/1/2016", "2/1/2016");
                          
                              var eventWeightedTotal = tag.Summary(timeRange, AFSummaryTypes.Total, AFCalculationBasis.EventWeighted, AFTimestampCalculation.Auto);
                              var timeWeightedTotal = tag.Summary(timeRange, AFSummaryTypes.Total, AFCalculationBasis.TimeWeighted, AFTimestampCalculation.Auto);
                          
                              Console.WriteLine(tag.Name);
                              Console.WriteLine($"StartTime: {timeRange.StartTime}");
                              Console.WriteLine($"EndTime: {timeRange.EndTime}");
                              Console.WriteLine($"Event Weighted: {eventWeightedTotal[AFSummaryTypes.Total].Value}");
                              Console.WriteLine($"Time  Weighted: {timeWeightedTotal[AFSummaryTypes.Total].Value}");
                          }
                          

                           

                          The Console output is:

                           

                          Test/Total

                          StartTime: 1/1/2016 12:00:00 AM

                          EndTime: 2/1/2016 12:00:00 AM

                          Event Weighted: 426829.556640625

                          Time  Weighted: 2359805.79010811

                           

                          If you round the event weighted value to 2 decimal places, the result is 426829.56, which is the value you want it to be.

                          1 of 1 people found this helpful
                            • Re: PiPoint.Summary Total value precision is off
                              skdesiraju

                              Exact same code, I am having different numbers that you are. I am on AF Client 2014.

                               

                              tag = PIPoint.FindPIPoint(piserver, TagName);

                               

                              var timeRange = new AFTimeRange("1/1/2016", "2/1/2016");

                               

                              var eventWeightedTotal = tag.Summary(timeRange, AFSummaryTypes.Total, AFCalculationBasis.EventWeighted, AFTimestampCalculation.Auto);

                               

                              var timeWeightedTotal = tag.Summary(timeRange, AFSummaryTypes.Total, AFCalculationBasis.TimeWeighted, AFTimestampCalculation.Auto);

                               

                              Console.WriteLine(tag.Name);

                               

                              Console.WriteLine($"StartTime: {timeRange.StartTime}");

                               

                              Console.WriteLine($"EndTime: {timeRange.EndTime}");

                               

                              Console.WriteLine($"Event Weighted: {eventWeightedTotal[AFSummaryTypes.Total].Value}");

                               

                              Console.WriteLine($"Time  Weighted: {timeWeightedTotal[AFSummaryTypes.Total].Value}");

                                • Re: PiPoint.Summary Total value precision is off
                                  skdesiraju

                                  When I get the recorded values as

                                   

                                  values = point.RecordedValues(time, AFBoundaryType.Inside, null, false, 0);

                                   

                                  these are values I get

                                   

                                  Recorded ValuesTimeStampActual Value
                                  7842.1901/1/2016 0:007842.190
                                  141327.1251/3/2016 12:20141327.100
                                  7567.5301/4/2016 2:207567.530
                                  110155.2501/15/2016 2:50110155.300
                                  7839.0401/19/2016 6:307839.040
                                  152098.3591/27/2016 5:35152098.400
                                  Sum= 426829.494426829.560
                                    • Re: PiPoint.Summary Total value precision is off
                                      Rick Davin

                                      Here is updated code.  My tag is a Float32 or Single.  The new code will display the recorded values as a Single as well as a Double.  It also shows a custom total.  Note that the PIPoint.Summary call will (a) convert all numeric inputs to Double, and (2) the output result will be a Double.

                                       

                                      public static void TestTotal(string dataArchiveName, string tagName)
                                      {
                                          const string dateFmt = "MM/dd/yyyy HH:mm:ss tt";
                                          var tag = GetPIPoint(dataArchiveName, tagName);
                                      
                                          var timeRange = new AFTimeRange("1/1/2016", "2/1/2016");
                                      
                                          Console.WriteLine(tag.Name);
                                          Console.WriteLine($"StartTime: {timeRange.StartTime.ToString(dateFmt)}");
                                          Console.WriteLine($"EndTime: {timeRange.EndTime.ToString(dateFmt)}");
                                      
                                          var values = tag.RecordedValues(timeRange, AFBoundaryType.Outside, null, true, 0);
                                          Console.WriteLine();
                                          Console.WriteLine("Recorded Values using OUTSIDE:");
                                          Console.WriteLine($"   Local Timestamp        \tSingle    \tDouble");
                                          Single sTotal = 0;
                                          double dTotal = 0;
                                          foreach (var pv in values)
                                          {
                                              var outside = pv.Timestamp < timeRange.StartTime || pv.Timestamp > timeRange.EndTime;
                                              Console.WriteLine($"   {pv.Timestamp.LocalTime.ToString(dateFmt)} \t{pv.ValueAsSingle()} \t{pv.ValueAsDouble()} {(outside ? "\tOUTSIDE" : "")}");
                                              if (pv.IsGood && !outside)
                                              {
                                                  sTotal += pv.ValueAsSingle();
                                                  dTotal += pv.ValueAsDouble();
                                              }
                                          }
                                      
                                          Console.WriteLine();
                                          Console.WriteLine($"Custom Event Weighted Totals");
                                          Console.WriteLine($"   Single: {sTotal}");
                                          Console.WriteLine($"   Double: {dTotal}");
                                      
                                          var eventWeightedTotal = tag.Summary(timeRange, AFSummaryTypes.Total, AFCalculationBasis.EventWeighted, AFTimestampCalculation.Auto);
                                          var timeWeightedTotal = tag.Summary(timeRange, AFSummaryTypes.Total, AFCalculationBasis.TimeWeighted, AFTimestampCalculation.Auto);
                                      
                                          Console.WriteLine();
                                          Console.WriteLine("Summary Calls on Total");
                                          Console.WriteLine($"   Event Weighted: {eventWeightedTotal[AFSummaryTypes.Total].Value}");
                                          Console.WriteLine($"   Time  Weighted: {timeWeightedTotal[AFSummaryTypes.Total].Value}");
                                      }
                                      

                                       

                                       

                                      Console output:

                                       

                                      Test/Total

                                      StartTime: 01/01/2016 00:00:00 AM

                                      EndTime: 02/01/2016 00:00:00 AM

                                       

                                      Recorded Values using OUTSIDE:

                                         Local Timestamp              Single          Double

                                         12/29/2015 23:10:00 PM       142104.5        142104.5        OUTSIDE

                                         01/03/2016 12:20:00 PM       7842.19         7842.18994140625

                                         01/04/2016 02:20:00 AM       141327.1        141327.09375

                                         01/15/2016 02:50:00 AM       7567.53         7567.52978515625

                                         01/19/2016 06:30:00 AM       110155.3        110155.296875

                                         01/27/2016 05:35:00 AM       7839.04         7839.0400390625

                                         01/27/2016 22:15:00 PM       152098.4        152098.40625

                                         03/01/2016 14:05:00 PM       126381.5        126381.5        OUTSIDE

                                       

                                      Custom Event Weighted Totals

                                         Single: 426829.6

                                         Double: 426829.556640625

                                       

                                      Summary Calls on Total

                                         Event Weighted: 426829.556640625

                                         Time  Weighted: 2359805.79010811

                                       

                                       

                                      That said, you may want to give serious consideration to upgrading your AF Client as well as PI Data Archive.

                                       

                                      The next thing you need to understand is that your desire for accuracy to the 2nd decimal place is not possible with Float32 (Single).  As per IEEE 754, the 32 bit binary floating point numbers will only have 7 significant digits.  A value such as 141327.1  is therefore only accurate to the 1st decimal place.  And that's the spec for just 1 number.  The 7th significant digit becomes more suspect when you perform a mathematical operation on 2 numbers.  It is incorrect to think that merely widening a Single to a Double somehow magically introduces more precision or accuracy.  It absolutely does not.

                                       

                                      Here's more links that contain other links regarding IEEE 754:

                                       

                                      Re: AFData.UpdateValues() gives more precision.. after decimal. Expected behavior

                                      Problems with Pi float Values conversion from and to C# double

                                      CompDevPercent and ExcDevPercent values different from configured values

                                       

                                       

                                      I am confused about your values list where you show Recorded Values and then Actual Value.  How was this list created?  Can you explain the difference between a Recorded Value and an Actual Value?  Can you share the code that produced this list?

                                        • Re: PiPoint.Summary Total value precision is off
                                          skdesiraju

                                          Sorry I should have been clearer. Recorded values column above is when I try to get the recorded values from the c# code.

                                          Here is the code:

                                           

                                          public List<TagValueContract> GetPiTagRecordedValues(string TagName, DateTime startDate, DateTime endDate, bool excludeStatusValues)
                                                  {
                                                      List<TagValueContract> recordedValues = new List<TagValueContract>();
                                                      PIPoint point = PIPoint.FindPIPoint(piserver, TagName);
                                                      AFTimeRange time = new AFTimeRange();
                                                      time.StartTime = startDate;
                                                      time.EndTime = endDate;
                                          
                                                      AFValues values = point.RecordedValues(time, AFBoundaryType.Inside, null, false, 0);
                                          
                                                      foreach (AFValue value in values)
                                                      {
                                                          if (value.IsGood )
                                                          {
                                                             
                                                              TagValueContract item = new TagValueContract();
                                                              item.RecordDate = value.Timestamp.LocalTime;
                                                              item.Value = value.Value;
                                                              item.TypeName = point.PointType.ToString();
                                          
                                                              if (excludeStatusValues)
                                                              {
                                                                  if(isDouble(value.Value))
                                                                      recordedValues.Add(item);
                                                              }
                                                              else
                                                                  recordedValues.Add(item);
                                                          }
                                          
                                                      }
                                                      return recordedValues;
                                                  }
                                          

                                           

                                           

                                            

                                          Recorded Values from code aboveTimeStampActual Recorded Values from PI SMT
                                          7842.1901/1/2016 0:007842.190
                                          141327.1251/3/2016 12:20141327.100
                                          7567.5301/4/2016 2:207567.530
                                          110155.2501/15/2016 2:50110155.300
                                          7839.0401/19/2016 6:307839.040
                                          152098.3591/27/2016 5:35152098.400
                                          Sum=426829.494426829.560


                                            • Re: PiPoint.Summary Total value precision is off
                                              Rick Davin

                                              See my last post with links regarding IEEE 754 Single.  It's still quite relevant in this conversation.

                                               

                                              What is the data type for TagValueContract.Value?  Is it double, object, or other?

                                               

                                              IsDouble is a custom method, right?  What does it do?  Is it something simple like:

                                               

                                              public bool IsDouble(value as object)
                                              {
                                                return value is double;
                                              }
                                              

                                               

                                              This code snippet:

                                               

                                                          if (value.IsGood ) 
                                                              { 
                                                                 
                                                                  TagValueContract item = new TagValueContract(); 
                                                                  item.RecordDate = value.Timestamp.LocalTime; 
                                                                  item.Value = value.Value; 
                                                                  item.TypeName = point.PointType.ToString(); 
                                              
                                                                  if (excludeStatusValues) 
                                                                  { 
                                                                      if(isDouble(value.Value)) 
                                                                          recordedValues.Add(item); 
                                                                  } 
                                                                  else 
                                                                      recordedValues.Add(item); 
                                                              } 
                                              

                                               

                                              On line 09, excludeStatusValues is not really excluding System state values.  You have already omitted System states on line 01 with the IsGood check.  But if your tag's PointType is a Digital or String or Timestamp, those values can be good but also not be Double.  Plus Float32's are not strictly equal to typeof(double) but they are a compatible type to double (again, value is double).  But when widening a Float32 to Double will introduce extra decimal places that are not accurate.

                                               

                                              The important thing to know here is whether or not TagValueContract.Value is a double.  If it is, then that explains why your Actual Value differs from your Recorded Value column.

                                               

                                              For the bad sum (426829.494), are you actually using the PIPoint.Summary call, or are you using the AFValues.Summary call? Or a custom sum on List<TagValueContract>?

                                                • Re: PiPoint.Summary Total value precision is off
                                                  skdesiraju

                                                  The bad sum irrespective of how I do I always get 426829.494. So far I have tried following to get the total and all approaches gave me bad total :

                                                  1. Get the total from PIPoint.Summary

                                                  2. Get average and count from PIPoint.Summary and multiply to get the total

                                                  3. Get PIPoint.Recordedvalues for the month and then aggregate them

                                                   

                                                  I have a feeling that this might be a bug from the AF Client 2014 as you were able to get the right total using AF Client 2017. We have used exact same logic of Summary with legacy PISDK API in the production and we always got the right totals. I also feel that it is not a float/double conversion issue because the original data did not have so many digits after decimals that they are lost by the precision. Also when I use PI plugin extension from the excel to pull in the recorded values they also come right.