6 Replies Latest reply on Feb 4, 2015 8:20 AM by Oysteinb

    Bug or incorrect usage of AFCalculation.CalculateSummaries method

    Oysteinb

      Hi

       

      I have an issue using the AFCalculation.CalculateSummaries method that I don't quite understand and I hope someone can help me.

       

      What I want to achieve is to do a calculation on a point and get the total for every day for the last week. For this test I am trying to get 1 day

       

      I added the following data for my test point. There is no earlier entries or later for that matter:

      value | timestamp

      001.02.2015 14:00:01
      001.02.2015 14:00:02
      101.02.2015 14:00:03
      101.02.2015 14:00:09
      001.02.2015 14:00:10
      001.02.2015 22:00:10

       

      this is my C# code:

      var startTime1 = new AFTime(new DateTime(2015,2,1,0,0,0, DateTimeKind.Utc));

                      var endTime2 = new AFTime(new DateTime(2015,2,2,0,0,0, DateTimeKind.Utc));

      string expression = String.Format(CultureInfo.InvariantCulture, @"('\\{0}\{1}')*86400", server.Name, point.Name); //86400 is the conversion unit when using TimeWeighted

                      var v1 = AFCalculation.CalculateSummaries(

                          server,

                          expression,

                          new AFTimeRangee(startTime1, endTime2),

                          new AFTimeSpan(hours: 1),

                          AFSummaryTypes.Total,

                          AFCalculationBasis.TimeWeighted,

                          AFSampleType.ExpressionRecordedValues,

                          new AFTimeSpan(seconds:1),

                          AFTimestampCalculation.Auto

                          );

         

      This returns a list off 24 values Where timestamps before 14:00 gives Bad data, 14. gives me 7.0019449847179773 (do not like the precision thing here...) and from 15 and rest of day: 0

      However when I change my return resolution to hours 24 I get 1 record. As expected, but the value is: 15.273112957397913. I dont understand why this is happening.

       

      If I set hours to 12 i get 2 records. Again as expected, but the total value is 7.6365564786989566?

       

      If I add a new value to the archive with value 0 and timestamp 31.01.2015 23:59:59 and set hours to 24 I get 1 record back with 7.0019449847179773.

       

      This makes me belive that there is some interpolation going on, but I can't see how this is calculated. If I want my data in 2hours intervals it looks like i take the value and multiply it with 2. 8 if I choose 8 hours...

       

      Is this a bug or is there something I have missed?

       

      Regards

        • Re: Bug or incorrect usage of AFCalculation.CalculateSummaries method
          bshang

          Hi Øystein,

           

          Thanks for your question. A couple of points here.

           

          1) AFSummaryTypes.Total will assume the tags units are in [x]/day when doing the totalization.


          2) When there are bad values (i.e. some values missing within the range), it will substitute the bad values with the average over the good summary range.


          3) Time-weighted totals will be calculated using the trapezoidal rule (assuming tag is stepped).



          With these points in mind, let's see how 7.0019449847179773 is determined.


          First, the input expression multiples the tag value by a conversion factor. I'll ignore what it's trying to do for now and just interpret the calculation based on the rules above (although I'd guess the input tags here are [x]/s). I will use x=gal for now as an example.


          For the time period 14:00 to 15:00, we can calculate the total as such:


          1*86400 gal/day * 6s * (1 day/86400s) + (0.5*86400 gal/day * 1day/86400s) * 2s


          = 7 gal


          The last term accounts for the boundary "triangles" due to the trapezoidal rule being used.


          Where does the remaining .0019449847179773 come from?


          Since there are no values before 14:00:01, the range 14:00:00 to 14:00:01 is considered bad and we substitute not with 0, but with the average over the good summary range (which is slightly more than zero). This average is 7gal/3599s and the total over that 1 second bad interval is .0019449847179773.


          I will work on trying to explain the rest of the results but let me know if you have any questions about the behaviors above.

            • Re: Bug or incorrect usage of AFCalculation.CalculateSummaries method
              Oysteinb

              Hi Barry

               

              Thanks for your reply.

               

              1. That's why I multiplied the value with 86400

              2. This will explain why I get the result doubled when I use 2 hours and times 8 when I use 8 hours.

               

              However since it does not looks like a bug, but incorrect usage is there a way to get the desired result? Or do I have to fall back and do the calculations myself?

                • Re: Bug or incorrect usage of AFCalculation.CalculateSummaries method
                  bshang

                  Hi Øystein,


                  What is the result you are looking for? Note that in general, choosing a larger time range will give a larger total (larger area under the curve).


                  To make the total calculation more "well-behaved", I'd make sure to have an archived value before the summary start time. Therefore, the boundary value can be determined via interpolation instead of the server assuming it is bad. That's why the 24 hr summary is larger (the substitution value for the bad range is being multiplied over the larger range).


                  It is interesting adding the event at 31.01.2015 23:59:59 gives the same result (=7.0019449847179773). What if you add an archived event of 0 at 31.01.2015 00:00:00 and then compute 1 hr and 24 hr summary ranges between Feb. 1 and Feb. 2? Is it the same result?

                    • Re: Bug or incorrect usage of AFCalculation.CalculateSummaries method
                      Oysteinb

                      Hi again

                       

                      What I want to achieve is to get the total value (i.e total rotation) pr. day. for the last week.

                       

                      Using the following expression solved my issue:

                       

                      string expression = String.Format(CultureInfo.InvariantCulture, @"If BadVal('\\{0}\{1}') < 1  then '\\{0}\{1}' * 86400 else 0", server.Name, point.Name);

                      and  using AFSampleType.Interval

                       

                      However I will have an issue when it comes to adding multiple points. The next thing I want to look into is setting the badval to a value for the whole expression.

                       

                      Regards

                        • Re: Bug or incorrect usage of AFCalculation.CalculateSummaries method
                          bshang

                          Hi Øystein,


                          That workaround was a good idea!


                          AFCalculation.CalculateSummaries supports PE syntax in the expression and for BadVal, you can reference multiple points or attributes depending on the target context. For example, you can have BadVal('tag1'*'tag2') or BadVal('attribute1'*'attribute2').


                          Is this what you are trying to do? Feel free to post the expression input for CalculateSummaries and we can take a closer look.

                            • Re: Bug or incorrect usage of AFCalculation.CalculateSummaries method
                              Oysteinb

                              Hi Barry

                               

                              What I would like to do is to set how AFCalulation should handle badvals. ie SetBadVal(0) or BadVal = 0. I can understand that it might be a correct way to use average when you have a gap in your data, but in my example where the first datapoint is at 14:00:01 then I think everything before that should be treated as 0. Or at least have the possibility to control the behavior

                               

                              One solution as you mentioned was to make sure that there is a datapoint so that AFCalculation could do a proper interpolation. For our solution that means before we insert our first value for a point we have to insert a 0 at ie 1970.1.1 and a 0 the second before the first value. For me this seems like a dirty hack.

                               

                              The other solution is to use PE syntax, but as I can't find a way to set the behavior of badvals I end up with a solution where the formula will be executed twice.

                               

                              Code:

                              string expression = String.Format(CultureInfo.InvariantCulture, @"if BadVal({0}) then 0 else {0}*86400", formula);