6 Replies Latest reply on Sep 23, 2017 8:31 AM by Jagan

    Getting Plotdata with filter expression

    Jagan

      This is about how to get the data to plot in most quickest/efficient way in two cases. One without any filter expressions and one with filter expression. Suggestions welcome. Here the accuracy of data is less important. What is important is to get the data in shortest possible time.

       

      -------------------

      Case1:

      Without any filter expression

      //String path = “temperature”;

      //Get context from path

      //AFContext context

      //AFTimeRange timeRange

      //UOM desiredUOM

      Int Intervals = 200;

      AFValues afValuesForPlottingCurve =  context.Attribute.Data.PlotValues(timeRange, Intervals, desiredUOM)

      afValuesForPlottingCurve are only used to show a plot on the screen. Is there any better/faster way to fetch the data?

      -------------------

      Case2:

      With filter expression the logic to get values for plotting curve has been this

       

      String filterExpression = “temperature > 20 “;

      AFValues afValuesForPlottingCurve =  =  context.Attribute.Data.RecordedValues(timeRange, AFBoundaryType.Interpolated, desiredUOM, filterExpression, true)

      AFValues  finalResutlForPlotting = afValuesForPlottingCurve.PlotValues(timeRange, Intervals);

       

      The disadvantage of this has been Case2 has been

      A. If timeRange is for longer duration say more than a year and if the data point has lot of values say 500 thousand+ values then only one value is returned with status: Event collection exceeded the maximum allowed

      B. Even if maximum limit is not exceeded this logic is trying to get/load lot of data here. Is there any better way to do this? possible to use PlotValues() along with filter expression?

      C. The data point on which this logic is applied are completely different from one another. Some might have data for every second, while some have data only once in 5 minutes. Also the time range differs for every call. Time range could be just 1 hour or it could also be 1 year. So should the logic to get data be changing based on amount of data present inside data point and based on the time range?

      -------------------

      Thanks,

      Jagan 

        • Re: Getting Plotdata with filter expression
          Rick Davin

          I think required reading should be KB01178 to have a foundation on filtered data.  For the plotting of data, PlotValues is the recommended method since after all the name says it all.  Your Case 2 is not the quickest nor most efficient way since its PlotValues call is done locally on the client.

           

          Also note that filtering via RecordedValues or retrieving data from PlotValues depends upon the attribute's data reference.  If those attributes use the PI Point data reference, this is not an issue.  But if these attributes use some other data reference, particular a custom DR, then the true efficiency and correctness of data rely upon the implementation of the CDR.

            • Re: Getting Plotdata with filter expression
              Jagan

              Thanks Rick. In my case all the attributes are using PI Point data reference. The problem with the Case 2 has been that, it' been time consuming at times when the time range is longer and there is lot of data in PI Point. The other problem is at times is only one value is returned with status: Event collection exceeded the maximum allowed.

                • Re: Getting Plotdata with filter expression
                  Rick Davin

                  I think you have answered your own question.  Go with Case 1, that is PlotValues.

                  • Re: Getting Plotdata with filter expression
                    vkaufmann

                    Sounds like your filtered call is running into the MaxArcCollect parameter on your PI server discussed in this KB: KB00646

                     

                    It seems pretty straightforward to me that if you are retrieving data to show on a plot somewhere then the PlotValues function is going to be your best bet. If the filtered values are important enough for your plot, you should consider creating an analysis that pre-filters that data that can then be retrieved by your PlotValues call. For best performance, you should write that filter data into a PI Point.

                     

                    --Vince

                      • Re: Getting Plotdata with filter expression
                        Jagan

                        Thanks Rick and Vincent for Comments.

                        Let me give more details. The code that I mentioned in the beginning of post runs as a custom API on the same server as AF. Then Custom UI makes API calls to get the data. Case 2 is still valid because custom UI should support use case which allows user to modify the filter expression on the fly and see the plot. For example this minute user wants to see the plot of temperature for last year having the values between 20 and 25. Then user is allowed to modify the expression in a next minute to see the plot of values between 20 to 30 or any other range. Also UI is providing flexibility to change time range from last minute to couple of year. Due to this not able to use analysis to pre-filter the data.

                  • Re: Getting Plotdata with filter expression
                    ernstamort

                    You could do the filtering of PlotValues in .NET ...?