8 Replies Latest reply on Jun 8, 2012 12:19 PM by hamort

    PI SDK Calls: Is there a more efficient way?

    bacherc

      We are using AF to search elements so that we can get PI Tag and PI Server information for three tags. We then use this PI Tag and Server information to look up data for PI tags.

       

      We need the data in two different ways:

       

      1)      Interpolated

       

      2)      Hourly Sums

       

      This data needs to be retrieved for one day, with 24 values being returned – one for each hour in the day -- for the three PI tags. We are using the PISDK to do this as it was not apparent that AF had anything appropriate to use in order to retrieve data in this way and PISDK allows for just one call to retrieve 24 interpolated values one hour apart. Below is the PISDK code being used to retrieve the data for one PI tag (where Interval is “1h”; StartTime like 5/1/2012 01:00; EndTime like 5/2/2012 0:00).

       

                  Dim MyPoints As PISDK.PIPoints = PIServer.PIPoints

       

                  Return CType(MyPoints(PiTag).Data, PISDK.IPIData2).InterpolatedValues2(StartTime, EndTime, Interval)

       

       

       

      To get the summed data, three calls (again, one for each PI tag) are made as well to retrieve 24 values (where Interval is “1h”; StartTime like 5/1/2012 00:00; EndTime like 5/2/2012 0:00).

       

       

       

                  Dim MyPoints As PISDK.PIPoints = PIServer.PIPoints

       

                  Return DirectCast(DirectCast(MyPoints(PiTag).Data, PISDK.IPIData2).Summaries2(StartTime, EndTime, Interval, _

       

                          PISDK.ArchiveSummariesTypeConstants.asTotal, PISDK.CalculationBasisConstants.cbTimeWeightedContinuous)(1).Value, PISDK.PIValues)

       

       

       

      Interpolated and Hourly Sums are going to be retrieved many times (we need to do the above routine about 250,000 times a day, so 250,000 x 3 PI Tags, you can imagine that's a lot of calls), as much data needs to be retrieved. As such, efficiency is very important when retrieving this data.

       

      Is there a more efficient way to go about doing this via use of OLEDB, ODBC, AF, or even another implementation using the PISDK?

       

      Thanks in advance!

        • Re: PI SDK Calls: Is there a more efficient way?
          andreas

          Hi Christine

           

          Christine

           

          Interpolated and Hourly Sums are going to be retrieved many times (we need to do the above routine about 250,000 times a day, so 250,000 x 3 PI Tags, you can imagine that's a lot of calls), as much data needs to be retrieved. As such, efficiency is very important when retrieving this data.

           

          this is a very challenging requirement - could you share why you need to request that data that often?

           

          Christine

           

           

           

           

          Is there a more efficient way to go about doing this via use of OLEDB, ODBC, AF, or even another implementation using the PISDK?

           

           

          PI OLEDB will use the PI SDK, so the only gain in effency would be that the PI OLEDB developer spent time in how they use the PI SDK calls in the most efficent way. To improve the performance you could use asynchronous calls to PI SDK, or you might want to take a look in retrieving this data via AF SDK and stay independent from PI SDK at all.

           

          If you have installed the current AF SDK CTP, here is a little code snippet:

           
                  ' the PI server
                  Dim myPIServers As OSIsoft.AF.PI.PIServers
                  myPIServers = New OSIsoft.AF.PI.PIServers()
                  Dim myPIServer As OSIsoft.AF.PI.PIServer
                  myPIServer = myPIServers.DefaultPIServer
          
                  ' the Pointlist
                  Dim myPIPointList As OSIsoft.AF.PI.PIPointList
                  myPIPointList = New OSIsoft.AF.PI.PIPointList(OSIsoft.AF.PI.PIPoint.FindPIPoints(myPIServer, "CD*158"))
          
                  For Each myPIPoint In myPIPointList
                      Dim a1 As OSIsoft.AF.Asset.AFAttribute
                      a1 = OSIsoft.AF.Asset.AFAttribute.FindAttribute(myPIPoint.GetPath(), Nothing)
                      ' Interpolated values
                      Dim myValues As OSIsoft.AF.Asset.AFValues
                      myValues = a1.Data.InterpolatedValues(New OSIsoft.AF.Time.AFTimeRange("1-Jun-2012 01:00", "2-Jun-2012 00:00"), New OSIsoft.AF.Time.AFTimeSpan(hours:=1), Nothing, Nothing, False)
                      ' ALL Summaries
                      Dim mySummaries As Dictionary(Of OSIsoft.AF.Data.AFSummaryTypes, OSIsoft.AF.Asset.AFValues)
                      mySummaries = a1.Data.Summaries(New OSIsoft.AF.Time.AFTimeRange("1-Jun-2012 01:00", "2-Jun-2012 00:00"), New OSIsoft.AF.Time.AFTimeSpan(hours:=1), OSIsoft.AF.Data.AFSummaryTypes.All, OSIsoft.AF.Data.AFCalculationBasis.TimeWeighted, OSIsoft.AF.Data.AFTimestampCalculation.Auto)
          
                      For Each myValue In myValues
                          Console.WriteLine("{0}: {1}: {2}", myValue.Attribute.Name, myValue.Timestamp, myValue.Value)
                      Next
          
                      ' Totals only
                      Dim myTotals As OSIsoft.AF.Asset.AFValues
                      myTotals = mySummaries(OSIsoft.AF.Data.AFSummaryTypes.Total)
          
                      For Each myTotal In myTotals
                          Console.WriteLine("{0}: {1}: {2}", myTotal.Attribute.Name, myTotal.Timestamp, myTotal.Value)
                      Next
          
                  Next
          

           

            • Re: PI SDK Calls: Is there a more efficient way?
              bacherc

              Andreas,

               

              We need to retrieve the data so often because the customer requires data for approximately 250,000 instruments, where each instrument has 3 desired tags/attributes. I apologize as I do not believe I was clear that the data would be retrieved for 250K different objects in my previous posting. The data for these instruments will be retrieved in batches, not all at once -- a process will call, for example, 100 or 1000 instruments at a time, requesting data for these instruments.

               

              Thank you for the code snippet. It looks like Interpolation and Summaries are now available in a desired implementation in the CTP release. However, I am not sure the customer is open to using a CTP release.

                • Re: PI SDK Calls: Is there a more efficient way?

                  @Christine: what is the end goal of these interpolations and hourly sums? I mean, do you need these to be performed in real time or this is for post-analysis? I'm asking because another approach might be to load all the data for the last day - or the last hour - and then perform these computations on locally cached data. At the end of the day, it's all about minimizing the number of roundtrips to the PI Server - every roundtrip means network latency on the way in, some processing time on the server end, network latency on the way back, then data copying on the client end.

                    • Re: PI SDK Calls: Is there a more efficient way?
                      bacherc

                      Steve,

                       

                      Thank you for your input.

                       

                      The end goal is to report hourly data for each instrument -- to compile this data we need interpolated or summed (depending on the tag) PI Tag data from three PI tags.

                       

                      The sums/interpolations are performed the day after; i.e. data for 5/1/2012 is retrieved on 5/2/2012. So, this is post-analysis. We would need to get an entire day for a given PI Tag (3 tags x 250,000 total tags, so approximately 750,000 tags will need cached locally). Plus, when batch processing we will not know until passed what tags are required (to get the tags we are querying AFElements which we have already cached locally). Would caching PI that much PI data all at once (this would be before any batch calls requesting data are made) be a gigantic burden?