7 Replies Latest reply on Mar 13, 2015 7:28 PM by aseck

    Basic question about how to pull data using the RESTFul API

    skedastic

      I’m very new to OSI, and have some hopefully basic questions about using the RESTFul API. 

      Our goal is to poll all devices for data on an interval, say every 100 ms and gather key metrics.

       

      It looks you can use the GET on  '/elements', and start drilling down into each of its 'children', to determine what devices are available that can provide data.

       

      So we can use that to build a map of the ids of each of the sensors, but the question is, what is the most effective way to get the current data for an element?

       

      I can see that when I have an ID I can request its snapshot.. which gives all attributes for an element, so now I have all devices and their possible readings (temperature, ect..), plus the current value.

       

      GET /snapshot

       

      Is there a different way to do this, or a more effective way to poll all sensors for data in realtime?

       

      I see there are also APIs for points and eventFrames, are either of them preferred?

      Is getting a Point via an ID the best programmatic way to get data?

      Whats the difference between a point and an eventFrame?

       

      GET /points

       

      GET /eventFrames

        • Re: Basic question about how to pull data using the RESTFul API
          sdriscoll

          Each of the GET requests you mentioned are for fundamentally different parts of the PI Server.

           

          The two main components of the PI Server are the PI Data Archive and the PI Asset Framework (AF). snapshot and points will access the PI Data Archive. elements and eventFrames will access PI AF. AF elements and event frames contain attributes which can be static values, pipoint data (from PI Data Archive), or a few other things. So you want to use the associated GET type to query a specific type (i.e. GET /eventframes for PI AF event frames or GET /snapshot for PI Data Archive snapshot values).

           

          We don't recommend grabbing and storing ID's as the best programmatic access. We recommend using the links item embedded in each JSON object.

          [EDIT] There have been several use-cases where it was necessary to store the webID's. The PI Web API webIDs are immutable and can be stored and reused when necessary. To store or not to store - a question which I do not have the answer for unfortunately. It will depend on the system architecture, network infrastructure, whether your PI System is semi-static or constantly changing, and most notably the design and usage of the application. Keep in mind some of the previous limitations that may have necessitated storing of the webID's are now gone: bulk calls and batch requests (as a CTP) are supported in 2015 R3.

           

          As an example, making a GET dataservers/{webid}/points request with a name filter, like so,

          https://<yourWebAPIServer>/piwebapi/dataservers/s0i025R-Sdg0mGFCKfjA0SygRElTQ09JTkZFUk5P/points?namefilter=sinusoid

          gives the following JSON object response,

          {

            "Links": {},

            "Items": [

              {

              "WebId": "P0i025R-Sdg0mGFCKfjA0SygAQAAAARElTQ09JTkZFUk5PXFNJTlVTT0lE",

              "Id": 1,

              "Name": "SINUSOID",

              "Links": {

                "Self": "https://webpotpie.osisoft.int/piwebapi/points/P0i025R-Sdg0mGFCKfjA0SygAQAAAARElTQ09JTkZFUk5PXFNJTlVTT0lE",

                "DataServer": "https://webpotpie.osisoft.int/piwebapi/dataservers/s0i025R-Sdg0mGFCKfjA0SygRElTQ09JTkZFUk5P",

                "Value": "https://webpotpie.osisoft.int/piwebapi/streams/P0i025R-Sdg0mGFCKfjA0SygAQAAAARElTQ09JTkZFUk5PXFNJTlVTT0lE/value",

                "InterpolatedData": "https://webpotpie.osisoft.int/piwebapi/streams/P0i025R-Sdg0mGFCKfjA0SygAQAAAARElTQ09JTkZFUk5PXFNJTlVTT0lE/interpolated",

                "RecordedData": "https://webpotpie.osisoft.int/piwebapi/streams/P0i025R-Sdg0mGFCKfjA0SygAQAAAARElTQ09JTkZFUk5PXFNJTlVTT0lE/recorded",

                "PlotData": "https://webpotpie.osisoft.int/piwebapi/streams/P0i025R-Sdg0mGFCKfjA0SygAQAAAARElTQ09JTkZFUk5PXFNJTlVTT0lE/plot",

                "SummaryData": "https://webpotpie.osisoft.int/piwebapi/streams/P0i025R-Sdg0mGFCKfjA0SygAQAAAARElTQ09JTkZFUk5PXFNJTlVTT0lE/summary"

                }

              }

              ]

          }

           

          You could then grab one of the Links by doing something like JSON.items[0].Links.Value

           

          If you were grabbing some recordeddata you could also append some filters on the Link as necessary. Also the same goes for grabbing the particular PI Data Archive or PI AF server that you want to work with, or the database for that matter.

           

          Please let us know if you have more questions or need further clarification.

          1 of 1 people found this helpful
          • Re: Basic question about how to pull data using the RESTFul API
            sdriscoll

            I have moved this thread to RESTful PI System Access

              • Re: Basic question about how to pull data using the RESTFul API
                Lonnie Bowling

                Hi Derek,

                 

                You might run into performance issues with the polling rates you are wanting. Something to think about is how fast is the underlying data actually changing. You really won't need a polling rate much faster than that. The next step is to do bulk calls, but the PI Web API does not support this feature. You would have to use the older PI Web Service. The best approach is to actually push data to the clients, like what you see in a chat room. That way only data changes are sent and the server is not bogged down with handling continuous client requests. To do that you will have to write some server side code, most likely subscribing to data events using the AF SDK. From there you will have to employ a realtime web technology, like SignalR, which uses websockets and a couple of other techniques to establish a connection to the client. Obviously this gets pretty complicated pretty fast... I'm working on an example of this, at some point I want to post some code on this when I'm finished.

                 

                Lonnie

              • Re: Basic question about how to pull data using the RESTFul API
                dng

                Hi Derek,

                 

                Sean provided a good explanation of the different data streams. In general,

                • If you are not using AF, get values from PI Points.
                • If you are using AF and have AF attributes mapped to those PI Points (using the PI Point Data Reference), get values from the attributes.
                • If you are using AF and your attributes are associated with specific event frames, get values from the event frame attributes. Event frames are generally used to store the start and end time for specific events (e.g. downtime) and attributes associated with event frames usually contain values important for that specific event frame.

                 

                With that said, there are many ways to get values. The determination on where to get values depend on whether the data is organized in AF (then you will want to use attributes) as well as where the data ultimately lie (e.g. whether it's associated with an event frame). Where does your device data reside?

                 

                From your description (poll all devices for data on an interval), you are likely looking at getting the interpolated values or recorded data. Interpolated values provide data in a regular time interval, while recorded values provide archived data within the specified time interval. If you are interested in getting the most current value (snapshot), use the "value" stream.


                If you are using PI Web API 2014 R2, a general rule of thumb is you have to retrieve values from each data stream individually (you can however retrieve multiple values in the same data stream, like in the recorded data call). PI Web API 2015 (scheduled for 2015 Q1 release) provides some bulk data retrieval functionality, mostly for attributes from the same parent element or event frame.


                If you are interested, there are some great blog posts about using PI Web API:

                1 of 1 people found this helpful
                • Re: Basic question about how to pull data using the RESTFul API
                  bshang

                  Hi Derek,

                   

                  Some good points have already been provided in this thread. Here are some of the issues to consider:

                   

                  1. As Lonnie mentioned, the 100ms polling interval could be a concern. Even with a bulk call (one round trip to the server), the call (and subsequent processing) may not be done in time. I've seen cases where hundreds of threads have been created as the application tries to keep up with the frequency of requests.

                   

                  2. What is the frequency of events on the data source? If you want to receive every value since the last call to the server, you can use AF SDK's AFDataPipe and AFDataCache. Therefore, you only need to call every 5 seconds for example, and get all the events from the data source since then (even at 100 ms resolution), avoiding the need to poll more frequently.

                   

                  3. What sorts of processing or calculations will be done with the data after it has been received? Will the raw data be stored in another database or just the calculation result? If it is a one-pass calculation through the retrieved values collection, the AF SDK has been optimized to support this use case with higher performance.

                   

                  4. Related to 3, if the application can be considered real-time or requires "high performance", then AF SDK really is the best option. One reason is that by working directly with the AF SDK, you are avoid unnecessary conversion of object types. For example, with PI Web API, the conversions might look like

                   

                  PI server--->network--->native PI events--->MDA events--->enumerable of AFValues--->JSON string--->network--->client object type--->calculation--->discard

                   

                  Note it is grossly over-simplified above but hopefully illustrates the idea.

                   

                  By working directly with the AF SDK, at least one object conversion is avoided. In addition, you will have greater flexibility for event signups, caching, paging, enumeration, etc. to optimize data retrieval.

                   

                  Therefore, if the application is intensive and real-time, then my recommendation is to consider the AF SDK first. However, there may be other requirements (HTTP/S only, web/cross-platform environment, etc.) here that preclude its use.

                   

                  5. If PI Web API must be used, for a bulk call, please refer to GET /streamsets for the 2015 CTP. As Daphne mentioned, this maps to a bulk call against attributes under a single parent element.

                  • Re: Basic question about how to pull data using the RESTFul API
                    aseck

                    Hello Derek,

                    Did the answers provided through though this thread answer your question satisfactorily? If so, it would help others if you mark the answer that helped you as correct, otherwise you can give us more details so we can further assist you.

                    Thanks!