15 Replies Latest reply on Jul 13, 2016 12:08 PM by Marcos Vainer Loeff

    Streaming data out of PI

    Roger Palmen

      Hi community,

      Probing if there are some inputs i can use to solve my latest task. The challenge is simple: send all snapshot values from PI out to an external system (not PI). The context provides the complexity: do that every minute for 1M tags.


      Let's see what we have available:

      • PI Cloud connect: Nah, only works if target is PI and we would need multiple nodes to handle the scale.
      • PI Web API: Could work, but not sure if there is any benefit over using AFSDK
      • PI ODBC / OLEDB: Could work, but really doubt if that would scale up to the required size
      • PI2PI: Would probably work, but only works if target is PI


      Is the only option i have left the AFSDK? Do a signup to all my tags and just process the events or should i just pull the snapshot every minute? Do i need multi-threading or multi-process? Lots of details to consider... just getting started on my ideas.

        • Re: Streaming data out of PI
          Rhys Kirk

          Assuming you are doing this for a client...then PI Cloud Connect might make some sense. If the destination is a cloud system then the exit point from your client could be via PI CC to your own hosted (intermediate) PI System. Then you probably have a bit more freedom as to how you extract and push the data to the end destination.


          Otherwise...yes, AF SDK is your new best friend. After all, that is all OSIsoft would do right inside of a PI Integrator/Connector (albeit using an intermediate protocol for buffering).


          Do we get a clue as to the destination?

          I've seen some neat developments with systems like Splunk that let you get directly at data in PI. In that case there is still a Windows Service, using AF SDK, involved to push data to Splunk.

          1 of 1 people found this helpful
          • Re: Streaming data out of PI
            Marcos Vainer Loeff

            Hi Roger,


            I don't agree with you that PI Web API hasn't any benefit over PI AF SDK for streaming data. If you can't develop in .NET Framework or if you can't install PI AF Client where your custom application would be running, PI Web API is for sure a good choice.


            Nevertheless, most of our PI Developer Technologies are developed on top of PI AF SDK. Therefore, PI AF SDK is the best option when are talking about performance, which in this scenario is very important.


            As you want to get all snapshots, there is no better programmatic option than using AFDataPipe. PI Web API Channels is still on CTP. Our development team is testing it for a large number of streams.


            Hope this helps!

            2 of 2 people found this helpful