8 Replies Latest reply on Feb 22, 2012 12:59 AM by jlakumb

    PI Server time range event pipes - pinched from StreamInsight

      I have been day dreaming again and my favourite topic popped up, the PI Server.  Here is the latest thought...


      We currently have the Update manager providing updates of individual values for individual PI tags for consumers, but I wonder if it is feasible to extend the update manager to include some more complex data update requests.  My recent inspiration has come from those clever folks at the Microsoft StreamInsight team - I am in two minds right now if what I describe is perfect for StreamInsight and no worry for the PI Server PM/developers, but I reckon it might have a use case to be core functionality of the PI Server.


      I am talking about creating data subscriptions with the update manager for "give me the average of this tag using this relative time period '*' -> '*-1d', oh and send me the result each time a new event for the PI tag is received that falls within my relative time period".  Now that is exactly what StreamInsight can give you or even Performance Equations, but why can't the core PI Server do this too without the overhead of StreamInsight or the additional PI tag for PE?  So from the up and coming rich data access, when we call RecoredValues against an AFAttribute we have the option to create an EventPipe from the method (e.g. RecordedValuesEventPipe) - if we use relative time the event pipe is a moving one, but if we specify absolute time the event pipe is based on archive values that arrive (if any) in the time frame; a calculation expression could supply the calculation to be performed, etc.



        • Re: PI Server time range event pipes - pinched from StreamInsight
          Ahmad Fattahi

          Mind teasing ideas as always Rhys! My question is why you want to get rid of the overhead. Is the efficient PE still posing enough overhead for your application that warrants the built-in capability in Update Manager? We would lose some modularity/abstraction compared with the existing architecture. How would you justify that?

            • Re: PI Server time range event pipes - pinched from StreamInsight

              Contacted the PM (Jay Lakumb) to chime in.

              • Re: PI Server time range event pipes - pinched from StreamInsight

                Here is one case I was thinking of...


                If an abnormal event occurs at a plant then the first thing that is going to happen is users of the PI data are all going to start to analyse data leading up to the event.  Likelihood is they will tend to be looking at the same type of data, maybe even the same time periods for that data.  I'm thinking more along the lines of the PI Server recognising the pattern of data retrieval and setting up some internals caches of those data retrieval signatures, and maintaing a moving set of data for those time periods - with a some kind of timeout where if no user requests that data it is removed from the updating cache; you could indirectly sign up for updates to that data i.e. the client data access product does it in the background.  So it is not necessarily the case you would have set up PE tags or similar knowing this situation occurs, but more a case of the PI Server reacting to data retrieval demands which I guess would be "easy" to implement with a distributed PI Server where load could be shared amongst member servers for recognising patterns.


                If it doesn't fit within the PI Server then maybe something like Coresight would benefit from this, where data retrieval is from one place (the Coresight server) and you could even have StreamInsight integrated there so when the Coresight server recognises patterns, it will set up standing queries in SI until demand drops off and the queries are then stopped.  Coresight would have some built in logic to know to use the output from SI or directly from the PI Server.



                  • Re: PI Server time range event pipes - pinched from StreamInsight
                    Ahmad Fattahi

                    IMHO caching on the client side (Coresight in your example) could be beneficial. I don't know about the PI Server caching itself. The superior performance of the PI Server 2012 may also diminish the merits of such a cache on the server side.


                    It's yet to be seen what the PM team has to say about this

                      • Re: PI Server time range event pipes - pinched from StreamInsight

                        Yes, much of what you're describing is smart, efficient caching.  Today, we leverage caching at many levels within the PI System - PI Interfaces (point configuration), PI Server (write cache before data is written to disk, read cache for recently accessed data), PI Analytics (recent data to reuse in calculations), PI Data Access (point configuration), and PI Clients (actually PI Data Services which caches both PI data and non-PI data).  I probably left out some places but at least that gives you some idea we are always thinking of how to intelligently cache data/metadata in PI System where it makes sense, to provide optimal performance/scalability.


                        By the way, in case you are wondering, we take great pains to ensure the cached data is only accessed by those users who should have access to that data.  In other words, security is properly enforced.  Of course it may still be beneficial for application developers to create their own cache and manage it based on their needs.  For this reason we offer update manager and event pipes, which exposes changes in data and configuration (tags, modules, security, digital sets, etc.).  Hopefully folks are familiar with this feature and are taking advantage of this key strength of the PI System.  If you need more info, look up event pipes in the PI SDK documentation.


                        That said, what you describe is a combination of a dynamic, updating cache and analytic.  This seems like a good use case for PI for StreamInsight, since it combines the power of PI System adapters (reading/writing PI System data) with the flexibility of StreamInsight (describe analytic using LINQ, include StreamInsight engine in a custom application).  In other words, I am suggesting this use case might be best addressed by an analytics tool rather than the PI Server itself.  Now that I teed it up for Glenn (Analytics PM) I am sure he would have more to add to this discussion...