4 Replies Latest reply on Apr 19, 2012 7:00 PM by RJKSolutions

    Lack of "PI" UDAs

      What are the plans, if any, to provide some of the PI related aggregates that we see in Performance Equations in to StreamInsight as User Defined Aggregates?  Seems that we can get PI data in, but then any common functions that are used in PI (typically PEs) are not exposed.  This leads to them being custom developed and potentially different to algorithms to what PI is using.


      Standard Deviation in PE is a good example.  I see there are examples of creating an UDA extension class for standard deviation out on the web for StreamInsight, but wouldn't it make sense to have all functions from PE, Notifications, etc available to StreamInsight?


      What are others doing?  Creating your own framework of UDAs?

        • Re: Lack of "PI" UDAs

          Hello Rhys,


          No plans to include this functionality in the adapters as it would appear at this time to be more appropriate in the StreamInsight engine, as a UDA for example - as you mention. The PI for StreamInsight adapters focus is on reading and writing data with the PI System.


          A library of UDAs targeting time series data? This would certainly compliment what StreamInsight is today and we have heard this feedback previously. Thanks for asking!


          That said, depending on what you want to do, you may be able to do that today utilizing an existing calculation engine of the PI System or via our SDKs


          One could ponder the idea where there is the ability to define a Performance Equation (PE)  (or more generally a calculation) and then specify or have the system determine the best place to run that (near the data - ala big data; on a dedicated engine engine - PE engine/PI ACE; or close to the data source - edge).
          This is just an idea...



            • Re: Lack of "PI" UDAs

              Thanks for the information, Glenn.


              As an experiment (remember, I'm a StreamInsight n00b right now) I created some basic UDAs just to look at some basic functions for now - not necessarily looking at what Performance Equations deliver.  It is actually easier than I thought to build and then consume the UDAs.  So to cut any development right down for statistical functions or any other more advanced math functions I decided to use the open source Math.Net project library.  As I am using this my code is minimal and extremely quick to develop, and I can use the functionality against data streaming from PI.


              Here is the [sarcasm]mammoth[/sarcasm] code to implement a Standard Deviation UDA:




              using MathNet.Numerics.Statistics;




              namespace MathNet.Numerics.Statistics.For.StreamInsight




                  public static class StandardDeviation






                      publicstaticdouble StdDev<T>(thisCepWindow<T> window, Expression<Func<T, double>> map)










                  public class StandardDeviationDouble : CepAggregate<double, double>




                      publicoverridedouble GenerateOutput(IEnumerable<double> source)














              Here is an example query using the UDA:



              var stdDevOutput = from e in dataStream
                                                group e by e.Path into pathGroup
                                                from window in pathGroup.TumblingWindow(TimeSpan.FromSeconds(30), HoppingWindowOutputPolicy.PointAlignToWindowEnd)
                                                select new { Path = pathGroup.Key + ".STDEV_SI", Status = 0, Value = window.StdDev(v => v.Value) };







              I think I am falling in love with StreamInsight.


              Maybe I have some questions being formed in my head over CTI and the PI Adapters getting events from the Snapshot (with exception testing), but I need to think on that some more.


              Do you think there is any value in a community project library that is just a StreamInsight wrapper for something like Math.Net, with examples to show how to use some of the functions against streaming PI data?




              Edit:  The code snippet thing sucks, it won't work properly for my post above.