4 Replies Latest reply on Apr 9, 2018 6:42 PM by TimCarmichael

    How to merge multiple datastreams into one?

    Roger Palmen

      I have a great puzzle for the weekend for you!

       

      I have two (or more) PI Points, each delivering events. I need one datastream combining the events of both these. E.g. (timestamp, PointName, value):

      • 6-4-2018 10:00, Point1, 32.6
      • 6-4-2018 11:00, Point2, 93.1
      • 6-4-2018 12:00, Point1, 27.2

       

      Should be merged into:

      • 6-4-2018 10:00, Output, 32.6
      • 6-4-2018 11:00, Output, 93.1
      • 6-4-2018 12:00, Output, 27.2

       

      I can't really change the datasource, so i was looking for a solution in AF Analytics. Best thing i came up with is cascading HasValueChanged (event triggered):

      Input1: If HasValueChanged(Point1) then Point1 else NoOutput()

      Input2: If HasValueChanged(input2) then Point2 else input1

      Input3: If HasValueChanged(input3) then Point3 else input2

       

      But i am not a great fan of HasValueChanged, and not sure how Analytics treats multiple triggers at the same timestamp, and probable some other concerns too.

       

      Any better ideas?

        • Re: How to merge multiple datastreams into one?
          TimCarmichael

          I had a series of elements that were set up as event based on any input; my situation was similar to yours, and I'm hoping close enough to be able to help.

          So... the basic scenario was: if the primary input is good, use it, otherwise if the secondary input is good, use if, otherwise if the tertiary input is good, use it.. and finally, we opting to drop to 0 if all inputs were good, but it could just as easily be nooutput().
          So...

          if (TimeStamp('OutputTag') < TimeStamp('Input1') then 'Input1'
          else if (TimeStamp('OutputTag') < TimeStamp('Input2') then 'Input2'

          else if (TimeStamp('OutputTag') < TimeStamp('Input3') then 'Input3'
          else noutput()

           

           

          And, have the analysis triggered on Input1, Input2 and Input3.

          If a value is received for more than 1 input with the same timestamp, the first to arrive will be used.

          If you have a delay in receiving values for one of them, the value will only if there is no value for the input timestamp.

          2 of 2 people found this helpful
            • Re: How to merge multiple datastreams into one?
              Roger Palmen

              Hi Tim,

              Thanks for your input. Given this some thought and i think your approach is the best alternate that i can find. I need to have a look at out-of-order behaviour as the data is received from a LIMS system that does not necessary provide timestamps in order.

               

              Still not too happy i need to build a functional solution to a technical problem, but can't really find a better way.

                • Re: How to merge multiple datastreams into one?
                  Roger Palmen

                  And then the obvious way... it just hit me when putting this into the target system...

                   

                  For each input create an analysis that writes the data to an output attribute, specific for that attribute. But point all the output attributes to the same PI Point:

                  • Attribute input1 used by Analysis1 to copy data to attribute Output1 that uses PI Point 'target'.
                  • Attribute input2 used by Analysis2 to copy data to attribute Output2 that uses PI Point 'target'.
                  • repeat...
                    • Re: How to merge multiple datastreams into one?
                      TimCarmichael

                      That should work... not sure how the system will handle having multiple output attributes with the same tag, but, in a large system, I had multiple AF/AA servers to split the load and had some analyses inadvertently enabled on multiple systems - same analysis, same output tag and it did result in duplicate values, so... your alternate solution sounds feasible.