9 Replies Latest reply on May 25, 2010 3:12 PM by dsabin

    Best Way to Store Min-Avg-Max in PI

    dsabin

      I have a sensor that provides a minimum value, average value, and maximum value of numerous data streams every five minutes.  I am using PI SDK to add these values to a PI Server.  Here is a simple example of the values:

       

      Time, Min, Avg, Max
      2010-05-23 00:00:00, 0.99, 1.01, 1.02
      2010-05-23 00:05:00, 0.98, 1.00, 1.03
      2010-05-23 00:10:00, 0.97, 1.01, 1.02
      2010-05-23 00:15:00, 0.85, 1.00, 1.03
      2010-05-23 00:20:00, 1.01, 1.02, 1.04
      2010-05-23 00:25:00, 1.00, 1.02, 1.05

       

      What is the best way to store the minimum, average, and maximum values?  I could store the minimum, average, and maximum values for each stream in different tags, but this would require that I use three tags for each channel of the sensor, which is costly.

       

      I could store the minimum, average, and maximum value in a single tag for a specified period of time, offsetting the min, avg, and max slightly in time so that they can be distinctly queried.  But then it makes it hard to write queries that handle the values differently.  I thought that I could annotate the minimum values with a description that identifies them as so, but from what I understand annotations are not cached as afficiently as the samples themselves so that would not be optimal.

       

      Do other PI users have some suggestions?

       

      Thanks!

        • Re: Best Way to Store Min-Avg-Max in PI
          mheere

          Well, I have no doubt that you're reaction to this will be "of course that's what the OSI folks think!" - but my opinion is to use the tags.  That's what they're for and how the system is designed.

           

          By the time you count the cost of the meters themselves, the infrastructure (telecoms and servers), your development time, and the time it will take the users to figure out how to use some "cost saving" structure...how much money have you already spent?  Is the licensing cost of three tags really a big delta in the scheme of this?  Compared to the ultimate usability I'm betting the answer to that is "no".

           

          If you really need to conserve the tags the best solution is to find a way to pull the underlying raw data from the meter and store that in PI.  This takes only one tag, and PI will provide you with the min, max and average (as well as a host of other stats).

           

          My advice to all who use PI is generally to think in terms of value rather than cost.  This is harder to do because you have to think through the entire lifecycle of what you're working on, rather than just the implementation.  In the end though, this is how you get the best ROI from your PI system (not to mention your own time!)

           

          -Matt

            • Re: Best Way to Store Min-Avg-Max in PI
              cescamilla

              Matt Heere

              pull the underlying raw data from the meter and store that in PI

               

              I would definitively suggest you do this. It is really the best solution for your problem.

                • Re: Best Way to Store Min-Avg-Max in PI
                  Børre Heggernes

                  I Agree with the other suggestions you've had above, but would like to add you should consider the PI-UFL interface to read your data. Seems you are reading ASCII files?

                    • Re: Best Way to Store Min-Avg-Max in PI
                      MichaelvdV@Atos

                      How many of these timelines are we talking about? As suggested, using multiple tags is the best solution. It will increase your pipoint count, and maybe thus increasing the license. But consider this: developing custom code/applications to handle the other solutions will also cost time, and will increase risk.

                        • Re: Best Way to Store Min-Avg-Max in PI
                          dsabin

                          I work with sensors that produce about sixty streams with data rates of a new min-avg-max sample every five minutes.  So it would be the difference between 60 tags per sensor and 180 tags per sensor.  Typical systems would have about 200 to 500 of these sensors, so it would mean the difference between 30,000 tags and 90,000 tags.

                           

                          I agree that handling the data in any other way than a simple tag would increase the complexity of my code; I was wondering if I was missing something. 

                            • Re: Best Way to Store Min-Avg-Max in PI
                              andreas

                              Daniel,

                               

                              I am not so sure that you have to store min,max,avg in PI - usually the PI System is fast enough to calculate min,max,avg on the fly - so you would store only the raw data, and when you need it you would use a dataset (PB) or calculated data (DL), etc.

                               

                              From you comment I could not tell if you have access to the raw data or not.

                               

                               

                                • Re: Best Way to Store Min-Avg-Max in PI
                                  Børre Heggernes

                                  The way I understand it he is not able to get to the raw values....

                                   

                                   

                                   

                                  Anyway Daniel, the price diff is not that great between a 30K and a 90K system Contact your local sales rep

                                  • Re: Best Way to Store Min-Avg-Max in PI
                                    dsabin

                                    Some of the meters do indeed have a interface like MODBUS that would allow another system like PI to get real-time values.  Assuming that PI could poll the meter fast enough, then we could use use its built-in functionality to derive a minimum, average, and maximum.  However, the meters have built-in functionality to log the minimum and maximum value, which might be experienced for only 10 to 20 milliseconds.  Scanning by MODBUS would probably miss the min and max value during these short but very important intervals.

                                     

                                    Thank you for your suggestion.  It would indeed be best to connect PI to the meter, but I do not think it is practical.

                              • Re: Best Way to Store Min-Avg-Max in PI
                                dsabin

                                No, not ASCII files.  I develop a system that integrates data from about fifty different systems.  Some produce binary flat files, some produce ASCII files, some store their data in SQL Server, some store their data in Oracle, and some store them in proprietary databases.  Each takes a different approach to storing time series data, so there is a querying/data cracking/remodeling step that is first completed by my application before the data looks like the simple table that I posted here.

                                 

                                PI-UFL looks like it might be useful for a few of the sources.  Thanks for the tip!