7 Replies Latest reply on Dec 2, 2013 8:30 PM by Roger Palmen

    PI Calculation Best Practice

    DavidMFairchild

      I am seeking the opinion of my fellow PI Administrators and OSI Professionals regarding choosing which PI Calculation technique to use under which circumstances.  With 13 years of PI Administrative experience I do have some opinions of my own.  But thought I would get yours.  I understand that this is a broad topic and not intended to address how to solve any problem.  Of the three calculating tools offered, PI-Performance Equations, PI-Totalizer, and PI-ACE what guidelines would you use for determining when to use each?

       

      Additionally, with PI-ACE is it preferrable to use one large Module with many calculations or a separate module for each calculation?  (Assuming many contexts per calculation in either case.)

       

      If there is already a good blog or discussion topic on this, please let me know.

        • Re: PI Calculation Best Practice
          Roger Palmen

          Hi David,

           

          Do you only need to perform a calculation, or do you also need to persist the calculations? And are these persisted calculations then timeseries data?

           

          In most cases, i prefer to use AF and not to persist the calculations. But that´s not always an option due to performance. It is impressive though how fast AF can produce data for a trend plot using large sets of calculations, table lookups, etc., so don´t dismiss this too quickly.

            • Re: PI Calculation Best Practice

              Hi David,

               

              PI Totalizer is considered the more accurate calculation engine compared to PI Performance Equation Scheduler because it includes snapshots. There's however an issue causing that events are not considered for periods PI Totalizer subsystem isn't running. The issue causes that rebooting a PI Server node e.g. because of Windows Patching causes Totalizer tags reporting wrong results until the next reset. For this reason I recommend using Totalizer subsystem only for calculations that have a reset at least once a day. The flexibility of PI Totalizer Subsystem is one of its downsides at the same time. For my understanding there are by far too much screws to drive.

               

              PI Performance Equation Scheduler is my personal preference as long as equations do not become too complex. With the help of PI Performance Equation Reference manual, it is pretty easy to set up equations.

               

              PI ACE is the engine I would use for more complex calculations or those that need to be applied to many contexts. Even the calculations require a development environment (Visual Studio) there's usually no deep programming knowledge required to code calculations. While PI Totalizer Subsystem and PI Performance Equation Scheduler run on the PI Server node PI ACE Scheduler can run on one or more separate PI ACE Servers. Redundancy and n-way buffering are supported.

               

              David M. Fairchild

              Additionally, with PI-ACE is it preferrable to use one large Module with many calculations or a separate module for each calculation?

               

              The ability to apply an existing calculation to other contexts without the requirement to write the same code again and again makes PI ACE very powerful. This however requires creating modules in module database and aliases pointing to PI tags. Reading from several dozens of PI Tags within a single ACE calculation is no good practice from my point of view. 

                • Re: PI Calculation Best Practice
                  lZheng

                  Hi David,

                   

                  In additional to what Gregor has mentioned, here are some more information

                   

                  About PE:

                   

                       - Tags in PE Syntax are specified explicitly by the name, which means if tag name is changed the calculation breaks

                   

                       - There is a limitation of the PE formula string.

                   

                       - PE Calculation cannot be used for other sets of tags, no context calculation available

                   

                       - Cannot create new calculation frequency/scan class without reconfiguring PIPESCHD.bat and restarting service

                   

                       - PE can only read and write to one Pi Server

                   

                       - Only one trigger tag for PE.

                   

                  About Totalizer

                   

                       - It uses snapshot data, therefore more accurate than PE

                   

                       - It has persistent memory to hold the running summary totals, therefore more efficient than PE in some cases

                   

                       - Totalizer holds tag by tag number instead of tag name, so renaming tags won't break the calculation

                   

                       - Like PE, totalizer can only read or write values from the PI server that it's currently running on.

                   

                       - There is no recalculator for Pi totalizer, for PE you can use PI Recalculator.

                   

                  About ACE

                   

                       - Easy to configure and efficient Scheduling & triggering

                   

                       - Can read and write data to different Pi Servers

                   

                       - High performance with compiled ACE dlls.  You can spread the load across multiple ACE Scheudler engines.

                   

                       - You can create one calculation and apply to many tag sets through PI MDB aliases, power of Context.

                   

                       - Compatible with PI Server High Availability and support HA over PI ACE Engine.

                   

                       - Easy to perform recalculation, support sophisticated debugging, logging, and performance diagnostics.

                   

                       - Flexibility of application, allow you to write complex calculation with .net language, integrate with 3rd party DLLs.

                   

                  To your question I would also use a separate module for each calculation, it can make the deployment of calculations to other contexts/modules easier.

                   

                  Hope this helps.

                    • Re: PI Calculation Best Practice

                      With AF 2.6 (Abacus) now in beta and slated for early 2014 release, several of the above discussions becomes moot.

                        • Re: PI Calculation Best Practice
                          DavidMFairchild

                          I thank you all for your replies and some excellant information.  I wanted to keep this generic and so I didn't go into our situation here.  Let me say that we have a crazy PI architecture that I inherited when I took this position.  I am currently pushing to move to a 4 node HA system in 2014 adding AF which we don't currently have.

                           

                          In response to Roger's reply, AF is not viable at the moment but should be in the future.  I see AF primarily as a tool to model our infrastructure.  But I am interested in it's efficiency in performing on-the-fly calculations vs. calculating a result and storing it in PI vs. on-the-fly calculations done in PI-Processbook using DataSets.

                           

                          One of the issues I deal with is tag name changes.  I typically see around 100 tag name changes every week.  Also I typically get 200 - 300 new tags each week with around 100 tags that are dropped from our SCADA system.  Some of the Add/Delete tags are actually tag name changes that I don't know about.  Occassionally I will find such a change weeks after it happened and am able to merge the two tags back into one.  Unfortunately, I don't believe there is a PI-PE function for finding the tag name of a PointID or that limitation could be resolved.  I would also be interested in how the Module Database or PI-AF handle PI Tag Name changes.

                           

                          Based on the discussions here, I think the inability of PI-Performance Equations and PI-Totalizer to write to multiple servers in a collective will drive me or anyone else using or migrating to an HA system to use PI-ACE exclusively.  I may have mis-understood Gregor and Lingli's comments though.  It occurs to me that OSI could concievably have designed either product to run on an interface server and write to all servers in a collection using the buffering system.

                           

                          On the idea of using a separate module for each calculation, I am glad to see that responses so far tend to agree with me.  Unfortunately, we have one Module with 17 poorly designed calculations.  It seems to me that in addition to improving deployment, using multiple modules may also improve performance by not running all of your calculations in the same process space.  This should make better use of multiple processors and extra memory.  

                           

                          It is my intention to install PI-ACE on Interface Servers and not on the PI-Server.  This will eliminate the requirement to run buffering on the PI Server, aid in compartmentalization (restricting the usages of a Windows server in order to improve the reliability of the server) and eliminate the need to reboot the PI server if an ACE calculation becomes hung.  (Note, this may not be an issue any more, but earlier versions of PI-ACE tended to require restarting the server in the event of a hung process.)  I also intend to run ACE on two or more servers taking advantage of multiple schedulers with failover to achieve redundancy and load sharing.

                            • Re: PI Calculation Best Practice
                              skwan

                              David:

                               

                              David M. Fairchild

                              In response to Roger's reply, AF is not viable at the moment but should be in the future.  I see AF primarily as a tool to model our infrastructure.  But I am interested in it's efficiency in performing on-the-fly calculations vs. calculating a result and storing it in PI vs. on-the-fly calculations done in PI-Processbook using DataSets.

                               

                              There are of course PRO and CON to on-the-fly calculations vs. pre-calculating and storing the results as PI tags.  Here are a few (non-exhaustive) points to consider:

                              • On-the-fly calculations are typically done client side, which means you incur the overhead of data transfer from data sources to client.  Now you have to do this every time someone initiates a calculation - often times you're reading the exact same inputs from the data source.  You're putting a load on each client machine, which may be doing the same calculations with the same results every time.  This becomes prohibitive as your calculations get complex or you have many calculations or many users (putting a load on your data sources and also over the wire).  As an example, we had a customer who was using 40,000 values in a client side on-the-fly calculation.  Let's just say it was less than performing.  You are at the mercy of the # of round trips to the slowest data source.  You are, however, guaranteed to have the latest inputs to your calculations.  This may or may not be what you want.  Imagine you have 500 DataLink sheets that are using the same known value for years and all of sudden this value changes.  Again, depending on your needs, this may or may not be desired.
                              • Pre-calculations and storing the results as PI tags will in most cases provide the fastest response as you're simply retrieving already calculated results.  You achieve performance because you're retrieving less values over the wires.  You're potentially not having to recalculate the same results over and over again.  However, if you have out of order data, the pre-calculated results may be incorrect requiring a recalc.  Doing the calculations on the server side also enables you to take advantage of the (typically) beefier hardware compared to a client.

                              Maybe I'm biased, but I see AF as more than just a tool to model your infrastructure.  It allows you to aggregate data from disparate sources into a common and standard way of looking at your data.  As an example, users don't need to memorize tagnames and also don't need to know where the data came from, whether it's a PI tag or from an external SQL database.  In addition, many of our users take advantage of AF templates, event frames and UOM support, just to name a few, in their applications and also our own (OSIsoft) clients.  With AF 2014, we're seriously expanding its analytics capabilities, which we're calling Asset Based Analytics.  I urge you to take a look at the AF 2014 beta, available from the download section of vCampus.

                               

                              David M. Fairchild

                              One of the issues I deal with is tag name changes.  I typically see around 100 tag name changes every week.  Also I typically get 200 - 300 new tags each week with around 100 tags that are dropped from our SCADA system.  Some of the Add/Delete tags are actually tag name changes that I don't know about.  Occassionally I will find such a change weeks after it happened and am able to merge the two tags back into one.  Unfortunately, I don't believe there is a PI-PE function for finding the tag name of a PointID or that limitation could be resolved.  I would also be interested in how the Module Database or PI-AF handle PI Tag Name changes.

                               

                              AF PI Point data reference currently prefers PointID over point name.  Therefore, if your users are accessing their data via AF, underlying name changes to PI tagnames wouldn't matter.  (Having said that, there is an issue with PI System Explorer in displaying the changed tagname, which we're fixing.)

                               

                              David M. Fairchild

                              Based on the discussions here, I think the inability of PI-Performance Equations and PI-Totalizer to write to multiple servers in a collective will drive me or anyone else using or migrating to an HA system to use PI-ACE exclusively.  I may have mis-understood Gregor and Lingli's comments though.  It occurs to me that OSI could concievably have designed either product to run on an interface server and write to all servers in a collection using the buffering system.

                               

                              I would urge you to take a look at the AF 2014 beta, which includes new capabilities to perform calculations using AF attributes.  The configuration follows PE syntax so it should be familiar to you.  The AF 2014 beta also includes a new (CTP) version of BufSS which allows the calculation results to be written to PI HA Collectives.  We feel the capabilities of these new capabilities would allow many of our users to build their calculations via a configuration experience and not requiring a programmatic experience while attaining improved scale and performance over using PI ACE.  In addition, we're removed some limitations and added some capabilities compared to PI Performance Equations, like sub-second calculation support.  Having said that, we realize some customers will require the flexibility of programmatic analytics, which we plan on addressing in future release(s).

                                • Re: PI Calculation Best Practice
                                  Roger Palmen

                                  Agree with Steve on his feedback on my preference to use AF. Instead of writing "in most cases", i should have written "in very tighly defined conditions". Using AF to run your calculations (meaning, the client runs the calculations) is only viable when e.g. your entire SW stack and usage patterns allow to use AF as the primary definition and excution point of your calculations.

                                   

                                  My apologies for being over-enthousiastic about AF! But AF2.6 with Abacus is quickly turning this around. Keen to do a large project with AF2.6.