13 Replies Latest reply on Jun 21, 2016 9:04 PM by skwan

    TimeEq without mapped output to save output history

    calmandrysdale

      Hello,

       

      I have an AF analyses producing some stats on alarms that I do not wish to historise.

       

      For example, I am wanting to just see but  not store the time an alarm has been on over today, yesterday, last week, last month etc.

       

      I have set this up in an AF Analyses to use the TimeEq function. Works beautifully, I can Evaluate it and it all works nicely.

       

      However, when i then got to map the output to a non-pi-point attribute i get the error:

      'Functions such as TimeEq are not supported unless all mapped outputs are configured to save output history'.

       

      This is a total pain. I have no desire to store these values just want to see the latest output from the analyses each day.

       

      any suggestions on how I could make this work without pi points.

       

      Thanks,

       

      Calman

        • Re: TimeEq without mapped output to save output history
          Kenji Hashimoto

          If you want to see just snapshot of value, then you can use Data Reference:<None> for calculating such expensive functions. (TimeEq)

          These expensive functions cannot be stored as a type <Analysis> for ad hoc calculations. So need to create PI tags for viewing historical calculated values.

          Followings are expensive functions.

          DeltaValue

          FindEq

          FindGE

          FindGT

          FindLE

          FindLT

          FindNE

          HasValueChanged

          NumOfChanges

          TimeEq

          TimeGE

          TimeGT

          TimeLE

          TimeLT

          TimeNE

          NoOutput()

            • Re: TimeEq without mapped output to save output history
              calmandrysdale

              All the attributes are set to Data Reference <None>.

               

              I can't see why you can run 'expensive functions' and store to pi tags but you can't run them and not store to pi tags. How is there any difference in loading?

               

              I'm looking for any alternate method to achieve this without using pi tags.

                • Re: TimeEq without mapped output to save output history
                  Kenji Hashimoto

                  Because expensive functions need a lot of burden of PI Data Archive machine. For example, if you use syntax = TimeEq ('attribute','*-1d','*', 10) then it needs to load 1 day data to get snapshot result.

                  If you try to show this attribute trend for 1 year as adhoc (Data Reference = <Analysis>) which is not supported, the calculation will be very big query.

                  So just snapshot (Data Reference = <None>) calculation is possible.

                  We have a plan to explain "Analysis DR performance considerations" by future release of PI Analysis user manual.

                  In your case, the question is why <None> does not work for you. Which version do you use?  In my environment, it works fine. (PI Analysis Service 2016)

                  Analysis1_Output shows 86400 which is correct value.

                  3 of 3 people found this helpful
                    • Re: TimeEq without mapped output to save output history
                      calmandrysdale

                      Thanks Kenji, I am somewhat of a noob at all this.

                       

                      When  I configured the attributes i set them to Data Reference = None then checked them in.

                       

                      When you map an attribute to an output it sets them as Data Reference = Analysis; you then need to go and reset those to Data Reference = None.

                      This was the step i had missed.

                       

                      I also understand now that this is only going to work over smaller time frames which can retrieve snapshot values.

                       

                      As I said before though, I  really am still struggling to understand how there is any difference in loading when you store this to a pi tag? Even if you are storing to a pi tag would you not have to retrieve all the values and recalculate it anyway? Or is the analyses engine smart enough to take the previously calculated value into account somehow?

                       

                      Thanks for all your help. much appreciated.

                       

                      Calman

                    • Re: TimeEq without mapped output to save output history
                      vkaufmann

                      Hi Calman,

                       

                      To help explain a little bit more, these functions were restricted from being able to use with the Analysis Data Reference for just the reason Kenji explained above. Analysis Data Reference outputs are all calculated client side, meaning the Analysis service goes out and fetches the data from the Data Archive just like in any other analysis, instead in this scenario, it is the responsibility of the client and not the server to generate the comparisons you are after. In most scenarios, the server is optimized to make some of these comparisons, as the way data is linked within the archives allows to do these calculations pretty efficiently (not to mention that servers are typically much more powerful machines). Now if I did these same calculations client-side, the Analysis service really has no way to search or perform any logic on the fetched data other than the time parameters of which you requested. In the case of the functions above, the data sets could be massive! Because of this, the developers decided to limit the capabilities of these functions with the Analysis Data Reference. The Analysis Data Reference itself was born out of performance issues with the None Data Reference as it has been known to flood writes to back end SQL servers from customers pushing the limits of AF (and probably trying to avoid creating tags).

                       

                      I have seen this error crop up from time to time and it can get a little pesky to clear. If in fact you can confirm that all of the output attributes of your analysis that are utilizing these functions are not configured to be Analysis Data References, then I would see if restarting the Analysis service clears this error.

                       

                      --Vince

                      2 of 2 people found this helpful
                        • Re: TimeEq without mapped output to save output history
                          calmandrysdale

                          Thanks Vince and Kenji.

                           

                          I think I understand somewhat better.

                           

                          So what I am gathering is there are basically three scenarios with analyses mapped outputs...

                           

                          * Mapped Output has Data Reference=PI Point

                          The analysis service passes the function (e.g. TimeEq) over to the server to update the pi point.

                          This is quite efficient and can handle large data sets.

                           

                          * Mapped Output has Data Reference = Analysis

                          The analysis service retrieves all the required data from the PI archive and runs the function (e.g. TimeEq) on the client side.

                          This is not as efficient and can't handle large data sets.

                           

                          * Mapped Output has Data Reference = None

                          The analysis service retrieves all the required data from the PI Snapshot Archive and runs the function (e.g. Timeq) on the client side.

                          This is ok but only works for data in the snapshot system.

                           

                          Does that sound like i have understood correctly what you and Kenji are describing to me?

                           

                          Sorry to be pedantic  i just like to understand.

                           

                          Cheers,

                           

                          Calman

                          1 of 1 people found this helpful
                            • Re: TimeEq without mapped output to save output history
                              vkaufmann

                              Hi Calman,

                               

                              All of the listed data references have the same data available to them (whatever is in the Data Archive including the snapshot). What is important to know about the differing data references is how and where they are computing their calculations and storing the resulting value.

                               

                              PI Point--the result of an analysis mapped to a PI Point Data Reference will write its value to a PI Point. These calculations are done on the Analysis server.

                               

                              None--the result of an analysis mapped to a None Data Reference will write its value to the PIFD database. These calculations are also performed on the Analysis server. These values are not indexed and can put a strain on SQL servers if used in bulk.

                               

                              Analysis DR--the result of an analysis mapped to an Analysis Data Reference does not store its value and will only appear on the client where it is being requested. These calculations are performed by the client machine (ie where PI System Explorer is installed).

                               

                              Does this make things a bit more clear?

                               

                              --Vince

                              • Re: TimeEq without mapped output to save output history
                                John Messinger

                                Hi Calman,

                                That's mostly correct

                                Functions like TimeEq will always require data from the Archive subsystem, not the Snapshot subsystem, as you are running the query over a time range, and the snapshot only holds the most recent value for a tag.

                                I wasn't aware that PI Analyses would pull all the archive data and perform the calculation client side when the output is configured with the Analysis data reference - learned something new today.

                                 

                                If you are feeling brave, you can always write your own custom data reference to perform these type of calculations without outputting to a tag, only downside would be that the call to the archive is made each time the attribute holding such a data reference is queried for its value, rather than having a stored result updated at specified intervals. Using the methods from the OSIsoft.AF.Data.AFCalculation class would have the calculations run server side (at least as far as I have understood when I have used these methods), meaning that while the data still needs to be accessed from the archives, you are only returning the results to the client (your data reference). While the best option is to use PI Analyses and write the output to a tag, the custom data reference approach allows for the possibility of a more ad-hoc calculation approach, despite the aforementioned drawbacks regarding archive data access (and the whole thing of having to write and maintain custom code).

                                  • Re: TimeEq without mapped output to save output history
                                    calmandrysdale

                                    Ooooh now that is an intriguing thought. I shall have to investigate this particular option - especially for something as repeatable as a TimeEq type function.

                                    I have often thought two things about AF calculations:

                                    * can we write custom analyses functions?

                                    * can we write custom formula functions?

                                    I believe the answer is no to both but that would be SO handy!

                                     

                                    Thanks!

                                      • Re: TimeEq without mapped output to save output history
                                        John Messinger

                                        At present you can't write custom analysis functions (plugins). It would be nice to be able to do so (see this thread). As for a custom formula function, I would interpret this as a data reference, of which the standard Formula data reference is a prime example. There are other examples of this throughout the forums (here, here, and here for example). Along with probably countless others, I've written rollup data references in the past, prior to the release of AF Analytics. So you can write your own custom analysis functions as data references, remembering that typically these calculate a result value 'on demand' whenever the value of the owning attribute is requested.

                                        If you truly want to investigate this as an option, there is a white paper called Implementing AF 2.0 Data References, which you can download from here.

                                          • Re: TimeEq without mapped output to save output history
                                            skwan

                                            Calman:

                                            The major driving force for implementing the Analysis Data Reference is so that users do not use "no data reference" for their analysis outputs.  Let me explain why this is the case.  When you configure an analysis output to have no data reference, here's what happens to the system.  PI Analysis Service will continue to run the analysis based on the Time Rule (event triggered or periodic).  When the analysis is executed, the output is written to the output attribute and since it has no data reference, the value is therefore written to the SQL Server.  When this happens, the SQL Server does many tasks, including updating many tables used by AF such as the FindChangedItems table.  Therefore, if you have many analyses configured this way and they are running at high frequency, you're causing a lot of havoc with the SQL Server.  From the user's perspective, you would only be able to see the last value that is written and if you're using PI System Explorer, you need to refresh the client in order to see these (last) values.  So you're wasting a lot of resources running these analyses per the schedule, yet you're not utilizing the majority of the results.  As you can see,  you may be causing a lot of unnecessary work on PI Analysis Service running these analyses all the time when you only want to see the most recent value.  Over time, this really adds up as you may have wastefully executed millions of analyses.  Many customers have had their FindChangedItems table grow to many hundreds of millions of rows because they have many high frequency analyses with outputs with no data reference and this often causes the AF system to slow way down.  With the Analysis Data Reference, you're evaluating the analyses client side and only on demand.  This prevents most of the problems described above if you only need to see the latest output value.  However, we know from experience that certain functions will not perform well on the client side so we made the decision not to support these functions with the Analysis Data Reference.  You rightfully asked what the difference is between calling these functions with PI Analysis Service vs. client side.  The difference is that with PI Analysis Service, we have made a lot of optimization which reduces potential performance problems.  So naturally the question is why would saving the output to a PI Point alleviate the problem since the data access is similar?  Saving the output of scheduled analyses to PI Points would allow users wanting to trend these outputs to do it in a performing manner since you would be trending the already calculated outputs, i.e. you're trending the pre-calculated results.

                                            Hope this is useful information.

                                            --

                                            Steve Kwan

                                            8 of 8 people found this helpful