21 Replies Latest reply on Mar 26, 2015 5:38 PM by Bannikov

    How To control the timestamp of ABACUS output tag?

    tboers

      Hi all, with an ABCUS calculation I’m looking to a way to
      control the timestamp of output tag. The use case is a daily total calculation
      with data which are delayed entered in to the PI historian. E.g.  the values of the PREVIOUS day are entered
      into the PI system at 10 am of the CURRENT day. So we like to schedule the Abacus
      at 11 am daily, but like to write the value with the timestamp of ‘Y’.  Anyone done this before?

        • Re: How To control the timestamp of ABACUS output tag?
          Mike Zboray

          This is not possible currently, however the next version of analysis (with AF 2.7) will support this.

           

          There is a work around that involves configuring an output attribute with a Relative Time configuration. This will cause the output to go into PI at the offset time instead of the output time. This might need to be an auxiliary attribute to the one that you normally use for visualization since it will always go back in time from the requested time stamp.

            • Re: How To control the timestamp of ABACUS output tag?
              tperovic

              Please elaborate on "configuring an output attribute with a Relative Time configuration". How do I configure such an attribute?

               

              PI Point Data Reference

               

               

               

              Value Retrieval Methods:

               

                   By Time:                ?

               

                        Relative Time:  ?

                • Re: How To control the timestamp of ABACUS output tag?
                  Mike Zboray

                  Yes, if,  for example, you want it to write at midnight today:

                   

                  pipointdr.png

                    • Re: How To control the timestamp of ABACUS output tag?
                      tboers

                      Thanks for the response. This configuration seemed logical to me. So I tried it before, but I got confused as the relative time 'y'  didn't write any values to the output tag when I backfilled the tag. Is this expeced behaviour? Also when I try this scheduled there is no result written to the output tag. I tested this with relative time=t => result was no value witten on the scheduled time, when made relative time = [empty] expected result was written but on the the time of the execution. So I'm not able to get this configuration to work.

                        • Re: How To control the timestamp of ABACUS output tag?
                          skwan

                          As I mentioned earlier, the next release will natively support overriding output time stamps.  Previously I did not want to discuss the workaround that Mike provided as it's a really poor workaround and can confuse the users very quickly.  But it looks like you are actually trying to use this workaround.  If you have to use this workaround, there are several things you need to consider.

                           

                          1) You need to be very careful and think through how you're scheduling your analysis because you can very easily write multiple output values to the same time stamp.  For example, if you configure the (analysis output) PI Point data reference with a relative time of 't', then every time it runs during a 24 hours period, you will write a value to the same time stamp.  Let's say you schedule the analysis to run every 5 minutes, then every 5 minutes, an output value will be written to 't'.  This is probably not what you want because the current behavior is that PI Data Archive will store all the values to the same time stamp.

                          2) Currently backfill only works if there are no existing output values during the time you're trying to backfill.  If you're trying to backfill, unless you're running the analysis only once per 24 hours, the backfill will attempt to write multiple values to the same time stamp, 't'.  Backfill will fail.

                          3) Depending on your application, you may need to write and read the output PI Point using different AF attribute configurations.  For example, if you write to an attribute with a PI Point configured with a relative time of 't', but then you read the same attribute, you will get back the same value, but with a time stamp of 't'.  Meanwhile, all your other AF attributes may be of a different time stamp.  This may or may not be your desired behavior.  If you're using PI System Explorer, add the Time Stamp column to the grid to ensure you're looking at the proper time stamp.  You may have to configure another AF attribute, pointing to the same PI Point, but without the relative time in order to sync up the behavior with your other AF attributes.

                           

                          TimeT.jpg

                           

                          In short, unless you have to have this capability now, I would advise you to wait until the next release.

                           

                          --

                          Steve Kwan

                          AF Product Manager

                          2 of 2 people found this helpful
                          • Re: How To control the timestamp of ABACUS output tag?
                            Bannikov

                            Hello,

                             

                            we've solved similar task: we need perform calculation every 2 hours, with 20 minutes phase shift (to ensure all input data are in place). Every calculation should put its result at the beginning of calculated interval. So our schedule is:

                            01.png

                            And our output tag is configured like this:

                            02.png

                            It's working fine.

                            Some disadvantage is that we need another attribute based on the same tag with zero Relative Time to retrieve our results properly.

                    • Re: How To control the timestamp of ABACUS output tag?
                      skwan

                      As Mike already pointed out, the next version will handle this.  The workaround from Mike would work but I would suggest you wait a little bit.  The current estimated release is Q2 so not very soon.

                       

                      --

                      Steve Kwan

                      AF Product Manager

                        • Re: How To control the timestamp of ABACUS output tag?
                          David Golverdingen

                          And what about a performance equation?

                          Does PE have a feature to manage the time stamp of the written value?

                          Or is the functionality of ABACUS equal to PE and not available yet?

                            • Re: How To control the timestamp of ABACUS output tag?
                              pthivierge

                              Hello David,

                               

                              With a Performance Equation you have the same limitation, it is not possible to control the timestamp of the output calculation.  The timestamp is set at the time when the calculation occurs. The only possibility it that you can chose whether the calculation is clock-scheduled or event-based (calculation is performed when a specific tag changes.).

                               

                              Asset Based Analytics (often called Abacus) offers more possibilities than Performance Equations:  ABA allows to have multiple tags to trigger a calculation when calculation is set to Event-Triggered, in opposition Performance Equations can only have on trigger tag when set to Natural. 

                               

                              Asset Based Analytics implements most of the performance equations functions, it is based on the AF structure thus you can use the power if the attribute to create some basic calculations at the first place prior to create your calculation logic with ABA. It is also much simpler to create complex equations with it because you can split your equation in several variables

                               

                              So Asset Based Analytics is really the way to go today and this is only the beginning of this product, the future will probably bring the missing pieces that many of you are requesting today.  And to conclude, most of the time we can find workarounds when we face a limitation that can help waiting for the feature you are looking for.  So do not hesitate to come ans search on this forum or to ask a new question about it.

                                • Re: How To control the timestamp of ABACUS output tag?
                                  tperovic

                                  With a PE, you have two options.

                                   

                                  Set timestamp of resulting expression to:

                                  a) The time when the expression is evaluated or

                                  b) The timestamp of the triggering event.

                                   

                                  ABACUS always uses the timestamp of the triggering event. I was hoping to get the time when the analysis is evaluated to determine the lag time.

                                   

                                  Tony Perovic

                                  Compumation, Inc.

                                    • Re: How To control the timestamp of ABACUS output tag?
                                      skwan

                                      There is a performance counter for "Maximum Latency", that should give you an idea on the load on the system.

                                       

                                      Note that there is a default calculation wait time of 5 seconds.  This is so that we can ensure all input values have arrived at the PI Data Archive.  This setting is editable.

                                       

                                      --

                                      Steve Kwan

                                      AF Product Manager

                                        • Re: How To control the timestamp of ABACUS output tag?
                                          tperovic

                                          My impression was that the 5 seconds for PI ACE was fixed. How do I speed it up?

                                           

                                          Tony Perovic

                                          Compumation, Inc.

                                            • Re: How To control the timestamp of ABACUS output tag?
                                              Eugene Lee

                                              Hi Tony,

                                               

                                              You can go to the Analyses plugin in PSE and right click to get a dialog box to edit the settings. The property that is relevant to you is "CalculationWaitTimeinSeconds".

                                               

                                              Capture.PNG

                                              • Re: How To control the timestamp of ABACUS output tag?
                                                skwan

                                                Tony:

                                                Not sure if the word "fixed" is appropriate.  In the case of inputs from multiple PI Data Archives, if the calculation with multiple inputs is initiated as soon as the first value arrives, there's a very good chance you would end up not using the most current values for all the inputs.  So it makes sense to wait some period of time before the calculation is executed.

                                                 

                                                In the case of asset analytics, the output timestamp is currently always the trigger time.  Therefore, even though the calculation is actually executed at some time after the trigger time, the output timestamp is always aligned with the inputs.

                                                 

                                                This setting is configurable.  Go to the Analysis Management Plugin (bottom left hand side of PI System Explorer), right-click on the right pane and you will find the settings for analysis service.  This information is also available in the documentation.  There's always a tendency to set this to zero because someone is going to think "faster is better".  If this is set to zero, there's risk to the calculations not using the proper inputs due to latency between all the inputs, even in the case where you have only one PI Data Archive.

                                                 

                                                --

                                                Steve Kwan

                                                AF Product Manager

                                                  • Re: How To control the timestamp of ABACUS output tag?
                                                    Roger Palmen

                                                    Any plans to make this setting specific for each analysis? Some require more latency than others. Thinking about online-interfaces (seconds) vs telemetry (minutes) vs manual logging (hours).

                                                      • Re: How To control the timestamp of ABACUS output tag?
                                                        skwan

                                                        Hi Roger,

                                                         

                                                        No, not at the moment.  But I think what you're describing is something else.  You want to make sure that the calculation only happens when there's an update to a specific attribute?  Or maybe only if there's at least one update to all the attributes for the analysis?

                                                         

                                                        --

                                                        Steve Kwan

                                                        AF Product Manager

                                                          • Re: How To control the timestamp of ABACUS output tag?
                                                            Roger Palmen

                                                            Hi Steve,

                                                             

                                                            It took me some time to think about that, as indeed at first look your response makes sense.

                                                             

                                                            Let's take one step back first.

                                                            • An analysis has a trigger, and at some time after that trigger, the analysis is run. That "some time after" is primarily due to the asynchronouis nature of the engine. No problems at all with that.
                                                            • Now about wait time (i'll call that latency). Latency is to me a minimum time to wait after the trigger to start the analysis. There is already a 5 second latency by default for each analysis, but for doubtful reasons if you think about it (and if i understand it correctly...).
                                                            • For clock-triggered analysis, a latency could be used to provide an offset to the clock. E.g. run hourly, but at 30 minutes past the hour. The terminology 'offset' is better placed here, and this is specific to the clock scheduling, so this one i would consider a requirement for the clock scheduler.
                                                            • A second reason could be for load distribution. Often bulk of calculations are run at midnight or full hours, but load distribution is better placed at the analytics manager (we already have analysis groups for that, with some magical, dark-art logic behind that). The analytics manager should know best how much performance is consumed, and where it is available. So this one is also not to be covered by the latency setting.
                                                            • A third reason is, as is the original post describes, nice one. E.g. a clock-scheduled calculation running hourly, using some PI Points that the user assumes are there at or around the hour. So a 5 seconds waiting time does make some sense, but this is actually hiding a more complex requirement in my opinion. Yes, it might be that all inputs should be present when the calculation is run. Or that all inputs should not be older than 5 seconds. Or.... So here you quickly move into the realm of business logic for data consistency, which should be handled in the analysis rules imho. For one, PI Points should be interpolatable if configured correctly, and if you need to wait, why not schedule to run 5 seconds later? I agree that 5 seconds are sufficient to compensate for clock-drift across interfaces etc. but that's indeed like it is currently implemented: just a technical solution to a minor technical problem.
                                                            • For Natural-triggered points, the current logic makes sense too. Interpolation should be OK all the time, and running on each event of defined trigger attributes should be sufficient.

                                                             

                                                            Now look at a real-world case:

                                                            • I have a system using e.g. 10K meters collected through telemetry. At any given point, just due to scale, we know that about 0.1% of meters is not providing data. That is still 100 meters. Also, telemetry introduces a latency of up to 15 minutes depending on network, etc. and just the time involved to push the data through all the layers in the telemetry network.
                                                            • Now i need to prepare the weekly report with some KPI for the monday meeting at 8. So i need to run the reporting, and any calculations not much later than at 6 or 7 in the morning. The weekly report needs values up to end-of-week.
                                                            • For dashboards, i need to have the most recent values of the KPI's.
                                                            • For some values, i need to revert to a forecast or nominal value or historical value if there is no current value available.

                                                             

                                                            I would like to implement this in the following way:

                                                            • Natural scheduling for the KPI's to support the dashboards. This will ensure we always have the most recent value based on received inputs. If there is time-logic on input attributes, then that should be covered by the analysis rules. E.g. Bad Value as output when any input has an Event Timestamp older than 5 minutes.
                                                            • Clock scheduling for the KPI to support the monday report. This to get values at midnight end of each week which can be easily extracted in reports without having any business logic in the reporting layer. We can schedule at 6 in the morning, allowing 6 hours for data to reach PI, and then manipulate the output timestamp to be at midnight (AF2.7). But i would need to perform the calculation at midnight, looking back 7 days to generate the weekly report KPI averages. So clock scheduling at midnight, and have a 6 hour waiting time, or being able to set the evaluation timestamp of the analysis, independant of the trigger timestamp. Or, build the date logic in the analysis rules (current solution) to look at the data from midnight to 7 days back. But this last one is not so easy to achieve.

                                                             

                                                            Conclusion:

                                                            • So do we really need latency / wait time to be set for each analysis? No.
                                                            • Do we need logic to wait for all inputs? I think more complex logic is required. E.g. max time to wait on each input, alternative values for inputs if there is no appropriate event.
                                                            • Do we need to be able to set the output timestamp of the analysis? Yes!
                                                            • Do we need to be able to set the evaluation timestamp of the analysis, OR set a wait time? Would be very handy, but that can be solved in the analysis itself. More out-of-box capabilities to work with dates, would be helpful.
                                                              • Re: How To control the timestamp of ABACUS output tag?
                                                                skwan

                                                                Roger Palmen wrote:

                                                                 

                                                                Conclusion:

                                                                • So do we really need latency / wait time to be set for each analysis? No.
                                                                • Do we need logic to wait for all inputs? I think more complex logic is required. E.g. max time to wait on each input, alternative values for inputs if there is no appropriate event.
                                                                • Do we need to be able to set the output timestamp of the analysis? Yes!
                                                                • Do we need to be able to set the evaluation timestamp of the analysis, OR set a wait time? Would be very handy, but that can be solved in the analysis itself. More out-of-box capabilities to work with dates, would be helpful.

                                                                Do you think some of the complex logic you described, such as using alternative values, should be part of the trigger logic for the software or should it be part of the logic within each expression?  For example, using if-then-else

                                                                 

                                                                --

                                                                Steve Kwan

                                                                AF Product Manager

                                                                  • Re: How To control the timestamp of ABACUS output tag?
                                                                    Roger Palmen

                                                                    Hi Steve,

                                                                     

                                                                    I think there is a bit of both. One the one hand you might not want to over-complicate the triggering as you need to evaluate that constantly,but on the other hand, you might need to push-back triggers on the evaluation stack to wait for inputs to arrive. In the end, an If-Then-Else in an analysis cannot suspend processing of a trigger. Unless that functionality is added (suspend analysis for x seconds or until timestamp y or until the next natural trigger arrives). We actually have built standard ACE modules that do exactly that (suspend executiuon until next natural trigger arrives).

                                                                     

                                                                    Now it often requires a lot of hacking and custom DR's to get what is needed. Evaluations on time are virtually non-existent in the formula DR, so doing some If-Then-Else on timestamps would help the most to take this one step forward.