9 Replies Latest reply on Jan 29, 2018 7:27 PM by Rick Davin

    AFEventFrameSearch Last 10 eventframes

    Jpkrohn

      I was wondering if how to find the last 10 results using AF SDK. We have a batch process that has several recipes ("|RecipeID") with 30 different steps("|StepID") . Each step generates an event. Overall we generate about 1200 events a day. I am trying to generate an average for the duration for the last 10 events for each step of each recipe. I can do it by manually looping through and counting by that is slow. I figure there is since I can do it in PICoresight event comparison. I think I could use AFEventFrame.FindEventFrames maxCount but this has been labeled as obsolete in the documentation says to use AFEventFrameSearch.

        • Re: AFEventFrameSearch Last 10 eventframes
          John Messinger

          Hi Jeff,

           

          I assume that the recipe steps are created as child event frames to the main recipe event? If you have a reference to the parent event frame, you could possibly look at LINQ to get your 10 final events for each recipe and then get your average duration.

           

          var eventsOfInterest = events.OrderByDescending(ef => ef.StartTime).Take(10);
          var avgDuration = Convert.ToInt64(eventsOfInterest.Average(ef => ef.Duration.Ticks));
          

           

          I haven't fully explored the AFEventFrameSearch class yet, so I don't feel qualified to comment on it's usage, but as the AFEventFrame.FindEventFrames() method is marked as obsolete then this would be the recommended method for searching for your initial list of event frames.

           

          I would definitely recommend reading Rick Davin's excellent blog series on Aggregating Event Frame Data. In fact, I might re-read it myself . Hopefully there will be something there that will help you with this requirement.

           

          Hope this helps.

           

          John

            • Re: AFEventFrameSearch Last 10 eventframes
              Jpkrohn

              Thanks I will look at LINQ. I did read the blog and think I will have to re-read again to see i I can figure out the aggregating multiple fields. Going to the lighter search (FindObjectFields) and breaking out of the foreach loop once I hit my target # of events (30 steps * 10 events each) have dropped my results time into acceptable range, but I would still like to improve it further. This is just one piece of a much larger program I am trying develop to teach myself C# and AF SDK at the same time:

               

              namespace EventFrameSearch
              {
                  public class EventFrameFields
                  {
                      // Field mapped using default name.
                      public Guid ID;                
                     
                      // Property mapped using default name.           
                      public string Name { get; set; }       
              
                      // Property mapped using 'ObjectField' attribute.
                      [AFSearch.ObjectField("StartTime")]
                      public AFTime EFStart { get; set; }
              
                      // Attribute value mapped to property using 'ObjectField' attribute.
                      [AFSearch.ObjectField("|_RecipeID")]
                      public AFValue RecipeID { get; set; }
              
                      // Attribute value mapped to property using 'ObjectField' attribute.
                      [AFSearch.ObjectField("|_ActiveStep")]
                      public AFValue StepID { get; set; }
              
                      // Attribute value mapped to property using 'ObjectField' attribute.
                      [AFSearch.ObjectField("|_Duration")]
                      public AFValue Duration { get; set; }
              
                      // Attribute value mapped to property using 'ObjectField' attribute.
                      [AFSearch.ObjectField("|_RunningTime")]
                      public AFValue RunningTime { get; set; }
                     
                      // Attribute value mapped to property using 'ObjectField' attribute.
                      [AFSearch.ObjectField("|_HeldTime")]
                      public AFValue HeldTime { get; set; }
              
                      // Attribute value mapped to property using 'ObjectField' attribute.
                      [AFSearch.ObjectField("|_StoppedTime")]
                      public AFValue StoppedTime { get; set; }
              
                      // Attribute value mapped to property using 'ObjectField' attribute.
                      [AFSearch.ObjectField("|_Feed_Flow_Average")]
                      public AFValue AverageLiquorFlow { get; set; }
              
                      // Attribute value mapped to property using 'ObjectField' attribute.
                      [AFSearch.ObjectField("|_55LB_Steam_Flow_Average")]
                      public AFValue Average55lbSteamFlow { get; set; }
              
                      // Attribute value mapped to property using 'ObjectField' attribute.
                      [AFSearch.ObjectField("|_160LB_Steam_Flow_Average")]
                      public AFValue Average160lbSteamFlow { get; set; }
              
                      // Attribute value mapped to property using 'ObjectField' attribute.
                      [AFSearch.ObjectField("|_Leg_Weight_ROC")]
                      public AFValue AverageLegWeightROC { get; set; }
                  }
              
                  public class AverageEventFrameFields
                  {
                      public string Name { get; set; }
                      public int RecipeID { get; set; }
                      public int StepID { get; set; }
              
                      public int count { get; set; }
              
                      public double Duration { get; set; }
                      public double RunningTime { get; set; }
                      public double HeldTime { get; set; }
                      public double StoppedTime { get; set; }
                      public double AverageLiquorFlow { get; set; }
                      public double Average55lbSteamFlow { get; set; }
                      public double Average160lbSteamFlow { get; set; }
                      public double AverageLegWeightROC { get; set; }
                  }
              
                  class Program
                  {
                      static void Main(string[] args)
                      {
                          PISystems myPISystems = new PISystems();
                          PISystem myPISystem = myPISystems["servername"];
                          if (myPISystem == null)
                              throw new InvalidOperationException("Default PISystem was not found.");
                          AFDatabase myDB = myPISystem.Databases["databasename"];
                          if (myDB == null)
                              throw new InvalidOperationException("Database was not found.");
                         
                          // Create a search to find all the event frames created from the 'Event'
                          // template and its 'Level' attribute value is less than 90.
                          int count;
                          int maxCount = 300;
                          AverageEventFrameFields[] Steps = new AverageEventFrameFields[30];
              
                          double average;
                          var sw = Stopwatch.StartNew();
                          using(var search = new AFEventFrameSearch(myDB, "FindEventFields", @"Template:'Step' End:<='*' SortField:EndTime SortOrder:Desc |_RecipeID:=7"))
                          {
                              search.CacheTimeout = TimeSpan.FromMinutes(10);
                             
                              // Do the search
                              // Return specified fields using dynamic type conversion.
                              count = 0;
              
                              var results = new EventFrameFields[maxCount];
              
                              var foundItems4 = search.FindObjectFields<EventFrameFields>();
                              Console.WriteLine("Find Object Fields using Dynamic Type Conversion:");
                              foreach (var row in foundItems4)
                              {
                                  results[count] = new EventFrameFields();
                                  results[count].ID = row.ID;
                                  results[count].RecipeID = row.RecipeID;
                                  results[count].StepID = row.StepID;
                                  results[count].AverageLiquorFlow = row.AverageLiquorFlow;
                                  results[count].Average55lbSteamFlow = row.Average55lbSteamFlow;
                                  results[count].Average160lbSteamFlow = row.Average160lbSteamFlow;
                                  results[count].AverageLegWeightROC = row.AverageLegWeightROC;
                                  results[count].RunningTime = row.RunningTime;
                                  results[count].Duration = row.Duration;
              
                                  count++;
              
                                  if (count >= maxCount)
                                      break;
                              }
                              Console.WriteLine("Found {0} EventFrames.", count);
                             
                              foreach (var result in results)
                              {
                                  if(Steps[result.StepID.ValueAsInt32()]==null)
                                  {
                                      Steps[result.StepID.ValueAsInt32()] = new AverageEventFrameFields();
                                      Steps[result.StepID.ValueAsInt32()].Name = result.Name;
                                      Steps[result.StepID.ValueAsInt32()].RecipeID = result.RecipeID.ValueAsInt32();
                                      Steps[result.StepID.ValueAsInt32()].StepID = result.StepID.ValueAsInt32();
                                      Steps[result.StepID.ValueAsInt32()].count = 0;
                                  }
              
                                  var refStep = Steps[result.StepID.ValueAsInt32()];
                                  ++refStep.count;
                                  refStep.Duration += (result.Duration.ValueAsDouble() - refStep.Duration) / refStep.count;
                                  refStep.RunningTime += (result.RunningTime.ValueAsDouble() - refStep.RunningTime) / refStep.count;
                                  refStep.AverageLiquorFlow += (result.AverageLiquorFlow.ValueAsDouble() - refStep.AverageLiquorFlow) / refStep.count;
                                  refStep.Average55lbSteamFlow += (result.Average55lbSteamFlow.ValueAsDouble() - refStep.Average55lbSteamFlow) / refStep.count;
                                  refStep.Average160lbSteamFlow += (result.Average160lbSteamFlow.ValueAsDouble() - refStep.Average160lbSteamFlow) / refStep.count;
                                  refStep.AverageLegWeightROC += (result.AverageLegWeightROC.ValueAsDouble() - refStep.AverageLegWeightROC) / refStep.count;
                              }
              
                              sw.Stop();
                              foreach(var step in Steps)
                              {
                                  if(step != null)
                                      Console.WriteLine("Step {0} Average 55 Steam flow {1}.", step.StepID, step.Average55lbSteamFlow);
                              }  
                              Console.WriteLine("Time to complete {0} seconds.", sw.Elapsed.Seconds);
                          }
                          Console.ReadKey();
                      }
                  }
              

               

               

                • Re: AFEventFrameSearch Last 10 eventframes
                  Rick Davin

                  Hi Jeff,

                   

                  I edited your post to add Syntax Highlighting (Use Advanced editor, then click on >>, select Syntax Highlighting, and finally C#) to your code.

                  • Re: AFEventFrameSearch Last 10 eventframes
                    Rick Davin

                    Hi Jeff,

                     

                    For someone learning C# and AF SDK at the same time, you are doing quite well.

                     

                    For the type of processing you are performing, the best performance will be if your event frames have captured values.  I am guessing that the |_RecipeID attribute is a static attribute, that is it does not use a data reference.  That's important because then all filtering will be performed server-side.  However, I am also pretty sure that not all attributes referenced by the EventFrameFields class will have static attributes, which means that those with data references will either have to resolve the values client-side (slower performance), or if you have captured values then the values have already been resolved and the server will quickly send you those cached values.

                     

                    I see you have a |_Duration attribute.  The AFEventFrameSearch.FindObjectFields also has a Duration property.  If these produce the same result, I would suggest using the built-in property.  This suggestion is made out of best practices, as I do not expect it to boost performance.

                     

                    In line 104, you could pass a pageSize to FindObjectFields:

                     

                    var foundItems4 = search.FindObjectFields<EventFrameFields>(pageSize: maxCount);

                     

                    You seem to only be interested in the first page of results.  I would suggest doing this when your expected maxCount is a reasonably small value, particularly if maxCount is less than the default pageSize (1000).  If your maxCount was much bigger, say 50K, I would stick with the default pageSize.

                    1 of 1 people found this helpful
                    • Re: AFEventFrameSearch Last 10 eventframes
                      Rick Davin

                      Hello again Jeff,

                       

                      Another suggestion that may boost performance would be to put a reasonable constraint on StartTime, perhaps something like "*-1d" or "*-2d".

                       

                      Your current query filter on line 94 only restricts EndTime.  Let's imagine there are 200K event frames with |_RecipeID=7.  This means the server must grab all 100K, and then sort it appropriately, before sending you various pages.  But if you were able to put a constraint on StartTime as well, maybe only 5K event frames would be needed to be fetched and sorted before paging to you.

                       

                      I am just making up numbers for illustration purposes.  But it is worth a consideration and hopefully would improve performance.

                      1 of 1 people found this helpful
                  • Re: AFEventFrameSearch Last 10 eventframes
                    Dan Fishman

                    I would recommend taking a look at several PI Square posts: Using the AFEventFrameSearch Class

                    and: OSIsoft.AF.Search Example

                     

                    You probably want to avoid maxCount and be sure to set a time range for best performance. It might make sense to obtain the last ten completed recipes and children, and then do some averaging on the step duration's. Also, I am not sure how comfortable you are programming (or if your restricted by a data access license), so you could use PI Datalink to do this.  I used PI Datalink to average "steps" in this presentation.  Jump to nine minutes in.

                    1 of 1 people found this helpful
                    • Re: AFEventFrameSearch Last 10 eventframes
                      Roger Palmen

                      Maybe off-topic, but if you generate the EFs using AF Analytics, and have 2017R2 you could output the duration to a PI Point and calculate an average over the last 10 values.

                      3 of 3 people found this helpful
                      • Re: AFEventFrameSearch Last 10 eventframes
                        JimGavigan

                        I have done something similar in your industry - as Roger Palmen mentions, there is a better way to do this in AF2017R2, but I can show you a way to at least capture the event durations as a tag in versions previous to 2017R2. I have to think about how to average the last 10, but pretty sure I can come up with a way.

                         

                        Shoot me a PM and we can get together next week and I will show you what I have done.