7 Replies Latest reply on Mar 9, 2015 5:43 PM by Dan Fishman

    EventFrame Attributes performance

    AlistairFrith

      We are developing an application that uses event frames and we are having performance issues when reading these event frames. Here's the costly code to convert an EventFrame into what we call a ValidationException:

       

         
              /// <summary>
              /// Convert an AF Event Frame into a Validation Exception
              /// </summary>
              /// <param name="frame"></param>
              /// <returns>The request Validation Exception or null the passed in EventFrame is null</returns>
              // Temporarily changed from private to public for unit tests
              // Changed from static to non-static
              public ValidationException ValidationExceptionFromEventFrame(AFEventFrame frame)
              {
                  if (frame == null)
                  {
                      return null;
                  }
                  ValidationException validationException = new ValidationException();
                  validationException.ID = frame.ID;
                  validationException.TemplateName = frame.Template.Name;
                  validationException.Start = frame.StartTime;
                  validationException.End = frame.EndTime;
                  //anomaly.CategoriesString = frame.CategoriesString;
      
                  // Get the attributes (doing them individually is slower, apparantly)
                  AFAttributeList attributes = new AFAttributeList(frame.Attributes);
                  AFValues values = attributes.GetValue();
                  int iTrigger = 0;
                  int iStatus = 1;
                  int iAction = 2;
                  int iComment = 3;
                  int iType = 4;
      
                  // Get the exception type. We default to 'Unspecified' if the event does not
                  // have the appropriate attribute or the attribute value can't be parsed
                  ValidationExceptionType exceptionType = ValidationExceptionType.Unspecified;
                  AFValue type = values[iType];
                  if (type != null && type.Value != null)
                  {
                      Enum.TryParse(type.Value.ToString(), out exceptionType);
                  }
                  validationException.ExceptionType = exceptionType;
      
                  // Get the exception status. We default to 'Open' if the event does not
                  // have the appropriate attribute or the attribute value can't be parsed
                  ValidationExceptionStatus exceptionStatus = ValidationExceptionStatus.Open;
                  AFValue status = values[iStatus];
                  if (status != null && status.Value != null)
                  {
                      Enum.TryParse(status.Value.ToString(), out exceptionStatus);
                  }
                  validationException.Status = exceptionStatus;
      
                  // Get the exception comment
                  AFValue comment = values[iComment];
                  validationException.Comment = (comment != null && comment.Value != null) ? comment.Value.ToString() : UndefString;
                  // Get the element causing the exception
                  if (frame.PrimaryReferencedElement == null)
                  {
                      validationException.ElementPath = UndefString;
                  }
                  else
                  {
                      validationException.ElementID = frame.PrimaryReferencedElement.ID;
                      validationException.ElementPath = frame.PrimaryReferencedElement.GetPath();
                  }
      
                  // Get the attribute causing the exception (first its path, then the attribute itself)
                  validationException.AttributePath = UndefString;
                  validationException.AttributeID = new Guid();
                  AFAttribute trigger = attributes["TriggeringInputPath"];
                  if (trigger != null)
                  {
                      AFValue triggerPath = values[iTrigger];
                      validationException.AttributePath = (triggerPath != null && triggerPath.ToString() != "") ? validationException.ElementPath + "|" + triggerPath.ToString() : UndefString;
                      if (frame.PrimaryReferencedElement != null) // if the referenced element is null, the path string will be invalid
                      {
                          if (validationException.AttributePath != UndefString)
                          {
                              IList<AFAttribute> triggerAttributes = AFAttribute.FindAttributes(validationException.AttributePath, null);
                              if (triggerAttributes != null && triggerAttributes.Count() > 0)
                              {
                                  validationException.AttributeID = triggerAttributes[0].ID;
                              }
                          }
                      }
                  }
                  return validationException;
              }
      
      
      
      
      
      
      
      

       

       

      This function takes several seconds to convert a single event frame and about 2 minutes to convert 350 of them. We were expecting to be able to load several hundred per second from AF. Is there a better way of doing this?

       

      Message was edited by: Alistair Frith Re-pasted the code after I realised I was not actually bulk-loading the values! New code performs just as badly though.

       

      Message was edited by: Alistair Frith - Re-pasted again since the formatting seems way screwy!

        • Re: EventFrame Attributes performance
          AlistairFrith

          Right, good news: the horrible performance was due to the fact that we are developing in a different country from the AF and PI Servers. If we deploy to the actual web server and run it where the final application will live, it takes 5 seconds to load 450 events. This is acceptable performance, however it is still a little slower than I expected. Is there anything further we can do to speed it up?

           

          Interestingly, loading the attributes into an AFAttributeList and doing a single GetValue() on that seems to make no difference from calling GetValue on each individual attribute: it's still 2 minutes locally and 5 seconds when deployed.

            • Re: EventFrame Attributes performance
              Dan Fishman

              Maybe try the AFEventFrame.LoadEventFrames( IList<AFEventFrame>) method for your the Event Frames you are passing into ValidationExceptionFromEventFrame().  You are getting all the attributes at once per Event Frames which is pretty good, but how about getting them all at once after you find your EF of interest.  You might want to chunk this and use parallelism if you truly need it but avoid it if you can.  What type of data reference are these attributes?  The only obvious other change is to use capturedValues if appropriate.

              • Re: EventFrame Attributes performance
                dng

                Hi Alistair,

                 

                Which part of the function is the bottleneck? Is most of the time spent on AFValues value = attributes.GetValue();?

                 

                Have you tried using AFEventFrame.LoadEventFrames before calling this ValidationException method? LoadEventFrames fully load the AFEventFrame objects from the server into the client. This will ensure that the event frames in the list are fully loaded to avoid a call to the server to load each of the event frames individually. If the values of your event frames are captured, loading the event frames into memory will cache the values as well. This might help with your development environment because it avoids round trips to the server since you have high network latency. You will have to test it on the local web server to see if a performance gain is achieved.

                 

                EDIT: Looks like Dan beat me to this answer just make sure to check if your event frame values are captured.

              • Re: EventFrame Attributes performance
                AlistairFrith

                Thanks Dan & Daphne, using LoadEventFrames() has bought it down to around 57 seconds. Doubling the speed is really quite helpful.

                 

                Most of the attributes were static (no DR), and no, we were not fully loading the event frames first. Having now done that, would there be much benefit in batching up the attribute calls from all the Event Frames into one? The LoadEventFrames call takes 12 seconds so there is still 45 seconds taken up elsewhere. AFAttributeList.GetValue() came back pretty instantly, but then I am calling it several hundred times so it could easily be that.

                 

                --- Alistair.

                  • Re: EventFrame Attributes performance
                    dng

                    Hi Alistair,

                     

                    If you use LoadEventFrames, subsequent GetValues call will hit the cache instead of making a round trip to the server. In this case, there is probably not much benefit in using AFAttributeList.GetValue over individual AFAttribute.GetValue. I did some tests before (see previous post: Loading EventFrames into DataTable (.Net, C#, AFSDK)) that seem to indicate otherwise, but upon repeating the tests I noticed no significant difference between the two.

                     

                     

                    You can probably use a profiler to determine your performance bottleneck. As a first step you can include a stopwatch to report the time it takes for specific blocks of code to further narrow down the bottleneck.

                      • Re: EventFrame Attributes performance
                        Dan Fishman

                        Perhaps it is a few EF that don't have static DR and the GetValue is slow.  Toss in a few stop watches as Daphne suggested and let us know where the lag is and we could try to optimize more. You might also find that your second run through will be faster due to the SQL Server cache or even PI Data Archive cache.  That makes performance testing a little harder.