4 Replies Latest reply on Sep 11, 2018 11:07 AM by Rick Davin

    Better efficiencies with loading AFAnalysis class properties

    natdavidson

      I'm making a c# app that will allow users to export a list of Analyses according to their statues, mostly because the PSE app doesn't have a very user friendly way to look at the Analysis in the management tab (no way to sort or export the list to do other manipulations).

      The biggest issue i'm having is after a get a collection of analyses with a particular status, and iterate through each Analysis, I get about five of the attributes for the analysis and write it out to a file, then loop back and get the next analysis. This seems to be taking a little bit more time that it should, and it seems the biggest time use is getting the different attributes for the analysis. Is there a better, more efficient way to get these attributes, some type of bulk or async call?

        • Re: Better efficiencies with loading AFAnalysis class properties
          gregor

          Hi Nate,

           

          Are you using AFAnalysisSearch Class?

          Do you like to share your code and identify where you experience bad performance?

            • Re: Better efficiencies with loading AFAnalysis class properties
              natdavidson

              I've been out for about a month, but here's some code from my project:

               

              private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
                      {
                          BackgroundWorker worker = sender as BackgroundWorker;
                          int j = 0;
                          AFStatus selectedStatus = (AFStatus)e.Argument;
                          string filePath = userPath + @"\MYAFServer_Analyses_" + selectedStatus + "_" + DateTime.Now.ToString("yyyyMMdd_HHmmss") + ".csv";
                          AFNamedCollection<AFAnalysis> AFColAnalyses = AFAnalysis.FindAnalyses(myDB, "*", null, null, null, null, null, selectedStatus, AFSortField.Name, AFSortOrder.Ascending, 0, 300000);
                          File.AppendAllText(filePath, "Status,Analysis Type,Element,Name,Template" + Environment.NewLine);
              
              
                          foreach (AFAnalysis item in AFColAnalyses)
                          {
                              if (worker.CancellationPending)
                              {
                                  e.Cancel = true;
                                  break;
                              }
                              else
                              {
                                  j++;
                                  string[] AnalysisEntry = new string[5];
                                  string AnalysisFullEntry = "";
                                  AFBaseElement itemBaseElement = item.Target as AFBaseElement;
              
              
                                  AnalysisEntry[0] = item.Status.ToString();
                                  AnalysisEntry[1] = item.AnalysisRule.Name;
                                  AnalysisEntry[2] = itemBaseElement.GetPath();
                                  AnalysisEntry[3] = item.Name;
                                  AnalysisEntry[4] = item.Template.ToString();
              
              
                                  for (int i = 0; i < 5; i++)
                                  {
                                      AnalysisFullEntry += AnalysisEntry[i] + ",";
                                  }
                                  File.AppendAllText(filePath, AnalysisFullEntry + Environment.NewLine);
              
              
                                  worker.ReportProgress((int)((j * 100) / AFColAnalyses.Count));
                              }
                          }
              

               

              I forgot how to post this formatted for code. I think the large amount of time is spent when getting the entries for the AnalysisEntry array. What i'm trying to do is format the exported file like how it looks on the management tab, this will help a lot when trying to trouble shoot analysis.

                • Re: Better efficiencies with loading AFAnalysis class properties
                  Rick Davin

                  Hi Nate,

                   

                  I formatted the code for you via the advanced editor.

                   

                  You seem to be using this older overload of AFAnalysis.FindAnalyses.  At the bottom of the provided link within its REMARKS section is this IMPORTANT text:  Consider using the new AFAnalysisSearch class for finding analyses instead of using this method.  From earlier comments, it sounds like a full load will be needed.  A page size of 1000 is a good place to start.  And most definitely, you will want to use server-side caching.  A quick re-working is:

                   

                  var query = $"Status:='{selectedStatus}'";
                  
                  // search is a disposable object so we wrap in a using block
                  using (var search = new AFAnalysisSearch(myDB, "", query))
                  {
                      search.CacheTimeout = TimeSpan.FromMinutes(10);  // VERY generous but immediately disposed at end of using block
                  
                      var analyses = search.FindAnalyses(fullLoad: true);
                  
                       var totalCount = search.GetTotalCount();  // works since all your filters are server-side
                  
                      foreach (var analysis in analyses)
                      {
                          // your other code
                      }
                  
                  } // search is disposed here
                  

                   

                  Note the above may require this usings to be at the top of your class:

                   

                  using OSIsoft.AF;
                  using OSIsoft.AF.Asset;
                  using OSIsoft.AF.Analysis;
                  using OSIsoft.AF.Search;
                  

                   

                  I am guessing your code omits a bunch of stuff because I really don't see anything regarding attributes, which is the whole reason why I use the more heavy weight full load.  If you don't do anything with attributes, you might get by with fullLoad: false.  I also considered if you could use the lightweight FindObjectFields, but the itemBaseElement.GetPath() call can't be run with FindObjectFields.

                   

                  Much of the code inside your foreach can be reduced a bit.  Just to offer some alternatives, this code:

                   

                                      string[] AnalysisEntry = new string[5];
                                      string AnalysisFullEntry = "";
                                      AFBaseElement itemBaseElement = analysis.Target as AFBaseElement;
                  
                                      AnalysisEntry[0] = analysis.Status.ToString();
                                      AnalysisEntry[1] = analysis.AnalysisRule.Name;
                                      AnalysisEntry[2] = itemBaseElement.GetPath();
                                      AnalysisEntry[3] = analysis.Name;
                                      AnalysisEntry[4] = analysis.Template.ToString();
                  
                                      for (int i = 0; i < 5; i++)
                                      {
                                          AnalysisFullEntry += AnalysisEntry[i] + ",";
                                      }
                  

                   

                  Could be this one statement spanning many lines:

                   

                  var fullEntry = string.Join(",", new[] { analysis.Status.ToString(),
                                                              analysis.AnalysisRule.Name,
                                                              ((AFBaseElement)analysis.Target).GetPath(),
                                                              analysis.Name,
                                                              analysis.Template.ToString()});
                  

                   

                  Or likewise be written as one statement on one line:

                   

                  var fullEntry = $"{analysis.Status},{analysis.AnalysisRule.Name},{((AFBaseElement)analysis.Target).GetPath()},{analysis.Name},{analysis.Template}";
                  

                   

                  I would rename variable j to be more meaningful, like currentCount.  Then your code becomes:

                   

                              int currentCount = 0;
                              var query = $"Status='{selectedStatus}'";
                              
                              // search is a disposable object so we wrap in a using block
                              using (var search = new AFAnalysisSearch(myDB, "", query))
                              {
                                  search.CacheTimeout = TimeSpan.FromMinutes(10);  // VERY generous but immediately disposed at end of using block
                  
                                  var analyses = search.FindAnalyses(fullLoad: true);
                  
                                  var totalCount = search.GetTotalCount();
                  
                                  foreach (var analysis in analyses)
                                  {
                                      if (worker.CancellationPending)
                                      {
                                          e.Cancel = true;
                                          break;
                                      }
                                      else
                                      {
                                          currentCount++;
                                          var fullEntry = $"{analysis.Status},{analysis.AnalysisRule.Name},{((AFBaseElement)analysis.Target).GetPath()},{analysis.Name},{analysis.Template}";
                                          File.AppendAllText(filePath, fullEntry + Environment.NewLine);
                  
                                         worker.ReportProgress((int)((currentCount * 100) / totalCount));
                                  }
                  
                              } // search is disposed here
                  

                   

                  If you would prefer to use search tokens instead of a query string, setting up your search object would look like:

                   

                  var tokens = new List<AFSearchToken>();
                  tokens.Add(new AFSearchToken(AFSearchFilter.Status, selectedStatus.ToString()));
                  
                  // search is a disposable object so we wrap in a using block
                  using (var search = new AFAnalysisSearch(myDB, "", tokens))
                  {
                      // other code here
                  }
                  

                   

                   

                  Explanation of why AFSearch is better

                   

                  There is no need for you to code any custom paging.  Or as you did with your original, you grabbed a HUGE page of up to 300K analyses, which meant that the older AFAnalysis.FindAnalyses had to take time to fully fetch 300K items, sort them, and send all of them to you as one big list.  The AFAnalysisSearch will efficiently stream results to you in pages of 1000 analyses.  All you do is just enumerate over each item in a foreach loop.  Because we use server-side caching, fetching the 2nd page and beyond is a very fast call.

                   

                  If you expect to have a huge amount of analyses, it is better to stream in pages that are consumed on demand.  But that means you will not have the full list in memory at once.

                  2 of 2 people found this helpful
              • Re: Better efficiencies with loading AFAnalysis class properties
                Rick Davin

                I agree with Gregor in that we need to see relevant pieces of code.  Have you performed any profiling where you know specific areas of poor performance, or is it more of a hunch?  Yes, RPC's to the AF Server can be slow.  Then again, the File I/O from your app can also be a slow down.  The AFAnalysisSearch class mentioned by Gregor is recommended. Since you mention attributes, it sounds like a full load will be needed.