4 Replies Latest reply on Jul 29, 2016 3:47 PM by gregor

    How to pull more than a million events from a pi point?

    Lstran

      I am using this method piPoint.RecordedValuesAsync(TimeRange, AFBoundaryType.Inside, "", false, 2147483646).

      I am getting this error {"[-11091] Event collection exceeded the maximum allowed"}.

      How can I avoid this error?

        • Re: How to pull more than a million events from a pi point?
          John Messinger

          This is caused by the value of the ArcMaxCollect tuning parameter on the PI server limiting the amount of archive data that can be retrieved in a single call. Depending on your PI version, the default value may be as small as 150,000. Try increasing the value of this tuning parameter to something higher than what you are expecting to retrieve. You will need to restart the Archive subsystem for this change to take effect.

          1 of 1 people found this helpful
            • Re: How to pull more than a million events from a pi point?
              Marcos Vainer Loeff

              Another approach is to call RecordedValuesByCount within an extension method for PIPoint. Here is an example:

               

              
              namespace ConsoleApplication3
              {
                  class Program
                  {
                      static void Main(string[] args)
                      {
                          PIPoint piPoint = PIPoint.FindPIPoint("\\\\marc-pi2016\\sinusoid");
              
              
                          AFTimeRange timeRange = new AFTimeRange(new AFTime("*-4000d"), new AFTime("*"));
                          AFValues valuesOneCall = piPoint.RecordedValues(timeRange, OSIsoft.AF.Data.AFBoundaryType.Inside, string.Empty, false, 0);
                          AFValues valuesManyCalls = piPoint.RecordedValuesManyCalls(timeRange, OSIsoft.AF.Data.AFBoundaryType.Inside, string.Empty, false, 0);        
                      }
                  }
              
              
                  public static class ExtClass
                  {
                      public static AFValues RecordedValuesManyCalls(this PIPoint piPoint, AFTimeRange timeRange, AFBoundaryType boundaryType, string filterExpression, bool includeFilteredValues, int maxCount = 0, int maxValuesPerCall = 149000)
                      {
                          AFValues allValues = new AFValues();
                          AFTime currentStartTime = new AFTime("*-4000d");
                          while (true)
                          {
                              AFValues values = piPoint.RecordedValuesByCount(currentStartTime, maxValuesPerCall, true, OSIsoft.AF.Data.AFBoundaryType.Inside, string.Empty, false);
                              allValues.AddRange(values);
                              if (values.Count == maxValuesPerCall)
                              {
                                  DateTime dt = values.OrderBy(m => m.Timestamp.LocalTime).Last().Timestamp.LocalTime;
                                  currentStartTime = new AFTime(dt.AddMilliseconds(1));
                              }
                              else
                              {
                                  break;
                              }
              
              
                          }
                          return allValues;
                      }
              
              
                  }
              }
              

               

              ArcMaxCollect tuning parameter has a purspose within the PI Data Archive. Raising this number is not always the best solution as it can cause other problems. The code above should solve your problem.

               

              Hope this helps!

                • Re: How to pull more than a million events from a pi point?
                  Marcos Vainer Loeff

                  Actually, I have improved a little it the code snippet above:

                   

                  namespace ConsoleApplication3
                  {
                      class Program
                      {
                          static void Main(string[] args)
                          {
                              PIPoint piPoint = PIPoint.FindPIPoint("\\\\marc-pi2016\\sinusoid");
                  
                  
                              AFTimeRange timeRange = new AFTimeRange(new AFTime("*-4000d"), new AFTime("*"));
                              AFValues valuesOneCall = piPoint.RecordedValues(timeRange, OSIsoft.AF.Data.AFBoundaryType.Inside, string.Empty, false, 0);
                              AFValues valuesManyCalls = piPoint.RecordedValuesManyCalls(timeRange, OSIsoft.AF.Data.AFBoundaryType.Inside, string.Empty, false);        
                          }
                      }
                  
                  
                      public static class ExtClass
                      {
                          public static AFValues RecordedValuesManyCalls(this PIPoint piPoint, AFTimeRange timeRange, AFBoundaryType boundaryType, string filterExpression, bool includeFilteredValues, int maxValuesPerCall = 149000)
                          {
                              AFValues allValues = new AFValues();
                              AFTime currentStartTime = timeRange.StartTime;
                              while (true)
                              {
                                  AFValues values = piPoint.RecordedValuesByCount(currentStartTime, maxValuesPerCall, true, boundaryType, filterExpression, includeFilteredValues);
                        
                                  if (values.Count == maxValuesPerCall)
                                  {
                                      allValues.AddRange(values);
                                      DateTime dt = values.OrderBy(m => m.Timestamp.LocalTime).Last().Timestamp.LocalTime;
                                      currentStartTime = new AFTime(dt.AddMilliseconds(1));
                                  }
                                  else
                                  {
                                      AFTimeRange lastTimeRange = new AFTimeRange(currentStartTime, timeRange.EndTime);
                                      AFValues lastValues = piPoint.RecordedValues(lastTimeRange, boundaryType, filterExpression, includeFilteredValues);
                                      allValues.AddRange(lastValues);
                                      break;
                                  }
                  
                  
                              }
                              return allValues;
                          }
                  
                  
                      }
                  }
                  

                   

                  Note, this works only if EndTime = "*". If this is not the case, the code needs to be changed/improved.