AnsweredAssumed Answered

Receiving sporadic "Timeout on PI RPC or System Call" errors when querying PI data using AF SDK

Question asked by TrentHuhn on Oct 11, 2016
Latest reply on Oct 14, 2016 by gregor

I am attempting to retrieve one-minute averages for a week's worth of data using the AFAttribute.Data.Summaries() function. Over 90% of the time, the call executes successfully with no issues. However, a small percentage of the time, the call fails to retrieve some or all of the expected data. Here is the code I am using:

 

            AFTimeSpan timeSpan = AFTimeSpan.Parse("1m");;
            AFTimeRange tempRange = timeRange;
            IDictionary<AFSummaryTypes, AFValues> summaryValues;
            AFValues valuesDataAverage = new AFValues();
            do
            {
                tempRange.EndTime = tempRange.StartTime.UtcTime.AddDays(7) > timeRange.EndTime.UtcTime ? timeRange.EndTime : new AFTime(tempRange.StartTime.UtcTime.AddDays(7));
                summaryValues = attributeToGetDataFrom.Data.Summaries(tempRange, timeSpan, AFSummaryTypes.Average, AFCalculationBasis.TimeWeighted, AFTimestampCalculation.MostRecentTime);
                valuesDataAverage.AddRange(summaryValues[AFSummaryTypes.Average]);
                tempRange.StartTime = tempRange.EndTime;
            }
            while (tempRange.EndTime != timeRange.EndTime);
            AFValue firstVal = valuesDataAverage[0];
            if(firstVal.IsGood)
                 logger.Info(string.Format("Added {0} points for {1} (First Value: {2} - {3})", valuesDataAverage.Count, attributeToGetDataFrom.GetPath(), (double)firstVal.Value, firstVal.Timestamp)); 

                      

 

 

where attributeToGetDataFrom is an AFAttribute object and timeRange is an AFTimeRange containing the time range in question. The code should break this time range into one-week chunks and call the Summaries method in succession in order to populate the valuesDataAverage object.

 

Here is a sample of the results I get when running this for a month's worth of data over several different AF attributes:

 

2016-10-11 11:18:41,286 INFO  - NRG.PIBIAS.LostEnergy.LostEnergyCalculationManager - Added 33121 points for \\WNTRNAFP01\Corporate\Texas Wind\STWD\STWD1\SUB1\FDRBKR-52-F1\BFDR1\WTG26|WindSpeed (First Value: 4.69192561320834 - 9/1/2016 12:01:00 AM)

2016-10-11 11:18:42,434 INFO  - NRG.PIBIAS.LostEnergy.LostEnergyCalculationManager - Added 43200 points for \\WNTRNAFP01\Corporate\Texas Wind\STWD\STWD1\SUB1\FDRBKR-52-F1\BFDR1\WTG26|WTOperationState (First Value: 0 - 9/1/2016 12:00:00 AM)

2016-10-11 11:18:43,637 INFO  - NRG.PIBIAS.LostEnergy.LostEnergyCalculationManager - Added 43200 points for \\WNTRNAFP01\Corporate\Texas Wind\STWD\STWD1\SUB1\FDRBKR-52-F1\BFDR1\WTG26|ErrorCode (First Value: 0 - 9/1/2016 12:00:00 AM)

2016-10-11 11:22:52,541 INFO  - NRG.PIBIAS.LostEnergy.LostEnergyCalculationManager - Added 33121 points for \\WNTRNAFP01\Corporate\Texas Wind\STWD\STWD1\SUB1\FDRBKR-52-F1\BFDR1\WTG26|ActivePower (First Value: 76.5700232357419 - 9/1/2016 12:01:00 AM)

2016-10-11 11:27:38,022 INFO  - NRG.PIBIAS.LostEnergy.LostEnergyCalculationManager - Added 2884 points for \\WNTRNAFP01\Corporate\Texas Wind\STWD\STWD1\SUB1\FDRBKR-52-F1\BFDR1\WTG27|WindSpeed (First Value: OSIsoft.AF.PI.PITimeoutException: [-10722]

PINET: Timeout on PI RPC or System Call.

   at OSIsoft.AF.PI.PIException.ConvertAndThrowException(PIServer piServer, Exception ex, String message)

   at OSIsoft.AF.PI.PIPoint.Summaries(IList`1 intervalDefinitions, Boolean reverseTime, AFSummaryTypes summaryType, AFCalculationBasis calcBasis, AFTimestampCalculation timeType)

   at OSIsoft.AF.Asset.DataReference.PIPointDR.Summaries(AFTimeRange timeRange, AFTimeSpan summaryDuration, AFSummaryTypes summaryType, AFCalculationBasis calcBasis, AFTimestampCalculation timeType)

   at OSIsoft.AF.Data.AFData.SummariesEvaluate(AFTimeRange timeRange, AFTimeSpan summaryDuration, AFSummaryTypes summaryType, AFCalculationBasis calcBasis, AFTimestampCalculation timeType) - 9/1/2016 12:00:00 AM)

2016-10-11 11:27:39,272 INFO  - NRG.PIBIAS.LostEnergy.LostEnergyCalculationManager - Added 43200 points for \\WNTRNAFP01\Corporate\Texas Wind\STWD\STWD1\SUB1\FDRBKR-52-F1\BFDR1\WTG27|WTOperationState (First Value: 0 - 9/1/2016 12:00:00 AM)

2016-10-11 11:27:40,202 INFO  - NRG.PIBIAS.LostEnergy.LostEnergyCalculationManager - Added 43200 points for \\WNTRNAFP01\Corporate\Texas Wind\STWD\STWD1\SUB1\FDRBKR-52-F1\BFDR1\WTG27|ErrorCode (First Value: 0 - 9/1/2016 12:00:00 AM)

2016-10-11 11:29:46,078 INFO  - NRG.PIBIAS.LostEnergy.LostEnergyCalculationManager - Added 43200 points for \\WNTRNAFP01\Corporate\Texas Wind\STWD\STWD1\SUB1\FDRBKR-52-F1\BFDR1\WTG27|ActivePower (First Value: 197.568193716581 - 9/1/2016 12:01:00 AM)

 

It appears that sometimes, it will only retrieve a small subset of the expected values (i.e. 2884) or, more commonly, will return only the first 33121 values (a full month's worth would be 1440 minutes/day x 30 days = 43200 values).

 

I took a look at KB3224OSI8, which recommended changing a couple tuning parameters. I adjusted the Archive_MaxQueryExecutionSec parameter from it's default value of 260 seconds to 600 seconds, but this did not seem to make any difference (it seems like any query that has issues will run for approximately 5 minutes before returning). I'm wondering what other things I could try (either at the PI archive level or within the AF SDK) that would increase the reliability of these queries. One other option I've considered is introducing a loop that will retry the query (up to a certain max # of tries) if it does not contain the expected number of results.

 

Thanks,

Trent

Outcomes