The Advanced AF SDK lab at UC SF 2017 was on this very topic.  The material in this 9-part series follows much of that lab which showcases AFEventFrameSearch methods new to PI AF SDK 2.9.

 

Blog Series: Aggregating Event Frame Data

Part 1 - Introduction

Part 2 - Let's Start at the End

Part 3 - Setting up the App

Part 4 - Classical FindEventFrames

Part 5 - Lightweight FindObjectFields

Part 6 - Summary per Model

Part 7 - GroupedSummary Per Manufacturer

Part 8 - Compound AFSummaryRequest

       Part 9 - Conclusion

 

Let's Make Only ONE Call

I'm going to assume that you haven't jumped blindly into this topic for the first time.  It should be a safe bet that you've read Parts 6 and 7 regarding Summary and GroupedSummary respectively.  It has boiled down to this: I don't want to make repeated calls, because we know each call to the server takes a performance hit.  Summary needed to be called 3 times for this use case, and GroupedSummary twice.  I want to issue one and only one call.  Plus I don't want to have know to know all Manufacturers or Models before I query for them.  Maybe I absolutely don't know that info ahead of time and that's why I'm doing this search in the first place.

 

Which brings us to AFSummaryRequest, which will perfectly fit the bill.  Technically it is not a method to the AFEventFrameSearch or AFSearch classes.  It's a concrete method in the OSIsoft.AF.Data namespace that implements the abstract method AFAggregateResult.  Other aggregation methods, like Summary and GroupedSummary, call AFSummaryRequest themselves for the most common method signatures that need a 1-level summary.  As we want a 2-level grouping, AFSummaryResult is the best choice for our use case.  With that in mind, we should be forgiving that the code to use it is a bit more complicated.  Note that AFSummaryRequest limits you to no more than 2 groupings.

 

public void GetSummaryByMfrAndModel(StatsTracker summary, AFDatabase database, IList<AFSearchToken> tokens)
{
    using (var search = new AFEventFrameSearch(database, "Compound Request", tokens))
    {
        //Opt-in to server side caching
        search.CacheTimeout = TimeSpan.FromMinutes(5);

        //While we eventually want an average, it will be calculated from Total and Count.
        var desiredSummaryTypes = AFSummaryTypes.Count | AFSummaryTypes.Total;

        //Here we make only 1 call to the server but we must build a compound AFSummaryRequest.
        //The GroupBy order is opposite than what you would intutively think: Model and then Manufacturer.
        //First we bundle the AFSummaryRequest.
        var compoundRequest = new AFSummaryRequest("Duration", desiredSummaryTypes)
                                    .GroupBy<string>("|Model")
                                    .GroupBy<string>("|Manufacturer");

        //We send the request as a member of IEnumberable<AFAggregateRequest>.
        //Since we pass a collection of one member, we get a collection of one member back.
        //So we grab that one member and cast it appropriately.
        var aggResult = search.Aggregate(new[] { compoundRequest })[0] as AFCompoundPartitionedResult<string, string>;

        //Unwrap the results.
        foreach (var kvp in aggResult.PartitionedResults)
        {
            var mfr = kvp.Key.PrimaryPartition;
            var model = kvp.Key.SecondaryPartition;

            var summaries = kvp.Value;

            var totalVal = summaries[AFSummaryTypes.Total];
            var countVal = summaries[AFSummaryTypes.Count];
            var stats = new DurationStats();

            if (countVal.IsGood)
            {
                stats.Count = countVal.ValueAsInt32();
                if (totalVal.IsGood)
                {
                    stats.TotalDuration = ((AFTimeSpan)totalVal.Value).ToTimeSpan();
                }
                summary.AddToSummary(mfr, model, stats.TotalDuration, stats.Count);
            }
        }
    }
}

 

 

There you have it.  Not exactly pretty.  But you can't argue with the results since this was the fastest method for my use case.

 

I want to reiterate that you may have no more than 2 levels of grouping for AFSummaryRequest.  And review lines 15-16 to see that the grouping is inside to outside.  That is if we want to group by Manufacturer first and Model second then when we compose the AFSummaryRequest the first GroupBy is by Model and the second GroupdBy is Manufacturer.

 

Metrics Comparison (from Part 2)

The numbers below are from a 2-core VM using Release x64 Mode.  The smaller values are better.  Caution that we sometimes have a difference in UOM between MB and KB, but I will bold KB when needed.

 

Resource Usage:

Values displayed are in MB unless noted otherwise

Method

Total GC Memory (MB)

Working Set Memory (MB)Network Bytes Sent
Network Bytes Received
FindEventFrames145.48257.089.13 MB190.08 MB
FindObjectFields1.2865.555.00 KB3.68 MB
Summary2.5455.358.58 KB261.81 KB
GroupedSummary9.8664.286.24 KB1.98 MB
AFSummaryRequest7.2965.365.00 KB3.68 MB

 

Performance:

MethodClient RPC CallsClient Duration (ms)Server RPC CallsServer Duration (ms)Elapsed Time
FindEventFrames12063337.011039118.102:27.8
FindObjectFields105360.8114547.600:06.0
Summary159484.6169310.900:10.1
GroupedSummary125527.2134938.500:06.2
AFSummaryRequest102992.2102222.200:03.7

 

We are at the same spot where we ponder if AFSummaryRequest is the fastest of the methods.  It appears to be so for my particular use case of reporting by Manufacturer and Model.  If we were to ignore my use case, and compare AFSummaryRequest to Part 6's Bonus Summary and Part 7's Bonus GroupedSummary, both of which issued one call on the same data set, here's how those metrics line up:

 

Metric
SummaryGroupedSummaryCompound AFSummaryRequest
Total GC Memory (MB)4.4812.247.29
Working Set Memory (MB)52.4861.8465.36
Network Bytes Sent4.77 KB4.85 KB5.00 KB
Network Bytes Received260.02 KB1.98 KB3.68 MB
Client RPC Calls101010
Client Duration (ms)534.01913.72992.2
Server RPC Calls101010
Server Duration (ms)353.81472.82222.2
Elapsed Time00:01.100:02.600:03.7

 

I know I sound like a broken record but the same advice applies: think through your application and pick the right tool for the right job.  Which one gives the correct results with making the fewest calls to the server?

 

Up Next: End of the Series

We conclude this 9-part series naturally enough with post that I call Part 9!