rdavin

What's new in AFSearch 2.10.5 (PI Server 2018 SP2)

Blog Post created by rdavin Employee on Jun 17, 2019

I have been talking about AFSearch 2.10.5 for well over a year now. I spoke about it at PI World SF in April 2018 and again at Barcelona in Sept 2018. However, both of those talks used a beta build. The great news is that for the week of PI World SF in April 2019, this is no longer beta and is now production! Sadly, when working with beta code one should accept that the final production version may be a bit different, and that's certainly the case here. As I dusted off my code from Sept 2018, I could see it had several compiler errors since there was some fairly significant changes on the journey to production. Such is the life of a beta tester.

 

In case you haven't been following AFSearch or PI Server 2018 SP2 at all in the past year, the big changes for AFSearch with AF SDK 2.10.5 will be concisely summarized as:

 

  • Overlapped event frames searches got a performance boost.
  • Search queries now support OR conditions.
  • A new generic AFSearch<T> base class was added to implement many of the common properties and methods in derived search classes. As a result of this change, several methods have been marked as obsolete.

 

Don't let the short list fool you. While the first item requires no changes upon your part, the other items do require code changes. Supporting OR conditions is easy to say, but it did cause some breaking changes. In particular the old AFSearchToken structure is insufficient to represent new filters with OR. To that end, the AFSearchToken structure is being marked as Obsolete and replaced with a new AFSearchTokenBase class. Likewise, a generic AFSearch<T> base class introduces an IAFSearch<T>.FindObjects method, which means the many of the older Find methods (e.g. FindElements or FindEventFrames) are also marked as Obsolete. This is all mentioned in the help file under the heading What's new in PI AF 2018 SP2, but this blog expands upon short summary to give examples.

 

Event Frame Search Performance Boost

 

Let's start with the simplest change. If you are using captured event frames, then some searches should be noticeably faster, particular searches for Overlapped event frames.  For those of you rusty on the subject, Overlapped event frames are also considered "Active Between", meaning that any portion of the event frame falls within a requested time range.  I refer you to the image found in this link if you want more information.  Side note: when I was a customer writing AF applications, Overlapped was our most frequently used mode, and I know this is true for many of you.

 

An anecdotal example of this performance boost can be found in the PI World talks in 2018. In April, it took me about 1 minute to find 320K event frames from an AF Server of 6 million event frames. In September, I was using the same hardware but updated the AF SDK build every couple of weeks. About 3 weeks before Barcelona, the 1 minute search suddenly dropped to around 15 seconds. I asked the dev team and they confirmed they had implemented new logic for event frame searches.  I then gratefully said "thank you" and wiped the tears from my eyes.

 

This requires no change on your part other than the acceptance that your code may run twice as fast. What a nice problem to have!

 

OR - small word, BIG Changes

 

To implement support for OR, it quickly became apparent that the old AFSearchToken was not going to get the job done. There was a simplicity with the original AFSearchToken, but this breaks with the complexity of OR searches. There was just no way to represent it with the old structure. AFSearchToken is marked as obsolete, and any Tokens properties that depended upon AFSearchToken are also marked as obsolete. If your search does not use OR, then it will will get parsed by the Tokens property just fine (or with a little compiler warnings). But if you do use OR in your search, then accessing the Tokens property will produce an exception. Instead, you should begin using the TokenCollection property.

 

The TokenCollection property returns an AFSearchTokenCollection object, which implements IList<AFSearchTokenBase>. If you dig deeper into the help, you may discover that AFSearchTokenBase is an abstract class with 4 derived classes:

 

  • AFSearchExpressionToken, representing a logical grouping of search tokens. These tokens would be grouped together based on an OR or the default AND. I jokingly add that this is also akin to adding virtual parenthesis around your expression. All joking aside, if a query string contains parentheses, then the tokenized equivalent will likely need to use AFSearchExpressionToken.
  • AFSearchFilterToken, representing one criteria of a search class. Most of the obsolete AFSearchToken searches would use the new AFSearchFilterToken.
  • AFSearchQueryToken, representing a nested search query. Some obsolete AFSearchToken searches did use nested queries but should now use this AFSearchQueryToken.
  • AFSearchValueToken, representing an attribute value query.

 

Let's walk through an example to see these new tokens in action. I have chosen an example that uses all 4 of the new tokens. Let's first state how we plan to filter using simple phrases and business terms.

 

  • I want to perform an AFAttributeSearch, which means I will need a nested element query as well.
  • I want to search attributes on elements that have been derived from the "Storage Tank" element template.
  • I want to refine the element search to limit tank names (element names) to those starting with the letter "T".
  • I want a value filter on the attribute named "|Fill Percent". That filter should be on any tank level <= 10% or >= 90%. That is I want to filter on tanks that are too low or too high.
  • I want to return attributes named "Fill Percent", which is the tank level per tank (element).

 

Let's see how that would look as a query string. I personally like query strings, and many of the devs on the Asset Team prefer them as well.

 

Let's cheat a little and start out with what the fully resolved query string would look like:

 

"Element:{Template:'Storage Tank' Name:T* ('|Fill Percent':<=10 OR '|Fill Percent':>=90)} Name:'Fill Percent'"

 

Note that anything between the red { } braces is the nested element query. Within the confines of those red braces, the context for Name (as in Name:T*) refers to the nested element query, so Name in this context means element name. Outside the red braces, you will see another Name (as in Name:'Fill Percent'), whose context refers to the attribute search, and therefore is an attribute name. (This example was specifically chosen to throw a curveball at you, forcing you to reconcile which Name is which).

 

You will note that any names containing embedded blanks are wrapped in quotes. You are free to create a longer and uglier string using escaped double quotes, but my eyeballs are much happier with the single quotes.

 

Okay, now let's see how that would be written with the new AFSearchTokenBase tokens. There are a few different ways, none of which make your code shorter. First, let's define some variables and constants to hold many of the names and limit values.

 

const string templateName = "Storage Tank";
const string elemName = "T*";
const string attrName = "Fill Percent";  // Note the pipe or bar symbol "|" is omitted
const double lowLimit = 10;
const double highLimit = 90;

 

Let me point out that attrName is used when a name is required. However, many of the value filters requires an attribute path. Later below you will see me preface attrName with a "|" to transform it into a path. Wherever this is done, you will also see me wrap the final text in single quotes to account for embedded blanks in the name.

 

This will be done in several pieces of code. I try to work with the innermost filter first, so that would be the OR clause wrapped in parenthesis.

 

AFSearchExpressionToken orExprToken = new AFSearchExpressionToken(AFSearchLogicalOperator.Or, new List<AFSearchTokenBase> {
    new AFSearchValueToken($"|{attrName}", lowLimit.ToString(), AFSearchOperator.LessThanOrEqual, AFSearchValueType.Numeric),
    new AFSearchValueToken($"|{attrName}", highLimit.ToString(), AFSearchOperator.GreaterThanOrEqual, AFSearchValueType.Numeric) });

 

That is one really long assignment. If you like breaking it down into more lines, then you may use this instead:

 

// First let's compose an OR condition for our limit extremes.
IList<AFSearchTokenBase> orChildTokens = new List<AFSearchTokenBase>();
orChildTokens.Add(new AFSearchValueToken($"|{attrName}", lowLimit.ToString(), AFSearchOperator.LessThanOrEqual, AFSearchValueType.Numeric));
orChildTokens.Add(new AFSearchValueToken($"|{attrName}", highLimit.ToString(), AFSearchOperator.GreaterThanOrEqual, AFSearchValueType.Numeric));

// Next the OR must be grouped as a single expression, which is akin to enclosing it in parenthesis.
AFSearchExpressionToken orExprToken = new AFSearchExpressionToken(AFSearchLogicalOperator.Or, orChildTokens);

 

This demonstrates 2 different ways to populate the expression token. The first way created the tokens argument inside the AFSearchExpressionToken constructor. The second way created the expression tokens before the AFSearchExpressionToken was constructed. There are 2 other noteworthy items along this topic. First, and this is a big change, is that you cannot modify the input tokens once the instance has been constructed. Second, is that you may expand this thinking to include any search-related constructor expecting an argument of IList<AFSearchTokenBase> such as constructors for AFSearchQueryToken, AFElementSearch, AFAttributeSearch, etc.

 

Either of the above code snippets fill in the first major piece towards a fully tokenized call. The second major piece will be to complete the nested element query token. Note we use the orExprToken object we defined above.

 

// Gather tokens for the nested element query, which will include the OR expression group.
IList<AFSearchTokenBase> nestedQueryTokens = new List<AFSearchTokenBase>();
nestedQueryTokens.Add(new AFSearchFilterToken(AFSearchFilter.Template, templateName));
nestedQueryTokens.Add(new AFSearchFilterToken(AFSearchFilter.Name, elemName));
nestedQueryTokens.Add(orExprToken);

 

The final piece will be to complete the attribute search query by combining the nested element query with other attribute search filters.

 

// Finally gather tokens for the AFAttributeSearch, which includes the nested element query
// and the attribute name filter.
List<AFSearchTokenBase> tokens = new List<AFSearchTokenBase>();
tokens.Add(nestedQueryTokens);
tokens.Add(new AFSearchFilterToken(AFSearchFilter.Name, attrName));

 

Alternatively, you could have also performed the last assignment with:

 

List<AFSearchTokenBase> tokens = new List<AFSearchTokenBase>() { nestedQueryTokens, new AFSearchFilterToken(AFSearchFilter.Name, attrName) };

 

That wraps it up for the new tokens. Again, the complexity of using the new tokens was a necessary evil to accommodate the robust requirements of OR clauses.

 

If you straddle the border on whether to use tokens or strings, I would encourage you to learn more about query strings. Meanwhile, let's return that original query string. As I said, this has been fully resolved:

 

"Element:{Template:'Storage Tank' Name:T* ('|Fill Percent':<=10 OR '|Fill Percent':>=90)} Name:'Fill Percent'"

 

But in many cases this won't be fully resolved as many of your parameters are most likely stored in variables. What if you wanted to write the string to have variable substitution? Using String Interpolation, it would look like:

 

string query = $"Element:{{Template:'{templateName}' Name:'{elemName}' ('|{attrName}':<={lowLimit} OR '|{attrName}':>={highLimit})}} Name:'{attrName}'";

 

You will note that creating query is where we insert the pipe symbol. If you are more familiar with String.Format, you could use:

 

string query = String.Format("Element:{{Template:'{0}' Name:'{1}' ('|{2}':<={3} OR '|{2}':>={4})}} Name:'{2}'", templateName, elemName, attrName, lowLimit, highLimit);

 

I personally favor the Interpolated String because I know which variable is being used in-place, rather than we sequential numbers that require more eye movement.

 

New Methods and Constructors

 

It should be no surprise that the introduction of new TokenCollection and associated AFSearchTokenBase classes require new constructors and methods. This touches on virtually every related search class. Nothing much to add other than to make you aware of this. There is no need to enumerate them here since it's already been covered in Live Library's What's New link.

 

The Obsolete Find Methods

 

Under the hood, there is a significant change to the AFSearch abstract class. Prior to AF SDK 2.10.5, the class signature was:

 

public abstract class AFSearch : IDisposable

 

Effective with 2.10.5, the signature is now:

 

public abstract class AFSearch<T> : AFSearch, IAFSearch<T> where T : AFObject

 

This means the AFSearch class can now implement many of the common properties and methods in each of the derived search classes. One such common method is the FindObjects method, which is an implementation of the IAFSearch<T>.FindObjects interface.

 

Given how each derived search class has a FindObjects method, there is no longer a need to have each derived search class with its own type-specific Find method. Towards that end, the type-specific methods are marked as obsolete. This means you should start using FindObjects in place of:

 

  • AFAnalysisSearch.FindAnalyses
  • AFAnalysisTemplateSearch.FindAnalysisTemplates
  • AFAttributeSearch.FindAttributes
  • AFCaseSearch.FindCases
  • AFElementSearch.FindElements
  • AFEventFrameSearch.FindEventFrames
  • AFNotificationContactTemplateSearch.FindNotificationContactTemplates
  • AFNotificationRuleSearch.FindNotificationRules
  • AFNotificationRuleTemplateSearch.FindNotificationRuleTemplates
  • AFTransferSearch.FindTransfers

 

Again, don't be thrown off by the generic sounding FindObjects. Calling AFElementSearch.FindObjects still returns IEnumerable<AFElement>, so only the method name has changed.  FindObjects still returns type-specific items.

 

Team Query String versus Team Tokens

 

AF SDK is flexible in that you may choose to use either query strings or tokens, just as you may choose to use C#, VB.NET, or managed C++. As mentioned earlier, the Asset developers strongly prefer query strings.

 

I admit the new AFSearchTokenBase class is much more complicated to work with than the older AFSearchToken structure. I talked to several customer developers and many said they like tokens that spread out over many lines rather than a single-line query string because they can quickly comment out individual lines. With the new AFSearchTokenBase, you can still comment individual lines. However, there's nothing stopping you from having a multi-line query string or better yet, a StringBuilder object. With a StringBuilder, you would still be passing a string to the search constructor, but you would have the query span multiple lines where you may comment a given line.

 

Let's see an example of this in action. We return once again to the fully-resolved, single-line query string:

 

"Element:{Template:'Storage Tank' Name:T* ('|Fill Percent':<=10 OR '|Fill Percent':>=90)} Name:'Fill Percent'"

 

We can convert that a multiple line StringBuilder object. We really should have a separate StringBuilder object for the nested query and another for the full query.

 

StringBuilder nested = new StringBuilder();
nested.Append("Element:{");  // begin nested query
// Each inner filter will start with a blank
nested.Append($" Template:'{templateName}'");
nested.Append($" Name:'{elemName}'");
nested.Append($" (");  // begin OR expression
nested.Append($" '|{attrName}':<={lowLimit}");
nested.Append($" OR");
nested.Append($" '|{attrName}':>={highLimit}");
nested.Append(")");  // end OR expression
nested.Append("}");  // end nested query

StringBuilder query = new StringBuilder();
query.Append(nested.ToString());
query.Append($" Name:'{attrName}'");

using (AFAttributeSearch search = new AFAttributeSearch(root.Database, "", query.ToString()))
{
    // do something here
}

 

(A brief aside before we go any further, I want to say this to anyone who wants to have multi-line queries (string or tokens) in their code: I HOPE YOU ARE HAPPY. We have taken a single elegant line of a query string, and turned it into over a dozen lines of ugly code. Sorry. I needed to get that off my chest. Back to the lesson at hand ... )

 

Now let's consider what we would do if we want to temporarily change the search with these new rules:

 

  • We do not want to filter tank elements by name
  • We do not want to filter on low limit, which means we no longer need an OR clause. Another way to say this is we only need a value filter on high limit.

 

We would take the above and temporarily comment out lines accordingly:

 

StringBuilder nested = new StringBuilder();
nested.Append("Element:{");  // begin nested query
// Each inner filter will start with a blank
nested.Append($" Template:'{templateName}'");
//nested.Append($" Name:'{elemName}'");
//nested.Append($" (");  // begin OR expression
//nested.Append($" '|{attrName}':<={lowLimit}");
//nested.Append($" OR");
nested.Append($" '|{attrName}':>={highLimit}");
//nested.Append(")");  // end OR expression
nested.Append("}");  // end nested query

StringBuilder query = new StringBuilder();
query.Append(nested.ToString());
query.Append($" Name:'{attrName}'");

using (AFAttributeSearch search = new AFAttributeSearch(root.Database, "", query.ToString()))
{
    // do something here
}

 

There you go! We have demonstrated 3 examples for declaring a search query:

 

  • Using the new AFSearchTokenBase class
  • Using a single line query string
  • Using multi-lines of StringBuilder(s)

 

We leave it to your personal coding style and preference for what works best for you.

 

Caution using different Server and Client Versions

 

Let me issue this general caution regarding using different AF Server and AF Client version: while your app may compile correctly and run on mismatched versions, the features available would depend upon which system has the lowest version.  A client running 2.10.5 or later may be aware of the OR expression but a server running 2.10.0 or earlier would not.  Thus that app would be limited to 2.10.0 features.  If the client app tried to send an OR expression to the 2.10.0 server, an unsupported feature exception would be returned.

 

A very general rule of thumb: an older client app running against a newer server version should most likely run without issues, but a newer client app running against an older server version could throw unexpected errors.

 

 

Past AFSearch Blogs or Posts

 

Getting the most out of AFSearch - PI World Barcelona - Sept 2018. This used a beta of AF 2.10.5 to introduce OR support. Great video to help you understand the inner workings of AFSearch. This customer question hits on all the key points of the Barcelona Live Coding talk, and shows where lightweight searches can be great event frame searches with captured values. This Tech Talk was a refined repeat of May's Live Coding Talk at PI World 2018 San Francisco.

 

What's new in AFSearch 2.10 (PI AF 2018) (June 2018) - attribute searches are no longer are restricted to templates.

 

What's new in AFSearch 2.9.5 (PI AF 2017 R2) (March 2018) - AFAttributeSearch is introduced.

 

Coding Tips for using the AFSearch IN Operator - you may search for multiple values using IN(value1; value2; etc.). Some code is offered to make this easier to generate. This explains embedded blanks and using escaped double quotes or single quotes.

 

Using the AFEventFrameSearch Class (Nov 2016) - Giving some attention to event frames since many of earliest AFSearch examples were element based.

 

Aggregating Event Frame Data Part 1 of 9 - Introduction (May 2017) - From the UC 2017 TechCon/DevCon hands on lab for Advanced Developers.

 

Why you should use the new AF SDK Search (June 2016) - The granddaddy of them all. The earliest post explaining AFSearch, which was new at that time.

Outcomes