Skip navigation
All Places > PI Developers Club > Blog > Authors Rick Davin
1 2 3 Previous Next

PI Developers Club

39 Posts authored by: Rick Davin Employee

The PI Geek Talks are generally presented by partners and customers.  The target audience is PI Admins and Developers.  All the talks will be on Wednesday, Day 2, at the Parc 55 Hotel in the Powell room located on Level 3.  You are invited to read the PI World Agenda for more information.  Along with Day 2 Tech Talks, these are great reasons to take the walk from the Hilton to the Parc 55.

 

 

Selecting the Right Analytics Tool

David Soll, Omicron

There are several analytics tools and approaches available for working with PI data: Performance Equations, AF analytics, custom data references, PI ACE, PI DataLInk and Business Intelligence (BI) tools. It can be a quandary in determining which tool should you use for what. Should you focus on only one tool or use a mix? As it turns out, the answer is not as simple as basing it on the specific analytic. Other considerations should be put into the decision including: scalability, reliability, maintainability, and future-proofing, to name a few.

This talk will discuss the various tools available for performing analytics on PI data and their strengths and weaknesses, their scalability, reliability, maintainability, and future-proofing. The tools will be separated into two major classes: server side (persistent) analytics and client side (query time) analytics and the general differences between the two classes. Attendees will learn practical guidelines to for selecting analytics tools

David Soll

 

 

Providing Enterprise Level Visibility for Snowflakes Using PI and AF

David Rodriguez, EDF Renewables, and Lonnie Bowling, Diemus

As part a larger project to monitor a large number of distributed wind farms throughout the US and Canada, the customer desired to have visibility into substation status information. This included showing substation one-line diagrams, voltage regulation status, breaker status, and events to notify them of any issues. Each wind project was design and installed by others which resulted in large differences between sites, include variability in networking, communications, and tag configuration. In other words, each project was like a snowflake. Using PI, AF Analytics, and Event frames, a solution was developed to normalized all wind projects. Once standardization was achieved, we then defined substation one-line circuits using an AF hierarchy. Data visualization was developed to provide on-demand, real-time rendering of circuits, voltage regulation trends, events, supporting information. This was implemented enterprise wide, and allowed for easy access and visibility for everyone in the organization.

David Rodriguez   Lonnie Bowling

 

Just Another Weather Application – Evaluating the OSIsoft Cloud System

Lonnie Bowling, Diemus

This session will showcase a weather application designed using the new OSIsoft Cloud System (OCS).

A backyard weather station was used as a data source for a live and historical data source. Forecasted data was then added to provide a complete picture of historical, current, and forecasted weather. Once all the data was streaming into an OCS sequencial data store, a full stack front-end solution was developed. This included an API layer in C#, Angular for the UI, and D3 for data visualization. A complete solution was developed to fully evaluate how OCS could be used in a real-life, purpose-built application. Key takeaways, including challenges, an architectural review, and source-code highlights will be shared.

Lonnie Bowling

 

Data Analytics to enhance Advanced Energy Communities planning and operation

John Rogers and Alberto Colombo, DERNetSoft

In today’s energy marketplace, poor energy awareness and a lack of data visibility coupled with the technical complexities of DER integration leads to a gap in local Advanced Energy Community development. DERNetSoft provides a scalable solution to this issue, making it possible to build advanced energy communities increasing energy awareness, enabling Distributed Energy Resources planning and supporting their operational optimization. We transform data into actionable insight and value-added advanced analytics and machine learning technique in the energy industry at the community level.

 

 

Data Quality & Shaping: Two Keys to Enabling Advanced Analytics & Data Science for the PI System

Camille Metzinger and Kleanthis Mazarakis, OSIsoft

Data quality is critical in the success of data-driven decisions. Issues with data will impact users across the organization- from operators, engineers, data scientists, to leaders. Answering business intelligence questions such as “which assets are performing well and which are under-performing” requires a birds-eye view of the data which may require (re)shaping of the data within the PI System. This talk and demo will explore the aspects of data quality and data shaping using PI System infrastructure by illustrating why they are so critical for success. We will also demonstrate the steps of how to improve Data Quality in the PI System and shape the PI System data to give it the right context for your advanced analytics.

Camille Metzinger   Kleanthis Mazarakis

In about a month, PI World 2019 will kick off in San Francisco.  Like the past many years, the events will be spread over 3 hotels.  Also like the past many years, the Parc 55 hotel is where you will find events catering specifically to developers and the data science community.  A year ago we introduced "Live Coding" sessions and "How To" walk-throughs to offer a more indepth talk (more steak, less sizzle).  This year we have collectively rebranded these new formats into a common track: Tech Talks.  These 90 minute talks hit the sweet spot for many.  If you leave a traditional 45 minute PowerPoint talk wishing for more details, then the longer 90 minute Tech Talk is for you.  If you feel a 3 hour lab is too slow, or you would rather be shown the material rather than typing it yourself, then the shorter 90 minute Tech Talk is for you too.

 

 

We have expanded the Tech Talks to begin on Day 2 rather than wait until Day 3.  Here's what you can find on the agenda.

 

Day 2 Tech Talks (Parc 55)

  • Using Stream Views for Real Time Analytics
  • Using PI Web API and PowerApps to Build Real World Apps With Your PI Data
  • Leveraging the Power of PI Vision Extensibility
  • Concurrent Programming for PI Developers
  • Generating API Clients with OpenAPI 2.0 (Swagger) specifications and interacting with REST endpoints
  • Effortlessly deploying a PI System in Azure or AWS

 

Day 3 Tech Talks (Parc 55)

  • OSIsoft Cloud Services for Developers
  • Writing High Performance Applications with AF SDK
  • Modernizing PI SQL For the Future
  • Create Dashboards to monitor PI Analysis Service

 

Check out the agenda for exact times and room locations.  While you are peeking at the agenda take a look at Day 2 PI Geek Talks and Day 3 Developer Talks too.  All are offered at the Parc 55.

 

And join us from 4:30-6:00 PM on Day 3 for the Developers Reception.

Dear Fellow PI Geeks,

 

It is with a heavy heart that I must announce that we have cancelled this year's PI World Innovation Hackathon.  It was not an easy decision.  We were given a cutoff date 8 weeks before PI World to make a decision to go forward or cancel.  It's hard to make firm predictions so far in advance.  While we were confident optimistic we could have a similar number of participants as last year, the tipping point in the decision was the bitter reality that the hackathon has been shrinking in attendance for many years.  When we started out on this brave new venture 7 years ago, we obviously were filled with hopes that it would grow rather than decline.

 

I have received emails asking if we will offer hackathons in the future.  YES, we will.  However, I do not see us offering hackathons during PI World.  Many of our partners and EA customers form the core of the developer community, but PI World has so many demands pulling those partners and customers in so many different directions.  Thus, we are currently considering a special hackathon-only event.

 

I will be hanging out at the Parc 55 hotel on Day 2 and Day 3, where the agenda has lots of developer related offerings from 90 minute Tech Talks, some cool PI Geek talks, and the traditional 45 minute Developer Talks.  Not to mention the Day 3 hands on labs!  So I invite all developers to come over to the Parc 55 to attend some in-depth talks. And if you happen to bump into me, I would love to talk to you about what you want to see in future hackathons.

 

There will be a Developer Reception from 04:30-06:30 PM at the Parc 55 with drinks and appetizers.  Come meet fellow developers or supporting members of the PI Developers Club.

 

If you have any concerns or comments, please email me and/or the entire team below.

 

Rick Davin    

rdavin@osisoft.com OR TechnologyEnablementTeam@osisoft.com

Great list, though the author admits the list is more exhausting than it is exhaustive.  A must-read for VB.NET developers.

 

An Exhausting List of Differences Between VB.NET & C# | Anthony's blog

Below is a link to the presentation given at Barcelona on September 27, 2018 for the LiveCoding session titled "Getting the Most Out of AFSearch".

 

Link to Recording:     LiveCoding--Getting-the-Most-Out-of-the-AFSearch

 

GitHub Repository:     AF-SDK -PIWorld-EMEA-2018-AFSearch-LiveCoding

 

While some of it is a repeat of what was presented at San Francisco in April, the beginning section features new material on the upcoming AF SDK 2.10.5 release.  The new features discussed:

 

  • OR clauses in searches.
  • FindObjects replaces the now deprecated FindElements, FindEventFrames, FindAttributes, etc.
  • AFSearchToken structure being replaced by new AFSearchBaseToken abstract class with 4 concrete classes: AFSearchFilterToken, AFSearchQueryToken, AFSearchOrToken, and AFSearchExpressionToken.
  • AFSearchToken will be mapped to new AFSearchBaseToken, if possible.
  • If your query uses an OR, the AFSearch.Tokens property will throw an exception.
  • New AFSearch.TokensCollection property is meant to replace Tokens functionality and works with new AFSearchBaseToken.
  • AFSearchToken and AFSearch.Tokens property are both marked as deprecated.

ATTENTION Developers, Data Scientists, IT CyberSecurity, and Power Users.  For lack of a better word, I will refer to you collectively as "developers".

 

Developers coming to PI World Barcelona may notice more offerings than ever before.  We proudly proclaim that Barcelona will have the most robust Developer Agenda (see link) ever seen at UC or PI World EMEA.  This includes the Developer Innovation Hackathon on Day 0 (see link), thanks to our data sponsor DEME.

 

Besides the traditional hands-on labs, which require an additional fee and pre-registration, the Day 3 agenda is chock full of technical content aimed specifically at developers, thanks to 90-minute Live Coding or How-To sessions.  These in-depth technical talks do not require a fee nor pre-registration.  You are free to come and go as you please.  Make no mistake about it ... just because we call the LiveCoding "talks" does not mean they contain less technical information than labs. We expect to offer even more such talks at future PI World events because we think it is better event for you.  Our reasoning: you can attend 2 labs on Day 3 for $300, or you can sit in on 4 LiveCoding talks for free.  Who can argue against more technical training for less cost?  (Tip: if you are trying to convince your boss to send you to PI World, presenting it as a major training event (which it is) could be a strong justification to attend.)

 

I invite you once again to review the Day 3 agenda.  You will see an Analytics Track and Developer Track.  One late correction I would like to make is the PI Admin Track, which is not really for PI Admins but should be considered a 2nd Developer Track.

AFSearch Barcelona.png

 

Day 3, Thursday September 27, 11:30 - 13:00

LiveCoding: Getting the Most Out of the New AFSearch

CCIB: Room 117 134, P1 Level

 

To any members of the PI Developer Community who will be at PI World Barcelona, you are invited to join me in a presentation on new features of AFSearch.  If you ask "Hey Rick, didn't you give this already in SF?", my answer would be "Yes BUT new sections were added to specifically cover some important NEW stuff."  PI AF 2018 R2 (AF SDK 2.10.5) will finally support OR conditions with AFSearch.  That is a highly anticipated new feature that many are looking forward to.  But in order to support OR conditions, it required replacing the older AFSearchToken structure with a new AFSearchTokenBase class that now has 4 different token instances.  Trust me, you will want to see how these new tokens will be used in code.  Everyone who has ever attended this talk has said they definitely learned something!

 

UPDATE: The room has been moved to 134 on Level P1.

PI AF 2018 (AF SDK 2.10) offers a very significant change in filtering on attribute values: there is no longer a restriction that the attribute must belong to an element template.  The allows for a greater flexibility for filtering.  For example, you may now search for attributes that don't belong to any template.  Or better yet, you may search for attributes with the same name but belonging to different templates!

 

Other Features New to AF SDK 2.10

 

The bulk of this blog will cover the ValueType used with attribute value filters.  Before we dig too deep into that topic, let's take a quick look at the other new features.

 

There is a new AFSearchFilter.ID search filter to allow searching for objects by their unique identifier (GUID).  This unique ID is the fastest way to locate a given object on the AF Server.  A much-welcomed addition is that the ID supports the Equal or IN( ) operator.  If you are developing code, the best way to pass a GUID is by using ID.ToString("B").

 

New Search Fields are ConfigString, DisplayDigits, IsManualDataEntry, Trait, UOM, SecurityString, and SecurityToken.  Note that with a SecurityToken field, the FindObjectFields method would return an AFSecurityRightsToken object.

 

PI AF 2017 R2 (AF SDK 2.9.5) introduced the AFAttributeSearch and the PlugIn filter.  You could combine that filter plus the new ability to search on attributes without specifying a template.  For example, you now have the ability to perform a completely server-side search of all attributes referencing the PI Point data reference!  Stay tuned for a blog dedicated to this topic from one of my colleagues.

 

And now, the remainder of the blog will discuss the new ValueType.

 

If using an element or event frame Template

 

As mentioned earlier, previous versions required a Template for any attribute value filters.  An additional requirement was that the Template needed to be specified before the attribute value filters.  If the Template was specified after, then an exception was thrown.

 

Since AF SDK 2.10 removes the restriction on the Template, an interesting artifact is that you may specify the Template after the attribute value filter - and an exception will not be thrown.  However, we strongly recommend against this practice.  If you want to filter on a Template, we highly recommend you specify the Template first - just as you did with AF SDK prior to 2.10.  Nothing has changed with the new version here (nor should your existing code).  If you follow this advice, then you should skip the "AS valuetype" cast (more below).

 

Now let's consider if you don't follow the advice and you specify the Template after the attribute value filters.  You will need to include the "AS valuetype" cast, and its behavior will be as described in the remainder of this document.  The search will still be limited to the specified Template but as far as the attribute value filters are concerned, they will be treated as if the template was entirely omitted.  Precisely how they are treated depends on the attribute's data type and the "AS valuetype" cast you declare, which is presented in detail below.

 

Casting AS ValueType - When not using a Template

(or the Template is specified after the attribute value filters)

 

To support this new capability, there are several new things to discover with AFSearch to address the issue of a filter attribute's data type.  When based on a template, the data type is easily inferred.  What happens if a template isn't specified and the attribute does not belong to a template and/or may span different templates?  How does the search know which data type to use in the filter?  The answer is that it is left up to you (the developer) to pass the desired value type as you build the search, either by a query string or search tokens:

 

  • New AFSearchValueType enumeration
  • A new AFSearchToken.ValueType property (a string)
  • Two new AFSearchToken constructors to allow you to indicate the ValueType (also a string)

 

Golden Rule:

  • If you DO specify the template, do NOT specify the value type. 
  • If you do NOT specify a template, then you SHOULD specify a value type.

 

How carved in stone is the above "SHOULD"?  If there is any possibility whatsoever of an ambiguous interpretation between a String versus a Numeric, then you absolutely should specify the value type.  For example,  AFSearch has no way of knowing whether 1 or '1' or "1" should be a Numeric versus a String value.  Best practice: anything numeric should always specify "AS Numeric".

 

If you are using search tokens, you would use the new AFSearchToken constructors.  If you are using a query string, you would use the new AS <value type> syntax.  The available values for value type are:

  • Numeric, i.e. the literal text "Numeric"
  • String, i.e. the literal text "String"
  • EnumerationSet, the name of the applicable AFEnumerationSet.  Do NOT use the literal text "EnumerationSet".

 

Typical Scenarios with Numeric or String

 

The brief examples below look very similar with the exception of the value type designator (Numeric or String).  This bears repeating: you only need to use the AS valuetype if you do not specify a template.

 

Data Type Numeric: Integer (Byte, Int16, Int32, Int64) or Floating Point (Float, Single, or Double)

Consider if you have an attribute named RunStatus, and its data type is an Int32, where 0 means not running and 1 means running.  The query string could look like either of these:

  • "|RunStatus:1 AS Numeric"
  • "|RunStatus:'1' AS Numeric"

 

Data Type String

And if RunStatus was a String where "0" means not running and "1" means running, you would use these:

  • "|RunStatus:1 AS String"
  • "|RunStatus:'1' AS String"

 

Typical Scenarios with EnumerationSet

 

We continue covering scenarios when you do not specify a template.  We turn to another typical scenario of when the attribute's data type is an Enumeration Set.  Here it doesn't matter where the data comes from, be it a PI point, a table lookup, formula, or even a static value.  What matters to the value being filtered on is that the attribute has been declared to use an enumeration set.  The important thing is to use the AS specifier followed by the name of the enumeration set; do NOT use the literal value "EnumerationSet".

 

string attrPath = "|Classification";
string attrValue = "Commercial";
string enumSetName = "Building Type";
string query = $"'{attrPath}':='{attrValue}' AS '{enumSetName}'";

 

The above snippet safely wraps anything I created in quotes.  Note that because the enumeration set name contains a blank, I absolutely must wrap it in quotes.  In this specific case where neither attrPath or attrValue contain a blank, I could have omitted the quotes.  The value in the variable query will be:

 

"'|Classification':='Commercial' AS 'Building Type'"

 

Later when passed into an AFAttributeSearch constructor, the search instance will resolve to:

 

{|Classification:Commercial AS "Building Type"}

 

 

Cases Needing Special Consideration

 

There are special cases you may need to keep in mind beyond the typical Numeric, String, or EnumerationSet.  Obviously an attribute that is a string should use "AS String" and an attribute that has a number data type should use "AS Numeric".  What about data types that aren't so obvious?  Boolean or DateTime, for example?

 

Data Type Boolean: cast AS String

Before we even touch on enumeration sets, let's investigate another area of caution.  An attribute with a data type was Boolean is not exactly a number and not exactly a string.  As far as AFSearch is concerned, you should compare the literal values of a Boolean, namely "True" and "False", as strings.  Therefore the following filters would all be correct for a Boolean data type:

  • "|RunStatus:True AS String"
  • "|RunStatus:'True' AS String"

 

It's absolutely important with Booleans to specify "AS String".  You need to be aware that the following will quietly fail by returning 0 items:

  • "|RunStatus:True"

 

Data Type DateTime: cast AS Numeric

This can be a bit tricky.  The safest practice when dealing with DateTime attributes, whether you have a DateTime or an AFTime instance in code, is to use Round Trip Formatting.  That is to say, use ToString("O") when converting a DateTime or AFTime to string for the AFSearchToken constructor or within a query string.  And despite passing it as a string, it will actually be treated AS Numeric.  So these snippets work:

 

Variable "date" can either be DateTime or AFTime instance

  • $"|Timestamp:>'{date.ToString("o")}'"
  • $"|Timestamp:>'{date.ToString("o")}' AS Numeric"

 

The above uses an Interpolated String.  If you prefer string.Format, it would be:

  • string.Format("|Timestamp:>'{0}'", date.ToString("o"))
  • string.Format("|Timestamp:>'{0}' AS Numeric", date.ToString("o"))

 

TimeSpan with Data Type Anything: Not Supported

Some customers have attributes with data type "<Anything>" to hold a TimeSpan object.  Value filtering on such time span attributes is not supported in AF SDK 2.10.

 

If your time span attribute is defined to hold an integer or a floating point, then it would be treated as a Numeric data type (see above).

 

Digital PIPoint with Data Type Anything (not using an AFEnumerationSet): cast AS String or Omit AS specifier

For an attribute using the PI Point data reference that grabs from a Digital tag, we recommend that the attribute data type be "<Anything>".  You do not have to map the digital tag to an AFEnumerationSet.  You can if you wanted to, but that means (1) you have to copy the PIStateSet as an AFEnumerationSet, and (2) what to do with the attribute falls under "AS EnumerationSet" discussed elsewhere in this document.

 

Assuming you have a string variable named  "stateText", which contains the text of a given digital state, the following would be used to filter:

  • $"|Digital:'{stateText}'"
  • $"|Digital:'{stateText}' AS String"

 

For instance, if your digital tag used the "Modes" StateSet and you wanted to filter on those attributes with a mode of "Manual", either of these should suffice:

  • "|Digital:'Manual'"
  • "|Digital:'Manual' AS String"

 

Past AFSearch Blogs

 

What's new in AFSearch 2.9.5 (PI AF 2017 R2)  (March 2018) - AFAttributeSearch is introduced.

 

Coding Tips for using the AFSearch IN Operator - you may search for multiple values using IN(value1; value2; etc.).  Some code is offered to make this easier to generate.

 

Using the AFEventFrameSearch Class (Nov 2016) - Giving some attention to event frames since many of earliest AFSearch examples were element based.

 

Aggregating Event Frame Data Part 1 of 9 - Introduction  (May 2017) - From the UC 2017 TechCon/DevCon hands on lab for Advanced Developers.

 

PI World 2018 SF Developer Track - Getting the most out of AFSearch - (May 2018) From PI World 2018, DevCon presentation for Intermediate Developers.

 

Why you should use the new AF SDK Search  (June 2016) - The granddaddy of them all.  The earliest post explaining AFSearch, which was new at that time.

A customer recently asked about filtering on multiple values of an attribute in AFSearch.  This is easily addressed using the IN search operator in a query string, or the equivalent AFSearchToken constructor expecting an array of values.  There is one major caveat: the IN operator is not allowed for floating point types such as Single or Double, since binary floating point values are considered approximations instead of exact numbers.

 

There are a few tips to get developers pointed in the right direction.  First of all, values within the IN operator are delimited by semi-colons.  If you (like me) accidentally use a comma-delimited list, then you will receive an odd error about missing a closing parenthesis.  Then if you carefully inspect the query string, then you (like me) will start pulling your hair out because you don’t see a missing closing parenthesis!  Hopefully you may benefit from my pain by quickly switching your delimiter to be a semi-colon.

 

Another tip is that while blanks are allowed within the IN operator - or more specifically around the delimiters - any values that contain embedded blanks must be wrapped in quotes (single or double quotes).  Consider this list of cities:

 

  • London
  • Paris
  • San Francisco

 

Obviously, San Francisco needs to be wrapped in quotes due to the blank inside the name.  If the attribute path is “|City”, then you could have this C# filter in your query string:

 

string uglyQuery = "|City:IN(London; Paris; \"San Francisco\")";

 

Though ... that is a wee bit ugly.  Besides being ugly, it also might be confusing to someone using a different language, for instance, VB.NET.  A couple of weeks ago, I taught a class where a VB coder thought the unsightly \" were delimiters for AFSearch.   I explained to him that it's how C# escapes double quotes.  This can look much easier on the eyes if you just use single quotes:

 

string prettyQuery = "|City:IN(London; Paris; 'San Francisco')";

 

And it avoids any needless confusion that \" is an AFSearch delimiter.

 

This is all fine-and-dandy, but how often do you hard-code values?  It does make the example simple and straight-forward, but what if the list of items you wish to filter upon are in a list or array?  Let’s look at some code that helps us out.  After all, this is a PI Developers Club, which demands for code.  Strictly speaking, all we need for the collection of values is that they be in an enumerable collection.  Your source collection doesn’t have to be strings, but keep in mind that eventually we will need to pass a string to the AFSearch query, or if we use the AFSearchToken, then we must actually pass an array of strings.

 

For our cities example, it makes sense that the city names are strings.  Let’s not take the easy way out and make it an array of strings.  Instead we will use a list so that we can see what we must do differently to make it all work together:

 

List<string> cities = new List<string>() { "London", "Paris", "San Francisco" };

 

A tiny bit of  code is needed to put that inside a query string:

 

// A little LINQ to wrap single quotes around each city, and separate them with "; "
string delimitedCities = string.Join("; ", cities.Select(x => $"'{x}'"));

// An Interpolated String to substitute the delimitedCities.
string query = $"|City:IN({delimitedCities})";

 

This resulting value in query would be:

 

"|City:IN('London'; 'Paris'; 'San Francisco')"

 

Or if you prefer working with search tokens, this would be the equivalent:

 

// Tip: here you must pass a string array, so we must ToArray() on the cities list.
AFSearchToken token = new AFSearchToken(AFSearchFilter.Value, AFSearchOperator.In, cities.ToArray(), "|City");

 

If we had defined cities to be a string array, we would not need the ToArray(), but then this example would be boring and less educational.

 

What if our enumerable collection isn’t a bunch of strings?  Let’s say we have a bunch of GUID ’s.  (Exactly how you got these GUID's is an interesting question not addressed here ; suffice to say this example takes a collection of things that aren't strings and converts them to one that is.)  There would now be an extra step needed where we must convert to string.  Once we have a collection of strings we can then implement code similar to the previous examples.  Let’s imagine we have something like this:

 

IEnumerable<Guid> ids = GetListOfGuids();  // magically get a list of GUID

 

Or it could maybe have been:

 

IList<Guid> ids = GetListOfGuids();

 

Let's say we want to filter on an attribute path of “|ReferenceID”.  First let’s tackle the problem of converting a GUID into a string that is compatible with AFSearch.  This is easy enough thanks to LINQ:

 

// Nicest way to convert a GUID a string compatible with AFSearch is to use GUID.ToString("B").
IEnumerable<string> idStrings = ids.Select(x => x.ToString("B"));

 

Okay, so now we have an enumerable collection of strings.  Using what we learned in previous examples, we can knock this out:

 

// A little LINQ to wrap single quotes around each string item, and separate them with "; "
string delimitedIds = string.Join("; ", idStrings.Select(x => $"'{x}'"));

// An Interpolated String to substitute the items.
string query = $"|ReferenceID:IN({delimitedIds})";

 

Fantastic.  If you prefer AFSearchTokens, that’s easy enough as well, but we do require the idStrings to generate a string array.

 

// Tip: here you must pass a string array, so we ToArray() on the idStrings collection.
AFSearchToken token = new AFSearchToken(AFSearchFilter.Value, AFSearchOperator.In, idStrings.ToArray(), "|ReferenceID");

 

Granted our example would have been simplified if we defined idStrings to be an array in the first place, but what fun would there be in that?

 

 

VB.NET Examples

 

Some of us supporting the PI Developers Club think there should be more VB.NET examples.  Towards that end, here are code snippets for the VB coders out there:

 

    Dim uglyQuery As String = "|City:IN(London; Paris; ""San Francisco"")"

    Dim prettyQuery As String = "|City:IN(London; Paris; 'San Francisco')"

    Dim cities As List(Of String) = New List(Of String) From {"London", "Paris", "San Francisco"}

 

Cities Query Example:

 

Query String

        ' A little LINQ to wrap single quotes around each city, and separate them with "; "
        Dim delimitedCities As String = String.Join("; ", cities.Select(Function(x) $"'{x}'"))

        ' An Interpolated String to substitute the delimitedCities.
        Dim query As String = $"|City:IN({delimitedCities})"

 

AFSearchToken

' Tip: here you must pass a string array, so we ToArray() on the cities list.
Dim token As AFSearchToken = New AFSearchToken(AFSearchFilter.Value, AFSearchOperator.In, cities.ToArray(), "|City")

 

Guids Example (or something that is not a collection of strings)

 

Query String

        Dim ids As IEnumerable(Of Guid) = GetListOfGuids()   ' magically get a list of GUID

        ' Nicest way to convert a GUID a string compatible with AFSearch is to use GUID.ToString("B").
        Dim idStrings As IEnumerable(Of String) = ids.Select(Function(x) x.ToString("B"))

        ' A little LINQ to wrap single quotes around each string item, and separate them with "; "
        Dim delimitedIds As String = String.Join("; ", idStrings.Select(Function(x) $"'{x}'"))

        ' An Interpolated String to substitute the items.
        Dim query As String = $"|ReferenceID:IN({delimitedIds})"

 

AFSearchToken

        ' Tip: here you must pass a string array, so we ToArray() on the idStrings collection.
        Dim token As AFSearchToken = New AFSearchToken(AFSearchFilter.Value, AFSearchOperator.In, idStrings.ToArray(), "|ReferenceID")

 

Summary

 

If you want to filter on a variety of values for an attribute, this requires the IN search operator.

  • You may use IN in a query string or an AFSearchToken.
  • Values must be exact.  Therefore, IN does not work on Single or Double floating point data types.
  • In query strings
    • The semi-colon is the delimiter between IN values.  Example: "|City:IN(London; Paris; 'San Francisco')"
    • Values containing blanks must be enclosed in single or double quotes.  Example: "|City:IN(London; Paris; 'San Francisco')"
  • AFSearchToken
    • Values must be passed as strings in an array

I have been asked on many occasions “How can I find the first recorded event for a tag?”  The direct answer to this may be as brief as the question.  However, usually there is a lurking question-behind-the-question about what they really want to do with that first event, and if you dig even slightly deeper you will uncover the overall task they are trying to accomplish.  What you may ultimately discover is there is no need to find the first event.

 

Let’s start off with the direct, simple answer in the form of a C# extension method:

 

public static AFValue BeginningOfStream(this PIPoint tag) => tag.RecordedValue(AFTime.MinValue, AFRetrievalMode.After);

 

This works because the PI Point data reference implements the Rich Data Access (RDA) method of RecordedValue.  The earliest timestamp to query is AFTime.MinValue, that is midnight January 1, 1970 UTC.  Thanks to the AFRetrievalMode, you ask for the first value after January 1, 1970.  If it’s the earliest recorded timestamp you are only concerned with, you can use this extension method:

 

public static AFTime BeginningTimestamp(this PIPoint tag) => BeginningOfStream(tag).Timestamp;

 

For PI points, this would give you the BeginningOfStream method to go along with the built-in EndOfStream.  Before the advent of future data, the EndOfStream was simply the Snapshot.  But there are oddities related to future data, which required different handling of data compared to the traditional historical data.  Hence, Snapshot was replaced by CurrentValue, and EndOfStream was added.

 

An AFAttribute could have a BeginningOfStream method, but it doesn’t have the same nice guarantees of PIPoint.  It all depends upon the data reference being used and whether it supports the RDA Data.RecordedValue method, which is why you should properly check that it is supported before attempting to call it:

 

public static AFValue BeginningOfStream(this AFAttribute attribute)
{
    if (attribute.DataReferencePlugIn == null) // static attribute
    {
        return attribute.GetValue();
    }
    if (attribute.SupportedDataMethods.HasFlag(AFDataMethods.RecordedValue))
    {
        // Depends on how well the data reference PlugIn handles AFRetrievalMode.After.
        return attribute.Data.RecordedValue(AFTime.MinValue, AFRetrievalMode.After, desiredUOM: null);
    }
    // Fabricated answer.  Not exact that one is hoping for.  
    return AFValue.CreateSystemStateValue(AFSystemStateCode.NoData, AFTime.MinValue);
}

 

Since the value being returned may be fabricated, I would be hesitant to include a BeginningTimestamp method as it would mask the inaccuracies.  To compensate, I would think further inspection of the returned value is needed, i.e. check for “No Data”.  Such are the difficulties of trying to create a BeginningOfStream method within your code libraries.  This is why we begin to probe more and ask about your use-case, or simply “What are you really wanting to do?

 

Virtually 100% of the people asking me how to find the first value in a stream want to find it for historical PI points only.  This greatly simplifies part of the problem because there is no need to be concerned with attributes or tags with future data.  Which brings us right back to the direct answer at the top, where you may be inclined to stop.  But if you take time to dig just a little deeper into what they are really doing, the true mission is revealed: they want to copy all historical data from an old tag to a new tag.  And trust me, there are several legitimate use-cases for doing this.

 

When I hear anything about copying of historical data, the first thought I have is “How much data is in the old tag?”  There are two particular answers that require the same special handling: (a) I don’t know, or (b) a lot.

 

The real problem they need to solve isn’t finding the earliest recorded timestamp.  Rather, they may have so much data they will bump into the ArcMaxCollect limitation (typically 1.5 million data values in a single data request).  There are programming ways around ArcMaxCollect (more below) and they rely upon the PIPoint.RecordedValues method specifying a maxCount > 0 (for instance, 100K works well).  The perceived issue of knowing the earliest timestamp becomes a moot point.  The more important date is knowing the end time, that is the switch-over date from the old tag to the new tag.  Depending upon how the old tag was populated, this may very well be the EndOfStream.  But if there is a chance that the old tag could still be receiving “InterfaceShut” or “IOTimeout” messages, you will need to explicitly specify the end time.  Worrying about the earliest recorded date has been a distraction to solving the real problem.

 

What of your start time?  I would think an in-house developer should know of the earliest start of their company's archive files.  A contracted developer could use AFTime.MinValue or go with a later, but still much safer date, such as “1/1/1980”.  Which brings us back to what they really want to do: copy large or unknown amounts of data.    This has been blogged about many times before:

 

Extracting large event counts from the PI Data Archive

 

PI DataPipe Events Subscription and Data Access Utility using AF SDK - PIEventsNovo

 

GetLargeRecordedValues - working around ArcMaxCollect

The recording for this Live Coding presentation may be found at: Getting the most out of AFSearch

 

Associated code may be found on this GitHub repository:

GitHub - Rick-at-OSIsoft/AF-SDK-TechCon-2018-AFSearch-LiveCoding: Presentation also titled "Getting the most out of AFSe…

 

GitHub also includes a PDF file of the slide deck.  Topics include:

 

Demo 1: What's new in AF SDK 2.9.5 and 2.10

Demo 2: Breaking it down line-by-line to see what's happening on the client versus the server

Demo 3: Caching versus Not Caching

Demo 4: Accurate counts especially when client-side filtering is required

Demo 5: The perils of LINQ

 

Along the way we touch upon AFAttributeSearch, using the skinny, lightweight FindObjectFields with AFEventFrames, why caching is so important to performance, and why you need to properly dispose of the AFSearch objects.

On behalf of Engineering's Technology Enablement team, and in conjunction with Technical Support's Developer Technologies team, it is my privilege to announce this year's Community All-Stars.  We give special recognition to these individuals for their prolific and relentless contributions to the PI Community at large.  We thank them for sharing their experiences and knowledge for all things PI, and in particular for any posts that are developer related.  Let me add that each recipient holds near and dear to each of them the concept of "community", and even if we did not hand out such awards, or if we did not hand out any prize other than simple recognition, these individuals would still post and contribute to our PI Square Community with the same dedication for the sheer sake of expanding the knowledge base of the community.

 

I ask my fellow PI Geeks to help congratulate our deserving 2018 All-Stars!  It's interesting to note that are 3 Community All-Star and one Rising Star are from 4 different continents.

 

PI All Stars 2018.jpg

Rick Davin (far left), Paurav Joshi, John Messinger, and Dan Fishman.  Not shown: Roger Palmen.

 

 

PI Developers Community All-Stars 2018

  • Dan Fishman, Exele Information Systems.  Dan is a first time winner as Community All-Star but won twice before as OSIsoft All-Star.
  • John Messinger, Optimate Pty Ltd.  John wins for the 2nd year in a row.
  • Roger Palmen, CGI.  I don't remember a year when Roger wasn't a winner.

 

Last year as an experiment, I began following a lot of people.  Anyone who has ever been an All-Star, honorable mention, or just nominated.  I can tell you that my Inbox was slammed all year by answers from Dan, John, and Roger.  Thanks, guys.  I don't mean that sarcastically.  Each "spam" in my Inbox from you is you helping another community member, so absolutely, sincerely THANKS!

 

PI Developers Club Community Rising Star 2018

We introduced this category last year to recognize contributions and efforts on well deserving contributors that steadily add to our content here but just fall short of the elite upper strata of All-Star.  A big difference between Rising Star and All-Star is a Rising Star may only be awarded once, whereas All-Star may be won year after year (I'm looking at you, Roger).  Anyone who has ever been an All-Star or Rising Star in the past is ineligible for further consideration as a Rising Star.  Last year we handed out 3 of these as a means of playing catch-up to some folks who have been solid contributors over the year.  This year we have only one winner:

 

 

Paurav along with Dan, John, and Roger win:

  1. Recognition in the community
  2. One year free subscription or renewal to PI Developers Club
  3. One-time free registration to a 2019 PI World event (SF or EMEA)
  4. A legal waiver for them to sign
  5. An Amazon gift card worth $400 USD granted after submitting the signed legal waiver back to us

 

And let's not forget there are a lot of fine contributors within OSIsoft.  While it is our job to help customers, we do want to recognize a select few for their efforts the past year.

 

PI Developers Club OSIsoft All-Stars 2018

 

Our OSIsoft recipients win:

  1. Recognition in the community
  2. Amazon gift card worth $200 USD

 

The clock has been reset and everyone's slate is clean

It's never too early to start thinking about 2019 All-Stars.  We monitor the quantity and quality of posts from one PI World SF to the next in order to pick our winners.  The new countdown has begun.  One thing I can tell you that all of our 2018 winners have in common are the following:

  • A high volume of posts
  • Posts of high quality (lots flagged as helpful or marked as answers)
  • A strong desire to help others.

 

Absolutely, positively candidates must contribute to threads other than their own.  In short, you must show strong participation in the community as a whole.  Without a doubt this year's crop of winners have done just that.

 

CONGRATULATIONS TO ALL WINNERS!

PI World NA is next week and I am quite excited.  Join us the afternoon of Day 3 (Thursday April 26) from 4:30 - 6:00 PM PDT for the first ever Developers Meet-Up Reception, where we will also be handing out a gaggle of awards for this year's crop of Community All-Stars, winners of the Innovation Hackathon, as well as the winners of the recent Visualization Virtual Hackathon!  If you consider yourself a developer, or just wish you were a developer in-training, or are a member of PI Developers Club, or at the very least are attending any DevCon presentations or labs at Parc 55, then by all means stop in for a beer or refreshing drink, have a snack, and mingle with fellow developers.  At 5:00 PM we will begin announcing awards, where you may congratulate the winners or commiserate with the losers other not-so-fortunate participants.  Don't overdo it because at 7:00 PM the party shifts to the Hilton for the always fun PI Geek Night, with more food and drinks, games, and other hijinks.

 

What:     Developer Meet-Up Reception & Awards

Where:   Parc 55, Cyril Magnin I & Foyer, Level 4

When:    4:30 PM - 6:00 PM (awards start at 5:00 PM)

 

You may see more at the bottom of this link:  PI World for Developers: PI World - OSIsoft Users Conference 2018 (SF)

 

Since it is being held in the late afternoon and to a limited segment of PI World attendees, this is not listed as an Evening Event.  However, it does appear on the Day 3 Agenda for Parc 55 hotel under the Sessions Agenda

 

We hope to see you there!

PI AF 2017 R2 (AF SDK 2.9.5) was released shortly before 2018.  There are some exciting new features with AFSearch that should interest developers.

 

First off, I would hope any developer would always go to the Live Library What's New page for any major PI AF release.  That page gives a summary of what's new with that particular release, not just for AFSearch but for all namespaces.  Specific to AFSearch namespace, you would see the following:

 

OSIsoft.AF.Search Namespace

Two new query based search classes have been added in this release: AFAttributeSearch and AFNotificationContactTemplateSearch. The AFSearchToken.AFSearchToken(AFSearchFilter, AFSearchOperator, String, IList<AFSearchToken> ) constructor and the AFSearchToken.Tokens property have also been added to support enhanced nested query filters for searches.

The following new search tokens have been added: EventFrame, IsInternal, Parent, PlugIn, and PlugInName.

 

As a fellow developer and PI geek, when I first read that my reaction was "Cool! Nested queries!"  If you think about it, an AFAttributeSearch means that attributes are being searched upon some element(s) or event frame(s).  This implies there will first be a search for elements or event frames, followed by a search upon those results for the attributes.  This does not require you to create 2 search objects on the client.  Rather you will have 1 search object, namely an AFAttributeSearch, and it will have a nested query to filter on the desired elements or event frames.  But the nested queries are not limited to AFAttributeSearch.   For example, you may have an AFEventFrameSearch that uses a nested query for its elements.

 

Each release of AF SDK since 2.8.0 has introduced new features and capabilities to put more efficient searches at your fingertips.  For example, if you were searching on an attribute category to return many attributes per element, you could additionally filter on PlugIn or PlugInName to restrict the returned attributes to be PI points.

 

Contacts Search Example

 

Here's a quick example where I search for any contact with a name beginning with "Davi*".

 

using (AFNotificationContactTemplateSearch search = new AFNotificationContactTemplateSearch(assetServer, "contact demo", " Name:'Davi*' "))
{
    foreach (AFNotificationContactTemplate contactTemplate in search.FindNotificationContactTemplates(fullLoad: true))
    {
        Console.WriteLine($"   {contactTemplate.Name,-24} {contactTemplate.Contact}");
    }
}

 

The output:

 

   Template                 Contact

   David Burns_Email        David Burns

   David Doll_OCS           David Doll

   David Moler_Email        David Moler

 

 

Nested Queries

 

Let's take a look at 3 different examples that all do the exact same thing.  We want to perform an attribute search for any attributes named "Feed Rate" that belong to any elements whose name starts with "Boiler".  We will perform the same search 3 times, but each time how we setup the search object will be different.  The 3 techniques we will briefly cover are:

 

  • Using Nested Search Tokens
  • Using Nested Query String
  • Using Interpolated Nested Query String

 

They example code is kept simple.  We will perform an AFAttributeSearch searching for attributes found within a nested element query with the following filters:

  1. Attribute Category of "Process Monitoring"  (note the blank)
  2. Element Category of "ProcessMonitoring" (does not have a blank)
  3. Element Template of "FCC Pump Process"

 

If you've ever worked with AFSearch before, then your previous experience should have been that you could not specify Category twice on a query prior to nested queries in 2.9.5

 

Using Nested Search Tokens

 

string templateName = "FCC Pump Process";
string elemCatName = "ProcessMonitoring";
string attrCatName = "Process Monitoring";

List<AFSearchToken> nestedTokens = new List<AFSearchToken>();
nestedTokens.Add(new AFSearchToken(AFSearchFilter.Template, templateName));
nestedTokens.Add(new AFSearchToken(AFSearchFilter.Category, elemCatName));

List<AFSearchToken> tokens = new List<AFSearchToken>();
// The Element uses the nested token(s)
tokens.Add(new AFSearchToken(AFSearchFilter.Element, AFSearchOperator.Equal, null, nestedTokens));
// The Attribute uses the non-nested tokens, in this case just Category.
tokens.Add(new AFSearchToken(AFSearchFilter.Category, attrCatName));

using (AFAttributeSearch search = new AFAttributeSearch(root.Database, "nested tokens example", tokens))
{
    search.CacheTimeout = TimeSpan.FromMinutes(10);
    foreach (AFAttribute item in search.FindAttributes())
    {
        // Do something
    }
}

 

 

Using Nested Query String

 

The trick with a nested query string is that it will be enclosed in {braces}.

 

// Notice how element has nested { }. 
// The Category depends on nesting level. 'Process Monitoring' with a blank is outside the nesting,
// so it will be an Attribute Category, whereas 'ProcessMonitoring' inside the nesting is an
// Element Category.
string query = "Element:{Template:'FCC Pump Process' Category:'ProcessMonitoring'} Category:'Process Monitoring'";

using (AFAttributeSearch search = new AFAttributeSearch(root.Database, "nested query string", query))
{
    search.CacheTimeout = TimeSpan.FromMinutes(10);
    foreach (AFAttribute item in search.FindAttributes())
    {
        // Do Something
    }
}

 

 

Using Interpolated Nested Query Strings

 

With Interpolated Strings, you may easily substitute a variable's value (technically, it substitute's the string returned from the variable's ToString() method).  If you are familiar with this in either C# or VB.NET, you know that {braces} are used.  This raises an interesting question of how the Interpolated String knows which brace is for the nested query, and which is for the value substitution.  You would denote that using an escape sequence of the braces themselves.

 

string templateName = "FCC Pump Process";
string elemCatName = "ProcessMonitoring";
string attrCatName = "Process Monitoring";

// This gives a compile error with an Interpolated String
// string query = $"Element:{Template:'{templateName}' Category:'{elemCatName}'} Category:'{attrCatName}'";

// Escape the { and } around literal braces with {{ and }}.
// Fun Fact: {{something}} is called the Mustache Template!
// See https://en.wikipedia.org/wiki/Mustache_(template_system)
string query = $"Element:{{Template:'{templateName}' Category:'{elemCatName}'}} Category:'{attrCatName}'";

using (AFAttributeSearch search = new AFAttributeSearch(root.Database, "interpolated nested query string", query))
{
    search.CacheTimeout = TimeSpan.FromMinutes(10);
    foreach (AFAttribute item in search.FindAttributes())
    {
        // Do Something
    }
}

 

An interesting bit of trivia: the {{something}} format of double braces is called the Mustache Template!

 

 

AFAttributeSearch

 

The nested query examples used AFAttributeSearch for searching upon elements.  The AFAttributeSearch may search on event frames instead.

 

One thing to note is the fullLoad parameter is missing from the FindAttributes method because it must always do a full load since the attributes cannot exist without the owning element or event frame.  However, the AFAttributeSearch.FindObjectFields is smart enough to know when it must make a full load to evaluate data references.  To state that a different way, it is smart enough to know when to skip a full load because of captured event frames. You don't have to do anything special like consult Oracles to figure this out for any given situation.  You would code it the same way regardless of the situation and let AFAttributeSearch.FindObjectFields make the right decision!

 

Why you should use the new search classes

 

Maintainability - some Find methods in AFAttribute or AFNotificationContactTemplate have already been marked as Obsolete, and it is reasonable to suspect that others may be marked so with future releases of AF SDK.  Make your applications more resilient by switching to these new methods sooner rather than later.

 

Performance - if you require fetching more than one page of results from the server and you opt-in to server-side caching, any of the AFSearch classes perform much faster than the older methods.  This means that AFAttributeSearch will be much faster than the AFAttribute.FindElementAttributes overloads.

 

Ease of Use - the AFSearch classes take care of paging issues seamlessly for you.  The new nested queries makes searching for attributes on either elements or event frames very easy with fewer lines of code.  There is previously mentioned smartness built into AFAttributeSearch.FindObjectFields. All of this without extra coding on your part.

 

Reliability - when searching on attribute values that might be evaluated client-side (due to needing to evaluate the data reference), it is impossible to do paging properly using the older search methods because you don’t reliably know what to use for the next page index.  By opting-in to server-side caching with the newer AFSearch methods, paging is more reliable.

The webinar scheduled for today (Wednesday March 21) will begin at 4:00 PM GMT or UTC.  Due to an unfortunate glitch with North America observing DST and Europe not, the previously published start times are incorrect.  Europe does not transition to Summer Time until March 25.

 

Recording and Slides are available at this link.

 

Topic: Asset Analytics native integration with MATLAB

 

North America 4:00 PM GMT is:

     Noon - 1 PM US Eastern

     9-10 AM US Pacific

 

Europe 4:00 PM GMT is:

     5:00 PM CET

 

UPDATE: A customer reported that the link in his confirmation mail 'click here to join' does not work, because it is missing the ':' after https. We don't know how widespread this is or whether it is a one-off for this particular customer.  However, he did report that the the URL in the Outlook invitation/appoint is OK.

Filter Blog

By date: By tag: