Skip navigation
All Places > PI Developers Club > Blog > 2018 > June
2018

Introduction

 

We're excited to announce to PI Dev Club members that we now have a PI Web API Client Library for Go, the Google-sponsored programming language specifically designed around concurrent processing.

 

You can visit the GitHub repository of this library here.

 

Requirements

 

go1.9.7 or later

 

Installation

 

If you haven't already, install the Go software development kit.

 

Run this line to install the PI Web API Client for go

 

go get -u github.com/christoofar/gowebapi

 

Note: You don't need the Go SDK after you have compiled a go program and wish to deploy it somewhere.  Go creates self-reliant executable programs that compact dependent libraries inside them.

 

Getting Started

 

Here is a sample Go program for retrieving the links from the Web API home page.

 

Create a directory under %GOPATH% and let's call it webapitest. Then create a new code file with the name webapitest.go

This will print all the version numbers of your PI Web API server plugins. Replace the string literals {in braces} with the appropriate values for your environment.

 

// webapitest.go
package main

import (
    "context"
    "fmt"
    "log"

    pi "github.com/christoofar/gowebapi"
)

var cfg = pi.NewConfiguration()

var client *pi.APIClient
var auth context.Context

func Init() {
    cfg.BasePath = "https://{your web api server here}/piwebapi"

    auth = context.WithValue(context.Background(), pi.ContextBasicAuth, pi.BasicAuth{
        UserName: "{user name here}",
        Password: "{password here}",
    })

    client = pi.NewAPIClient(cfg)
}

func main() {
     Init()
    response, _, fail := client.SystemApi.SystemVersions(auth)
    if fail != nil {
        log.Fatal(fail)
    }

    fmt.Println("Here's all the plugin versions on PI Web API")
    for i := range response {
        fmt.Println(i, response[i].FullVersion)
    }
}

 

You can run the program by issuing the following commands

 

~/go/webapitest $ go build
~/go/webapitest $ ./webapitest

 

Your output should look something like this

 

~/go/webapitest $ ./webapitest
Here's all the plugin versions on PI Web API
OSIsoft.REST.Documentation 1.11.0.967
OSIsoft.REST.Services 1.11.0.967
OSIsoft.Search.SvcLib 1.8.0.3651
OSIsoft.PIDirectory 1.0.0.0
OSIsoft.REST.Core 1.11.0.967

 

Coding examples

 

There are some simple examples on how to start probing the PI Web API Client library over here.

 

Developing in Go

 

Golang programmers tend to develop using Visual Studio Code on Windows which has great golang support and is also available on MacOS and Linux.   There is great golang support available for emacs (configure emacs from scratch as a Go IDE) and vim as plugins which also give you function templates, IntelliSense, syntax checking, godoc (the documentation system for go), gofmt (code formatting/style) and support Delve, the debugger for the go language which cleanly handles the concept of go routines.

 

You can also build Go code with nothing but your web browser using the Go Playground.   This is a very handy tool where you can experiment with Go code snippets and compile and run them directly in a web browser, viewing the output.

 

Caveats

 

A WebID wrapper has been added to the library.  You can review the unit test code to see how you can create WebIDs from paths.

 

Final Thoughts

 

Most everyone's exposure with Go is minimal (including myself!).  But this language is expected to grow in popularity.   The reason: Go's awesome power to simplify concurrent programming is making it spread quickly, particularly within the realm of sensors and other lightweight devices.   Go code is also quite fast and produces programs that are lightweight yet powerful.

 

It's also a very simple programming language to learn (so simple you can become a Go programmer in 15 minutes if you have exposure to any other programming language).  Considering that there is also a rich library of data adapters written in Go, it made obvious sense to open a portal to the world of OSIsoft in golang.

 

Happy Gophering!

datanerd.jpg

 

A gopher is a euphemism Go programmers use to describe one another.

 

With the announcement of the PI Web API Client Library for the Go programming language I have hope that we can all broaden our understanding of concurrent programming.  Go isn't just the "programming language of the {date_year}".   It really is an exciting time to be coding, particularly with a programming language as simple to implement and understand as this one.

 

What is Go?

 

package main

import (
    "fmt"
)

func main() {
    fmt.Println("Hello, playground")
}

Go (known as golang) is a language funded by Google with a design goal of making concurrent programming much easier to write, debug and manage.  The principle designers at Google are Robert Griesemer, Rob Pike, and Ken Thompson formerly of AT&T and Bell Labs.   It's easier to explain what Go is by describing what it's not:

 

What isn't in Go

 

Objects

Yes, you can survive totally fine in a computer language these days with no objects.  It certainly lowers the barrier to new programmers who don't have the patience to memorize Design Patterns.   As a functional programming language that mimics C, avoiding holding on to state as much as possible is primarily the point.  So in the case of web server-based REST programming and other types of concurrent processing--avoiding holding on to state as much as possible is ideal and makes concurrent code easier to understand.

 

Threads

Well, threads as you've always known them to be.   Intel/ARM processor threads are used in Go programs as you would expect but go runtimes utilize a concept called a goroutine, which is a much more lightweight concept than a thread and they live within your processor's threads.   The main advantage of using a lightweight goroutine is massive scalability on a single server instance.   A Java service that might support 5,000 threads could theoretically scale to hundreds of thousands of threads when implemented in Go.   Some more benefits of goroutines:

  • You can run more goroutines on a typical system than you can threads.
  • Goroutines have growable segmented stacks.
  • Goroutines have a faster startup time than threads.
  • Goroutines come with built-in primitives to communicate safely between themselves (channels).
  • Goroutines allow you to avoid having to resort to mutex locking when sharing data structures.
  • Goroutines are multiplexed onto a small number of OS threads, rather than a 1:1 mapping.
  • You can write massively concurrent servers without having to resort to evented programming.

 

A process that normally would have to live on an expensive cloud VM instance or on a medium-sized server can be scaled down to an Arduino device or a much smaller VM instance.    This gives you an unparalleled amount of parallel power (pun intended), not only taking full advantage of all that hardware you paid for but it also affords you the capability to go cheaper down the hardware cost curve in future hardware.

 

An even more convincing selling point for Go is the power of race-condition debugging, which is difficult to do in nearly every evolved programming language.  The Data Race detector is built-in to Go and can pick up your memory conflict points in your code as you run a production workload through it.  To invoke the detector you just kick off your program with the go command using the -race option.

 

Anyone who has had to hunt down race conditions in .NET languages or C++ only to download loads of third-party tools to assist with locating offending race condition code would kill to have this feature built into the language.

 

Exception handling

One of the behaviors of Go that it definitely inherited from C is the concept of exception handling, or rather the lack of it.  Just like you must do in C and Visual Basic, in Go you will need to check returns from functions and handle error states immediately after calls that have a non-zero probability of failure.   The only thing you can really trust being available are memory cells, your local variables and the CPU.   Just about everything else you touch can fail (disk, network, http calls, etc).

 

To get around this though, Go supports multiple return variables from functions which is a very pleasing feature of the language.   A typical call to a function that might fail often looks like this:

 

var theData, err, numRecords := GetRecordsFromDatabase(userContext, sql)
if err != nil
{
     go myapp.PrintWarning("The database is down right now.  Contact Support. " + err)
     return
}

... // Begin processing records

 

What Go feels like to code in

 

The designers of Go definitely went on a shopping trip; starting with C and picking off concepts found in Pascal (the := assignment operator), inferred assignments from Javascript and C# using the same var keyword in both languages.  It also has pointers, breakouts to assembly, and the compiler condenses raw-metal binary executables that are freed from the need to host a JVM or a .NET runtime kit to have your programs launch.

 

One of the strongest benefits is that Go has garbage collection--a feature in hosted languages that need frameworks (Java and .NET).   So there aren't any calls to .free() and destructors are not necessary since there are no objects.  Instead, Go uses the defer keyword so you can run a cleanup routine when a function ends.   And unlike C there is no malloc() nonsense to worry about.   It's the fun of C without the items that make C frustrating.

 

The Go language is also surprisingly simple to wrap your brain around to the point that I am seeing people who have learned Go as their first programming language.  The Go code spec has a strict convention on formatting and gofmt comes with Go which can lint your code.   It's also customary to use godoc to heavily document what you're doing inside go routines.  Once again this comes with Go; no 3rd party tools are necessary to stylize your code.   The burden of code reviews that developers must do inside teams is greatly simplified thanks to these standards.

 

These standards combined with the lightweight thread power this design offers make it easy to understand why this language is taking off so rapidly and why Google invested in it.

 

Where Should I Start?

 

These are places that helped me get started in Go:

 

IDE Developing in Go

 

Golang programmers tend to develop using Visual Studio Code on Windows which has great golang support and is also available on MacOS and Linux.   There is great golang support available for emacs (configure emacs from scratch as a Go IDE) and vim as plugins which also give you function templates, code completion, syntax checking, godoc (the documentation system for go), gofmt (code formatting/style) and support Delve, the debugger for the go language which cleanly handles the concept of go routines.

 

You can also build Go code with nothing but your web browser using the Go Playground. This is a very handy tool where you can experiment with Go code snippets and compile and run them directly in a web browser, viewing the output.

 

Happy Gophering!

Introduction

 

A few years ago, we've announced the public PI Web API endpoint in order to:

 

  • Provide developers access to a PI System who may not be able to access PI otherwise
  • Create code samples against the public endpoint and to host them under OSIsoft organization on GitHub
  • Offer developers a playground to exercise with PI Web API
  • Create a streamlined way to offer datasets in a PI System

 

Although we were able to achieve the goals above, I felt that visualizing the data only through PI Web API is a challenge since common tools like PI System Explorer and PI Vision are not available to work with PI data stored on a remote PI Web API endpoint. Given this scenario, I've developed the PI Web API Data Reference which allows local attributes to access data from remote attributes through PI Web API endpoints. As a result, now I can see PI data from the public PI Web API endpoint within the PI System Explorer.

 

This custom data reference (CDR) was developed on top of the PI Web API client library for .NET Framework and PI AF SDK. The basic idea is to make HTTP requests against PI Web API and convert the responses into PI AF SDK objects which PSE will be able to process.

 

Requirements

 

  • PI Web API 2017 R2+ installed within your domain using Kerberos or Basic Authentication. If you are using an older version, some methods might not work.
  • PI AF 2017 R2+
  • .NET Framework 4.6.2

 

GitHub repository

Please visit the GitHub repository for this project where you can find its source code, download the CDR under the GitHub release section and read about the new features added. There are two folders: src and dist. The src folder has the source code package. If you compile the VS Solution in the Release mode, the binaries will be created on the dist folder.  Due to the settings of the .gitgnore file, this folder is empty.

 

Video to get started:

 

 

 

 

Installation

  • Copy all files from the dist folder to %PIHOME%\PIWebApiDR
  • Register the CDR using the following command (you can also run register_plugin.bat):

 

"%pihome%\AF\regplugin" "OSIsoft.PIDevClub.PIWebApiDR.dll" "OSIsoft.PIDevClub.PIWebApiClient.dll" "Newtonsoft.Json.dll" /own:OSIsoft.PIDevClub.PIWebApiDR.dll /PISystem:PISYSTEMNAME 

 

 

Uninstallation

  • Unregister the CDR using the following command (you can also run unregister_plugin.bat):

 

"%pihome%\AF\regplugin" -u "OSIsoft.PIDevClub.PIWebApiDR.dll" /PISystem:PISYSTEMNAME 

 

  • Delete all files from the %PIHOME%\PIWebApiDR folder.

 

ConfigString and ConfigStringEditor

 

The config string of this CDR has the following structure:RemoteDatabaseId={RemoteDatabaseId};WebId={WebId}

  • RemoteDatabaseId is the ID of the RemoteDatabase which is an AF Element with many attributes with the configuration to connect to a remote PI Web API.
  • WebId is the Web ID 2.0 of the remote AF Attribute.

 

The ConfigStringEditor (the window that is opened when you click on the "Settings..." button on PI System Explorer generally used to edit the attribute config string) does not allow the user to change the ConfigString of the attribute. It is used only to visualize the information. Please use the Remote Database Manager to delete and create databases mapping remote ones.

 

 

 

The remote databases are stored under the "OSIsoft Technology" root element of the Configuration AF database.

 

 

 

Remote Database Manager

The Remote Database Manager should be used to manage the remote database on the system as well as to create local AF databases mapped to remote AF databases through PI Web API. Below you can find a screenshot of this application that comes with the PI Web API Data Reference.

 

 

Visualizing PI data from remote AF databases locally in PSE

On the screenshot below, you can see a PI System Explorer accessing data from the "Universities Mapped" database. This database was created by the Remote Database Manager, mapping the "Universities" AF database from the public PI Web API.

 

 

Visualizing PI data from remote AF databases in PI Vision 3

 

This custom data reference works with PI Vision 3 as long as the following procedure is followed:

  1. On the PI Vision 3 machine, edit the file C:\ProgramData\OSIsoft\Coresight\PIDS_Settings.config  by adding the following key:   <add key="AFDRSetting:PI Web API" value="DisableInputs"/>
  2. Edit the PI Vision 3 web config (C:\Program Files\PIPC\PIVision) by making sure the file attribute of pidsSetting node is defined as:  <pidsSettings file="C:\ProgramData\OSIsoft\Coresight\PIDS_Settings.config" />
  3. Save and restart IIS

 

The procedure above will make sure that PI Vision will read data using the default PI AF SDK methods.

 

Features of the CDR in PI AF SDK

 

When writing our UnitTests, we have create a sample AF database with attributes using the PI Point DR and then created mapped AF database with PI Web API DR using the Remote Database Manager. Each test will get data from the PI Point DR and PI Web API DR and compare the results to make sure they are the same. Below you can find the features implemented on the PI Web API data reference:

  • Asynchronous
  • DataPipe (please refer to PIWebApiEventSource.cs)
  • InterpolatedValue
  • InterpolatedValues
  • InterpolatedValuesAtTimes
  • PlotValues
  • RecordedValue
  • RecordedValues
  • RecordedValuesAtTimes
  • Summary
  • Summaries
  • Bulk calls

 

If you use AFDataPIpes with Attributes with PI Web API Data Reference, the CDR will use PI Web API Channels to get new real-time values under the hood.

 

Troubleshooting

If you are having issues make sure:

  • .NET Framework 4.6.2 is installed on the computer registering the plugin.
  • Run the Remote Database Manager with administrative privileges.
  • Unblock the files after extracting them from the compressed file.

 

 

Disclaimer

 

In case you are using Basic authentication, the username and password will be stored in fixed attributes. Although the password will be encoded, it won't be safe. Every user with read access to the Configuration database will be able to get the password of the remote PI Web API endpoint. For Kerberos authentication, the credentials won't be stored on the AF database.

 

This is not an official product. As described, there are security risks involved. Use it carefully.

 

Conclusion

I hope that the PI Community will benefit a lot from using the library as another tool to share data. Please test yourself and let me know if it works fine. I plan to record a video with some tips to use this great feature.

PI AF 2018 (AF SDK 2.10) offers a very significant change in filtering on attribute values: there is no longer a restriction that the attribute must belong to an element template.  The allows for a greater flexibility for filtering.  For example, you may now search for attributes that don't belong to any template.  Or better yet, you may search for attributes with the same name but belonging to different templates!

 

Other Features New to AF SDK 2.10

 

The bulk of this blog will cover the ValueType used with attribute value filters.  Before we dig too deep into that topic, let's take a quick look at the other new features.

 

There is a new AFSearchFilter.ID search filter to allow searching for objects by their unique identifier (GUID).  This unique ID is the fastest way to locate a given object on the AF Server.  A much-welcomed addition is that the ID supports the Equal or IN( ) operator.  If you are developing code, the best way to pass a GUID is by using ID.ToString("B").

 

New Search Fields are ConfigString, DisplayDigits, IsManualDataEntry, Trait, UOM, SecurityString, and SecurityToken.  Note that with a SecurityToken field, the FindObjectFields method would return an AFSecurityRightsToken object.

 

PI AF 2017 R2 (AF SDK 2.9.5) introduced the AFAttributeSearch and the PlugIn filter.  You could combine that filter plus the new ability to search on attributes without specifying a template.  For example, you now have the ability to perform a completely server-side search of all attributes referencing the PI Point data reference!  Stay tuned for a blog dedicated to this topic from one of my colleagues.

 

And now, the remainder of the blog will discuss the new ValueType.

 

If using an element or event frame Template

 

As mentioned earlier, previous versions required a Template for any attribute value filters.  An additional requirement was that the Template needed to be specified before the attribute value filters.  If the Template was specified after, then an exception was thrown.

 

Since AF SDK 2.10 removes the restriction on the Template, an interesting artifact is that you may specify the Template after the attribute value filter - and an exception will not be thrown.  However, we strongly recommend against this practice.  If you want to filter on a Template, we highly recommend you specify the Template first - just as you did with AF SDK prior to 2.10.  Nothing has changed with the new version here (nor should your existing code).  If you follow this advice, then you should skip the "AS valuetype" cast (more below).

 

Now let's consider if you don't follow the advice and you specify the Template after the attribute value filters.  You will need to include the "AS valuetype" cast, and its behavior will be as described in the remainder of this document.  The search will still be limited to the specified Template but as far as the attribute value filters are concerned, they will be treated as if the template was entirely omitted.  Precisely how they are treated depends on the attribute's data type and the "AS valuetype" cast you declare, which is presented in detail below.

 

Casting AS ValueType - When not using a Template

(or the Template is specified after the attribute value filters)

 

To support this new capability, there are several new things to discover with AFSearch to address the issue of a filter attribute's data type.  When based on a template, the data type is easily inferred.  What happens if a template isn't specified and the attribute does not belong to a template and/or may span different templates?  How does the search know which data type to use in the filter?  The answer is that it is left up to you (the developer) to pass the desired value type as you build the search, either by a query string or search tokens:

 

  • New AFSearchValueType enumeration
  • A new AFSearchToken.ValueType property (a string)
  • Two new AFSearchToken constructors to allow you to indicate the ValueType (also a string)

 

Golden Rule:

  • If you DO specify the template, do NOT specify the value type. 
  • If you do NOT specify a template, then you SHOULD specify a value type.

 

How carved in stone is the above "SHOULD"?  If there is any possibility whatsoever of an ambiguous interpretation between a String versus a Numeric, then you absolutely should specify the value type.  For example,  AFSearch has no way of knowing whether 1 or '1' or "1" should be a Numeric versus a String value.  Best practice: anything numeric should always specify "AS Numeric".

 

If you are using search tokens, you would use the new AFSearchToken constructors.  If you are using a query string, you would use the new AS <value type> syntax.  The available values for value type are:

  • Numeric, i.e. the literal text "Numeric"
  • String, i.e. the literal text "String"
  • EnumerationSet, the name of the applicable AFEnumerationSet.  Do NOT use the literal text "EnumerationSet".

 

Typical Scenarios with Numeric or String

 

The brief examples below look very similar with the exception of the value type designator (Numeric or String).  This bears repeating: you only need to use the AS valuetype if you do not specify a template.

 

Data Type Numeric: Integer (Byte, Int16, Int32, Int64) or Floating Point (Float, Single, or Double)

Consider if you have an attribute named RunStatus, and its data type is an Int32, where 0 means not running and 1 means running.  The query string could look like either of these:

  • "|RunStatus:1 AS Numeric"
  • "|RunStatus:'1' AS Numeric"

 

Data Type String

And if RunStatus was a String where "0" means not running and "1" means running, you would use these:

  • "|RunStatus:1 AS String"
  • "|RunStatus:'1' AS String"

 

Typical Scenarios with EnumerationSet

 

We continue covering scenarios when you do not specify a template.  We turn to another typical scenario of when the attribute's data type is an Enumeration Set.  Here it doesn't matter where the data comes from, be it a PI point, a table lookup, formula, or even a static value.  What matters to the value being filtered on is that the attribute has been declared to use an enumeration set.  The important thing is to use the AS specifier followed by the name of the enumeration set; do NOT use the literal value "EnumerationSet".

 

string attrPath = "|Classification";
string attrValue = "Commercial";
string enumSetName = "Building Type";
string query = $"'{attrPath}':='{attrValue}' AS '{enumSetName}'";

 

The above snippet safely wraps anything I created in quotes.  Note that because the enumeration set name contains a blank, I absolutely must wrap it in quotes.  In this specific case where neither attrPath or attrValue contain a blank, I could have omitted the quotes.  The value in the variable query will be:

 

"'|Classification':='Commercial' AS 'Building Type'"

 

Later when passed into an AFAttributeSearch constructor, the search instance will resolve to:

 

{|Classification:Commercial AS "Building Type"}

 

 

Cases Needing Special Consideration

 

There are special cases you may need to keep in mind beyond the typical Numeric, String, or EnumerationSet.  Obviously an attribute that is a string should use "AS String" and an attribute that has a number data type should use "AS Numeric".  What about data types that aren't so obvious?  Boolean or DateTime, for example?

 

Data Type Boolean: cast AS String

Before we even touch on enumeration sets, let's investigate another area of caution.  An attribute with a data type was Boolean is not exactly a number and not exactly a string.  As far as AFSearch is concerned, you should compare the literal values of a Boolean, namely "True" and "False", as strings.  Therefore the following filters would all be correct for a Boolean data type:

  • "|RunStatus:True AS String"
  • "|RunStatus:'True' AS String"

 

It's absolutely important with Booleans to specify "AS String".  You need to be aware that the following will quietly fail by returning 0 items:

  • "|RunStatus:True"

 

Data Type DateTime: cast AS Numeric

This can be a bit tricky.  The safest practice when dealing with DateTime attributes, whether you have a DateTime or an AFTime instance in code, is to use Round Trip Formatting.  That is to say, use ToString("O") when converting a DateTime or AFTime to string for the AFSearchToken constructor or within a query string.  And despite passing it as a string, it will actually be treated AS Numeric.  So these snippets work:

 

Variable "date" can either be DateTime or AFTime instance

  • $"|Timestamp:>'{date.ToString("o")}'"
  • $"|Timestamp:>'{date.ToString("o")}' AS Numeric"

 

The above uses an Interpolated String.  If you prefer string.Format, it would be:

  • string.Format("|Timestamp:>'{0}'", date.ToString("o"))
  • string.Format("|Timestamp:>'{0}' AS Numeric", date.ToString("o"))

 

TimeSpan with Data Type Anything: Not Supported

Some customers have attributes with data type "<Anything>" to hold a TimeSpan object.  Value filtering on such time span attributes is not supported in AF SDK 2.10.

 

If your time span attribute is defined to hold an integer or a floating point, then it would be treated as a Numeric data type (see above).

 

Digital PIPoint with Data Type Anything (not using an AFEnumerationSet): cast AS String or Omit AS specifier

For an attribute using the PI Point data reference that grabs from a Digital tag, we recommend that the attribute data type be "<Anything>".  You do not have to map the digital tag to an AFEnumerationSet.  You can if you wanted to, but that means (1) you have to copy the PIStateSet as an AFEnumerationSet, and (2) what to do with the attribute falls under "AS EnumerationSet" discussed elsewhere in this document.

 

Assuming you have a string variable named  "stateText", which contains the text of a given digital state, the following would be used to filter:

  • $"|Digital:'{stateText}'"
  • $"|Digital:'{stateText}' AS String"

 

For instance, if your digital tag used the "Modes" StateSet and you wanted to filter on those attributes with a mode of "Manual", either of these should suffice:

  • "|Digital:'Manual'"
  • "|Digital:'Manual' AS String"

 

Past AFSearch Blogs

 

What's new in AFSearch 2.9.5 (PI AF 2017 R2)  (March 2018) - AFAttributeSearch is introduced.

 

Coding Tips for using the AFSearch IN Operator - you may search for multiple values using IN(value1; value2; etc.).  Some code is offered to make this easier to generate.

 

Using the AFEventFrameSearch Class (Nov 2016) - Giving some attention to event frames since many of earliest AFSearch examples were element based.

 

Aggregating Event Frame Data Part 1 of 9 - Introduction  (May 2017) - From the UC 2017 TechCon/DevCon hands on lab for Advanced Developers.

 

PI World 2018 SF Developer Track - Getting the most out of AFSearch - (May 2018) From PI World 2018, DevCon presentation for Intermediate Developers.

 

Why you should use the new AF SDK Search  (June 2016) - The granddaddy of them all.  The earliest post explaining AFSearch, which was new at that time.

A customer recently asked about filtering on multiple values of an attribute in AFSearch.  This is easily addressed using the IN search operator in a query string, or the equivalent AFSearchToken constructor expecting an array of values.  There is one major caveat: the IN operator is not allowed for floating point types such as Single or Double, since binary floating point values are considered approximations instead of exact numbers.

 

There are a few tips to get developers pointed in the right direction.  First of all, values within the IN operator are delimited by semi-colons.  If you (like me) accidentally use a comma-delimited list, then you will receive an odd error about missing a closing parenthesis.  Then if you carefully inspect the query string, then you (like me) will start pulling your hair out because you don’t see a missing closing parenthesis!  Hopefully you may benefit from my pain by quickly switching your delimiter to be a semi-colon.

 

Another tip is that while blanks are allowed within the IN operator - or more specifically around the delimiters - any values that contain embedded blanks must be wrapped in quotes (single or double quotes).  Consider this list of cities:

 

  • London
  • Paris
  • San Francisco

 

Obviously, San Francisco needs to be wrapped in quotes due to the blank inside the name.  If the attribute path is “|City”, then you could have this C# filter in your query string:

 

string uglyQuery = "|City:IN(London; Paris; \"San Francisco\")";

 

Though ... that is a wee bit ugly.  Besides being ugly, it also might be confusing to someone using a different language, for instance, VB.NET.  A couple of weeks ago, I taught a class where a VB coder thought the unsightly \" were delimiters for AFSearch.   I explained to him that it's how C# escapes double quotes.  This can look much easier on the eyes if you just use single quotes:

 

string prettyQuery = "|City:IN(London; Paris; 'San Francisco')";

 

And it avoids any needless confusion that \" is an AFSearch delimiter.

 

This is all fine-and-dandy, but how often do you hard-code values?  It does make the example simple and straight-forward, but what if the list of items you wish to filter upon are in a list or array?  Let’s look at some code that helps us out.  After all, this is a PI Developers Club, which demands for code.  Strictly speaking, all we need for the collection of values is that they be in an enumerable collection.  Your source collection doesn’t have to be strings, but keep in mind that eventually we will need to pass a string to the AFSearch query, or if we use the AFSearchToken, then we must actually pass an array of strings.

 

For our cities example, it makes sense that the city names are strings.  Let’s not take the easy way out and make it an array of strings.  Instead we will use a list so that we can see what we must do differently to make it all work together:

 

List<string> cities = new List<string>() { "London", "Paris", "San Francisco" };

 

A tiny bit of  code is needed to put that inside a query string:

 

// A little LINQ to wrap single quotes around each city, and separate them with "; "
string delimitedCities = string.Join("; ", cities.Select(x => $"'{x}'"));

// An Interpolated String to substitute the delimitedCities.
string query = $"|City:IN({delimitedCities})";

 

This resulting value in query would be:

 

"|City:IN('London'; 'Paris'; 'San Francisco')"

 

Or if you prefer working with search tokens, this would be the equivalent:

 

// Tip: here you must pass a string array, so we must ToArray() on the cities list.
AFSearchToken token = new AFSearchToken(AFSearchFilter.Value, AFSearchOperator.In, cities.ToArray(), "|City");

 

If we had defined cities to be a string array, we would not need the ToArray(), but then this example would be boring and less educational.

 

What if our enumerable collection isn’t a bunch of strings?  Let’s say we have a bunch of GUID ’s.  (Exactly how you got these GUID's is an interesting question not addressed here ; suffice to say this example takes a collection of things that aren't strings and converts them to one that is.)  There would now be an extra step needed where we must convert to string.  Once we have a collection of strings we can then implement code similar to the previous examples.  Let’s imagine we have something like this:

 

IEnumerable<Guid> ids = GetListOfGuids();  // magically get a list of GUID

 

Or it could maybe have been:

 

IList<Guid> ids = GetListOfGuids();

 

Let's say we want to filter on an attribute path of “|ReferenceID”.  First let’s tackle the problem of converting a GUID into a string that is compatible with AFSearch.  This is easy enough thanks to LINQ:

 

// Nicest way to convert a GUID a string compatible with AFSearch is to use GUID.ToString("B").
IEnumerable<string> idStrings = ids.Select(x => x.ToString("B"));

 

Okay, so now we have an enumerable collection of strings.  Using what we learned in previous examples, we can knock this out:

 

// A little LINQ to wrap single quotes around each string item, and separate them with "; "
string delimitedIds = string.Join("; ", idStrings.Select(x => $"'{x}'"));

// An Interpolated String to substitute the items.
string query = $"|ReferenceID:IN({delimitedIds})";

 

Fantastic.  If you prefer AFSearchTokens, that’s easy enough as well, but we do require the idStrings to generate a string array.

 

// Tip: here you must pass a string array, so we ToArray() on the idStrings collection.
AFSearchToken token = new AFSearchToken(AFSearchFilter.Value, AFSearchOperator.In, idStrings.ToArray(), "|ReferenceID");

 

Granted our example would have been simplified if we defined idStrings to be an array in the first place, but what fun would there be in that?

 

 

VB.NET Examples

 

Some of us supporting the PI Developers Club think there should be more VB.NET examples.  Towards that end, here are code snippets for the VB coders out there:

 

    Dim uglyQuery As String = "|City:IN(London; Paris; ""San Francisco"")"

    Dim prettyQuery As String = "|City:IN(London; Paris; 'San Francisco')"

    Dim cities As List(Of String) = New List(Of String) From {"London", "Paris", "San Francisco"}

 

Cities Query Example:

 

Query String

        ' A little LINQ to wrap single quotes around each city, and separate them with "; "
        Dim delimitedCities As String = String.Join("; ", cities.Select(Function(x) $"'{x}'"))

        ' An Interpolated String to substitute the delimitedCities.
        Dim query As String = $"|City:IN({delimitedCities})"

 

AFSearchToken

' Tip: here you must pass a string array, so we ToArray() on the cities list.
Dim token As AFSearchToken = New AFSearchToken(AFSearchFilter.Value, AFSearchOperator.In, cities.ToArray(), "|City")

 

Guids Example (or something that is not a collection of strings)

 

Query String

        Dim ids As IEnumerable(Of Guid) = GetListOfGuids()   ' magically get a list of GUID

        ' Nicest way to convert a GUID a string compatible with AFSearch is to use GUID.ToString("B").
        Dim idStrings As IEnumerable(Of String) = ids.Select(Function(x) x.ToString("B"))

        ' A little LINQ to wrap single quotes around each string item, and separate them with "; "
        Dim delimitedIds As String = String.Join("; ", idStrings.Select(Function(x) $"'{x}'"))

        ' An Interpolated String to substitute the items.
        Dim query As String = $"|ReferenceID:IN({delimitedIds})"

 

AFSearchToken

        ' Tip: here you must pass a string array, so we ToArray() on the idStrings collection.
        Dim token As AFSearchToken = New AFSearchToken(AFSearchFilter.Value, AFSearchOperator.In, idStrings.ToArray(), "|ReferenceID")

 

Summary

 

If you want to filter on a variety of values for an attribute, this requires the IN search operator.

  • You may use IN in a query string or an AFSearchToken.
  • Values must be exact.  Therefore, IN does not work on Single or Double floating point data types.
  • In query strings
    • The semi-colon is the delimiter between IN values.  Example: "|City:IN(London; Paris; 'San Francisco')"
    • Values containing blanks must be enclosed in single or double quotes.  Example: "|City:IN(London; Paris; 'San Francisco')"
  • AFSearchToken
    • Values must be passed as strings in an array

I have been asked on many occasions “How can I find the first recorded event for a tag?”  The direct answer to this may be as brief as the question.  However, usually there is a lurking question-behind-the-question about what they really want to do with that first event, and if you dig even slightly deeper you will uncover the overall task they are trying to accomplish.  What you may ultimately discover is there is no need to find the first event.

 

Let’s start off with the direct, simple answer in the form of a C# extension method:

 

public static AFValue BeginningOfStream(this PIPoint tag) => tag.RecordedValue(AFTime.MinValue, AFRetrievalMode.After);

 

This works because the PI Point data reference implements the Rich Data Access (RDA) method of RecordedValue.  The earliest timestamp to query is AFTime.MinValue, that is midnight January 1, 1970 UTC.  Thanks to the AFRetrievalMode, you ask for the first value after January 1, 1970.  If it’s the earliest recorded timestamp you are only concerned with, you can use this extension method:

 

public static AFTime BeginningTimestamp(this PIPoint tag) => BeginningOfStream(tag).Timestamp;

 

For PI points, this would give you the BeginningOfStream method to go along with the built-in EndOfStream.  Before the advent of future data, the EndOfStream was simply the Snapshot.  But there are oddities related to future data, which required different handling of data compared to the traditional historical data.  Hence, Snapshot was replaced by CurrentValue, and EndOfStream was added.

 

An AFAttribute could have a BeginningOfStream method, but it doesn’t have the same nice guarantees of PIPoint.  It all depends upon the data reference being used and whether it supports the RDA Data.RecordedValue method, which is why you should properly check that it is supported before attempting to call it:

 

public static AFValue BeginningOfStream(this AFAttribute attribute)
{
    if (attribute.DataReferencePlugIn == null) // static attribute
    {
        return attribute.GetValue();
    }
    if (attribute.SupportedDataMethods.HasFlag(AFDataMethods.RecordedValue))
    {
        // Depends on how well the data reference PlugIn handles AFRetrievalMode.After.
        return attribute.Data.RecordedValue(AFTime.MinValue, AFRetrievalMode.After, desiredUOM: null);
    }
    // Fabricated answer.  Not exact that one is hoping for.  
    return AFValue.CreateSystemStateValue(AFSystemStateCode.NoData, AFTime.MinValue);
}

 

Since the value being returned may be fabricated, I would be hesitant to include a BeginningTimestamp method as it would mask the inaccuracies.  To compensate, I would think further inspection of the returned value is needed, i.e. check for “No Data”.  Such are the difficulties of trying to create a BeginningOfStream method within your code libraries.  This is why we begin to probe more and ask about your use-case, or simply “What are you really wanting to do?

 

Virtually 100% of the people asking me how to find the first value in a stream want to find it for historical PI points only.  This greatly simplifies part of the problem because there is no need to be concerned with attributes or tags with future data.  Which brings us right back to the direct answer at the top, where you may be inclined to stop.  But if you take time to dig just a little deeper into what they are really doing, the true mission is revealed: they want to copy all historical data from an old tag to a new tag.  And trust me, there are several legitimate use-cases for doing this.

 

When I hear anything about copying of historical data, the first thought I have is “How much data is in the old tag?”  There are two particular answers that require the same special handling: (a) I don’t know, or (b) a lot.

 

The real problem they need to solve isn’t finding the earliest recorded timestamp.  Rather, they may have so much data they will bump into the ArcMaxCollect limitation (typically 1.5 million data values in a single data request).  There are programming ways around ArcMaxCollect (more below) and they rely upon the PIPoint.RecordedValues method specifying a maxCount > 0 (for instance, 100K works well).  The perceived issue of knowing the earliest timestamp becomes a moot point.  The more important date is knowing the end time, that is the switch-over date from the old tag to the new tag.  Depending upon how the old tag was populated, this may very well be the EndOfStream.  But if there is a chance that the old tag could still be receiving “InterfaceShut” or “IOTimeout” messages, you will need to explicitly specify the end time.  Worrying about the earliest recorded date has been a distraction to solving the real problem.

 

What of your start time?  I would think an in-house developer should know of the earliest start of their company's archive files.  A contracted developer could use AFTime.MinValue or go with a later, but still much safer date, such as “1/1/1980”.  Which brings us back to what they really want to do: copy large or unknown amounts of data.    This has been blogged about many times before:

 

Extracting large event counts from the PI Data Archive

 

PI DataPipe Events Subscription and Data Access Utility using AF SDK - PIEventsNovo

 

GetLargeRecordedValues - working around ArcMaxCollect

The 2.10 release of AF SDK unveiled DisplayDigits as a property you can investigate and set on an AF DataReference. Let’s do a quick investigation of what this property entails.

 

Further, there is a new .DisplayValue() method on the AFValue object for rendering single and double-precision floating point numbers.

 

What DisplayDigits Is

DisplayDigits is a setting that you can place on floating-point number tags to convey how much precision you wish to be displayed on-screen by downstream applications.

 

pse_example.png

Valid settings are any integer number from -20 up to 10.

 

What it’s used for

Most of your floating-point data will be expressed either in 32-bit or 64-bit floating point numbers. This allows for a large variety of decimal expression with varying degrees of precision. Controlling what precision you get back is what DisplayDigits is for.

Numerical precision

Consider the floating-point data types in the PI Data Archive…

TypeFloat16Float32Float64
MinimumZero of tag1.175494351 E-382.2250738585072014 E-308
MaximumSpan of tag3.402823466 E +381.7976931348623158 E +308
Exponent-8 bits11 bits
Mantissa-23 bits52 bits
Accuracyn/a7 digits15 digits

Floating point datatypes make a representation of a decimal number with relative increasing levels of precision given the number of bits used.1

Let’s experiment:

The SINUSOID tag on your Data Archive by default is stored as a 32-bit floating-point number. Let’s investigate what all the possible DisplayDigits settings exposed in AF SDK reveal.

First you will need to use an AF Database and set an attribute to the SINUSOID tag on your PI Data Archive. The DisplayDigits property hangs off the AFAttribute class and not off of PIPoint.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using OSIsoft.AF;
using OSIsoft.AF.Asset;
using OSIsoft.AF.Data;
using OSIsoft.AF.PI;


namespace DisplayDigitsExample
{
    class Program
    {
        static void Main(string[] args)
        {


            // Let's get the server
            PISystems pisystems = new PISystems();


            // Get an injection well asset
            AFElement well = pisystems["CLSAF"].Databases["TransformedWells"]
                .Elements["Injection Wells By Contractor"]
                .Elements["Paisano Transmission And Service Inc"]
                .Elements["Site1.F100_IW01"];


            // Get the line pressure
            AFAttribute linepressure = well.Attributes["Line Pressure"];
            
            Console.WriteLine($"Line Pressure uses PI tag {linepressure.DataReference.ConfigString}");
            Console.WriteLine($"Present Value: {linepressure.GetValue().ValueAsSingle()}");
            Console.WriteLine($"Display Digits Setting: {linepressure.DisplayDigits}");


            // Display all the DisplayDigits settings for this PI value
            for (int i = 10; i >= -20; i--)
            {
                Console.WriteLine($"Value with DisplayDigits Set to {i}: {linepressure.GetValue().DisplayValue(i, null, true)}");
            }


            Console.ReadLine();


        }
    }
}

 

Here's the same example in VB.Net...

 

Imports OSIsoft.AF
Imports OSIsoft.AF.Data


Module Module1
    Sub Main()
        Dim piServers = New PISystems


        Dim wellsDB = piServers("CLSAF").Databases("TransformedWells")


        Dim well = wellsDB.Elements("Injection Wells By Contractor") _
            .Elements("Paisano Transmission And Service Inc") _
            .Elements("Site1.F100_IW01")


        Dim linePressure = well.Attributes("Line Pressure")


        Console.WriteLine("Line pressure uses PItag " & linePressure.DataReference.ConfigString)
        Console.WriteLine("Present value: " & linePressure.GetValue().ValueAsSingle())
        Console.WriteLine("Display Digits Setting: " & linePressure.DisplayDigits)


        For i = 10 To -20 Step -1
            Console.WriteLine($"Value with DisplayDigits Set to {i}: {linePressure.GetValue().DisplayValue(i, Nothing, True)}")
        Next
        Console.ReadLine()
    End Sub
End Module

 

This yields the following:

img

 

So what’s with the negative and positive DisplayDigits?

After you run some example code against floating point data you have you will notice how DisplayDigits applies zero-padding and rounding.

Positive DisplayDigits settings will force a floating point number to be returned with a fixed number of digits to the right of the decimal point, up to 10 and include padding zeroes (0) where necessary.

If the number is more significant than what you specificed then the number may be rounded or truncated. For instance, if the value stored is 63.90804 but DisplayDigits is set at 2, the number is returned back as 63.91

A negative DisplayDigits setting instead determines the number of significant digits to display, with no zero padding.  If a rounding occurs the number of digits displayed may be less than the setting.

Why do I see more digits when I inspect .Value than when I use -7 or -15 DisplayDigits?

Floating point types are inherently imprecise as it’s a reflection on how your CPU represents floating-point values, given that the binary representation of the number is likely to not be exact.

That’s why you should use the new DisplayValue() method that’s been added to the AFValue object.

Footnotes

1 The PI Data Archive uses 32-bit single precision and 64-bit double precision as well as 16-bit

Note: Development and Testing purposes only. Not supported in production environments.

 

Link to other containerization articles

Containerization Hub

 

Introduction

During PI World 2018, there was a request for a PI Analysis Service container. The user wanted to be able to spin up multiple PI Analysis Service container to balance the load during periods where there was a lot of back filling to do. Unfortunately, this is limited by the fact that each AF server can only have exactly one instance of PI Analysis Service that runs the analytics for the server. But this has not discouraged me from making a PI Analysis Service container to add to our PI System compose architecture!

 

Features of this container include:

1. Ability to test the presence of AF Server so that set up won't fail

2. Simple configuration. The only thing you need to change is the host name of the AF Server container that you will be using.

3. Speed. Build and set up takes less than 4 minutes in total.

4. Buffering ability. Data will be held in the buffer when connection to target PI Data Archive goes down. (Added 13 Jun 2018)

 

Prerequisite

You will need to be running the AF Server container since PI Analysis Service stores its run-time settings in the AF Server. You can get one from Spin up AF Server container (SQL Server included).

 

Procedure

1. Gather the install kits from the Techsupport website. AF Services

2. Gather the scripts and files from GitHub - elee3/PI-Analysis-Service-container-build.

3. Your folder should now look like this.

4. Run build.bat with the hostname of your AF Server container.

build.bat <AF Server container hostname>

5. Now you can execute the following to create the container.

docker run -it -h <DNS hostname> --name <container name> pias

 

That's all you need to do! Now when you connect to the AF Server container with PI System Explorer, you will notice that the AF Server is now enabled for asset analysis. (originally, it wasn't enabled)

 

Conclusion

By running this PI Analysis Service container, you can now configure asset analytics for your AF Server container to produce value added calculated streams from your raw data streams. I will be including this service in the Docker Compose PI System architecture so that you can run everything with just one command.

 

Update 2 Jul 2018

Removed telemetry and added 17R2 tag.

1. Introduction

Every day more and more customers get in contact with us asking how does PI could be used to leverage their GIS data and how their geospatial information could be used in PI. Our answer is the PI Integrator for Esri ArcGIS. If your operation involves any sort of georeferenced data, geofencing or any kind of geospatial data, I encourage you to give a look at what the PI Integrator for Esri ArcGIS is capable of. But this is PI Developers Club, a haven for DIY PI nerds and curious data-driven minds. So, is it possible to create a custom data reference that provides access to some GIS data and functionalities? Let's do it using an almost-real-life example.

 

2018-06-07 11_38_41-pisquare - QGIS.png1.1 Scenario

The manager of a mining company has to monitor some trucks that operate at the northmost deposit of their open-pit mine. Due to recent rains, their geotechnical engineering team has mapped an unsafe area that should have no more than three trucks inside of it. They have also provided a shapefile with a polygon delimiting a control zone (you can download the shapefile at the end of this post). The manager wants to be notified whenever the number of trucks inside the control area is above a given limit.

 

relationship.pngCaveat lector, I'm not a mining engineer, so please excuse any inaccuracy or misrepresentation of the operations at a zinc mine. It's also important to state that the mine I'm using as an example has no relation to this blog post nor the data I'm using.

 

1.2 Premises

If you are familiar with GIS data, you know it's an endless world of file formats, coordinate systems, and geodetic models. Unless you have a full-featured GIS platform, it's very complicated to handle all possible combinations of data characteristics. So, for the sake of simplicity, this article uses Esri's Shapefile as its data repository and EPSG:4326 as our coordinate system.

 

1.3 A Note on CDRs

As the name implies, a CDR should be used to get data from an external data source that we don't provide support out-of-the-box. Simple calculations can be performed, but you should exercise caution as, depending on how intensive your mathematical operations are, you can decrease the performance of an analysis using this CDR. For our example, shapefiles, GeoJsons, and GeoPackages can be seen as a standalone data source files (as they contain geographic information in it) and the math behind it is pretty simple and it won't affect the server performance.

 

1.4 The AF Structure

Following the diagram on 1.1, our AF structure renders pretty simply: a Zinc Mine element with Trucks as children. The mine element has three attributes: (a) the number of trucks inside the control area (a roll-up analysis), (b) the maximum number of trucks allowed in the control area (a static attribute) and (c) the control area itself.

 

2018-06-06 09_37_25-__RBORGES-AF_GISToolBox - PI System Explorer.png

 

The control area is a static attribute with several supporting children attributes holding the files of the shapefile. Due to shapefile specification, together with the SHP file you also need the other three.

 

2018-06-06 09_39_00-__RBORGES-AF_GISToolBox - PI System Explorer.png

 

Finally, the truck element has two historical attributes for its position and the one using our CDR to tell if it's currently inside the control area or not (this is the one used by the roll-up analysis at the zinc mine element). Here I'm using both latitude and longitude as separated attributes, but if you have AF 2017 R2 or newer, I encourage you to have this data stored as a location attribute trait.

 

2018-06-06 16_55_42-__RBORGES-AF_GISToolBox - PI System Explorer.png

 

 

2. The GISToolBox Data Reference

The best way to present a new CDR by showing its config string:

 

Shape=..\|control area;Method=IsInside;Latitude=Latitude;Longitude=Longitude

 

Breaking it down: Shape is the attribute that holds the shapefile and its supporting files. It's actually just a string with a random name. What is important are the children underneath it that are file attributes and hold the shape files. Method is the method we want to execute. Latitude and Longitude are self-explanatory and they should also point to an attribute. If you don't provide a lat/long attribute, the CDR will use the location attribute trait defined for the element. There are also two other parameters that I will present later.

 

The code is available here and I encourage you to go through it and read the comments. If you want to learn how to create a custom data reference, please check the useful links section at the end of this post.

 

2.1 Dataflow

The CDR starts by overriding the GetInputs method. There we use the values passed by the config string, to get the proper attributes. You should pay close attention to the way the shapefile is organized, as there are some child attributes holding the files (these child attributes are AFFiles). Once this is done, the GetValue is called. It starts by downloading the shapefile from the AF server to a local temporary folder and creating a Shapefile object. Although Esri's specification is open, I'm using DotSpatial to incorporate the file handling and all spatial analysis we do. Once we have the shapefile, it goes through some verifications and we finally call the method that gets the data we want: GISGelper.EvaluateFunction(). For performance reasons, I'm also overriding the GetValues method. The reason is that we don't need to recreate the files for every iteration on the AFValues array sent by the SDK.

 

2.2 Available Methods

Taking into account what I mentioned on 1.3, we should not create sophisticated calculations so the CDR doesn't slow down the Analysis engine. To keep it simple and with good performance, I have implemented the following methods:

NameDescriptionOutputRepresentation
IsInsideDetermines whether a coordinate is inside a polygon in the shapefile. If your shapefile contains several polygons, it will check all of them.

1 if inside

0 if outside

inside.png
IsOutsideDetermines whether a coordinate is outside a polygon in the shapefile. If your shapefile contains several polygons, it will check all of them.

1 if outside

0 if inside

outside.png
MinDistanceDetermines the minimum distance from a coordinate to a polygon in the shapefile. If your shapefile contains several polygons, it will check all of them and return the shortest of them all.A double with the distance in the units defined by the used CRSmindist.png
CentroidDistanceDetermines the distance from a coordinate to a polygon's centroid in the shapefile. If your shapefile contains several polygons, it will check all of them and return the shortest of them all.A double with the distance in the units defined by the used CRScentdist.png

 

2.3 CRS Conversion

The GISToolbox considers that both lat/long and shapefiles are using the same CRS. If your coordinate uses a different base from your shapefile, you can use two other configuration parameters (FromEPSGCode and ToEPSGCode) to convert the coordinate to the same CRS used by the shapefile.

 

Let's say you have a shapefile using EPSG:4326, but your GPS data comes on EPSG:3857. For this case, you can use:

Shape=..\|control area;Method=IsInside;Latitude=Latitude;Longitude=Longitude;FromEPSGCode=3857;ToEPSGCod=4326

 

2.4 Limitations

  • It doesn't implement an AF Data Pipe, so it can't be used with event-triggered analysis (only periodic).
  • It handles temporary files, the user running your AF services must have read/write permissions on the temporary folder.
  • It only supports EPSG CRS
  • It only supports shapefiles.

 

3. Demo

Let's go back to our manager who needs to monitor the trucks inside that specific control area.

 

3.2 Truck Simulation

In order to make our demo more realistic, I have created a small simulation. You can download the shapefile at the end of this post (Trucks_4326.zip). Here's a gif showing the vehicles' position

 

gismap.gif

 

The trucks start outside of the control area and they slowly move towards it. Here's a table showing if a given truck is inside the polygon at a specific timestamp:

TSTruck 001Truck 002Truck 003Truck 004Total
000000
101001
201102
301102
411103
511114
611114
711013
811002

 

The simulation continues until the 14ᵗʰ iteration, but note how the limit is exceeded on the timestamp 5, so we should get a notification right after entering the 5ᵗʰ iteration.

 

3.3 Notification

The notification is dead simple: every 4 seconds I check the Active Trucks attribute against the maximum allowed. And as I mentioned before, the Active Trucks is a roll-up counting the IsInside attribute of each truck.

 

2018-06-07 16_05_09-__RBORGES-AF_GISToolBox - PI System Explorer.png

 

Shall we see it in action?

 

notif.gif

Et voilà!

 

The simulation files are available at the end of this post. Feel free to download and explore it.

 

4. Conclusion

This proof of concept demonstrates how powerful a Custom Data Reference can be. Of course, it doesn't even come close to what the PI Integrator for Esri ArcGIS is capable of, but it shows that for simple tasks, we can mimic functionalities from bigger platforms and can be used as an alternative while a more robust platform is not available.

 

If you like this topic and think that AF should have some basic support to GIS, please chime in on the user voice entry I've created to collect ideas from you.

Introduction

 

This is a MATLAB toolbox that integrates the PI System with MATLAB through PI Web API. With this toolbox you can retrieve PI data without having to generate the URL for each request. This version was developed on top of the PI Web API 2018 Swagger specification.

 

In the new upcoming 2018 release, PI Asset Analytics will introduce native connectivity to MATLAB enabling users to schedule and run their MATLAB functions fully integrated into their analyses. In other words, you will be able to integrate the PI System with MATLAB on production using a model that you have already built. This tool was developed for you to create new models with PI System data before using it on production.

 

Finally, this client library is read-only since only the HTTP GET request methods were added to the library.

 

Requirements

 

  • PI Web API 2018 installed within your domain using Kerberos or Basic Authentication. If you are using an older version, some methods might not work.
  • MATLAB 2018a+

 

GitHub repository

 

Please visit the GitHub repository for this project where you can find its source code, download the toolbox and read about the new features added.

 

Installation

 

This MATLAB toolbox is not available on MATLAB central servers. You should download it directly from this GitHub repository on the release section.

 

Please use the command below to install the toolbox:

 

matlab.addons.toolbox.installToolbox('piwebapi.mltbx')

 

If the installation is successful, you should see this toolbox inside matlab.addons.toolbox.installedToolboxes:

 

toolboxes = matlab.addons.toolbox.installedToolboxes;

 

If you want to uninstall this toolbox, use the command below:

 

matlab.addons.toolbox.uninstallToolbox(toolboxes(i))

 

 

Documentation

All the methods and classes from this MATLAB Toolbox are described on its documentation, which can be opened by typing on the console:

 

demo toolbox 'PI Web API client library for Matlab'

 

Notes

 

It is highly recommended to turn debug mode on in case you are using PI Web API 2017 R2+ in order to receive more detailed exception errors. This can be achieved by creating or editing the DebugMode attribute's value to TRUE from the System Configuration element.

 

Examples

 

Please refer to the following examples to understand how to use this library:

 

Create an instance of the piwebapi top level object using Basic authentication.

 

username = 'webapiuser';
useKerberos = false;
password = 'password'
baseUrl = 'https://devdata.osisoft.com/piwebapi';
verifySsl = false;
client = piwebapi(baseUrl, useKerberos, username, password, verifySsl);

 

Only the Basic authentication is available on this initial version. Please make sure to set up PI Web API properly to make it compatible with this authentication method.

If you are having issues with your SSL certificate and you want to ignore this error, set verifySsl to false.

 

Retrieve data from the home PI Web API endpoint

 

pilanding = client.home.get();

 

Get the PI Data Archive object

 

dataServer = client.dataServer.getByName(serverName);

 

Get the PI Point, AF Element and AF Attribute objects

 

point = client.point.getByPath("\\PISRV1\sinusoid");
attribute = client.attribute.getByPath("\\PISRV1\Universities\UC Davis\Buildings\Academic Surge Building\Electricity|Demand");
element = client.element.getByPath("\\PISRV1\Universities\UC Davis\Buildings\Academic Surge Building\Electricity"); 

 

Get recorded, interpolated and plot values from a stream

 

webId = point1.WebId;
startTime = "*-10d";
endTime = "*";
interval = "1h";
intervals = 30;
maxCount = 100;
desiredUnits = '';
selectedFields = '';
associations = '';
boundaryType = '';
filterExpression = '';
includeFilteredValues = '';


recordedValues = client.stream.getRecorded(webId, associations, boundaryType, desiredUnits, endTime, filterExpression, includeFilteredValues, maxCount, selectedFields, startTime);
interpolatedValues = client.stream.getInterpolated(webId, desiredUnits, endTime, filterExpression, includeFilteredValues, interval, selectedFields, startTime);
plotValues = client.stream.getPlot(webId, desiredUnits, endTime, intervals, selectedFields, startTime);

 

Get recorded, interpolated and plot values from a streamset in bulk

 

sortField = '';
sortOrder= '';

webIds = { point1.WebId, point2.WebId, point3.WebId, attribute.WebId};

recordedValuesInBulk = client.streamSet.getRecordedAdHoc(webIds, boundaryType, endTime, filterExpression, includeFilteredValues, maxCount, selectedFields, sortField, sortOrder, startTime);
interpolatedValuesInBulk = client.streamSet.getInterpolatedAdHoc(webIds, endTime, filterExpression, includeFilteredValues, interval, selectedFields, sortField, sortOrder, startTime);
plotValuesInBulk = client.streamSet.getPlotAdHoc(webIds, endTime, intervals, selectedFields, sortField, sortOrder, startTime);

 

 

Conclusion

 

If you want to create models to be used in production, please use this library and let me know if it works fine. You can send me an e-mail directly (mloeff@osisoft.com) or leave a comment below.

Filter Blog

By date: By tag: