Skip navigation
All Places > PI Developers Club > RESTful PI System Access > Blog

First, the results:

 

When the Channels controller was introduced we gave you the ability to open a direct stream of PI data over a WebSockets connection. Since then, questions have formed around how far this technology can be stretched in scope.  We have wondered for ourselves how far we can push WebSockets and have results for you.

 

We found a noticeable degradation in performance with two particular load types.

 

For 200 concurrent users with one subscription each, each subscription listening to 1,000 AF Attributes, performance began to decline where it was perceptible.

 

 

For a single user (such as a service account) creating subscriptions for a single AF Attribute, degradation in performance was observed after 2,500 channels were instantiated.

 

 

What we've discovered

Our testbed reveals the CPU and Memory usage of Channels scaling roughly linearly with the number of subscriptions/channels.

 

However, the performance of Channels certainly shows room for improvement; especially in the case of multiple channels.  The use of multiple channels provides far less throughput than that of a single channel with an equivalent number of subscriptions.

 

One potential where improvement that could be made is in the serialization process.  This requires the whole message be serialized to a string before sending.  The web socket implementation used on the server does provide a way to send large messages in “chunks”, which could be used to avoid large allocations and hopefully reduce memory usage.  The scheduling of the underlying AF calls could also potentially be improved in order to reduce CPU usage.

 

How we performed the tests

Two experiments were performed to collect data on the performance of Channels.  For both experiments, the results were collected using Performance Monitor, which was configured with the “% Processor Time” and “Private Bytes” performance counters for the “OSIsoft.REST.Host.exe” process.  The PI Web API instance itself was hosted on a HyperV virtual machine with the following specifications:

 

•             Operating System: Windows Server 2016 Standard

•             Virtual Processors: 6 Cores

•             Memory: Dynamic 8GB (8192 MB)

•             Hard Drive: 100 GB

 

For the first experiment, varying numbers of unique users were tested with a subscription of a single element with 1,000 attributes.  The source PI Points generated data with a 1 Hz frequency.  This test was intended to show how PI Web API performed under load with multiple unique user identities.  The stream set’s “GetChannel” endpoint was used on several AF elements, each of which had 1,000 attributes, all referring to a unique PI Point.  This PI Point was updated with values approximately every second to provide data for the channel.

 

For the second experiment, a single user was tested with varying numbers of channels created with a single subscription each.  This test was intended to show how PI Web API performed under load with a single user identity.  The stream’s “GetChannel” endpoint was used on the same unique PI Points as the first experiment.

 

Authentication Limitations

Authentication is perhaps the most severe limitation of Channels right now.  Currently, Channels uses the same authentication schemes as normal HTTP requests. This is done for simplicity and consistency.  While the web socket specification permits these authentication schemes, browser support for them is still limited.  The main problem is the HTTP challenge mechanism.  Basic, Kerberos, and Bearer authentication schemes require an “Authorization” header in the HTTP request with information about the user’s credentials.

 

Basic Authentication

Here, a browser would normally respond by prompting the user for their name and password; then generating the proper “Authorization” header.  But none of the tested browsers displayed this prompt for web socket connections and merely failed the request.  One workaround for this limitation is to first make a normal HTTP request so the browser can cache the “Authorization” header and use it for subsequent web socket requests.  This workaround was successful for all of the tested browsers.

 

Kerberos Authentication

If Single Sign-On is enabled, this “Authorization” header can be generated from the user’s current Windows account, so no user input is necessary.  When it is not enabled the browser must prompt the user for credentials.  In these cases the same workaround for Basic also works for Kerberos; a normal HTTP request can be made beforehand so the browser can cache the “Authorization” header.

 

Kerberos is not supported by default in Firefox (and sometimes Chrome), but it can be enabled using some configuration options.  See the OSIsoft Knowledge Base article KB01223 for instructions on how to configure Kerberos.

 

Bearer Authentication

For the Bearer scheme this “Authorization” header needs to be added programmatically. The browser will not cache or automatically generate it.  Unfortunately the web socket client API defined in the HTML specification (which is the interface most browsers implement) does not allow adding headers to the initial HTTP handshake request for security reasons.  As a result, Bearer authentication cannot currently be used in a browser with Channels.  However, web socket clients that do support adding headers manually can still use Bearer authentication.  For example, the C# ClientWebSocket class supports this functionality:

 

Other Limitations

Internet Zones (MSIE and Edge)

Internet Explorer and Edge both disallow web socket connections between “internet zones” by default.  In these cases, the following error messages will be logged in the console:

•             Internet Explorer: “SCRIPT5022: SecurityError”

•             Edge: “SCRIPT12017: WebSocket Error: SECURITY_ERR, Cross zone connection not allowed”

 

This issue can be resolved using the following steps on the client machine:

  •             Open “Internet Options” in Windows
  •             Click the “Security” tab
  •             Click the “Trusted Sites” zone
  •             Click the “Sites” button
  •             Add the domain name of the PI Web API server to the list

 

This will allow inter-zone web socket connections to the PI Web API server specified.

 

I really need the benefits of WebSockets.  Can I get around these limitations by scaling horizontally?

You can certainly explore using a WebSockets-friendly traffic director in either hardware or software if your project comes close to, but just above, the limitations that we have discovered here.  You might want to check out NGINX WebSockets-friendly traffic direction to load-balance multiple instances of PI Web API.

We have introduced claims-based authentication! The technique we used is OpenID Connect which is a simple identity layer on top of the OAuth 2.0 protocol. It provides excellent support for developers (both us and you) to authenticate users and exchange standards-based identity tokens securely between systems, even on the Internet. This means it is possible to create a secure PI Web API deployment that is open to the Internet! We currently support 3 OpenID Connect providers: Active Directory Federation Services (ADFS), Azure Active Directory, and PingFederate. We hope to support identity providers more in the future.

 

A Caveat: At the moment, a user claim obtained from the identity provider must match the User Principal Name (UPN) in the PI Web API Server's Windows domain. In my case, OSIsoft's Active Directory knows me as ray@osisoft.com. To use claims-based authentication, ray@osisoft is what must be returned by an identity provider. This means we can't currently support other OpenId Connect providers such as Google because is there is no way to configure ray@osisoft.com to be my identity claim in Google.

 

Documentation: Configuration of claims-based authentication is documented in the PI Web API 2017 User Guide. Search for the words "claims" and "bearer." You must download this document from our Technical Support website because OSIsoft Live Library has not yet been updated. We are drafting administrator guide documents specific to the 3 supported identity providers. Until they are published, please contact Technical Support if you need help. Our support engineers will get you in touch with the right people!

 

Another Caveat: Unfortunately, none of this frees you from having to configure Kerberos delegation. Our implementation works by using the Claims to Windows Token Service (C2WTS) to translate the identity claim to a Windows token which is needed by the PI System servers.

I am pleased to announce the release of PI Web API 2016, a member of the PI Developer Technologies suite of products. This release includes support for new PI AF 2.8 Event Frame features and access to AF Security information. You can now reduce REST response payload size by requesting only the data of interest (look for "selectedFields" in the Controller documentation in PI Web API Help). You can now retrieve data from multiple PI points or AF attributes by either Web ID or Path (look for "GetMultiple" in PI Web API Help). The Batch request function initially launched as a Community Technology Preview (CTP) is now fully supported. Thanks for your feedback on this significant feature!

 

We are also launching a Community Technology Preview (CTP) of a Shared Index feature in Indexed Search. This means that several instances of PI Web API in a cluster (with a network load balancer) can share the same index. It also means that only a single Crawler is necessary to scan the PI System databases. All search results returned to clients will be consistent regardless of which cluster member performs the search. We have published a special white paper to help you configured the Shared Index. If you are interested in the Shared Index CTP, or in cluster configurations in general, please contact me by either replying to this post or by emailing me directly.

 

As always, full details of this release can be found in the PI Web API 2016 Release Notes.

This blog post is an extension of Using PI Web API on HTML5 with AngularJSMarcos Vainer Loeff's post on using AngularJS with PI Web API. For the details on how to write the actual sample application, please refer to his post. This post will only cover some extra information regarding what needs to be checked when connecting to a remote installation of PI Web API, in particular how to make sure to avoid CORS conflict. The code used below is available at this GitHub repository.

 

This information will also be used at this year UC 2016's Hackathon: Join us at Programming Hackathon 2016 - Innovation around Smart Cities! Do make sure to join if you attend the UC!

If you are coming to this year UC, also make sure to consider joining Marco's sure to be excellent course on PI Web API.

Course and registration details below:

Join our Develop Cross-Platform Mobile Apps using PI Web API TechCon Lab

 

 

The remote installation

Our goal is to use data that is available via the following PI Web API endpoint: https://proghackuc2016.osisoft.com/piwebapi. (The credentials to log in to this site will be given at the UC.)

Aside from having the credentials to access the endpoint, to be able to write an application that gets data out, we will need to first need to see how it is configured. In particular, let us get the version information and the CORS settings.

 

Version information

To access the version information, we can access the following page: https://proghackuc2016.osisoft.com/piwebapi/system/versions and we see the following data, which is, as always, formatted in JSON.

 

{
  "OSIsoft.REST.Channels": {
  "FullVersion": "1.7.0.327",
  "MajorMinorRevision": "1.7.0",
  "Build": "327"
  },
  "OSIsoft.REST": {
  "FullVersion": "1.7.0.327",
  "MajorMinorRevision": "1.7.0",
  "Build": "327"
  },
  "OSIsoft.REST.Documentation": {
  "FullVersion": "1.7.0.327",
  "MajorMinorRevision": "1.7.0",
  "Build": "327"
  },
  "OSIsoft.Search.SvcLib": {
  "FullVersion": "1.4.0.1294",
  "MajorMinorRevision": "1.4.0",
  "Build": "1294"
  }
}

So we can see the following, PI Web AI 1.7.0.327 (this is an internal beta of PI Web API 2016) is installed and also that the Channel features is installed, the Batch feature is now part of the default installation in PI Web API 2016. We also see that the search was installed. We now known exactly which functions we can use when interacting with PI Web API.

To Chagelog for PI Web API 2016 is a good place to get started to be familiar with the new features it offers.

 

CORS information

The next information that we need is with respect to CORS setting. We can get them by looking up: https://proghackuc2016.osisoft.com/piwebapi/system/configuration

Which starts with the following data:

"AuthenticationMethods": [
  "Basic"
  ],
  "CorsExposedHeaders": "Allow,Content-Encoding,Content-Length,Date,Location",
  "CorsHeaders": "*",
  "CorsMethods": "GET,POST,PATCH, OPTIONS",
  "CorsOrigins": "http://localhost:54621,http://localhost:3626",
  "CorsSupportsCredentials": true,
  "DisableWrites": false,

 

Thus in our applications we can send Get, Post, Patch and Options HTTP request. Thus, for example we can send requests relating to the creation of new elements, but we cannot send requests to delete any elements (as delete methods all use the Delete HTTP request). And this only means that we can send such requests without generating basic errors, we may not have such rights to create any elements in the whichever AF database we are interested in. The new with PI Web API 2016 CorsExposedHeaders setting will also us to retrieve, for example, the location header that is part of a response from PI Web API, this can contain, for example, the url of a newly created Event Frame.

 

We also see that we are required to use Basic authentications and that CORS request can only originate from port 54621 and 3626. Basic authentications will be taken care of automatically in browsers like Chrome (i.e. Chrome will automatically display a text box asking you for your credentials). So, let's turn our attention to the ports mentioned in CorsOrigin.

 

Setting up ports

 

How to set the ports a web application will use very much depend on which environment you are using to do your development in. Here, we only show the required setting for an ASP.net project built in Visual Studio (The screenshots below are for Visual Studio 2015 Community). To do so, first access the project properties by going, for example, selecting it from the build menu. In the Web section of the configuration, you can see the project Url. You now need to enter either http://localhost:3626/ or http://localhost:54621/. (and not anything else such as http://127.0.0.1:3626) Once done, you now need to click on Create Virtual Directory, to allow IIS Express to use this address.

 

 

Now, next time you run the project in a browser, Visual Studio should open the following page for you: http://localhost:3626/index.html.

 

You can now go through Marcos' post on using AngularJS or download a slightly updated version here GitHub repository. The updated version allows you to point to set the base PI Web API endpoint URL directly from the main html page and now uses AngularJS 1.5.0. This sample code was also tested on Mac OSX using Xamarin. The only browser that was tested on both Mac and Windows is Google's Chrome.

ray

Introducing PI Web API Channels

Posted by ray Sep 11, 2015

Until now, PI Web API has been strictly a request-response developer technology. You make a request to the PI Web API server and it responds. This is fine so far, but you have asked us for a way to let your applications know when PI System data values have changed. In PI Web API 2015 R3, we will release a Community Technology Preview (CTP) of Channels. This is our name for a technology that will send new and changed data values to your application through web sockets.

 

Web sockets behave much like TCP sockets except that they are initiated with an HTTP request from a client. Once open, the server is free to send unsolicited data to the client. As with REST services, there are a number of client libraries available for many languages and platforms to consume data from web sockets.

 

To open a web socket to a single AF Attribute or PI Point, you would submit this:

 

wss://myhost.com/piwebapi/streams/{webId}/channel

 

The protocol identifier is "wss" which means secure web sockets (equivalent to "https"). Once open, you will get a payload from the PI Web API server whenever new or changed data values arrive. The design of the payload in this example is the same as the streams/{webId}/end function. This means you can use the code you already have to process updates from your web socket. We also support updates from: (A) all Attributes of an Element or Event Frame (equivalent to /streamsets/{webId}/end), and (B) updates from an unrelated set of Attributes or PI Points (equivalent to /streamsets/end).

 

Since the Channel open request is a standard HTTP request, we can support URL parameters. One of them will be includeInitialValues, a boolean which defaults to false. If true, the PI Web API Server will respond with current values for all Attributes or PI Points in its first response. Use this with care because this could generate a lot of data!

 

There's a lot of detail still to share. We'll post more detailed documentation and code samples soon. As I said, the feature will be released as a CTP within the PI Web API 2015 R3 release in October. What CTP means is that we will make the feature available but reserve the right to change the design based on your feedback.

Edit 2017-12-11:

 

Introduction and available resources

This post will give you basic knowledge about PI Web API and the Web's world.  This is a quick starter to master the basic concepts.

However you will also be interested by the following resources:

 

--

 

Getting Started with PI Web API - The post

 

I was asked lately to provide an example that shows how to create an event frame with PI Web API.  This was my first time I had to work with the PI Web API and I thought it would be useful it to share my experience in this blog post.

 

Things you absolutely need to know and be familiar with

I will first start by the list of things that I think you need to know before you dive in the subject:

  • PI Web API is "like" a web site, you interact with it by using urls.  Ex: http://www.osisoft.com.  Each data query you need to do against PI Web API will be a different url.
  • Depending on the PI Web API query you need to do, you will either need to do an HTTP Request of type GET, POST, DELETE or PATCH.
  • Every HTTP Request has a header. The header will be different weather you are performing a POST, a GET Request, etc...
    • Requests have a method: GET/POST
    • a Host i.e. PI Web API Server,
    • And other possible parameters that we will cover as needed.

 

Example - GET request SENT to the server that hosts PI WEB API

GET /piwebapi/system/userinfo HTTP/1.1
Host: tst-srv-pi2015
Connection: keep-alive
Cache-Control: max-age=0
Accept: */*
Origin: http://localhost:63342
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36
Referer: http://localhost:63342/JavascriptFramework/JavascriptFramework/src/index.html
Accept-Encoding: gzip, deflate, sdch
Accept-Language: fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4
                               

 

  • The client you will use ( Browser + JavaScript, c#, Postman Google Add In) will have a HUGE impact on CORS, if you don't understand CORS you absolutely need to stop here and take 5-10 minutes to understand it. Mozilla Foundation gives a great explanation.  Please keep it in mind and make sure you understand when it applies.  Wikipedia article is not bad neither.  But to be simple if your application is not built to run in a web browser you should not care to much about CORS.

 

I am doing a little summary here in case you don't look

 

What is CORS ? - read this if you are going to request data with JavaScript (ajax / XMLHTTPRequest)

I'll give my little quick and dirty explanation of the thing, this is not my field of expertise so it may not be as accurate as the articles presented above:

 

At the very first there is the Cross-site HTTP requests limitation.  This is implemented in the latest browsers and prevents JavaScript code to execute (ajax) queries to an external website (domain).

ex: a script that is inside the web page www.osisoft.com could not (without proper CORS settings on the remote site) request data on www.google.com.  This was implemented for security reasons.

Then the W3C group has worked on a recommendation to see how resources can be shared between different domains, and they came up with CORS. It stands for Cross-Origin Resource sharing.

CORS is a way to configure the "foreign" web server to accept Cross-site HTTP requests from the client, the client exposes the website origin to the foreign server with its header.

 

There are two types of HTTP requests that can occur with CORS - Reference here

 

Simple Requests

  • Only uses GET, HEAD or POST. If POST is used to send data to the server, the Content-Type of the data sent to the server with the HTTP POST request is one of application/x-www-form-urlencoded, multipart/form-data, or text/plain.
  • Does not set custom headers with the HTTP Request (such as X-Modified, etc.)
                    

 

Preflighted Requests

  • It uses methods other than GET, HEAD or POST.  Also, if POST is used to send request data with a Content-Type other than application/x-www-form-urlencoded, multipart/form-data, or text/plain, e.g. if the POST request sends an XML payload to the server using application/xml or text/xml, then the request is preflighted.
  • It sets custom headers in the request (e.g. the request uses a header such as X-PINGOTHER)
                   

Preflighted requests will first send an OPTION HTTP request, to get details on what is allowed to send to the "foreign" (PI Web API) server.  This is where the PI WEB API Server will send us back : "CorsHeaders": "accept, content-type, Access-Control-Allow-Origin" for our Ajax query to work!

 

Ok! so if we have something to remember here it is:

  • If I am create a web client using JavaScript and a browser, I need to make sure I understand CORS.  GET may work easily and POST will be pre-flighted.
  • If I am using other technologies (.net, python, etc) this is less of a problem

 

Enough with CORS!

 

How to work with PI Web API?

 

You will need the documentation, it covers installation, configuration debugging and more...

 

When you navigate to you PI Web Api Server using your browser, you will be performing GET queries, browser do not execute other types of HTTP requests such as POST unless there is a form and a button on the page, or there is javascript that does that for you. This means you cannot do any type of actions from your browser by simply clicking.

 

This is the very first approach you should have, so from https://YOUR_SERVER/piwebapi you can find links that will lead you to

This is the starting point, you have to get in one of these links and select a server, from within the server scope, you can then call the data you need.

 

Ex: I want to Element Attribute values?

I select the AssetServer, then I select "Databases" and it returns me the list of all AF Databases and their possible "actions", once you get there you can go further down and you will see at some point the values you are interested with...

You can also look at the documentation (/piwebapi/help ) to see how to use different methods:

"Links": {
  "Self": "https://tst-srv-pi2015/piwebapi/assetdatabases/D0gaIE9drlp06O5SHUQQ87kgLQVMmlQYQ0q0cOOuZUedhgVFNULVNSVi1QSTIwMTRcU1VQUE9SVA",
  "Elements": "https://tst-srv-pi2015/piwebapi/assetdatabases/D0gaIE9drlp06O5SHUQQ87kgLQVMmlQYQ0q0cOOuZUedhgVFNULVNSVi1QSTIwMTRcU1VQUE9SVA/elements",
  "ElementTemplates": "https://tst-srv-pi2015/piwebapi/assetdatabases/D0gaIE9drlp06O5SHUQQ87kgLQVMmlQYQ0q0cOOuZUedhgVFNULVNSVi1QSTIwMTRcU1VQUE9SVA/elementtemplates",
  "EventFrames": "https://tst-srv-pi2015/piwebapi/assetdatabases/D0gaIE9drlp06O5SHUQQ87kgLQVMmlQYQ0q0cOOuZUedhgVFNULVNSVi1QSTIwMTRcU1VQUE9SVA/eventframes",
  "AssetServer": "https://tst-srv-pi2015/piwebapi/assetservers/S0gaIE9drlp06O5SHUQQ87kgVFNULVNSVi1QSTIwMTQ",
  "ElementCategories": "https://tst-srv-pi2015/piwebapi/assetdatabases/D0gaIE9drlp06O5SHUQQ87kgLQVMmlQYQ0q0cOOuZUedhgVFNULVNSVi1QSTIwMTRcU1VQUE9SVA/elementcategories",
  "AttributeCategories": "https://tst-srv-pi2015/piwebapi/assetdatabases/D0gaIE9drlp06O5SHUQQ87kgLQVMmlQYQ0q0cOOuZUedhgVFNULVNSVi1QSTIwMTRcU1VQUE9SVA/attributecategories",
  "TableCategories": "https://tst-srv-pi2015/piwebapi/assetdatabases/D0gaIE9drlp06O5SHUQQ87kgLQVMmlQYQ0q0cOOuZUedhgVFNULVNSVi1QSTIwMTRcU1VQUE9SVA/tablecategories",
  "EnumerationSets": "https://tst-srv-pi2015/piwebapi/assetdatabases/D0gaIE9drlp06O5SHUQQ87kgLQVMmlQYQ0q0cOOuZUedhgVFNULVNSVi1QSTIwMTRcU1VQUE9SVA/enumerationsets",
  "Tables": "https://tst-srv-pi2015/piwebapi/assetdatabases/D0gaIE9drlp06O5SHUQQ87kgLQVMmlQYQ0q0cOOuZUedhgVFNULVNSVi1QSTIwMTRcU1VQUE9SVA/tables"
  }
                  

 

You will see that the documentation talks a lot about Web IDs, if you are building your own application, it would be a clever approach to Cache them on the client, so you can re-use them and avoid to perform so many queries on the PI WEB API Server.

This way you can do data calls directly.

 

How do I test other type of HTTP requests then?

You should use Postman REST client to do that, it is an add in for chrome.

 

Here is an example that shows how to use it for post request: you must notice the Content-Type header that is required.

2015-04-16_19-02-09_Postman.png

This is the result "201 created" after the Ssend button was pressed.

2015-04-16_19-01-58_Postman.png

 

 

Make PI WEB API ready for ajax calls - This is The Configuration I used for my development environment - DO NO USE RED PARAMETERS FOR A PRODUCTION SERVER THAT IS EXPOSED TO THE INTERNET!

 

You can simply use PI System Explorer to configure that, this is in the Configuration Database: \\YOUR_AFSERVER\Configuration\OSIsoft\PI Web API\YOUR_PI_WEB_API_SERVER\System Configuration.

Here are the settings you need to have to use jquery + ajax with CORS, please take these settings as for development purpose only.  In production I would change red settings to tighten the security at the maximum I can.

  "AuthenticationMethods": [

  "Anonymous"

  ],

  "CorsHeaders": "accept, content-type, Access-Control-Allow-Origin",

  "CorsMethods": "*",

  "CorsOrigins": "*",

  "CorsSupportsCredentials": false,

 

 

Example?

I told you that I was looking to create an event frame out of this right? Here is my little example.

A little bonus, it displays the user information too.

This little app does POST and GET HTTP requests and there is also a .js file that wraps content related to PI WEB API

 

To make it work you will need to change one line in the file app-pi_web_api.js line 12, you need to replace the server name for yours.

var piwebapiBaseUrl = "https://tst-srv-pi2015/piwebapi/";

            

 

The code is definitely not perfect, but as I am studying for the MS Exam 70-480 - Programming in HTML5 with JavaScript and CSS3 at the moment, I am trying to put things as right as I can.

( you can see from the screenshot below that I did not work the CSS3 part very hard )

 

 

2015-04-17_17-22-07_PI Web API Example.png

This is just the beginning for this example, I am planning to add more features such as tag search, and get data as I get "howto's" requests!

 

 

Other useful resources:

 

 

Hope this helps!

https://pisquare.osisoft.com/community/developers-club/blog/2016/09/21/pi-web-api-useful-links-nexus