The PI Web API 2015 R3 is released! There are many exciting new features that come with this release, including the much discussed batch CTP and channel CTP. The purpose of this blog post is to test out some of the new features, as well as write a simple JavaScript application that incorporate batch and channel to see how they work.

 

Enhancements

Let’s take a look at some of the enhancements in this release (see release notes here).

 

Search AF Elements or Event Frames by their attribute value

It looks like this functionality is enabled in the actions CreateSearchByAttribute and ExecuteSearchByAttribute in the Element and EventFrame controllers. Let’s test this out in a REST client.

 

I’m using the NuGreen database:

 

In this database, Houston, Little Rock, Tucson, and Wichita belongs to the element template “Plant”. I would like to search for all element where the Environment attribute is above 0.

 

Using the CreateSearchByAttribute action in the Element Controller:

Sending HTTP POST request to https://MyPIWebAPIServer/piwebapi/elements/searchbyattribute with payload:

 

{
    "SearchRoot":"E0G1eh97OAw0KXIEzJrhPzTAzVOQ_L9z4xGT8wAVXYKvGgRE5HLUFGMjAxNFxOVUdSRUVOXE5VR1JFRU4",
    "ElementTemplate":"T0G1eh97OAw0KXIEzJrhPzTALcH_J_O6eUyAvrBdmUbkgARE5HLUFGMjAxNFxOVUdSRUVOXEVMRU1FTlRURU1QTEFURVNbUExBTlRd",
    "ValueQueries":
    [
        {
            "AttributeName":"Environment",
            "AttributeValue":"0",
            "SearchOperator":"GreaterThan"
        }
    ]
}

 

SearchRoot contains the WebId of NuGreen, and ElementTemplate contains the WebId of the “Plant” Element Template. It looks like only the root template’s child elements are searched, but not the full hierarchy (e.g. if SearchRoot is not specified, it defaults to the Asset Database; however, this search returns no results because the city elements are not direct children of the root asset database). In the response’s “location” header, I was able to get the link which store the SearchId. The SearchId is simply an encoded string of the query.

 

Using the ExecuteSearchByAttribute action in the Element Controller:

From the previous request, I was able to get the search result in the response content (first page only). To get the response by a separate GET request, I sent a HTTP GET request to https://MyPIWebAPIServer/piwebapi/elements/searchbyattribute/{searchId}(where searchId is obtained from the previous request). The response body contains the resulting elements based on the search criteria!

 

 

Allow any HTTP method request as HTTP POST request

In this release, you can submit a HTTP POST request to the home controller of PI Web API with the appropriate header to reach any resources. E.g.

 

Sending a HTTP POST request to: https://MyPIWebAPIServer/piwebapi with headers:

 

You will receive the same response as submitting a GET request to https://MyPIWebAPIServer/piwebapi/system/versions directly.

 

 

Batch CTP and Channel CTP

Now, onto the much anticipated Batch and Channel CTPs! I have decided to write a JavaScript plotting application to test out these new functionalities. This is going to be an application where:

  • User can enter a tag mask and get a list of tag (batch demo).
  • User can then select a tag, enter a time interval, and plot a trend (normal HTTP request).
  • The plot will automatically update based on new values coming in (channels demo).

 

Environment

I am using Visual Studio 2015 to write this application. Create a new empty ASP.NET Web Application without any template.

 

To start, let’s add the following NuGet packages:

  • Bootstrap (Install-Package bootstrap)
  • jQuery (Install-Package jQuery)
  • Flot (Install-Package flot)

 

Flot is an open-sourced JavaScript plotting library. I picked it based on its simplicity and support for time-series plot. For more information, please refer to the Flot documentation. If you are interested in other plotting options, check out this previous discussion.

 

Let’s also create the files needed to build this application:

  • index.html (under the root): main HTML page
  • app.js (under the Scripts folder): main JavaScript file
  • piwebapi_wrapper.js (under the Scripts folder): PI Web API wrapper.

 

HTML page

Let’s write the HTML page (index.html):

 

<!DOCTYPE html>
<html>
<head>
    <title>Plotting with PI Web API Channels</title>
    <meta charset="utf-8" />
    <link rel="stylesheet" type="text/css" href="Content/bootstrap.min.css" />
</head>
<body>
    <div class="container">
        <!-- User login form -->
        <div id="auth-view-mode">
            <h1>Plotting with PI Web API Channels: Login Page</h1>
            <form>
                <div class="form-group">
                    <label for="username">User name</label>
                    <input type="text" class="form-control" id="username" placeholder="User name" />
                </div>
                <div class="form-group">
                    <label for="password">Password</label>
                    <input type="password" class="form-control" id="password" placeholder="Password" />
                </div>
                <button type="button" id="go-to-plot-btn" class="btn btn-default">Submit</button>
            </form>
        </div>

        <!-- Plotting screen -->
        <div id="plot-view-mode" style="display: none;">
            <h1>Plotting with PI Web API Channels</h1>
            <br />
            <div id="nav">

                <!-- Tag search -->
                <form class="form-inline">
                    <input type="text" class="form-control" id="tagmask-text" placeholder="Tag Mask" />
                    <button type="button" id="search-btn" class="btn btn-default">Search for tags</button>
                </form>
                <br />

                <!-- Information to plot -->
                <form class="form-inline">
                    <select class="form-control" id="tag-select"></select>
                    <div class="form-group">
                        <input type="text" class="form-control" id="time-interval-text" placeholder="Time Interval" />
                        <select class="form-control" id="time-interval-unit">
                            <option value="h">hour(s)</option>
                            <option value="m">minute(s)</option>
                            <option value="s">second(s)</option>
                        </select>
                    </div>
                    <button type="button" id="plot-btn" class="btn btn-primary" disabled>Start Plot</button>
                    <button type="button" id="stop-btn" class="btn btn-danger" disabled>Stop Update</button>
                    <button type="button" id="back-btn" class="btn btn-default">Back to login page</button>
                </form>
                <br />
            </div>

            <!-- Placeholder for plot -->
            <div id="plot" style="width:800px; height:400px"></div>
        </div>
    </div>

    <script type="text/javascript" src="Scripts/jquery-2.1.4.min.js"></script>
    <script type="text/javascript" src="Scripts/flot/jquery.flot.min.js"></script>
    <script type="text/javascript" src="Scripts/flot/jquery.flot.time.min.js"></script>
    <script type="text/javascript" src="Scripts/bootstrap.min.js"></script>
    <script type="text/javascript" src="Scripts/piwebapi_wrapper.js"></script>
    <script type="text/javascript" src="Scripts/app.js"></script>
</body>
</html>


 

You will notice that the web application is divided in two main divs. The first div is the login form that allows user to enter in the user name and password. The second div, initially hidden, is the main page of the plot application. (Credit to Marcos for the original source of the login form.) To see the html page, remove the style="display: none;" in the “plot-view-mode” div.

Don’t forget to put the style attribute back in the div after visualizing the HTML page.

 

Login – Basic Authentication

When user enters the username and password, we would like to make a HTTP GET request to the home controller to check if the credentials are correct. If not, an error alert is displayed.

 

First, let’s add some methods in the PI Web API wrapper (piwebapi_wrapper.js) to send ajax request based on the provided username and password.

var basePIWebAPIUrl = "https://YourPIWebAPIServer/piwebapi";
var piDataArchiveName = "YourPIDataArchive";

// PI Web API wrapper
var piwebapi = (function () {
    var currentUserName = null;
    var currentPassword = null;

    // Send ajax request
    var processJsonContent = function (url, type, data, successCallBack, errorCallBack) {
        return $.ajax({
            url: encodeURI(url),
            type: type,
            data: data,
            contentType: "application/json; charset=UTF-8",
            beforeSend: function (xhr) {
                xhr.setRequestHeader("Authorization", makeBasicAuth(currentUserName, currentPassword));
            },
            success: successCallBack,
            error: errorCallBack
        });
    };

    // Return authorization header value for basic authentication
    var makeBasicAuth = function (user, password) {
        var tok = user + ':' + password;
        var hash = window.btoa(tok);
        return "Basic " + hash;
    };

    // Publicly available methods
    return {
        // Store current username and password
        SetCredentials: function (user, password) {
            currentUserName = user;
            currentPassword = password;
        },

        // Test basic authentication
        Authorize: function (successCallBack, errorCallBack) {
            return processJsonContent(basePIWebAPIUrl, 'GET', null, successCallBack, errorCallBack);
        },

        // Reset username/password and web socket
        Reset: function () {
            currentUserName = null;
            currentPassword = null;
        }
    }
})();

 

Note that the above is written as a JavaScript module. (For more information, please refer to this previous blog post. We have two private functions and three public functions:

  • Private
    • processJsonContent: send an ajax request to the specified URL, supply different callbacks when the request is successful or not.
    • makeBasicAuth: return the authorization header for basic authentication. The authorization header is the base64-encoded string of the username:password.
  • Public
    • SetCredentials: store the username and password.
    • Authorize: submit a GET request to the home controller to test credentials.
    • Reset: reset private variables in the module.

 

In the app.js file:

// Reset buttons, text boxes and plot in the plot view
var resetPlotView = function () {
    $('#tagmask-text').val('');
    $('#tag-select').empty();
    $('#time-interval-text').val('');
    $('#plot-btn').attr('disabled', 'disabled');
    $('#stop-btn').attr('disabled', 'disabled');
    $('#plot').empty();
};

// Test username/password and go to plot screen
$('#go-to-plot-btn').click(function () {
    // Store username and password
    var username = $("#username").val();
    var password = $("#password").val();
    piwebapi.SetCredentials(username, password);

    // Check authentication
    var authSuccessCallBack = function (data, statusMessage, statusObj) {
        if (statusObj.status == 200) {
            $('#auth-view-mode').hide();
            $('#plot-view-mode').show();
        }
    };
    var authErrorCallBack = function (data) {
        if (data.status == 401) {
            alert("Invalid username and password.");
        }
        else {
            alert("Error during validation.");
        }
    };
    piwebapi.Authorize(authSuccessCallBack, authErrorCallBack);
});

// Go back to login screen
$("#back-btn").click(function () {
    $("#username").val('');
    $("#password").val('');

    $("#plot-view-mode").hide();
    $("#auth-view-mode").show();

    piwebapi.Reset();
    resetPlotView();
});


 

Testing the application, we can now test authentication from the login screen by providing the correct pair of username and password. We can also go back to the login page by clicking the “Back to login page” button on the main plotting page. Note that the username and password will be used to make every HTTP request in the application via basic authentication (see processAjaxContent function).

 

Tag Search – Batch Demo

Let’s build the functionality on the “Search for tags” button. In a PI Data Archive, we would like to search the list of tags by specifying a tag mask. (Of course, this can easily be achieved using the indexed search, but let’s take the opportunity to test out the new batch functionality here). Looking at the GetPoints action in the DataServer controller, we can submit a GET request to dataservers/{webId}/points and specify a nameFilter. This will give us a list of all the matching points. As part of the URL, we need to specify the WebId for the PI Data Archive, which can be achieved by using the GetByName action in the DataServer controller. Why don’t we bundle these requests together to test out Batch?!

 

Notes about batch: We can now batch multiple REST requests into a single HTTP request. This could be helpful in scenarios where network speed between the REST client and the PI Web API server is the bottleneck. The help file provides a detailed example of a sample batch request. In the payload, you can specify a list of requests with specific IDs for each. You can also specify dependency by listing the parent IDs in each request (i.e. parent requests must complete before the child request).

 

In the piwebapi_wrapper.js file, let’s add a new private function:

// BATCH demonstration: get list of tags based on particular PI Data Archive and tag mask
var getTagListRequest = function (piDataArchive, tagMask, successCallBack, errorCallBack) {
    var url = basePIWebAPIUrl + "/batch";
    var data = {};
    data["1"] = {
        "Method": "GET",
        "Resource": encodeURI(basePIWebAPIUrl + "/dataservers?name=" + piDataArchive)
    };
    data["2"] = {
        "Method": "GET",
        "Resource": "{0}?nameFilter=" + encodeURIComponent(tagMask),
        "Parameters": [
            "$.1.Content.Links.Points"
        ],
        "ParentIds": [
            "1"
        ]
    };
    return processJsonContent(url, "POST", JSON.stringify(data), successCallBack, errorCallBack);
};

 

and a public function:

 

        // Get the list of tags based on tagmask
        GetTagList: function (tagMask, processFunction, errorFunction, errorCallBack) {
            var successCallBack = function (data) {
                var filteredTags = {};
                if (data["1"].Status == 200 && data["2"].Status == 200) {
                    for (var i = 0; i < data["2"].Content.Items.length; i++) {
                        filteredTags[data["2"].Content.Items[i].Name] = data["2"].Content.Items[i].WebId;
                    }
                    processFunction(filteredTags);
                }
                else {
                    errorFunction();
                }
            };
            getTagListRequest(piDataArchiveName, tagMask, successCallBack, errorCallBack);
        }

 

The getTagListRequest method bundles 2 dependent requests into 1 HTTP request:

  1. First, to get PI Web API endpoints for a specific PI Data Archive
  2. Then, use the Points endpoint to get the list of tags satisfying the provided tag mask.

 

The GetTagList public method calls the private getTagListRequest method and process the results. Since a batch request will return a status code of 207 even if some of the batched requests failed, we need to check individual status code for each request from the response body. Since the response will contain an Id identical to the request Id (1 and 2 in this case), we can easily determine which response correlates with each request.

 

In the app.js file, call the GetTagList function when user clicks on the “Search for tags” button:

 

var piTagWebIds = {}; 

// General error call back
var errorCallBack = function (xhr) {
    console.log(xhr.responseText);
}

// Search for tags using tag mask
$('#search-btn').click(function (data) {
    $('#plot-btn').attr('disabled', 'disabled');
    $('#tag-select').empty();
    var tagMask = $('#tagmask-text').val();
    if (tagMask.length > 0) {
        var populateTags = function (tags) {
            $.each(tags, function (name, webId) {
                $('#tag-select').append("<option value=\"" + name + "\">" + name + "</option>");
            });
            piTagWebIds = tags;
            $('#plot-btn').removeAttr('disabled');
        };
        var error = function () {
            alert("Error during tag search.");
        };
        piwebapi.GetTagList(tagMask, populateTags, error, errorCallBack);
    }
});

 

Testing the application, you can now enter a tag mask and see the list of tags populated:

 

Getting Plot Data

Next, we need to get the plot data from PI Web API and plot a time-series graph using Flot.

 

In the piwebapi_wrapper.js file, add a private function:

    // Normal HTTP request: get plot values based on webId
    var getValuesRequest = function (tagWebId, startTime, pixels, successCallBack, errorCallBack) {
        var url = basePIWebAPIUrl + "/streams/" + tagWebId + "/plot?intervals=" + pixels;
        if (startTime != null) {
            url += "&starttime=" + startTime;
        }
        return processJsonContent(url, "GET", null, successCallBack, errorCallBack);
    };

 

and a public function:

        // Get plot values based on webId
        GetValues: function (tagWebId, startTime, pixels, successCallBack, errorCallBack) {
            getValuesRequest(tagWebId, startTime, pixels, successCallBack, errorCallBack);
        }

 

These functions allow us to submit a GET request to the Streams Controller > GetPlot action.

 

In the app.js file, we need to transform the response content into a format consumable by our plotting library Flot.

 

Data in Flot is an array of data series:

    [ series1, series2, ... ]

 

In this case, we only have 1 series (1 PI tag). To plot time series data, we need to express our timestamps and values in an array as follow (represent 1 series):

    [ [time1, value1], [time2, value2], ... ]

 

After we received the response body, we will loop through each item and add the timestamp and value to an array if the quality of the value is “good”.

 

In app.js

var plotData = [[]];
var width = "800";

// Plot options
var options = {
    xaxis: {
        mode: "time",
        timeformat: "%Y/%m/%d %H:%M:%S",
        timezone: "browser"
    }
};

// Start plot
$('#plot-btn').click(function (data) {
    var tag = $('#tag-select option:selected').text();
    var timeInterval = $('#time-interval-text').val();
    var timeIntervalUnit = $('#time-interval-unit option:selected').val();

    if (tag.length > 0) {
        // Get initial values
        if (timeInterval.length <= 0 || isNaN(timeInterval)) {
            timeInterval = 1;
            $('#time-interval-text').val(timeInterval);
        }
        var startTime = "*-" + timeInterval + timeIntervalUnit;

        var successCallBack = function (values) {
            plotData = [[]];
            for (var i = 0; i < values.Items.length; i++) {
                if (values.Items[i].Good) {
                    plotData[0].push([new Date(values.Items[i].Timestamp).getTime(), values.Items[i].Value]);
                }
            }
            $.plot($('#plot'), plotData, options);
        };

        piwebapi.GetValues(piTagWebIds[tag], startTime, width, successCallBack, errorCallBack)

        $('#plot-btn').attr('disabled', 'disabled');
        $('#stop-btn').removeAttr('disabled');
    }
});

// Stop update
$('#stop-btn').click(function (data) {
    $('#plot-btn').removeAttr('disabled');
    $('#stop-btn').attr('disabled', 'disabled');
});

 

Testing the application, you should be able to specify a time interval, and plot the values for a specific PI tag until current time.

For now, we will disable the start button and enable the stop button after the start button is pressed (and vice versa) in preparation for our next section on channels demonstration. The idea is once the start button is pressed, the trend will be updating in real-time as the PI tag gets new values. For now, you can toggle the start and stop button to refresh the trend for a fast updating tag.

 

Channels CTP

Next, let’s make this plot update on its own when new values come in. With traditional REST HTTP requests, the client will have to initiate the request. This means that the client would have to periodically poll the server for new information. In PI Web API 2015 R3, channels are introduced. Channels use the WebSocket protocol to allow the PI Web API server send messages containing new stream value changes. Once a channel is opened, the client will receive continuous updates from the server without having to poll. This can be more efficient by reducing unnecessary requests, as well as reducing the overhead that comes with HTTP headers.

 

You can find out more information about channels in the PI Web API help file. Look at the GetChannel (and GetChannelAdHoc) action in the Stream and StreamSet controllers respectively. In addition, example usage is shown in the Channels Topic section.

 

Since I’m new to WebSocket, I decided to test out this functionality first using a pre-built tool available at https://www.websocket.org/echo.html. I have provided a URL for testing: wss://MyPIWebAPIServer/piwebapi/streams/{webId}/channel where {webId} is for an interface heartbeat tag.

 

I see new values coming in once the WebSocket is opened!

 

A few caveats that I have discovered:

  • Since WebSocket is still a relatively new technology, not all browsers (and browser versions) support WebSockets. (e.g. I tested this successfully on Chrome 46.0.2490.80)
  • If you have trouble connecting, open up the browser developer tools (usually F12) to take a look at the error message. If you see an authentication failure, try to open up a new tab in the same browser and navigate to any PI Web API endpoint (and enter credentials if necessary). This is because we cannot customize WebSocket headers from JavaScript currently, so we are relying on the authorization headers sent from the browser. (Here, I’m bypassing the limitation by using the basic authentication cached by the browser.)
  • Since response only comes in when a new value comes in for the tag; if your tag is slow updating, you will not see any response until a new value comes in. You can consider adding the includeInitialValues=true URL parameter to see the current value of the stream when the channel is opened.

 

Once we are familiar with the functionality of channels, let’s implement it in our JavaScript application.

 

In piwebapi_wrapper.js, add the following public variable:

var basePIWebAPIChannelUrl = "wss://YourPIWebAPIServer/piwebapi";

 

Add the following private variable in the module:

var webSocket = null;

 

Add the following private functions in the module:

    // Channels demonstration: open web socket based on webId
    var openStreamChannel = function (tagWebId, channelOpenCallBack, channelErrorCallBack, channelMessageCallBack, channelCloseCallBack) {
        var url = basePIWebAPIChannelUrl + "/streams/" + tagWebId + "/channel";
        webSocket = new WebSocket(encodeURI(url));
        webSocket.onopen = channelOpenCallBack;
        webSocket.onerror = channelErrorCallBack;
        webSocket.onmessage = channelMessageCallBack;
        webSocket.onclose = channelCloseCallBack;
    };

    // Channels demonstration: close web socket
    var closeStreamChannel = function () {
        if (webSocket != null) {
            webSocket.close();
        }
    };

 

Add the following public functions in the module:

       // Open channel for a stream
        OpenChannel: function (tagWebId, channelOpenCallBack, channelErrorCallBack, channelMessageCallBack, channelCloseCallBack) {
            openStreamChannel(tagWebId, channelOpenCallBack, channelErrorCallBack, channelMessageCallBack, channelCloseCallBack);
        },

        // Close existing opened channel
        CloseChannel: function () {
            closeStreamChannel();
        }

 

Add code to reset websocket in the public Reset function:

        Reset: function () {
            currentUserName = null;
            currentPassword = null;
            webSocket.close();
            webSocket = null;
        },

 

The above JavaScript simply opens a channel based on a specific URL, and allow for callbacks when the channel:

  • Just opened (onopen)
  • Encounters an error (onerror)
  • Receives a new message from the server (onmessage)
  • Just closed (onclose)

 

In app.js, add the following in the click handler for #plot-btn after calling piwebapi.GetValues:

        // Open channel
        var messageCallBack = function (event) {
            var values = JSON.parse(event.data);
            var plotStartTime = getPlotStartTime(timeIntervalUnit, timeInterval);

            // Remove old values before new start time
            var removeCount = 0;
            while (plotData[0][removeCount][0] < plotStartTime.getTime()) {
                removeCount++;
            }
            plotData[0].splice(0, removeCount);

            // Add new values
            for (var i = 0; i < values.Items[0].Items.length; i++) {
                if (values.Items[0].Items[i].Good) {
                    var timestamp = new Date(values.Items[0].Items[i].Timestamp);
                    if (timestamp.getTime() >= plotStartTime.getTime()) {
                        plotData[0].push([timestamp.getTime(), values.Items[0].Items[i].Value]);
                    }
                }
            }

            // Sort array
            plotData[0].sort(function (a, b) {
                if (a[0] === b[0]) {
                    return 0;
                }
                else {
                    return (a[0] < b[0]) ? -1 : 1;
                }
            });

            // Plot
            $.plot($('#plot'), plotData, options);
        };

        var errorCallBack = function () {
            alert("Error getting updates.");
        };

        var openCallBack = function () {
            $('#search-btn').attr('disabled', 'disabled');
        };

        var closeCallBack = function () {
            $('#search-btn').removeAttr('disabled');
        };

        piwebapi.OpenChannel(piTagWebIds[tag], openCallBack, errorCallBack, messageCallBack, closeCallBack);

 

Note that we need to parse the string as JSON (by JSON.parse) returned by WebSocket. In addition, the above code does the following when a new value is received:

  • Removed values outside of the time interval of plotting
  • Add new values that are within the plotting interval
  • Sort arrays because there can be out-of-order data, and channel does not guarantee that the values are in chronological order.

 

The getPlotStartTime function is designed to get the start time of the plot by subtracting a specific interval from the current time:

 

// Subtract time interval from now
var getPlotStartTime = function (intervalUnit, intervalVal) {
    var ret = new Date();
    switch (intervalUnit.toLowerCase()) {
        case 'h': ret.setTime(ret.getTime() - intervalVal * 3600000); break;
        case 'm': ret.setTime(ret.getTime() - intervalVal * 60000); break;
        case 's': ret.setTime(ret.getTime() - intervalVal * 1000); break;
        default: ret = undefined; break;
    }
    return ret;
};

 

Finally, we need to close the channel when the stop update button is clicked.

// Stop update
$('#stop-btn').click(function (data) {
    piwebapi.CloseChannel();
    $('#plot-btn').removeAttr('disabled');
    $('#stop-btn').attr('disabled', 'disabled');
});

 

Testing the application with a fast moving tag:

liveupdate.gif

 

The trend is now live updating whenever new values come in! Note that this trend will not update unless there is a new value that comes in, so your trend can look stale if the tag is not updating frequently. If you do not have a fast updating tag, consider plotting a test tag and manually inputting data to the tag.

 

 

Conclusion

I hope you have as much fun using the new PI Web API 2015 R3 as I am! The full project is available in this GitHub repository. Please note that both batch and channel are currently CTP releases, which means that if you would like to see them as official release, please give us your comments and feedback!