Skip navigation
All Places > PI Developers Club > Blog > 2015 > March

Interested to see how the PI System and Esri ArcGIS maps can be used to show cool visualizations like the one below?



At OSIsoft, we’ve developed an internal application PI Fitness to store geolocation data (such as latitude and longitude) collected during activities such as running, walking, and even mountain biking!

Other than secretly spying on co-workers’ weekend activities , it is a great tool for us to find new exercise routes or discover shared interests. Using Asset-Based Analytics, we've also created a "leaderboard" of total points to provide some competitive flavor.


The architecture is like such:



One of the data sources is geolocation data stored as GPX files. GPX is an XML schema for exchanging GPS and related data. Free mobile apps such as RunKeeper and Strava allow you to track your geolocation during an activity and later export them as GPX files. Wearable devices such as Garmin watches also expose their data as GPX files.


We can bring this data into PI and leverage Esri Maps Javascript API to share this data via a website.


The first part of this blog series will show how to parse the GPX file and use the PI System to store this information and bring more context. The next part will show how to retrieve this data from PI and display routes on a map using the Esri Maps API.


The full Visual Studio 2013 solution is attached below, along with an AF Element template and sample GPX file.


Special thanks to Paul Pirogovsky for creating the GPX parser!


Taking a peek at the GPX file


The GPX file is actually very simple. It is just an XML text file with a specific schema. Here’s a sample below:


<?xml version="1.0" encoding="UTF-8"?>
  creator="RunKeeper -"
<trkpt lat="37.808881000" lon="-122.256421000"><ele>9.0</ele><time>2015-03-30T02:34:18Z</time></trkpt>
<trkpt lat="37.808849000" lon="-122.256279000"><ele>9.2</ele><time>2015-03-30T02:34:28Z</time></trkpt>


You can see it stores a timestamp, latitude, longitude, and elevation.


Generate C# classes to represent GPX data


Our goal is to generate C# classes that will encapsulate the GPX data after it's been deserialized. We can use the tool xsd. From the Visual Studio Developer Command Prompt, run


xsd GpxXmlType.xml


(I named my sample file GpxXmlType.xml)


Then run


xsd GpxXmlType.xsd /classes


This generates GpxXmlType.cs, which you want to include in your project.


To deserialize, we can run


string fileContents = File.ReadAllText(pathToFile);

XmlSerializer xmlSerializer = new XmlSerializer(typeof(GpxXmlType));
using (TextReader reader = new StringReader(fileContents))
        GpxXmlType _gpxData = xmlSerializer.Deserialize(reader) as GpxXmlType;


Once we have _gpxData, we can navigate through the properties of _gpxData to browse the XML structure.

To assist us later, we have also created a simple POCO (plain ol' C# object) called TrackPoint for storing a coordinate event, representing a 4-tuple of Timestamp, Elevation, Latitude, and Longitude.


public class TrackPoint
    public DateTime Timestamp { get; set; }
    public double Elevation { get; set; }
    public double Latitude { get; set; }
    public double Longitude { get; set; }

    public TrackPoint(DateTime timestamp, double elevation, double latitude, double longitude)
        this.Timestamp = timestamp;
        this.Elevation = elevation;
        this.Latitude = latitude;
        this.Longitude = longitude;



AF Asset Structure


Before parsing the GPX data, it is good to decide what our AF structure will look like. Here is mine below.




I created a "My Routes" element and four attributes underneath. Each attribute corresponds to a data stream exposed in the GPX file.


Also, we will create an enumeration type, where the names reflect the AF attributes that store our data. This makes it easier to key each AFValues collection and associate them with an AF Attribute.

public enum GpxAttributes


We will also want to create an AF Event Frame for each route to store the start and end times. This makes retrieval of the route data from PI easier later on, as we have a start/end time to "bookmark" the data. It also enables us to perform configured analytics should we want to do so, given the time context provided by an event frame.


To bring data into PI from GPX, we will use our custom GpxXmlType class (generated from xsd) and AF SDK to send these values to the PI Data Archive.


Parsing the XML into AFValues

First, we will create a client class that services calls to read GPX data and convert them to AFValue objects. It is called GpxToPIClient.


public class GpxToPIClient
    private GpxParser _parser;
    public GpxToPIClient(GpxParser parser)
          _parser = parser;

    public IDictionary<GpxAttributes, AFValues> GetTimeSeries()
          return _parser.ParseString();


It has one dependency GpxParser and exposes one method GetTimeSeries(). GpxParser does the actual work. Let's take a look at its header and constructor.


public class GpxParser
    private GpxXmlType _gpxData;
    private string _filePath;

    public GpxParser(string filePath)
          this._filePath = filePath;
          string fileContents = File.ReadAllText(this._filePath);

          XmlSerializer xmlSerializer = new XmlSerializer(typeof(GpxXmlType));
          using (TextReader reader = new StringReader(fileContents))
              _gpxData = xmlSerializer.Deserialize(reader) as GpxXmlType;


GpxParser holds a reference to GpxXmlType, our class for holding XML data. The ParseString() method on GpxParser browses the XML data via GpxXmlType, creates AFValues, and updates them to the PI System. Let's take a look at the full method below.


internal IDictionary<GpxAttributes, AFValues> ParseString()
    string _activityType = GetActivityType();

      //pull all geopositional data into the class
      List<TrackPoint> _wayPoints = new List<TrackPoint>();
      foreach (trksegType trackSeg in _gpxData.trk[0].trkseg)
            foreach (wptType waypoint in trackSeg.trkpt)
                wayPoints.Add(new TrackPoint(waypoint.time, (double)waypoint.lon, (double), (double)waypoint.ele));

      //create AF objects
      AFValues _elevationValues = new AFValues();
      AFValues _latitudeValues = new AFValues();
      AFValues _longitudeValues = new AFValues();
      AFValues _activityValues = new AFValues();

      foreach (TrackPoint point in _wayPoints)
            AFTime timeStamp = new AFTime(point.Timestamp);
            _elevationValues.Add(new AFValue(point.Elevation, timeStamp));
            _latitudeValues.Add(new AFValue(point.Latitude, timeStamp));
            _longitudeValues.Add(new AFValue(point.Longitude, timeStamp));

      //update the activity tag
      DateTime temp_time = (from pt in _wayPoints
                            orderby pt.Timestamp ascending
                            select pt.Timestamp).FirstOrDefault();

      _activityValues.Add(new AFValue(_activityType, temp_time));

      //reset the activity tag to "Idle"
      temp_time = (from pt in _wayPoints
                    orderby pt.Timestamp descending
                    select pt.Timestamp).FirstOrDefault();

      //increment by one second
      temp_time = temp_time.AddSeconds((double)1.0);
      _activityValues.Add(new AFValue("Idle", temp_time));

      //create output dictionary
      IDictionary<GpxAttributes, AFValues> dict = new Dictionary<GpxAttributes, AFValues>();
      dict.Add(GpxAttributes.Activity, _activityValues);
      dict.Add(GpxAttributes.Elevation, _elevationValues);
      dict.Add(GpxAttributes.Latitude, _latitudeValues);
      dict.Add(GpxAttributes.Longitude, _longitudeValues);

      return dict;


Whew! That is a long method. It certainly doesn't follow the SRP (Single Responsibility Principle) but it does the trick


All we are doing is using our member variable _gpxData to load the XML data and create AFValues from it. We are using our enumeration GpxAttributes to key the AFValues collections. One detail is that we want to update our AFValues denoting the Activity (running, walking, etc.) by resetting it to "Idle" once we are done with the actual activity.


Creating Event Frames


AFEventFrames are perfect objects for providing a time context for our routes. Without them, it would be difficult to determine from the raw PI Point events when one activity starts and ends. We would need to look for transitions within the Activity AF Attribute which can be cumbersome. With event frames, though, it is easy to retrieve the relevant time series data for an activity. We simply get the start/end time for the event frame and pass that into RecordedValues to get the associated timeseries data.


Using event frames to store activity meta data, it is also possible to perform additional statistics on each activity, such as average speed, distance, and total time. We can also perform statistics on multiple event frames, looking at the distribution of speeds and distances based on activity and see how they vary over time.


We've created a custom EventFrameHelper class to create an event frame from each parsed GPX file. This class also extracts out a custom Route object from our stored AF data. The Route object can be conveniently used by the Esri Maps API to show the route on a map. We won't show the full methods here, but just the class stub. Please take a look at the full Visual Studio solution posted below for details.


public class EventFrameHelper
    public EventFrameHelper()
    public AFEventFrame CreateEventFrame(AFDatabase db, IDictionary<GpxAttributes, AFValues> dict, AFElement primaryRefElement)
    public IList<Route> GetRoutes(AFElement element, object startSearchTime, object endSearchTime)


The GetRoutes method returns a list of Route objects. The Route class combines the AF Event Frame metadata with the PI timeseries data. Here is the class below.

public class Route
    public string Name { get; set; }
    public DateTime StartTime { get; set; }
    public DateTime EndTime { get; set; }
    public IList<TrackPoint> Coordinates{ get; set; }

    public Route(string name, DateTime startTime, DateTime endTime, IList<TrackPoint> coordinates)
          this.Name = name;
          this.StartTime = startTime;
          this.EndTime = endTime;
          this.Coordinates = coordinates;



Sample console application


Below is a console application that shows how to use every class and method that we have described above. It demonstrates the following:


1. Set up the GPX parser classes

2. Parse the file and return a keyed collection of AFValues

3. Send these values to PI

4. Create an AFEventFrame from the GPX data

5. Retrieve the data from PI and create a Route class


class Program
     static void Main(string[] args)
          //Set up gpx parser
          string filePath = @"C:\example.gpx";
          GpxParser parser = new GpxParser(filePath);
          GpxToPIClient rkClient = new GpxToPIClient(parser);

          //Parse the Gpx file and return a dictionary of AFValues
          IDictionary<GpxAttributes, AFValues> rkData = rkClient.GetTimeSeries();

          //Update data to PI System
          PISystem ps = new PISystems().DefaultPISystem;
          AFDatabase db = ps.Databases["My Activities"];
          AFElement element = db.Elements["My Routes"];

          foreach (var stream in rkData)
              AFValues vals = stream.Value;
              AFAttribute attr = element.Attributes[stream.Key.ToString()];
              AFErrors<AFValue> errors = attr.Data.UpdateValues(vals, AFUpdateOption.Replace);   

          //Create AFEventFrame on AF server
          EventFrameHelper efHelper = new EventFrameHelper();
          AFEventFrame eventFrame = efHelper.CreateEventFrame(db, rkData, element);

          //Retrieve routes from last 60 days
          IList<Route> routes = efHelper.GetRoutes(element, "*-60d", "*");

          Console.WriteLine("Press any key to exit");



Want to play with our data? Come join the Programming Hackathon!


At PI Fitness, we've already collected over 300 routes!


Here is just a small gallery:


Running                                                                                              Mountain Biking


Walking                                                                                              Running




If you want to play with the data or learn more, please join our Programming Hackathon!


Registration is now open!




Next time, Marcos Vainer Loeff will continue with Part 2 of this blog series and show how to display data from PI using Esri ArcGIS maps!

We just made a minor change to the name of our upcoming PI OLEDB Enterprise release. We had been planning to call it 2014 but it seemed a better idea to wait until we could incorporate the features of the upcoming AF 2.7 release which includes future data and support for excluded and hidden AF Attributes. We are also implementing support for AF's delta time strings which means a monthly report built using PI OLEDB Enterprise will cover the month and not just 30 days. The last reason is, well, it's 2015...


Happy PI Day 2015!

Posted by pthivierge Mar 14, 2015

At this precise moment, I take the opportunity to wish you an happy PI Day 2015 folks!

This year is exceptional (3/14/15... ).



PI Day 2015.png


Filter Blog

By date: By tag: