In the past, in order to capture real-time weather data, I have always had to use PowerShell or Python to capture the data from an API and then reformat it before a using a configuration file (INI) that would never end. With the new functionalities added to the PI Connector for UFL, I can now parse the JSON file directly from the API call and create my dynamic AF Structure.

 

For this example, I have used OpenWeatherMap to collect real-time data. The same could be adapted for different calls (ie. forecast) or another API. Just note that at the time of writing, the UFL connector cannot create future data PI Points, so the PI Points would need to be created prior to doing the steps below. Once the points are created, the future data will flow normally.

 

Get an API key

Create an account on https://openweathermap.org/api and get your free API key. The free API key allows you to access Current Weather data and 5d/3h forecast and you get up to 60 calls per minute.

All your calls will need to include &APPID=<YourAPIKey>

 

Define the call you want to make

For example, if I want to query the current weather in Philadelphia, San Leandro, Montreal, and Johnson City in metric units:

http://api.openweathermap.org/data/2.5/group?id=6077243,4560349,5392263,4633419&units=metric&appid=<apikey>

 

Below is an example data file:

 

{
"cnt": 4,
"list": [{
"coord": {
"lon": -73.59,
"lat": 45.51
},
"sys": {
"type": 1,
"id": 3829,
"message": 0.005,
"country": "CA",
"sunrise": 1518091524,
"sunset": 1518127940
},
"weather": [{
"id": 801,
"main": "Clouds",
"description": "few clouds",
"icon": "02d"
}
],
"main": {
"temp": -12.02,
"pressure": 1027,
"humidity": 78,
"temp_min": -13,
"temp_max": -11
},
"visibility": 24140,
"wind": {
"speed": 3.6,
"deg": 200
},
"clouds": {
"all": 20
},
"dt": 1518107649,
"id": 6077243,
"name": "Montreal"
}, {
"coord": {
"lon": -75.16,
"lat": 39.95
},
"sys": {
"type": 1,
"id": 2361,
"message": 0.0048,
"country": "US",
"sunrise": 1518091260,
"sunset": 1518128952
},
"weather": [{
"id": 800,
"main": "Clear",
"description": "clear sky",
"icon": "01d"
}
],
"main": {
"temp": -1.36,
"pressure": 1030,
"humidity": 54,
"temp_min": -3,
"temp_max": 0
},
"visibility": 16093,
"wind": {
"speed": 3.1,
"deg": 300
},
"clouds": {
"all": 1
},
"dt": 1518107649,
"id": 4560349,
"name": "Philadelphia"
}, {
"coord": {
"lon": -122.16,
"lat": 37.72
},
"sys": {
"type": 1,
"id": 397,
"message": 0.0117,
"country": "US",
"sunrise": 1518102305,
"sunset": 1518140465
},
"weather": [{
"id": 701,
"main": "Mist",
"description": "mist",
"icon": "50d"
}, {
"id": 741,
"main": "Fog",
"description": "fog",
"icon": "50d"
}
],
"main": {
"temp": 8.57,
"pressure": 1024,
"humidity": 66,
"temp_min": 5,
"temp_max": 13
},
"visibility": 16093,
"wind": {
"speed": 1.41,
"deg": 44.5012
},
"clouds": {
"all": 1
},
"dt": 1518107649,
"id": 5392263,
"name": "San Leandro"
}, {
"coord": {
"lon": -82.35,
"lat": 36.31
},
"sys": {
"type": 1,
"id": 2512,
"message": 0.004,
"country": "US",
"sunrise": 1518092621,
"sunset": 1518131039
},
"weather": [{
"id": 803,
"main": "Clouds",
"description": "broken clouds",
"icon": "04d"
}
],
"main": {
"temp": 0.49,
"pressure": 1032,
"humidity": 69,
"temp_min": 0,
"temp_max": 1
},
"visibility": 16093,
"wind": {
"speed": 2.6,
"deg": 280,
"gust": 5.7
},
"clouds": {
"all": 75
},
"dt": 1518107649,
"id": 4633419,
"name": "Johnson City"
}
]
}

 

 

Parse the JSON file from the API

With the 1.2 version of the PI Connector for UFL, there is native support for JSON formatted file using the new FOREACH(), JsonGetValue() and JsonGetItem() functions.

 

JSONGetItem() function is used as part of the FOREACH statement and allow to capture the objects within the array. For those unfamiliar with JSON formatting, objects are enclosed with curly brackets {} and contain unordered data. On the other hand, arrays are enclosed with square brackets [] and are used for to store ordered objects.

As the name indicates, JSONGetValue() is used to obtain a value from a name/value pair within an object.

 

Since the data file contains an array for the 4 cities we are interested in, we will use the FOREACH() function with JsonGetItem to loop through each object (all the information for each city).

 

The first step is to capture the message using JsonGetItem("Json_input","Selector"). In this example, we are grabbing the content of "list", so the Selector is simply "list[]" and we will read the entire message using __MESSAGE.

 

Once we are within the object of the array, we simply have to use a series of JSONGetItem() to get the level where the value is stored.

 

Take advantage of the Collection data type to store the values in PI and AF

In the [FIELD] section, we created TagNames, Values, and AttributeNames. Those three collections will simply be an array of the tagnames, values, and attribute names. This will allow us to simply make one StoreEvents() call for each object. The ADD() function is used to add an item to the collection. One of the things I really like about collections is that they can store multiple data types at once. In this example, the Values collection stores DateTime and Numbers. You simply have to define the data type in [FIELD] section for the values to be added to the collection.

 

Create the AF Structure:

In order to create the AF Structure, it is important to create each level of the hierarchy. Since I wanted my attributes to be under "\Weather_Monitoring\<CityName>", I had to create the "\Weather_Monitoring" element first. Since the UFL Connector does not read the AF structure, it is necessary to do so even if the parent element already exists.

 

Final INI file:

[FIELD] 
FIELD(1).NAME="TagNames"
     TagNames.TYPE="Collection"
FIELD(2).NAME="Values"
     Values.TYPE="Collection"
FIELD(3).NAME="AttributeNames"
     AttributeNames.TYPE="Collection"
FIELD(4).NAME="temp"
     temp.TYPE="Number"
FIELD(5).NAME="pressure"
     pressure.TYPE="Number"
FIELD(6).NAME="humidity"
     humidity.TYPE="Number"
FIELD(7).NAME="windspeed"
     windspeed.TYPE="Number"
FIELD(8).NAME="sunrise"
     sunrise.TYPE="DateTime"
     sunrise.FORMAT="SECONDS_LOCAL"
FIELD(9).NAME="sunset"
     sunset.TYPE="DateTime"
     sunset.FORMAT="SECONDS_LOCAL"
FIELD(10).NAME="main"
FIELD(11).NAME="City"
FIELD(12).NAME="wind"
FIELD(13).NAME="sys"
FIELD(14).NAME="ElementName"


[MSG]
MSG(1).NAME="Data"

[Data]
Data.FILTER=C1=="*"
'Going through each object in the array

FOREACH (JsonGetItem(__MESSAGE, "list[]")) DO 
'Initialize the Collection variables
TagNames = Clear()
Values = Clear()
AttributeNames = Clear()
City = JsonGetValue(__ITEM, "name")

'Getting the variables of interest under main{}
main = JsonGetValue(__ITEM, "main")
temp = JsonGetValue(main, "temp")
     Tagnames = Add(CONCAT(city,"_temp"))
     Values = Add(temp)
     AttributeNames=Add("Temperature")
pressure = JsonGetValue(main, "pressure")
     Tagnames = Add(CONCAT(city,"_pressure"))
     Values = Add(pressure)
     AttributeNames=Add("Pressure")
humidity = JsonGetValue(main, "humidity")
     Tagnames = Add(CONCAT(city,"_humidity"))
     Values = Add(humidity)
     AttributeNames=Add("Humidity")
'Getting the variables of interest under wind{}
wind = JsonGetValue(__ITEM, "wind")
windspeed = JsonGetValue(wind, "speed")
     Tagnames = Add(CONCAT(city,"_windspeed"))
     Values = Add(windspeed)
     AttributeNames=Add("Wind Speed")
'Getting the variables of interest under sys{}
sys = JsonGetValue(__ITEM, "sys")
sunset = JsonGetValue(sys, "sunset")
     Tagnames = Add(CONCAT(city,"_sunset"))
     Values = Add(sunset)
     AttributeNames=Add("Sunset Time")
sunrise = JsonGetValue(sys, "sunrise")
     Tagnames = Add(CONCAT(city,"_sunrise"))
     Values = Add(sunrise)
     AttributeNames=Add("Sunrise Time")

'Store the values in PI
   StoreEvents(TagNames,AttributeNames,,Values)
        'Create the AF Element to store the attributes in
ElementName=CONCAT("Weather_Monitoring\", City)
StoreElement("Weather_Monitoring") 'Parent Element
            'We use tagnames collection as the Dynamic attributes collection
StoreElement(ElementName, "WeatherTemplate",tagnames) 'Child Element
ENDFOR

 

 

Configuration in the connector admin page:

  • Configuration File: The .ini file built previously
  • Data Source Type: REST Client
  • Address: <YourAPIcall>
  • Scan Time: 600 (I am getting a new value every 10 minutes)
  • Word Wrap: -1; This is very important. In order to read JSON format, we need to use "-1" so that it interprets the formatted file as 1 line

 

Results:

An element was created for each city, including our attributes of interest. Since they all share the same template, the UOMs can be defined and further calculations/displays can be built at the template level.

Since the API call defines the entire structure, we could add any other location and the structure would automatically update on the next scan.