Part 1 covered creating a data logger by connecting an Arduino to a raspberry PI and using a text file to log data from a temperature probe streaming data through the USB port. In this blog we'll walk through how to turn a data logging device into an Internet of Things device using the PI Web API.

 

The architecture used for this project consists of a Windows 2008 server with PI AF, PI archive and the PI Web API installed. The Web API is dependent on PI AF. On the same intranet network I have the Raspberry PI connected via Ethernet with an Arduino Uno connected to a USB port.

 

To begin with, make sure you have opened ports 443 (Web API), 5450 (PI Archive) and 5457 (AF) in the firewall of your server.

 

firewall.png


To test the Web API you can use any web browser or download Postman, a free HTTP development tool.

Type the following URL into your browser or Postman to verify the installation of the Web API.

 

https://<server-name>/piwebapi/

 

You should get a response like this for a successful install:

{

    "Links": {

        "Self": "https://<server-name>/piwebapi/",

        "AssetServers": "https://<server-name>/piwebapi/assetservers",

        "DataServers": "https://<server-name>/piwebapi/dataservers",

        "Search": "https://<server-name>/piwebapi/search",

        "System": "https://<server-name>/piwebapi/system"

    }

}

 

Next, we need to configure the Web API. You'll need admin access to your AF server to complete the configuration. An AF database is created during the Web API install that holds the PI Web API configuration. The following configuration settings discussed in this blog are not intended for production environments and should only be used for development and testing.

Open AF Explorer and select the Configuration database. Go to the System Configuration element.

 

AF_Piwebconfig.png

 

 

The first setting that most likely will need to be revised is the attribute AuthicationMethods. The default is Kerberos. This will probably need to be revised to "Basic". There is a third choice of using "Anonymous" but it is not recommended even for testing. In the string settings you can remove Kerberos or simply revise the order and make Basic=0.

 

AF_PiwebAuth.png

 

The next few settings refer to Cross Origin Resource Sharing (CORS). These settings are used for a client and server that reside on different domains. Since this is not the case for this project we can open things up.

CorsHeaders: Set to *
CorsMethods: add POST
CorsOrigins: set to *
CorsSupportsCred: True

DisableWrites: set to False to complete POST requests.
EnableCSRFDefence: set to False.

 

Now that our web server is configured to receive requests, let's build a simple AF model to hold the attribute that we want to write data to. In the image below we have a simple database with one element and one attribute.

Database: Arduino

Element: RaspPI

Attribute: TempProbe1

 

AF_arduino.png

 

Next, use Postman to make GET and POST requests to test the Web API. Initially we use GET request to dive into the hierarchy of our AF model. Using successive GET requests, we retrieve the URL and WebId needed to fabricate our POST request to add data to PI. Every object in AF is referenced through the Web API using a unique WebId. The WebId for all objects can be retrieved through GET request demonstrated below in the screenshot from Postman.

Initially we plug the URL shown below into Postman and click Send to initiate a response. In the window a number of links are returned. We can click on these links to drill further down into our model. Click on the database link to expose the AF databases. Select the database you have built for your project and continue clicking on links until you drill down to the attribute for the probe. Each click on a link opens a new tab in Postman.

 

psdemo.png

 

 

Assuming that you have clicked your way down to the Temperature Probe attribute we can see in the returned data the WebId. Copy and paste this ID into a text file for future use.

 

Lets's try a POST command in Postman to verity that our Web API is configured correctly and that we have adequate permissions to write data to PI. This will also help determine the structure of our POST URL with a data payload to send with it. A successful post will return a Status 202 message.

 

Enter the POST URL using this format:

https://<SERVER>/piwebapi/streams/<WEBID>/value

where the <WebId> is the WebId of your temperature probe.

Select the POST command from the dropdown.

On the Authorization tab, select Basic Auth and enter the Windows username and password.

 

ps.png

 

Under the Headers tab select "application/json". All payloads are sent in JSON format.

 

 

ps1.png

Under the Body tab select the radio button "raw" and JSON from the dropdown. Add the payload, in json format as shown below. Click on SEND. In this example we are not providing a timestamp. Pi will automatically timestamp the value when it arrives if no time is provided.  if the request is successful you receive a status 202. If there is an error it displays in POSTMAN which makes it much easier to test and resolve. Copy and paste the POST URL in the same text file as the WebID for the probe attribute.

 

ps2.png

 

 

Finally its time to start writing code and working on the Raspberry PI. We'll create a Python 2 script and use the same HTTP POST request as shown above to push streaming data to the PI system. Save the text file (containing the WebId and URL) we saved above to a USB drive. Insert the USB drive into the Raspberry PI and copy the text file to the file system.

In the script below revise the URLs using the WebId from the text file and enter the username and password for authentication. Run the following Python 2 code with the Arduino plugged in and collecting data. The data should display in the python terminal and update in PI AF. Use SMT to watch the values stream into PI.

 

 

import serial
import datetime
import time
import sys
import urllib, urllib2
from urllib2 import Request, urlopen, URLError, HTTPError

print "Starting rPI2PI"

baudrate = 9600
portaddr = "/dev/ttyACM0"

#Headers
req_headers = {'Content-Type': 'application/json'}


# test Web API connection using simple GET
url = 'https://<server_name>/piwebapi'
request = urllib2.Request(url, headers=req_headers)
password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_manager.add_password(None,url,'<username>','<password>')

auth = urllib2.HTTPBasicAuthHandler(password_manager)
opener = urllib2.build_opener(auth)
urllib2.install_opener(opener)

try:
        responce=urllib2.urlopen(request).read()
        print 'Responce: ', responce
except  HTTPError as e:
        print 'Error: ', e.code
except  URLError as e:
        print 'Reason: ', e.reason


#URL for POST command to write a value to an AF Attribute with PI point reference
url = 'https://<server_name>/piwebapi/streams/<WebId>/value'
request = urllib2.Request(url, headers=req_headers)

# define serial port and read temperature value.
ser = serial.Serial(portaddr,baudrate)
# read and skip first line of data which is usually not complete
tempValue=ser.readline() 


while 1:
        ts = time.time()
        st = datetime.datetime.fromtimestamp(ts).strftime('%Y-%m-%d %H:%M:%S')
        tempValue=ser.readline()
        print(st + "," + tempValue)
        #This data string works as well and PI will provide the timestamp
        #req_data = str({"Value": x})
        
        # use the generated timestamp from the raspPI
        # package the request
        req_data = str({"Timestamp": st, "Value":tempValue})
        request = urllib2.Request(url, headers=req_headers)
        request.add_data(req_data)

        # Use try/catch to get telemetry on errors
        try:
                responce=urllib2.urlopen(request).read()
        except  HTTPError as e:
                print 'Error: ', e.code
        except  URLError as e:
                print 'Reason: ', e.reason


        #Arduino sends data approx every 3 seconds.
        time.sleep(3)
        
        # flush serial cache or data will lag the timestamp.
        ser.flush()
        ser.flushInput()
        ser.flushOutput()

ser.close

 

 

Below is a sample of output over several hours.

 

 

trend.png

 

This is significant step in data collection because we go from data statically held in a text file to migrating data in real time to any server that we can reach through the internet. The data can be exposed to one or many users. Additionally, the same PI Web API requests that we make to send the data can be revised to also retrieve data in real time. Our Raspberry PI can be configured to send values to PI. PI AF can do analytics on that data and return a value which can be used to send an output signal to another device connected to the Raspberry PI. A user that is authorized to revise data in AF can control this device from anywhere in the world. This is the fundamentals of the Internet of Things. In the future most devices will have the ability to be managed remotely and devices will be controlled by other devices. We are at the beginning of a IOT revolution within our current technology revolution.

 

Below is a picture of the Arduino (packaged in its cardboard case) and Raspberry PI with heat sink temperature values streaming to the console.

 

ard_rpi.png

 

* A "Thanks" goes out to Barry Shang for his blog post on this topic. I formatted my requests calls using basic authentication based on his post.