Skip navigation
All Places > PI Developers Club > Blog > Author: Eugene Lee
1 2 Previous Next

PI Developers Club

23 Posts authored by: Eugene Lee

Note: Development and Testing purposes only. Not supported in production environments.

 

Link to other containerization articles

Containerization Hub

 

Introduction

During PI World 2018, there was a request for a PI Analysis Service container. The user wanted to be able to spin up multiple PI Analysis Service container to balance the load during periods where there was a lot of back filling to do. Unfortunately, this is limited by the fact that each AF server can only have exactly one instance of PI Analysis Service that runs the analytics for the server. But this has not discouraged me from making a PI Analysis Service container to add to our PI System compose architecture!

 

Features of this container include:

1. Ability to test the presence of AF Server so that set up won't fail

2. Simple configuration. The only thing you need to change is the host name of the AF Server container that you will be using.

3. Speed. Build and set up takes less than 4 minutes in total.

4. Buffering ability. Data will be held in the buffer when connection to target PI Data Archive goes down. (Added 13 Jun 2018)

 

Prerequisite

You will need to be running the AF Server container since PI Analysis Service stores its run-time settings in the AF Server. You can get one from Spin up AF Server container (SQL Server included).

 

Procedure

1. Gather the install kits from the Techsupport website. AF Services

2. Gather the scripts and files from GitHub - elee3/PI-Analysis-Service-container-build.

3. Your folder should now look like this.

4. Run build.bat with the hostname of your AF Server container.

build.bat <AF Server container hostname>

5. Now you can execute the following to create the container.

docker run -it -h <DNS hostname> --name <container name> pias

 

That's all you need to do! Now when you connect to the AF Server container with PI System Explorer, you will notice that the AF Server is now enabled for asset analysis. (originally, it wasn't enabled)

 

Conclusion

By running this PI Analysis Service container, you can now configure asset analytics for your AF Server container to produce value added calculated streams from your raw data streams. I will be including this service in the Docker Compose PI System architecture so that you can run everything with just one command.

 

Update 2 Jul 2018

Removed telemetry and added 17R2 tag.

Note: Development and Testing purposes only. Not supported in production environments.

 

Link to other containerization articles

Containerization Hub

 

Introduction

In one of my previous blog posts, I was spinning up an AF Server container using local accounts for authentication. For non-production purposes, this is fine. But since Kerberos is the authentication method that we recommend, I would like to show you that it is also possible to use Kerberos authentication for the AF Server container. To do this, you will have to involve a domain administrator since a Group Managed Service Account (GMSA) will need to be created. Think of GMSA as a usable version of the Managed Service Account. A single gMSA can be used for multiple hosts. For more details about GMSA, you can refer to this article: Group Managed Service Accounts Overview

 

Prerequisite

You will need the AF Server image from this blog post.

Spin up AF Server container (SQL Server included)

 

Procedure

1. Request GMSA from your domain administrator. The steps are listed here.

Add-KDSRootKey -EffectiveTime (Get-Date).AddHours(-10) #Best is to wait 10 hours after running this command to make sure that all domain controllers have replicated before proceeding
Add-WindowsFeature RSAT-AD-PowerShell
New-ADServiceAccount -name <name> -DNSHostName <dnshostname> -PrincipalsAllowedToRetrieveManagedPassword <containerhostname> -ServicePrincipalNames "AFServer/<name>"

2. Once you have the GMSA, you can proceed to install it on your container host.

Install-ADServiceAccount <name>

3. Test that the GMSA is working. You should get a return value of True

Test-ADServiceAccount <name>

4. Get script to create AF Server container with Kerberos.

Invoke-WebRequest "https://raw.githubusercontent.com/elee3/AF-Server-container-build/master/New-KerberosAFServer.ps1" -UseBasicParsing -OutFile New-KerberosAFServer.ps1

5. Create a new AF Server container

.\New-KerberosAFServer.ps1 -ContainerName <containername> -AccountName <name>

 

Usage

Now you can open up PI System Explorer on your container host to connect to your containerized AF Server with the <name> parameter that you have been using in the procedure section. On the very first connect, you should connect with the afadmin user (password:qwert123!) so that you can set up mappings for your domain accounts. Otherwise, your domain accounts will only have 'World' permissions. After you set up your mappings, you can choose to delete that afadmin user or just keep it. With the mappings for your domain account created, you can now disconnect from your AF Server and reconnect to it with Kerberos authentication. From now on, you do not need explicit logins for your AF Server anymore!

 

Conclusion

We can see that security is not a limitation when it comes to using an AF Server container. It is just more troublesome to get it going and requires the intervention of a domain administrator. However, this will remove the need of using local accounts for authentication which is definitely a step towards using the AF Server container for production. I will be showing how to overcome some limitations of containers in future posts such as letting containers have static IP and the ability to communicate outside of the host.

 

New updates (3 Aug 2018)

Script updated to allow GMSA to work in both child and parent domains. For example, mycompany.com and test.mycompany.com.

Script now uses the new image with 18x tag based on a newer version of Windows Server Core.

Note: Development and Testing purposes only. Not supported in production environments.

 

Link to other containerization articles

Containerization Hub

 

Introduction

In this blog post, I will be giving an overview of how to use Docker Compose to create a PI System compose architecture that you can use for

 

1. Learning PI System development

2. Running your unit tests with a clean PI System

3. Compiling your AF Client code

4. Exploring PI Web API structure

5. Testing out Asset Analytics syntax

5. Other use cases that I haven't thought of (Post in the comments!)

 

What is Compose?

It is a tool for defining and running multi-container Docker applications. With Compose, you use a single file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. It is both easy and convenient.

 

Setup images

The Setup involved is simple. You can refer to my previous blog posts set up these images. Docker setup instructions can be found in the Containerization Hub link above.

Spin up PI Web API container (AF Server included)

Spin up PI Data Archive container

Spin up AF Client container

Spin up PI Analysis Service container

 

Compose setup

In Powershell, run as administrator these commands:

 

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest "https://github.com/docker/compose/releases/download/1.21.2/docker-compose-Windows-x86_64.exe" -UseBasicParsing -OutFile $Env:ProgramFiles\docker\docker-compose.exe

 

Obtain Compose file from docker-compose.yml. Place it on your desktop.

 

Deployment

Open a command prompt and navigate to your desktop. Enter

docker-compose up

 

Wait until the screen shows

Once you see that. You can close the window. Your PI System architecture is now up and running!

 

Usage

There are various things you can try out. If you are experiencing networking issues between the containers, turn off the firewall for the Public Profile on your container host.

 

1. You can try browsing the PI Web API structure by using this URL (https://eleeaf/piwebapi) in your web browser. When prompted for credentials, you can use

username: afadmin

password: qwert123!

 

2. Test network connectivity from client container to the PI Data Archive and AF Server by running

docker exec -it desktop_client_1 afs

The hostname of the AF Server is eleeaf. When prompted to use NTLM, enter q. The hostname of the PI Data Archive is eleepi. You should see the following results.

 

3. You can install PI System Management Tools on your container host and connect to the PI Data Archive via IP address of the container. Somehow, PI SMT doesn't let you connect with hostname.

 

4. You can also install PI System Explorer and connect to the AF Server to create new databases.

 

5. You can try compiling some open source AF SDK code found in our Github repository using the AF Client container. (so that you do not have to install Visual Studio)

 

6. You can use PI System Explorer to experiment with some Asset Analytics equations that you have in mind to check if they are valid.

 

Destroy

Once you are done with the environment, you can destroy it with

docker-compose down

 

Limitations

This example does not persist data or configuration between runs of the container.

These applications do not yet support upgrade of container without re-initialization of the data.

This example relies on PI Data Archive trusts and local accounts for authentication.

AF Server, PI Web API, and SQL Express are all combined in a single container. 

 

Conclusion

Notice how easy it is to set up a PI System compose architecture. You can do this in less than 10 minutes. No more having to wait hours to install a PI System for testing and developing with.

The current environment contains PI Data Archive, AF Server, AF Client, PI Web API, a AF SDK sample application (called afs) and PI Analysis Service. More services will be added in the future!

Eugene Lee

Spin up AF Client container

Posted by Eugene Lee May 21, 2018

Note: Development and Testing purposes only. Not supported in production environments.

 

Link to other containerization articles

Containerization Hub

 

In this blog post, the instructions for building an AF Client image will be shown. For instructions on how to install Docker, please see the link above.

 

1. Please clone this git repository. GitHub - elee3/AF-Client-container-build

2. Download AF Client 2017R2 from the Techsupport website. AF Client 2017 R2

3. Extract AF Client into the cloned folder.

4. Run build.bat

 

If you prefer us to build the image for you so that you can docker pull it immediately (less hassle). Please post in the comments!

 

Usage

This container can be used to compile your AF SDK code (so that you do not have to install Visual Studio) and you can use the container to pack an AF SDK application with its AF Client dependency for easier distribution. An AF SDK sample application (called afs) has been included in the image for you to try compiling it.

 

Limitations

Containers cannot run applications with GUI such as WPF and Windows Forms applications.

 

Update 27 Jun 2018

Fixed an issue with the registry links breaking.

Eugene Lee

Containerization Hub

Posted by Eugene Lee May 21, 2018

Good day everyone, welcome to the Containerization Hub. Here you will find containerization articles that have already been published in PI Square and also the titles of those articles that are planned for the future (subject to changes). Users will just need to bookmark this page rather than bookmark all the individual articles.

 

Important: As of May 30, 2019. We are no longer distributing the source code for the Docker container images. Please refer to Important update to OSIsoft's GitHub Policy, November 2018

 

Standalone

Spin up AF Server container (SQL Server included)

Spin up PI Web API container (AF Server included)

Spin up PI Data Archive container

Spin up AF Client container

Spin up PI Analysis Service container

Overcome limitations of the PI Data Archive container

Spin up PI to PI Interface container

 

Upgrading

Upgrade to PI Data Archive 2018 container with Data Persistence

Upgrade to AF Server 2018 container with Data Persistence

 

Security

Spin up AF Server container (Kerberos enabled)

Container Kerberos Double Hop

 

Health

AF Server container health check

PI Data Archive container health check

 

Cloud

AF Server container in the cloud

 

Orchestration

Compose PI System container architecture

Containers and Swarm Part 1 (Setup, Service)

 

Install Docker

The steps to setup Docker are below.

 

For Windows 10,

You can install Docker for Windows. Please follow the instructions here.

 

For Windows Server 2016/2019,

You can use the OneGet provider PowerShell module. Open an elevated PowerShell session and run the below commands.

Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
Install-Package -Name docker -ProviderName DockerMsftProvider -Force
Restart-Computer -Force

Note: Development and Testing purposes only. Not supported in production environments.

 

Link to other containerization articles

Containerization Hub

 

Introduction

I now present to you another blog post in the containerization series on spinning up PI Web API in less than 3 minutes (My test came out to be 2 min 44 sec!).

 

I will repeat the steps here for setting up Docker for your convenience. If you have already done so while using the AF Server image, then you do not need to repeat it again. The PI Web API image offered here is fully self contained. In other words, you do not have to worry about any dependencies such as where to store your PI Web API configuration. In a later blog post, I will be posting on a PI Web API image that only contain the application service for those of you who want the application service to be separate from the database service. In that image, you will need to furnish your own AF Server then. For now, you do not have to care about that.

 

Set up

Install PI Web API image

Run the following command at a console. When prompted for the username and password during login, please contact me (elee@osisoft.com) for them. Currently, this image is only offered for users who already have a PI Server license or are PI Developers Club members (try it now for free!). You will have to login before doing the pull. Otherwise, the pull will be unsuccessful.

docker login  
docker pull elee3/afserver:piwebapi  
docker logout  

Remember to check digest of image to make sure it is not tampered with.

Update 2 Jul 2018: Please use the fast version with tag webapifast17R2 as that image is better in every possible way. Boot up time 15 sec compared to 3 minutes.

 

Deployment

Now that the setup is complete, you can proceed to running the container image. To do so, use the following command. Replace <DNS hostname> and <containername> with one of your own picking. Remember to pick a DNS hostname that is unique.

docker run -it --hostname <DNS hostname> --name <containername> elee3/afserver:piwebapi  

 

After about 3 minutes, you will see that the command prompt indicates that both the PI Web API and AF Server are Ready.

This indicates that your PI Web API is ready for usage. At this point, you can just close the window.

Update 2 Jul 2018: Please use the fast version with tag webapifast17R2 as that image is better in every possible way. Boot up time 15 sec compared to 3 minutes.

 

Usage

Now you can open a browser on your container host and connect to it with the DNS hostname that you chose earlier.

https://<DNS hostname>/piwebapi

 

When prompted for credentials, you can use

User name: afadmin

Password: qwert123!

 

Browsing your PI Data Archive

You can use a URL of the form

https://<DNS hostname>/piwebapi/dataservers?path=\\<PI Data Archive hostname>

to access your PI Data Archive. Of course, you need to give access permissions by creating a local user on the PI Data Archive machine with the same username and password above and give a PI mapping to that user.

 

Browsing your AF Server

You can use a URL of the form

https://<DNS hostname>/piwebapi/assetservers?path=\\<AF Server hostname>

to access your AF Server. Again, you need to give access permissions by creating a local user on the AF Server machine with the same username and password above. By default, everyone has World identity in AF Server so you do not need to give any special AF mapping.

 

Multiple PI Web API instances

You can spin up several PI Web API instances by using the docker run command multiple times with a difference hostname and containername.

You can see above that I have spin up several instances on my container host.

 

Destroy PI Web API instance

If you no longer need the PI Web API instance, you can destroy it using

docker stop <containername>  
docker rm <containername>  

 

Limitations

AF Server, PI Web API, and SQL Express are all combined in a single container. There will be an upcoming blog post for a container with just PI Web API in it.

This example relies on local accounts for authentication.

 

Conclusion

Observe that the steps to deploy both the AF Server and PI Web API containers are quite similar and can be easily scripted. This helps to provision testing environments quickly and efficiently which helps in DevOps.

 

New updates (12 Jun 2018)

In the never ending quest for speed and productivity, every minute and second that we save waiting for applications to boot up can be better utilized elsewhere such as taking a nap or watching that cat video clip that your friend sent you. Therefore, I present to you a faster PI Web API container image that is more than 60% faster than the original one.

 

docker pull elee3/afserver:webapifast17R2

Remember to check digest of image to make sure it is not tampered with.

 

3 test runs were performed to compare the boot up time.

 

Run 1

Start time was 13:48:00 for both. The original image finished in 2 min 36 sec while the new one finished in 55 sec.

 

Run 2

Start time was 13:58:00 for both. The original image finished in 2 min 27 sec while the new one finished in 55 sec.

 

Run 3

Start time was 14:29:00 for both. The original image finished in 2 min 28 sec while the new one finished in 57 sec.

 

Summary of results

Run
Original (s)
New (s)
115655
214755
314857
Average15055

 

The results show that the new image is about 63% faster than the original one.

 

New updates (18 Jun 2018)

1. Added reminder to check digest of the image to make sure image has not been tampered with.

 

New updates (2 Jul 2018)

1. Removed telemetry and changed tag from webapifast to webapifast17R2. Took down image with tag piwebapi from repository. Boot up time for webapifast17R2 has been further reduced to 15 sec!!

 

New updates (26 Oct 2018)

1. New tag webapi18s for version 2018.

Note: Development and Testing purposes only. Not supported in production environments.

 

Link to other containerization articles

Containerization Hub

 

Introduction

Currently, in order to set up an AF Server for testing/development purposes, you have two choices.

 

1. Install SQL Server and AF Server on your local machine

The problem with this method is that there is no isolation from the host operating system. Therefore, you risk the stability of the host computer if something goes wrong. You also can't spin up multiple AF Servers this way.

 

2. Provision a VM and then install SQL Server and AF Server on it

While this method provides isolation, the problem lies in the time it takes to get it set up and also the size of the VM which includes many unnecessary components.

 

There is a better way!

Today, I will be teaching you how spin up AF Server instances in less than 1 minutes (after performing the initial setup which might take a bit longer). This is made possible by the usage of containerization technology.

 

Requirements

Windows Server build 1709, Windows Server 2016 (Core and with Desktop Experience) or Windows 10 Professional and Enterprise (Anniversary Edition). Ensure that your system is current with the Windows Update.

 

Benefits

1. Portability. Easy to transfer containers to other container hosts that meet the prerequisites. No need to do tedious migrations.

2. Side by side versioning. Ability to run multiple versions of AF Server on the same container host for compatibility testing and debugging purposes.

3. Speed. Very fast to deploy.

4. Resource efficiency and density. More AF Servers can run on the same bare metal machine compared to virtualization.

5. Isolation. If you no longer need the AF Server. You can remove it easily. It won’t leave any temporary or configuration files on your container host.

6. Able to use with container orchestration systems such as Swarm or Service Fabric.

 

Set up

Install Docker

For Windows 10,

You can install Docker for Windows. Please follow the instructions here

 

For Windows Server 2016,

You can use the OneGet provider PowerShell module. Open an elevated PowerShell session and run the below commands.

Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
Install-Package -Name docker -ProviderName DockerMsftProvider -Force
Restart-Computer -Force

 

Install AF Server image

Run the following command at a console. When prompted for the username and password during login, please contact me (elee@osisoft.com) for them. Currently, this image is only offered for users who already have a PI Server license or are PI Developers Club members (try it now for free!). You will have to login before doing the pull. Otherwise, the pull will be unsuccessful.

docker login
docker pull elee3/afserver:18x
docker logout

Remember to check digest of image to make sure it is not tampered with.

18x: digest: sha256:4ce64e6dafa4fe93d3cf2ef4fb3a38ddadd4493e10e7a68efd809b92d946a77f

 

Deployment

Now that the setup is complete, you can proceed to running the container image. To do so, use the following command.

Replace <DNS hostname> and <containername> with one of your own picking. Remember to pick a DNS hostname that is unique in your domain.

docker run -d --hostname <DNS hostname> --name <containername> elee3/afserver:18x

 

You can now open up PI System Explorer on your local machine and connect to the AF Server by specifying the DNS Hostname that you chose earlier. When prompted for credentials, use

User name: afadmin

Password: qwert123!

Check the box to remember the credentials so that you won't have to enter it every time.

 

You can choose to rename the AF Server if you wish.

 

And you are done! Enjoy the new AF Server instance that you have created!

 

Using with AF SDK

To connect to the AF Server from code using AF SDK, the following Connect overload can be utilized with the same credentials as above.

PISystem.Connect Method (NetworkCredential)

 

Multiple AF Servers

In order to spin up another AF Server instance, follow the steps above. When you get the new container running. You have to change the ServerID. You can do this via

docker exec -i <containername> cmd /c "cd %pihome64%\af&afdiag.exe /cid:<guid>"

You can generate a new guid using this.

 

You do not need to manually generate a new Server ID anymore. The image does it automatically for you.

 

Destroy AF Server

If you no longer need the AF Server, you can destroy it using

docker stop <containername>
docker rm <containername>

 

Limitations

This example uses a public SQL Express container image which is currently not available for use in a production environment. (Changed to a SQL Express image that I built myself)

This example relies on local accounts for authentication. Refer to the following article if you want to use Kerberos. Spin up AF Server container (Kerberos enabled)

 

New updates (14 Feb 2018)

1. 2017R2 tag is now available

2. Image has been updated with ability to import in an existing AF Server backup in the form of PIFD.bak file. To do this, run

docker run -di --hostname <DNS hostname> --name <containername> -v <path to folder containing PIFD.bak>:c:\db elee3/afserver:2017R2 migrate.bat

 

New updates (30 May 2018)

1. Local account is no longer in the administrators group. Only a mapping to an AF Identity is done (better security)

 

New updates (18 Jun 2018)

1. Added reminder to check digest of the image to make sure image has not been tampered with

 

New updates (2 Jul 2018)

1. Changed tag from 2017R2 to 17R2

2. Removed telemetry

 

New updates (13 Jul 2018)

1. Changes to facilitate upgrading to 2018 container

 

New updates (4 Sep 2018)

1. Updated to use tag 18x which comes with health check and some performance improvements

 

New updates (26 Oct 2018)

1. New tag 18s comes with SQL data files persisted externally

Note: Development and Testing purposes only. Not supported in production environments.

 

Link to other containerization articles

Containerization Hub

 

Introduction

Today, I will be teaching you a recipe for cooking a PI Data Archive container. Please see my previous blog posts above on how to get Docker installed. We will be mixing the ingredients in a folder to create an image. After we have the image, we can bake the image to obtain a container.

 

Ingredients

For the source code to build the PI Data Archive container. Please send an email to technologyenablement@osisoft.com. This is a short term measure to obtain the source code while we are revising our public code sharing policies. Apologies for any inconvience caused.

1. PI Data Archive 2017 R2A Install Kit Download from techsupport website (contains the software)

2. Dockerfile (describes the mixing steps to form the image)

3. build.bat (script to start the mixing)

4. generateid.txt (reference commands for changing the Server ID)

5. pilicense.dat (many ways to obtain one such as through Account Manager/Partner Manager/Academic Team/PI DevClub membership, best to get a demo license that doesn't require a MSF)

6. temp.txt (adds host trust)

7. trust.bat (adds host trust)

 

Recipe

1. Gather all the required ingredients as listed above

2. Extract out the Enterprise_X64 folder from the PI Data Archive Install Kit.

3. Add pilicense.dat into the Enterprise_X64 folder. Override the existing files if needed.

4. Put the other ingredients into the parent folder of the Enterprise_X64 folder.

 

Your folder structure should now look like this.

5. Execute build.bat. The mixing will take less than 5 minutes.

 

6. Once the image is formed, you can now execute

 

docker run -it --hostname <DNS hostname> --name <containername> pida

 

at the command line to bake the image. This will take about 15 seconds.

You will see the IP address of the PI Data Archive listed in the IPv4 Address field. Use this IP address with PI SMT on your container host to connect. Your PI Data Archive container is now ready to be consumed!

 

Hint: Multiple PI Data Archive instances

If you want to bake another instance of the PI Data Archive container (just repeat step 6 with a different hostname and containername), you will need to change the Server ID too. The following procedure can be done in piconfig to accomplish this.

 

@tabl pisys,piserver
@mode ed
@istr name,serverid
hostname,
@quit

 

Replace hostname with the real hostname of your container. For more information about this, please refer to this KB article. Duplicate PI Server IDs cause PI SDK applications to fail

 

Limitations

1. This example does not persist data or configuration between runs of the container image.

2. This example relies on PI Data Archive trusts and local accounts for authentication.

3. This example doesn't support VSS backups.

4. This example doesn't support upgrading without re-initialization of data.

 

For Developers

Here is an example to connect with AF SDK and read the current value of a PI Point.

 

Conclusion

Notice how quick and easy it is to cook a PI Data Archive container. I hope you find it delicious. I like it because I can easily cook up instances for testing and remove them when I do not need them. (Please don't waste food)

 

Update 31 May 2018

Local account is no longer in the administrators group. Only a mapping to a PI Identity.

 

Update 2 Jul 2018

Added 17R2 tag.

 

Update 21 Nov 2018

For the source code to build the PI Data Archive container. Please send an email to technologyenablement@osisoft.com. This is a short term measure to obtain the source code while we are revising our public code sharing policies. Apologies for any inconvience caused.

Important update to OSIsoft's GitHub Policy, November 2018

Filter Blog

By date: By tag: