jonasengedal

PI Web API with multiple channels

Discussion created by jonasengedal on Oct 8, 2019
Latest reply on Oct 16, 2019 by vkaufmann

I have investigated to use PI Web API streamset channels for monitoring 1500 elements with 10 attributes each using a single service account. The elements are child elements of around 25 parent elements.

 

When opening 1500 streamset channel websockets and a poll interval of 10 seconds the CPU usage was 20-30% with an Intel® Xeon® Gold 6140 CPU @ 2.30GHz (4 processors) which is really great.

 

I have however done some findings I want to share that make me less excited.

 

  1. During the process of opening/closing websockets on streamsets channels the other PI Web API controllers and channels are blocked
    1. This might not be an issue when you open/close a few channels but when you open/close hundreds of channels it starts to take minutes where PI Web API is in-responsive
  2. When opening channels they are opened in chunks of varying size (10-300). Within each chunk the PI Web API is blocked.
  3. When closing channels, more or less all channels are closed in one big chunk where PI Web API is in-responsive for a long time.
    1. When closing all open channels, the OSIsoft.REST.Host CPU usage quickly drops to 0%. So what is the time spent at when CPU usage is 0%?

 

I also experienced that after 1500 channels were closed (which took 6 minutes), the PI Web API controllers were forever in-responsive and another channel I had opened for verification was also closed. I had to restart OSIsoft.REST.Host to solve these issues.

 

So my questions are:

  1. Why does open/close channels block the PI Web API controllers?
  2. Why does it take that long to open/close channels?
    1. If I do my own AFDataPipe implementation will I experience the same waiting times in AddSignups/RemoveSignups?
  3. Will OSIsoft at some point support a streamset value query on a parent element that also looks at child element attributes?
    1. Then the number of channels can be reduced drastically to 25.

 

Alternatives I consider:

  1. Change my AF model to have fewer elements.
    1. Not the right solution as I then need to map my elements into parent attributes which really does not feel right and will be a nightmare to maintain in the AF Library
  2. Horizontal scaling of PI Web API
    1. Will most likely lower the waiting times, but will not solve the root problems
  3. Use the PI Web API HTTP controllers to pull data on a schedule in multiple threads
    1. Then I might not be able to support 10 second update frequency
  4. Writing my own streaming service with WebSockets based on AFDataPipe
    1. Might be the right solution at the moment as I then will be able to decrease the amount of web sockets drastically to 1-25.

 

Below you see some measures I have done regarding open/close channels.

Channels

Open Time (sec)

Close Time (sec)

1502

210

375

111

 

13

139

 

10

171

 

12

249

 

~60

305

 

75

335

 

90

385

 

110

482

110

110

 

Best regards

 

Jonas

Outcomes