I have investigated to use PI Web API streamset channels for monitoring 1500 elements with 10 attributes each using a single service account. The elements are child elements of around 25 parent elements.
When opening 1500 streamset channel websockets and a poll interval of 10 seconds the CPU usage was 20-30% with an Intel® Xeon® Gold 6140 CPU @ 2.30GHz (4 processors) which is really great.
I have however done some findings I want to share that make me less excited.
- During the process of opening/closing websockets on streamsets channels the other PI Web API controllers and channels are blocked
- This might not be an issue when you open/close a few channels but when you open/close hundreds of channels it starts to take minutes where PI Web API is in-responsive
- When opening channels they are opened in chunks of varying size (10-300). Within each chunk the PI Web API is blocked.
- When closing channels, more or less all channels are closed in one big chunk where PI Web API is in-responsive for a long time.
- When closing all open channels, the OSIsoft.REST.Host CPU usage quickly drops to 0%. So what is the time spent at when CPU usage is 0%?
I also experienced that after 1500 channels were closed (which took 6 minutes), the PI Web API controllers were forever in-responsive and another channel I had opened for verification was also closed. I had to restart OSIsoft.REST.Host to solve these issues.
So my questions are:
- Why does open/close channels block the PI Web API controllers?
- Why does it take that long to open/close channels?
- If I do my own AFDataPipe implementation will I experience the same waiting times in AddSignups/RemoveSignups?
- Will OSIsoft at some point support a streamset value query on a parent element that also looks at child element attributes?
- Then the number of channels can be reduced drastically to 25.
Alternatives I consider:
- Change my AF model to have fewer elements.
- Not the right solution as I then need to map my elements into parent attributes which really does not feel right and will be a nightmare to maintain in the AF Library
- Horizontal scaling of PI Web API
- Will most likely lower the waiting times, but will not solve the root problems
- Use the PI Web API HTTP controllers to pull data on a schedule in multiple threads
- Then I might not be able to support 10 second update frequency
- Writing my own streaming service with WebSockets based on AFDataPipe
- Might be the right solution at the moment as I then will be able to decrease the amount of web sockets drastically to 1-25.
Below you see some measures I have done regarding open/close channels.
Open Time (sec)
Close Time (sec)