Hello All,
While monitoring system via performance counters, we saw that some of the counters were showing more than 100%.
For example, PI Web API counter: \\<machine_name>\Process(OSIsoft.REST.Host)\% Processor Time.
Trend of this is as shown below:
Is it possible or do we need to miss something in the configuration part or what?
It will be great if somebody can explain this.
Regards,
Paurav Joshi
You likely have more than 1 core on the machine.
If you had 2 cores on the machine and you see 116% then you can think of it as 1 core is 100% utilized and the other is 16% utilized.
Edit: Something else to think about. %Processor Time is the sum of processor time on each processor. If you have 2 cores then the max would be 200%. At 116% your system is only 58% utilized. If you have 4 cores then 29% utilized, ...
Thanks Rhys .
-Paurav
Edit:
you have 2 cores then the max would be 200%. At 116% your system is only 58% utilized. If you have 4 cores then 29% utilized, ...
In this context, Is it correct to assume that if I have 4 cores than CPU utilization of this PI process is 29%?
Yep that's right, it is using 29% of the total available processing power, and that's what you'd see if you were to look at Task Manager > Processes
Edit: Something else to think about. %Processor Time is the sum of processor time on each processor. If you have 2 cores then the max would be 200%.
Some surprising values we got as output of perfmon tags, so that I changed question status from answered .
Below is machine specification:
We can find out that it has 4 cores, so maximum value of perfmon tags _%Processor Time can be 400%.
Below are the surprising values:
| <machine_name>_Process(java#10)_% Processor Time | |
| 08-Jun-16 11:14:21 | 606.6098633 |
| <machine_name>_Process(java#3)_% Processor Time | |
| 08-Jun-16 11:39:21 | 672.156311 |
| <machine_name>_Process(java#4)_% Processor Time | |
| 08-Jun-16 11:29:51 | 558.30896 |
| 08-Jun-16 11:39:21 | 409.9706421 |
| <machine_name>_Process(java#9)_% Processor Time | |
| 08-Jun-16 11:15:51 | 606.6099243 |
| <machine_name>_Process(java#8)_% Processor Time | |
| 08-Jun-16 11:25:51 | 957.4956665 |
| <machine_name>_Process(java#5)_% Processor Time | |
| 08-Jun-16 11:25:51 | 547.6170654 |
| 08-Jun-16 11:29:51 | 533.6616211 |
Is there something more we should consider or what? Please guide us on how should we interpret these values.
Regards,
Paurav Joshi
Your machine specification states it has 4 processors. This does not necessarily equate to cores. How many cores does each processor have? As an example, a server I am working on at the moment looks like the following:
Here I'm looking at 1 x processor with 4 cores. In this case my maximum % Processor Time would be 400%.
If you open Task Manager and look at the Performance tab, making sure that the graph is showing logical processors, how many graphs can you see? Alternatively, look at the CPU tab in the Resource Monitor application. My guess is that there are more than 4 logical processors on your system.
Sorry for the delay Paurav.
The clue is in your processor, they are Xeon E5-2650 v2 which have 8 cores. With Hyper Threading that equates to 16 threads per CPU, but we'll ignore HT for now. So 4 sockets with 8 cores each = theoretically 3200% max.
If you are on a VM then the VM host may be stealing away CPU time from your server and because the Windows OS doesn't know it is virtualized then it may report higher % processor time than it has actually used.
Understood at some length after John and your explanation.
So coming to basic question where all this thing started. For PI process related counters like pisqldas64, rest.exe (for PI WebAPI) how should we consider the actual % consumed?
For example, pisqldas64%Procesor_time is 100%, then should we divide it by 8 or 4 to calculate actual CPU utilization by this process?
Should we consider other process like java threads to monitor how much actual CPU utilization done by these processes?
Thanks,
Paurav
Hi Paurav,
Not sure that dividing these values by either 4 or 8 is the right approach. Doing a bit more reading, came across a couple of things that might be helpful here. First is a VMWare KB article on Using Virtual CPU Performance Monitoring Counters that may have some relevance to your situation. The other one was an article that discusses in a bit more detail a concept that Rhys mentioned, regarding the fact that the guest Windows OS doesn't know it's virtualised and thus may report different CPU usage than what is actually uses. The article I came across is here if you want to have a read (and I'm not specifically promoting the company or technology at this linked site). At the end of the article is mention of monitoring CPU usage under a "VM Processor" counter. I don't have handy access to a VMWare hosted VM at present, so can't have a look at this myself, but it also may bear some investigation as potentially a more accurate measure of processor usage for you.
Hey John,
Really appreciate your time and efforts to explain this issue (thumbs up emoticon has been removed from editor ) .
Bare with me for some time, so that I will analyse them and get back on thread for query/concern if any.
Hello Paurav,
For PI process related counters like pisqldas64, rest.exe (for PI WebAPI) how should we consider the actual % consumed?
I really like how your are looking into this to get the most accurate reading for these values
My opinion on this is that when troubleshooting an issue, the percentage / value of a unique counter does not really matter. As these values will depend on hardware/VM and other different machine's characteristics.
What does matter though is how those values compare to each others on the specific machine at a specific time when an issue occurs.
A last comment: virtualization introduced another layer of technology that adds up complexity, I'd try to see if this could be fixed on the virtualization engine with a patch. Otherwise I would leave it as is, and make performance analysis relatively to each others values, because the next update may break your conversion factors.
This is my personal opinion when investigating for performance issues.
Hope this helps,
Today I stumble upon this same query and because of great in-depth knowledge shared here by John Messinger and Rhys Kirk , was able to navigate through it easily.
Thanks for sharing your time and knowledge.
Cheers,
Paurav Joshi
on Jun 6, 2016
Your machine specification states it has 4 processors. This does not necessarily equate to cores. How many cores does each processor have? As an example, a server I am working on at the moment looks like the following:
Here I'm looking at 1 x processor with 4 cores. In this case my maximum % Processor Time would be 400%.
If you open Task Manager and look at the Performance tab, making sure that the graph is showing logical processors, how many graphs can you see? Alternatively, look at the CPU tab in the Resource Monitor application. My guess is that there are more than 4 logical processors on your system.