understanding total CPU used on a RedHat VM running in Hyper-V 2012 host

[SysEng]

I have a Red Hat Enterprise Linux Server release 6.4 (Santiago) [Linux 2.6.32-358.11.1.el6.x86_64 x86_64 x86_64 x86_64 GNU/Linux] virtual machine running on Hyper-V 2012 hypervisor. We have a SCOM monitor that monitors CPU usage on that VM and it continuously reporting that the CPU being used is at around 98%. When I log onto the VM and run "top" command, it shows 74.6%id (idle?) meaning that only 25.4% of total CPU on the VM is being used, right? In Windows it is very easy to see the total amount of CPU being consumed by the system, but in Linux it is a bit confusing.

What's even more confusing is that a different command, like "mpstat -P ALL" shows a different amount of CPU usage.

So my question is, how do I see the total amount of CPU being used at the moment on a Linux system?


top - 11:18:00 up 212 days, 1:47, 54 users, load average: 5.16, 5.21, 5.21
Tasks: 416 total, 6 running, 410 sleeping, 0 stopped, 0 zombie
Cpu(s): 12.0%us, 13.2%sy, 0.0%ni, 74.6%id, 0.0%wa, 0.2%hi, 0.0%si, 0.0%st
Mem: 3916152k total, 3810532k used, 105620k free, 112396k buffers
Swap: 4063224k total, 28208k used, 4035016k free, 3375732k cached



Linux 2.6.32-358.11.1.el6.x86_64 (machine) 05/12/2016 _x86_64_ (4 CPU)

11:21:19 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle
11:21:19 AM all 4.91 0.00 5.68 1.63 0.07 0.06 0.00 0.00 87.65
11:21:19 AM 0 1.97 0.00 1.88 2.19 0.27 0.16 0.00 0.00 93.53
11:21:19 AM 1 15.92 0.00 19.98 1.10 0.00 0.03 0.00 0.00 62.98
11:21:19 AM 2 0.88 0.00 0.47 1.67 0.00 0.03 0.00 0.00 96.95
11:21:19 AM 3 0.90 0.00 0.43 1.58 0.00 0.03 0.00 0.00 97.06

 

[Captain_Chicken]

Your SCOM monitor is showing cpu usage across the cores assigned to the VM, or it may be incorrect, while the VM sees all four cores and sees that one is at 100% usage, and see four physical cores, so that 25% cpu usage is total.

 

[SysEng]

Wait, looking at the output from "mpstat -P ALL", I don't see a single core running anywhere close to 90%. Or are you saying "mpstat -P ALL" does not display correct CPU usage? The core #1 is at 37.02% which is the highest. I have no idea how SCOM monitor collects its data, but it looks way off to me.

P.S. Why the result from "top" and "mpstat -P ALL" for total CPU used are different? "top showed 74.6% idle and "mpstat -P ALL" showed 87.65% idle for all CPUs.