Monday 7 June 2010

C#: Per-process CPU Usage and Network IO Tracking using PerformanceCounter

I have been tacking per-process CPU percentage and Network IO data bytes in my recent project using System.Diagnostics.PerformanceCounter class. I felt the online materials I found does not explain the usage of the PerformanceCounter very well.

Some basics about PerformanceCounter.
1. A performance category has many performance counters underneath. A performance counter may be single-instanced or contain multiple running instances. The structure is like this:
PerformanceCategory
    -- PerformanceCounter
        -- Instances
To get all the built-in categories:
PerformanceCategory.GetCategories()

To get all the counters under the category that has exactly one running instance:
PerformanceCategory.GetCounters()

2. The constructor to create a performance counter is:
PerformanceCounter(string categoryName, string counterName, string instanceName.

To monitor the IO (file or network) data bytes sent/received by FireFox:
PerformanceCounter counter = new PerformanceCounter("Process", "IO Data Bytes/sec", "firefox");

To monitor the CPU percentage used by FireFox:
PerformanceCounter counter = new PerformanceCounter("Process", "% Processor Time", "firefox");

3. To take a sample and get the calculated value:
counter.NextValue();
Before calling this method, make sure you wait for at least one second.
counters.Add(new PerformanceCounter("Process", "IO Data Bytes/sec", "firefox"));
counters.Add(new PerformanceCounter("Process", "% Processor Time", "firefox"));

System.Threading.Thread.Sleep(1000);

foreach(PerformanceCounter counter in counters)
        Console.WriteLine(string.Format("{0}:{1}}, counter.CounterName, counter.NextValue());

Some lessons I have learned:
1. The calculated value of the first sample is always 0. E.g. the very first time you call NextValue(), the returned value is always 0.

2. If an application has more than one running instances, even though from the Task Manager, these running instances all have the same process name, E.g. "TestProcess", their instance names are not the same. The instance started first has its name as "TestProcess", the second "TestProcess#1", the third "TestProcess#2"...the nth, "TestProcess#n". So make sure the right instance name gets passed into the performance counter constructor.

3. If any of the running instances of the same application exits, the instance names will be changed. If your code doesn't update the instance name accordingly, an "Instance not found" error will occur.

E.g. If "TestProcess" of the 2 running instances, "TestProcess" and "TestProcess#1", stopps running, "TestProcess#1" will be renamed to "TestProcess". If the counter still has the name "TestProcess#1", it will throw an exception when NextValue() is called. The instance name can be updated by simply calling counter.InstanceName = "new instance name".

4. In my case, I am only interested in getting per-Process statistics. Unfortunately, the PerformanceCounter does not operate against the process id. To get around this limitation, I have created my own wrapper class called ProcessPerformanceCounter.
public class ProcessPerformanceCounter
{
    ...
    public ProcessPerformanceCounter(string counterName, int processId) {...}
    public string CounterName {get{...}}
    public string InstanceName {get{...} set{...}}
    public NextValue(){...}
}

Based on the process id, I can get the process name. I can then get all running processes under the same name. From there, I can get the correct instance name for that particular process:
private string GetInstanceName(int processId)
{
        string instanceName = GetProcessNameById(processId);
        bool found = false;
        if (!string.IsNullOrEmpty(instanceName))
        {
                Process[] processes = Process.GetProcessesByName(instanceName);
                if (processes.Length > 0)
                {
                        int i = 0;
                        foreach (Process p in processes)
                        {
                                instanceName = FormatInstanceName(p.ProcessName, i);
                                if (PerformanceCounterCategory.CounterExists("ID Process","Process"))
                                {
                                        PerformanceCounter counter = new PerformanceCounter("Process", "ID Process", instanceName);

                                        if (processId == counter.RawValue)
                                        {
                                                found = true;
                                                break;
                                        }
                                }
                                i++;
                        }
                }
        }

        if (!found)
                instanceName = string.Empty;

        return instanceName;
} 
 
private string FormatInstanceName(string processName, int count)
{
        string instanceName = string.Empty;
        if (count == 0)
                instanceName = processName;
        else
                instanceName = string.Format("{0}#{1}", processName, count);
        return instanceName;
} 
 
5. When calling counter.NextValue(), do not forget to handle the instance name changed scenario:
public float NextValue()
{
        float nextValue = 0;
        try
        {
                nextValue = counter.NextValue();
        }
        catch (Exception ex)
        {
                if (InstanceNotExistError(ex))
                {
                        // adjust instance name based on process id
                        instanceName = GetInstanceName(processId);
                        counter.InstanceName = instanceName;
                        nextValue = counter.NextValue();
                }
                else
                        throw;
        }

        return nextValue;
} 

7 comments:

Neil Franken said...

Hi Yan

I am a Java/C++ programmer most of the time and I have a open source project that I want to code in C#. Basically it is a Windows client which reports performance data to a server(for more information see www.xymon.com). AS this client needs to report accurate data and not be a resource drain I have to say I am a little dissapointed in the .Net performance counters classes.

After searching on the web I have found much frustration around performance counters and C#. Anyway I have some questions for you.

1) Do you always need to make the thread sleep between measurements?
2) Have you tied working with the raw values? If so what are your experiences with it?
3) The performance counter objects seems to scew results in some cases as the resource usage jumps when using these classes.
4) Have you looked at the API as a alternative?

From what I can gather the PerformanceCounters object is a easy way to work with the performance data however I feel the implementation done by MS is a little lacking. The current client used by the project is written in C++ and uses the performance counters via the API and uses less memory and less cpu during the test. However the current client has been abandoned and before I fork the code I wanted to try it in C#.

Maybe I am just doing something wrong but when I go the harder API route I seem to get more reliable performance data.

I would love to hear your thoughs and experiences on this subject.


Regards
neil

Neil Franken said...

Hi Yan

Its Neil again. I found something interesting in the MSDN checkout http://msdn.microsoft.com/en-us/library/system.diagnostics.performancecounter.aspx seems you dont have to make the thread sleep for a second. Just call the nextvalue() twice. It needs to values to do the calculations on.

Thought it might help
Cheers
N

Yan (Pamela) Yang said...

Hi Neil, regarding to your questions, please see my comments;

1) Do you always need to make the thread sleep between measurements?
No, you don't. Only the first sample.
2) Have you tied working with the raw values? If so what are your experiences with it?
In my project, I used the calculated values to get what I want - I want the CPU usage to be the calculated CPU percentage between the previous sample and the current sample. Raw value of this counter can't give me the ratio. I used raw value for the counter "ID Process", which has a static value, to get the process ID. It depends on what data you want to get.
3) The performance counter objects seems to scew results in some cases as the resource usage jumps when using these classes.
I don't know what performance data you are gathering. You may want to make sure the right counters are used, and these counters you are interested in have been installed.

What I have encountered recently was while the app was running and collecting performance data, the event log of the server was filled with Perflib errors and warnings.

MS documentation doesn't say very clearly what damage these errors/warnings will do. But vaguely, it states that they may cause some unexpected errors if your application expects some performance data that are not present. It also states that for a warning message (event ID 2003), it can be ignored safely but it will slow down the perf data collecting process. You may want to make sure your application log is not filled with Perflib related errors/warning.

The cause of these errors mostly due to the performance counters info stored in registry are out of sync with what actually installed on the server (maybe due to some system update). MSDN have a very detailed article about how to re-build the performance counters which you can easily google out.

I didn't rebuild all performance counters. I only unload and reload those reported errors to have their registry values updated.

4) Have you looked at the API as a alternative?
We don't have the luxury to go this route as we are not building a standalone tool, but incorporated in an existing C# app.

Hope all these help...

Yan (Pamela) Yang said...

I found something interesting in the MSDN checkout http://msdn.microsoft.com/en-us/library/system.diagnostics.performancecounter.aspx seems you dont have to make the thread sleep for a second. Just call the nextvalue() twice. It needs to values to do the calculations on.

I think you are right. Actually I called NextValue() twice the first time to get a non-zero first sample value ;)

Neil Franken said...

Hi Pamela.

Thanks for the replies. I did some more research and I am pretty sure on how to deal with the Performance Counter objects.

However I think I would need to switch to C++ as the .net application I am writting needs to take snapshots of the entire system(CPU,Network,Memory,Threads,Ports,Processes,event log and so on) every 5 minutes. Currently the application is just monitoring CPU and I am allready consuming a massive 20MB in memory.

While a hell of a lot easier to use than C++ the C# performance counters are a bit pricey to use in terms of what I need to do.

Anyway if I do find some technique to use less memory I will sure as hell let you know. Hopefully I will be able to create a nice library in C++ that I can share.

Regards
Neil

Yan (Pamela) Yang said...

Great, looking farword to your library!

Anonymous said...

Every weekend i used to pay a quick visit this website, as i want enjoyment,
since this this web page conations actually nice funny data too.
My web page :: Casino