Monitoring Apache Server Status with JMeter

Posted by on September 21, 2010
Filed Under Performance Engineering | 8 Comments

We’ve been using JMeter to run some tests of a web application and wanted to monitor the Apache web server’s performance.  JMeter has a performance monitor for Tomcat, but not for the Apache web server.  We didn’t need JMeter to graph the data in the UI.  We just wanted a .CSV file that we could chart in Excel.  I couldn’t find any plug-ins on the Internet that do what we wanted (I find it hard to believe, but maybe I didn’t search well enough).  It was pretty easy to create a monitor using the built in functionality of JMeter.  Here’s what I did.

 The basic steps to create the monitor were

  1. Configure Apache to show the server status page.
  2. Use a JMeter HTTP sampler to request the status page periodically.
  3. Use a BeanShell script to parse the response from Apache and it to a .CSV file.

Configure the Apache Status Page

The Apache web server status page provides information about the number of accesses, bytes sent, workers, etc.  I find the number of workers value particularly useful when tuning the ServerLimit, MaxClients, and KeepAliveTimeout parameters in Apache.  The mod_status section of the Apache web server documentation explains how to turn on the status page http://httpd.apache.org/docs/current/mod/mod_status.html

The BeanShell script I attached to this post assumes that ExtendedStatus is ON.  You would have to modify the script if you don’t want to use ExtendedStatus.  This script also requires you to pass the “auto” parameter to the status page so the result that comes back is more easily parsed.  For example, the URL for your http request sampler in JMeter would look like http://myserver/server-status?auto.

Configure JMeter to Monitor Apache Status

Create a Thread Group with a Single User

Add a thread group to your test plan that has a single user (Number of Threads = 1).  Set the ramp up time to 1.  You can set the loop count to a number you calculate based on how frequently you want the request to occur and the length of your test (60 minute test * 4 req. per minute = 240 loop count), or just set it loop forever.  If you loop forever you will have to stop the group by manually stopping the test. 

Add a Constant Timer

Add a constant timer to the thread group to delay the frequency of requests for the status page.  I find that a delay of 15 seconds is a good balance of getting enough data to make accurate assessments vs. overloading the server with status requests.

Create an HTTP Request Sampler

 Create an HTTP Request Sampler in the thread group that does a GET request for the Apache status page.  The “auto” parameter needs to be passed in the request.  It would probably work putting it in the “Send Parameters” section of the request configuration, but I just put in in the URL path, i.e. /server-status?auto. 

Add a BeanShell Post Processor

Create a BeanShell Post Processor for the HTTP Request that will parse the result and save it to a file.  The BeanShell code I used is attached to this post.  Download the script and put the file in the ../bin directory of your JMeter install.  In the “Script File” field of the Post Processor config, put the file name of the script.  The script file  location is relative to the JMeter home directory.  I didn’t fully investigate the directory configurations for JMeter, but if you start JMeter by running the scripts in the ../bin directory, then the script location is relative to that directory.  There may be a config parameter to change the script directory, but I haven’t found it.  Entering the absolute file path will not work.

In the “Parameters” field of the Post Processor configuration you must put the absolute path to the file name that you want for the output, e.g. C:\jakarta-jmeter-2.4\projects\www\apache_stats.csv.  The BeanShell script will write the output to that file in .CSV format.  It will append the data to the file if it already exists.

Create The Rest of Your Test Plan as Normal

The Apache monitor thread group will run along with the other thread groups for your tests.  If you set the monitor thread group to loop forever, you will need to stop the test manually to stop the monitor thread.  Below is an example of a test plan.  It shows the monitor user and the thread group with the load test users.

 Apache Monitor Test Plan

Monitoring Multiple Servers

There are at least a coule of approaches to monitoring multiple servers.  I just set up a different thread group for each server I wanted to monitor with a different HTTP request and different filename for the output script.  Each thread group fires the request for 1 server and outputs the data to different files.

You could also modify the BeanShell script to get the server name out of the SampleRequest object and write that out in another column in the .CSV file.  That should allow you to have a single thread group for all of the Apache monitors you need.  You would set up each HTTP request in that one thread group, and the PostProcessor would write out the server name in the .CSV data.  One drawback to this approach is that the constant timer will cause a pause between each HTTP request, so you need to consider that in setting the delay.  The output data will not all correspond to the same time with this approach.

Considerations

Of course you could write a  plug-in to do this instead.  Since most of the functionality was already in JMeter so I decided on the BeanShell approach for simplicity.   In addition to the benefits of being compiled instead of interpreted at runtime, the code could be made more efficient as a plug in.  For example, the BeanShell script checks for the file existence each time it is executed.  It would be much more efficient to do it only when the test starts.  Another possibility for improving this BeanShell could would be to put some of the code in methods and put them in an initialization script, e.g. the pre-processor initilization script.  That script is only parsed once at startup.  You could probably use a BeanShell Listener instead of the PostProcessor and implement the testStarted method.

Other Possibilities

You should be able to use the same approach to monitor Linux system stats as well.  You could write a shell script to run a command like vmstat on your web server.  You could then enable CGI on your Apache instance and have it run the shell script and return the results in HTML.  You could parse the results in a similar manner that this BeanShell script does, and put the results in a .CSV file.  Of course running the vmstat, sar, iostat, etc. on the box itself will provide you more information, but this could be a good way to pull some key data from several servers and collect it in one place.  Using CGI to do it would would prevent you from having to write a plug-in to do a Telnet or SSH to the server to get the data.

BeanShell Code – click the link below to download the PostProcessor BeanShell Code discussed above

Apache Status Monitor Beanshell Code 



Comments

8 Responses to “Monitoring Apache Server Status with JMeter”

  1. Faraz on January 7th, 2011 1:32 pm

    Dear,
    That’s a nice blog that enabled me to go ahead with Jmeter and Apache web server. But I cannot find the performance graph in JMeter for my Apache server. I’ve enabled all requisite settings for server status page but jmeter does not show up anything. Kindly help me in this regard to make Jmeter display the apache web server performance. I’ll be grateful.

    Regards,
    Faraz

  2. Doug Baber on January 7th, 2011 5:24 pm

    Hi Faraz,
    Unfortunately, this approach for monitoring Apache does not provide a real-time graph. The data is output to the .CSV file that you specify in the “Parameters” field of the PostProcessor configuration. You need to open the .CSV file with Excel or some other spreadsheet tool and graph the data manually.

  3. faraz ahmad on January 13th, 2011 2:54 pm

    Hi Babaer,

    I’ll make you bother a bit more.. if you don’t mind. Its about the your beenshell script. How do you find the order of information received from apache? and secondly, CPU usage is not retrieved through this script or perhaps Jmeter does not get it. Apache server status page if viewed through browser displays CPU usage info. what is its reason? and thirdly, memory is the most important performance element for a web server. But there’s nothing about it either on server status page or through beanshell. Why?

    Regards,
    Faraz

  4. Doug Baber on January 13th, 2011 6:09 pm

    Hi Faraz,
    The column names are output in the .csv file as the first line in the file. They are in the same order as they are displayed on the Apache status page with the addition of the date and time of the sample as the first column of the output. The URL for the status page MUST include the ?auto parameter. That parameter displays the status output in an easier to parse format. If you are not seeing the correct data in the output file, it is probably due to not having the ?auto parameter in the URL. There could be some other difference with your install as well that is causing the beanshell not to work.

    The CPU load data that is provided by the Apache status page is output to the .CSV file, but that probably isn’t the percent utilization data that you are used to seeing from other tools. You’d have to check the Apache documentation to see exactly what the data means.

    This beanshell script can only report the data that is provided by the Apache status page. Since memory usage isn’t displayed, the script can’t output it. I would think that the Apache folks didn’t include it because calculating that data would be too much of a performance impact. You would have to monitor memory usage, as well as CPU and disk usage on the web servers themselves. Unfortunately, this tool isn’t a comprehensive approach for monitoring, just a way to gather Apache status page data over time, and compiled into one location.

    I hope this blog entry and the beanshell script provide some value for you.

  5. Faraz on January 17th, 2011 11:04 am

    Can we parse and convert the information displayed in the server status page without auto? parameter using and modifying the beanshell script that you have provided with this blog??

  6. Faraz on January 18th, 2011 9:08 am

    Dear Baber,

    Can you please tell me what “CPU Usage: u55.15 s7.5 cu0 cs0 – .0138%” means on the server status page of Apache. I mean what is “u” in u55.15, “s” in s7.5 and what is cu0 and cs0 and what is this ratio .0138%??

  7. Doug Baber on January 18th, 2011 11:23 am

    Yes you can modify the beanshell to parse the human readable formatted status page. The beanshell script is provided as freeware. Do whatever you like with it. However, the information that is on the ?auto page is pretty much the same as the human readable page, and should be sufficient for performance testing analysis.

  8. Doug Baber on January 18th, 2011 11:30 am

    I’m not completely sure what those numbers mean. They look like they are probably a breakdown of the CPU time used by Apache into user and system time, or something like that. The percentage number may be of use, and is provided by the beanshell script as the CPULoad. The Apache documentation says the CPULoad number is “The current percentage CPU used by each worker and in total by Apache”. I have never found much use for this number in performance testing. Tools such as vmstat, top, mpstat, or Windows Perfmon are far better at monitoring CPU utilization. If your web server is used for lots of other processes, this number could help you figure out what portion of the CPU load is attributed to Apache, but other than that, the OS provided tools should be used to monitor CPU utilization.