Scalability Testing using JMeter

Sripathi Krishnan

08 Jun 2017

https://static.hashedin.com/wp-content/uploads/2017/05/Sripathi-150x150.png

In this blog post, we will discuss how we did scalability testing on Charcha, an internal discussion forum. We use JMeter – A very powerful tool capable of complex load tests, but can be very unfriendly to new users. We will start with the installation process followed by a quick introduction to the JMeter GUI and finally, we will get our hands dirty with JMeter load testing.

Configuring JMeter JUI

JMeter has a lot of things to offer but for this blog, we are going to focus on Thread groups and samplers.

Thread Groups

Thread Group is an initial stage of the test plan. The name, Thread Groups represent a group of Threads. Under this group, each thread simulates one real user requests to the server.

adding a thread group
adding a thread group

According to the figure above:

Samplers

Samplers facilitate JMeter to deliver explicit types of requests to the server. It simulates a user’s request for a page from the target server. So that, to avail POST, GET, DELETE functions on an HTTP service, the user can add HTTP Request sampler.

HTTP Request Defaults

Here we specify the configuration which we want to be applied to all the HTTP requests JMeter makes.

http request default
http request default

Listeners

Finally, we will add listeners to view the results of our test cases. There are a lot of listeners the JMeter provides for performance testing but we are going to add only two listeners – View Results Tree and Summary Report.

View result tree
Summary report

An important thing to note here is that we are going to use View Results Tree only for debugging purposes because it holds every request and response in memory. We will use the Summary Report listener for drawing load testing conclusions.

Load Testing Setup

Now that we have our GUI configured, let’s start with JMeter load testing. We will start with the scalability testing of Charcha’s home page. To do that go ahead and create some HTTP request sampler as shown below. We are going to add three HTTP requests for now – The home page, discussion page and the profile page

Profile page

Note that I have added HTTP requests for login and starting a discussion but it is disabled for now because these are POST request and we need to do some configurations for POST requests when load testing a Django application. Lets see how to do that.

CSRF Configuration for POST Requests

Authentication in JMeter is a bit tricky due to Django’s CSRF protection. To address that, JMeter provides a way to access cookies using variables.
There are two ways to enable cookies in JMeter:

  1. In the file JMeter.properties add the following line CookieManager.save.cookies=true
  2. Launch JMeter from terminal ./jmeter -JCookieManager.save.cookies=true

I would prefer the former
Now to access the cookies, JMeter provides a config element called HTTP Cookie Manager. Right click on the thread group and add the config element – HTTP Cookie Manager

Cookie Manager

Now that we have our cookie manager setup, we can save cookie variables which can be referenced like this – ${COOKIE_cookiename}
Now let’s enable our login and start discussion post request and add the csrf token so that it can be sent in the post requests.

Login

Now that we have the setup done, click on the run button and view the results in the Summary Report listener.

Things to Remember

  1. Only enable “Summary Results”. Use the treeview only for debugging purposes, because it holds every request and response in memory
  2. Always run for a fixed number of threads. Do not provide any delay / ramp-up time. Other tools also call this as “think time”
  3. You must run the tests for at least 30 minutes
  4. You only need to capture 4 things from every test run. It’s best to capture these results at a steady state – that is when you do not see the numbers changing much, usually a couple of minutes before the test ends.
    • Number of threads
    • Average response time and standard deviation
    • Throughput
    • Total duration
  5. ImportantAverage Response Time x Throughput ~= Number of Threads 10-15% deviation is okay. This is called Little’s law, and if it doesn’t hold true – something is definitely wrong in your test cases

Scalability Testing Methodology

  1. Run tests for 10 threads, 20 threads, …. 100 threads. Each run must be at least 10 minutes, and you must pass little’s law described above
  2. Throughput must increase in proportion to the number of threads. Response time must not increase as the number of threads increases
  3. If the throughput does not increase – STOP and Investigate
  4. Look at graphs for client network bandwidth, server network, I/O, CPU, Memory, database graphs, graphs for Redis/cache, number of workers configured on web servers, etc.
  5. You will find that one of these components is slowing down the entire system. Find the culprit
  6. Once you know the culprit, fix the problem
  7. Now run the tests again, and increase load till throughput becomes constant.
  8. Once you get your target throughput and an acceptable 95 percentile response time, you are done

Summary

You should now be able to create a detailed scalability test for your Django application using JMeter. Use the JMeter config elements (for example – HTTP Request Defaults) to avoid configuring repetitive elements. Remember to run your tests with a fixed number of threads for at least 30 minutes and observe the throughput and response time to find the culprit.

 


Have a question?

Need Technology advice?

Connect

+1 669 253 9011

contact@hashedin.com

facebook twitter linkedIn youtube