In this blog post we will discuss how we did scalability testing on Charcha. We will use JMeter – A very powerful tool capable of complex load tests, but can be very unfriendly to new users. We will start with the installation process followed by a quick introduction to the Jmeter GUI and finally, we will get our hands dirty with jmeter load testing.
Configuring JMeter JUI
JMeter has a lot of things to offer but for this blog we are going to focus on Thread groups and samplers.
Thread Group is an initial stage of test plan. The name,Thread Groups represent a group of Threads. Under this group, each thread simulates one real user requests to the server.
According to the figure above:
- Number of Threads: 100 numbers of users are connected to target website
- Loop Count: execute testing 10 numbers of times
- Ramp-Up Period: 100
Samplers facilitate JMeter to deliver explicit types of requests to the server. It simulates a user’s request for a page from the target server. So that, to avail POST, GET, DELETE functions on a HTTP service, user can add HTTP Request sampler.
HTTP Request defaults
Here we specify the configuration which we want to be applied to all the HTTP requests JMeter makes.
Finally we will add listeners to view the results of our test cases. There are a lot of listeners the JMeter provides for performance testing but we are going to add only two listeners – View Results Tree and Summary Report.
An important thing to note here is that we are going to use View Results Tree only for debugging purposes because it holds every request and response in memory. We will use the Summary Report listener for drawing load testing conclusions.
Load Testing Setup
Now that we have our GUI configured, let’s start with JMeter load testing. We will start with scalability testing of Charcha’s home page. To do that go ahead and create some HTTP request sampler as shown below. We are going to add three HTTP requests for now – The home page, discussion page and the profile page
Note that I have added HTTP requests for login and starting a discussion but it is disabled for now because these are POST request and we need to do some configurations for POST requests when load testing a Django application. Lets see how to do that.
CSRF Configuration for POST Requests
Authentication in JMeter is a bit tricky due to Django’s CSRF protection. To address that, JMeter provides a way to access cookies using variables.
There are two ways to enable cookies in JMeter:
- In the file jmeter.properties add the following line
- Launch JMeter from terminal
I would prefer the former
Now to access the cookies, JMeter provides a config element called HTTP Cookie Manager. Right click on the thread group and add the config element – HTTP Cookie Manager
Now that we have our cookie manager setup, we can save cookie variables which can be referenced like this –
Now let’s enable our login and start discussion post request and add the csrf token so that it can be sent in the post requests.
Now that we have the setup done, click on the run button and view the results in the Summary Report listener.
Things to Remember
- Only enable “Summary Results”. Use the treeview only for debugging purposes, because it holds every request and response in memory
- Always run for a fixed number of threads. Do not provide any delay / ramp-up time. Other tools also call this as “think time”
- You must run the tests for at least 30 minutes
- You only need to capture 4 things from every test run. It’s best to capture these results at a steady state – that is when you do not see the numbers changing much, usually a couple of minutes before the test ends.
- Number of threads
- Average response time and standard deviation
- Total duration
Average Response Time x Throughput ~= Number of Threads10-15% deviation is okay. This is called Little’s law, and if it doesn’t hold true – something is definitely wrong in your test cases
Scalability Testing Methodology
- Run tests for 10 threads, 20 threads, …. 100 threads. Each run must be at least 10 minutes, and you must pass little’s law described above
- Throughput must increase in proportion to the number of threads. Response time must not increase as the number of threads increases
- If the throughput does not increase – STOP and Investigate
- Look at graphs for client network bandwidth, server network, I/O, CPU, Memory, database graphs, graphs for redis/cache, number of workers configured on web servers, etc.
- You will find that one of these components is slowing down the entire system. Find the culprit
- Once you know the culprit, fix the problem
- Now run the tests again, and increase load till throughput becomes constant.
- Once you get your target throughput and an acceptable 95 percentile response time, you are done
You should now be able to create a detailed scalability test for your Django application using JMeter. Use the JMeter config elements (for example – HTTP Request Defaults) to avoid configuring repetitive elements. Remember to run your tests with a fixed number of threads for atleast 30 minutes and observe the throughput and response time to find the culprit.