EDIT: Part 2 is now published(distributed tests and
monitoring)
Funkload is a functional and stress test tool that can be used on
your web applications.
It's my favorite stress tool for these reasons:
- stress tests are implemented as PyUnit tests, so they can also be
used as functional tests
- the test runner is very light, it's dead easy to run a stress test
on your box against a local app
- it's simple to do a distributed test with a few extra options --
Funkload will drive the other boxes using SSH
- Funkload provides nice reporting features out of the box: you can do
trends and diff reports
- You can monitor the server being tested
This blog post demonstrates how to use Funkload on a dummy app and
generate report. A second blog post will explain how to run the test
using several test servers, and how to monitor the server that runs the
application.
The tested application
The dummy application we'll stress test for this demo is a very simple
WSGI application that can be run in plain Python, or using Gunicorn:
from wsgiref.simple_server import make_server
Every time the application is called, no matter what's the request path
or method, it will sleep for 100 ms then return a Hello World.
Let's run our application:
$ python wsgiapp.py
The application is ready to get some hits !
Installing Funkload
Let's open a new shell and create a local environment in a directory,
using Virtualenv, and install Funkload in it:
$ virtualenv --no-site-package --distribute .
Once everything's installed, you will get in the local bin
directory a few Funkload scripts:
$ ls bin/fl*
The two scripts that interest us right now are fl-run-test and
fl-run-bench.
A first Funkload test
Let's create a test_simple.py module in our directory:
import unittest
This test simply checks that the server returns Hello World and
that the response status code is 200.
To run this test, Funkload needs a few options to run. These options
can be placed in a configuration file. Let's create a Simple.conf
file with this content:
[main]
Those are defining options Funkload will use when it's running.
Now we can try out our Funkload script with the fl-run-test
script, which will run the tests just once:
$ bin/fl-run-test test_simple.py
Victory ! The test is working nicely.
Let's now try a full bench, using fl-run-bench. The bench script
takes an extra option which is the test method to use to run the bench:
$ bin/fl-run-bench test_simple.py Simple.test_simple
The script runs three cycles of respectively 5, 10 and 20 virtual
users, for 10 seconds each. This is configured with the cycles and
duration options in the [bench] section of the configuration
file (cycles=5:10:20 and duration=10), but you can also provide
these options through the command line.
Let's say you want to run 2 then 5 users for 5 seconds each:
$ bin/fl-run-bench --cycles=2:5 --duration=5 test_simple.py Simple.test_simple
Now let's check the reporting features..
Reporting
Everytime you are running a bench, an XML file is produced. In our case
it's simple-test.xml
The XML file contains raw results and can be used to produce reports.
fl-build-report takes these XML file and produce reports out of them.
For example, you can create an HTML report with the html option.
Make sure you have gnuplot installed, then run:
$ bin/fl-build-report --html --output-directory=html simple-bench.xml
The result is a nice HTML page containing various diagrams, like the
number of requests per seconds depending on the number of virtual
concurrent users.
[
][]
Another nice report is the diff report, which takes two already
generated reports, and build a diff one -- if you get some errors, make
sure you have the latest gnuplot installed.
$ bin/fl-build-report -o html --diff html/test_simple-20110727T123642 html/test_simple-20110727T123718
The diagram you get will provide a clear overview of the differences
between the two runs. This is useful if you want to check for speed
regression when you've changed some code.
[
][]
The trending report has the same goal, but can be built using more that
two runs:
$ bin/fl-build-report -o html --trend html/*
That's useful to see how your application is doing over time.
[
][]
In the next post we see two extra features Funkload provides:
- run distributed tests
- monitor the benched server
Go to part 2
[]: http://tarekziade.files.wordpress.com/2011/07/requests_rps.png
"rps_diff"
[]: http://tarekziade.files.wordpress.com/2011/07/rps_diff.png
"trend_avg"
[]: http://tarekziade.files.wordpress.com/2011/07/trend_avg.png