If you monitor your system performance with Apache jMeter test suite, you can easily make it a part of your Continuous Integration and monitor the performance on a regular basis. You need to do three things:
- Put your Apache jMeter binary files on Jenkins machine
- Run jMeter tests in headless mode from Jenkins job, generating the xml test report
- Analyze reports with Jenkins Performance Plugin
Let’s get into details!
1. Put your jMeter binaries on Jenkins
Apache jMeter does not require installation so you can just copy the files that you currently use to the Jenkins server. Thanks to it you will have all the plugins ready to use. Then make sure that user that runs jenkins has access rights to execute jMeter.
2. Run jMeter tests from Jenkins job
Create Jenkins job and, as a Build Step, Execute Shell commands:
#remove previous reports
rm jMeter/reports/*.jtl -f
#run tests
$JMETER_PATH/jmeter -n
-t jMeter/suites/sampleSuite.jmx
-l jMeter/reports/sampleSuiteReport.jtl
-p jMeter/properties/user.propertieswhere:
- $JMETER_PATH is a path where you put jMeter binaries (for example /opt/apache-jmeter-2.13/bin/)
- -n param means that jMeter will be run in headless mode
- – t jMeter/suites/sampleSuite.jmx is a test suite that you are going to run
- -l jMeter/reports/sampleSuiteReport.jtl is a result report file that will be generated. This file will be interpreted by Jenkins Performance Plugin
- -p jMeter/properties/user.properties is a properties file where you configure the report file format to XML (interpreted by default in Jenkins Performance Plugin)
By default the .jtl report file is in the csv format. Jenkins Performance Plugin can read the csv, however the default format for it is xml. By the same token I use user.properties file with one line indicating that the jtl output should be xml:
jmeter.save.saveservice.output_format=xml3. Analyze reports with Jenkins Performance Plugin
Install the Performance Plugin on your Jenkins instance. To use it, add the Post Build Action to your job, providing the path to .jtl report files:
Jtl files will be interpreted each time the job is executed. Moreover in the job page on Jenkins you have new option in menu: Performance Trend, where you can find (on graphs) how your performance was changing over time at each build:
Looks OK! 
Another approach that you can take is that in Spring Cloud Sleuth we’re running benchmark tests with JMeter from the command line as a part of a build. https://github.com/spring-cloud/spring-cloud-sleuth/blob/master/benchmarks/pom.xml#L270 That way you don’t need any plugins whatsover and you can run the performance tests locally.
cute :) and there is a gradle jmeter plugin as well.
Can you compare the performance of subsequent builds in that approach to monitor how the performance was changing over time?
Good question! I don’t know the answer :P I assume that you would have to store the results as an artifact and compare it.
So there is a place for yet-another-gradle-plugin! :P
All in all it depends what are your goals, what you want to automate and measure and how.
And that being said, I like your gradle / maven approach to fire jMeter tests! Since it is platform independent and integrated into the build process.
And it can also integrate with Performance Monitoring. That plugin in fact just reads jtl reports and compares the results :)
Thanks for feedback!
We’re checking out Jenkins 2.0 in Pivotal so I’m definitely going to check that combination. The information about the change of performance in time is really crucial to have!
BTW – I’ve just updated our Groovy DSL to support the Performance Plugin – https://github.com/spring-cloud/spring-cloud-jenkins-dsl/commit/fa5a6def9b942c5c3bfedecd408baebde6003373 . Thanks for the suggestion!
cool!