4. In short, load testing is a simulation of multiple users working with a web
application at the same time.
Load testing
Load testing can be performed for a number of purposes, but the main goal
is always to check the performance of the application being tested.
6. Types of load testing
Performance testing
Purpose : To determine or validate speed, scalability, and/or stability.
Goal :
A performance test is a technical investigation done to determine or validate
the responsiveness, speed, scalability, and/or stability characteristics of the
product under test.
7. Types of load testing
Capacity testing
Purpose : To determine how many users and/or transactions a given system
will support and still meet performance goals.
Goal
Capacity testing is conducted in conjunction with capacity planning, which you use
to plan for future growth, such as an increased user base or increased volume of
data. For example, to accommodate future loads, you need to know how many
additional resources (such as processor capacity, memory usage, disk capacity, or
network bandwidth) are necessary to support future usage levels.
Capacity testing helps you to identify a scaling strategy in order to determine
whether you should scale up or scale out.
8. Types of load testing
Stress testing
Purpose : To determine or validate an application’s behavior when it is pushed
beyond normal or peak load conditions.
The goals of stress testing are:
• Find the capacity limit;
• Check that when it is reached, the application handles the stress
correctly: produces graceful overload notifications and does not crash;
• When the load is reduced back to regular level, the application should
return to normal operation retaining all its performance characteristics.
9. Types of load testing
Volume testing
Volume tests are targeted at loading the application with significant amount of
data and maximizing the complexity of each transaction.
For example, if the application can upload files, try using very large ones. If it has a
search function, try complex keyword combinations and queries producing a very
long list of results.
10. Types of load testing
Endurance testing
This type of testing (also called “soak testing”) is used to check that the system
can stand the load for a long time or a large number of transactions.
It usually reveals various types of resource allocation problems. For example, a
small memory leak will not be evident from a quick test even with a high load.
For endurance testing it is recommended to use changing periodic load and long
duration.
11. Types of load testing
Regression testing
Make the load testing a part of your regular development process by creating
regression load tests and applying them to every new version of your
application.
12. What is Jmeter?
The Apache JMeterTM is pure Java open source software, which was first
developed by Stefano Mazzocchi of the Apache Software Foundation, designed
to load test functional behavior and measure performance. You can use JMeter
to analyze and measure the performance of web application or variety of
services. Performance testing means testing a web application against heavy
load, multiple and concurrent user traffic. JMeter originally is used for testing
Web Application or FTP application. Nowadays, it is used for functional test,
database server test etc.
14. X 1000 = $35000
X 1000 = $25000
Performance Testing without Tool
15. X 5 = $1500
X 5 = $250
Performance Testing with Tool
16. What Can You Do With It?
• JMeter lets you set up test plans that simulate
logging into a web site, filling out forms,
clicking buttons, links, etc.
• You can simulate the number of users doing
this, the rate that they do it.
17. JMeter
Tool for performance testing your web application
Java based open-source desktop application
Platform Independent – can run on any environment accepting JVM
E.g; Windows, Linux, Mac
Graphical interface using Swing graphical API
18. JMeter Features
Simple and Intuitive GUI
Java based open-source software
Full Swing and lightweight component support
Can load and performance test different server like –
HTTP
HTTPS
Database (JDBC)
SOAP etc.
Can perform automated & functional testing of your application
Creates its test plan in XML format, so you can generate a test plan in text
editor
20. JMeter Use
Performance Test : Determine speed, scalability, stability
Load Test : Verify application behaviour under normal & peak load condition
Stress Test : Validate application behaviour beyond normal & peak load condition
Volume Test : Determine how many users/transactions a given system will
support and still meet performance goal
Scalability Test : Measure System performance while steadily increasing user load
21. JMeter Installation
Download Jmeter http://jmeter.apache.org/download_jmeter.cgi
For using jmeter you must have Java Installed
Download Java
First Install Java
Set path for Java
Start Jmeter
26. Thread Group
Thread Groups is a collection of Threads. Each thread represents one user using the
application under test. Basically each Thread simulates one real user request to the
server.
27. Sampler
As we know already that JMeter supports testing HTTP, FTP , JDBC and many other
protocols.
We already know that Thread Groups simulate user request to the server
But how does a Thread Group know which type of requests (HTTP, FTP etc.) it needs to
make ?
The answer is Samplers
28. Listeners
Listeners: shows the results of the test execution. They can show results in different
format such as tree, table, graph or log file
29. Reports are called Listeners in JMeter vocabulary. Listeners are used to collect the
performance test results and display it to the performance tester.
number of requests sent
Average is the average response
time for that particular http request.
This response time is in millisecond.
Min denotes to the minimum
response time taken by the http request
Max denotes to the maximum
response time taken by the http request
This shows how many exceptional cases
were found which were deviating from
the average value of the receiving
time. The lesser this value more
consistent the time pattern is assumed.
This denotes the
error percentage in
samples during run
the amount of
transactions
produced over
time during a
test.
Throughput =
(number of
requests) /
(total time).
32. Latency: Time from invoking the request to the first
part (packet) of the response returning.
Time from invoking the request to the last byte of the
response coming back
JMeter measures the time it took to establish the
connection, including SSL handshake
View Result Table
34. Creating a Web Test Plan (Testing a Single Page)
Select Test Plan R.C Select Thread Thread Group
Select Thread R.C Add Sampler Http Request
Select Thread R.C Add Listener Summary Report
35. Creating a DataBase Test Plan (Testing a Single Page)
Select Test Plan R.C Select Thread Thread Group
Select Thread R.C Add Config Elements Jdbc Connection Configuration
36. Select Thread R.C Add JDBC Request Summary Report
Select Thread Add Listener Summary Report
This name is same as Pool Named Earlier
Created in JDBC Connection
Configuration
37. Creating FTP Test Plan
Select Test Plan R.C Select Thread Thread Group
R.C On Thread Add Config FTP Request Default
R.C On Thread Add Sampler FTP Request
43. Using RegEx
Regular expressions are used to search and manipulate text, based on
patterns
With the use of regular expressions, we can certainly save a lot of time
and achieve greater flexibility as we create or enhance a Test Plan.
Regular expressions provide a simple method to get information from
pages when it is impossible or very hard to predict an outcome.
contains means that the regular expression matched at least some part
of the target, so 'alphabet' "contains" 'ph.b.' because the regular
expression matches the substring 'phabe'.
matches means that the regular expression matched the whole target. So
'alphabet' is "matched" by 'al.*t'.
44. Using Timer in Jmeter
By default, JMeter sends the request without pausing between each request. In that
case, JMeter could overwhelm your test server by making too many requests in a short
amount of times.
Let imagine that you send thousands request to a web server under test in a few
seconds. This is what happens!
45. Timers allow JMeter to delay between each request which a thread makes. Timer can
solve the server overload problem.
Also, in real life visitors do not arrive at a website all at the same time, but at different
time intervals. So Timer will help mimic the real time behavior.
Following are some common types of timer in Jmeter.
Constant timer delays each user request for the same amount of time.
46. Gaussian Random timer delays each user request for a random amount of time.
Uniform random timer delays each user request for a random amount of time.
47. WebService Timer Based TestPlan
Select Test Plan R.C Select Thread Thread Group
R.C On Thread Add Sampler Http Request
R.C On Thread Add Timer Constant Timer
R.C On Thread - Add Listener Summary Report
49. This is for the response - which can be
displayed in a browser
If Pdf, Doc etc coming as a
response
JMeter out-of-the-box considers all 4xx and 5xx responses as failures. If your test
case is “negative” and, for example, a “404” error is expected, you’ll need to
check this box to suppress JMeter’s built-in status code check and substitute it
with your own status code assertion.
51. Duration Assertion
The Duration Assertion is a very simple one! When used alongside the Response
Assertion, it covers 90% of use cases where assertions are required. The usage is very
straightforward.
It provides the maximum duration in milliseconds
If any request lasts longer than the value specified, the sample is marked as ‘failed’
52. Size Assertion
Size Assertion checks the response length for being equal/not equal/greater/less
than the expected size in bytes
53. XML Assertion
The XML Assertion checks that the returned response body is XML-compliant. Only
the syntax is checked - any external resources are not downloaded and validated.
When there is an invalid XML, the reason for failure will be reported in an ‘Assertion
Failure’ message.
54. HTML Assertion
The HTML Assertion checks that the response HTML syntax is a well-formed
HTML/XHTML/XML document. So it’s handy if your web application or site requires
zero HTML validation errors. The majority of modern browsers are able to render even
invalid HTML pages - but search engine robots or 3rd-party integrations may not be so
tolerant.
56. Loop Controller makes the user request run specified number of times or
run forever .
57. Interleave Controller will make one of samplers contained in it run in each loop of
the thread and but the samplers in each loop will be in sequential order.
63. The goal of Module Controller is to add modularity to JMeter.
The general idea is that web applications consist of small units of functionality (i.e.
Logon, Create Account, Logoff...).
65. First, create a CSV file with the logins and passwords. You can
just do this in a text editor, and the format should be:
username,password
username2,password2
username3,password3
Save that file in the bin directory where your JMeter
installation lives. For this example, I saved the file as
“logins.csv”.
Now go into JMeter and find the HTTP request step that you
want to modify to use the values in this CSV file. Right click on
it and go to Add > Config Element > CSV Data Set Config. This
adds the CSV Data Set Config as a child of the HTTP request.
66. Because I had defined my variables as “login” and “password” in the
previous step, in this step I need to use those variables as the value of
the parameters that are being sent with the request. To do that I
replaced the email address with ${login} and the password with
${password}.
67. Preprocessor and Postprocessors in Jmeter
Pre-processor executes some action before making Sampler Request.
Consider a simple example: let's say you wanted JMeter to "spider" through website
under test, parse link(check all links on the page) and return the HTML. You would
add some action such as "HTML link parser" to your controller before creating an
HTTP request.
68. Consider a simple example: JMeter send HTTP request to the web server under
test (etc www.google.com) and get the response. You want JMeter to stop the
testif the server response is error. You can use the post-processor to do above
task as following:
69. Do WebTesting
Step - 1 Select Test Plan R.C Select Add Thread Thread Group
Step-2 R.C on Thread Group Add Config Element Http Request Default
Step -3 In Http Request Default Element Server Name Enter www.google.com
Step- 4 R.C on Thread Group Add a Logic Controller Recording Controller
Step -5 Next Select the WorkBench
Step- 6 R.C on WorkBench and add the recorder Add Non Test Elements Http(s)
Test Script Recorder
Step- 7 OnHttp(s) Test Script Recorder , Click the Add button in URL Pattern to include
Step- 8 Right Click on Http(s) Test Script Recorder and Add a Listener Add- Listener
View Result Tree
70. Configure the Browser
Tools Perferences Select the Advance Tab NetworkTab - Settings
Manual Proxy-
71. Using CSV DATA SET CONFIG
If you need to test web-site, the first thing you should do is login operation.
JMeter has an element that allows you to read different parameters from text
file. It is “CSV Data Set Config” which is used to read lines from a file, and split
them into variables
74. Jmeter with Continuous Integration
./jmeter.sh -n -t ${path to your .jmx script} -l ${path to your results file}
Place Jenkins War File in Tomcat
Download the Performance Plugin in Jenkins
Editor's Notes
Let’s call this gas station “Joe’s Gas,” and envision that it only has three gas pumps. Naturally, if we have three gas pumps and three cars, it follows that Joe’s attendants can only fill up three cars per minute. So, if we were to fill out a performance report for Joe’s gas station, it would show that Joe’s throughput is three cars per minute.
Note : Place MySql Jar in the Following Location
E:\Softwares\apache-jmeter-2.13\apache-jmeter-2.13\lib\ext
*********************************************************************************************
Sample - number of requests sent
Avg - an Arithmetic mean for all responses (sum of all times / count)
Minimal response time (ms)
Maximum response time (ms)
Deviation - see Standard Deviation article
Error rate - percentage of failed tests
Throughput - how many requests per second does your server handle. Larger is better.
KB/Sec - self expalanatory
Avg. Bytes - average response size
Label - The label of the sample. If "Include group name in label?" is selected, then the name of the thread group is added as a prefix. This allows identical labels from different thread groups to be collated separately if required.
# Samples - The number of samples with the same label
Average - The average elapsed time of a set of results
Min - The lowest elapsed time for the samples with the same label
Max - The longest elapsed time for the samples with the same label
Std. Dev. - the Standard Deviation of the sample elapsed time
Error % - Percent of requests with errors
Throughput - the Throughput is measured in requests per second/minute/hour. The time unit is chosen so that the displayed rate is at least 1.0. When the throughput is saved to a CSV file, it is expressed in requests/second, i.e. 30.0 requests/minute is saved as 0.5.
Kb/sec - The throughput measured in Kilobytes per second
Avg. Bytes - average size of the sample response in bytes. (in JMeter 2.2 it wrongly showed the value in kB)
ftp://ftp.domain.com/
Change the Name to "File2".
Change the Remote File field to "/directory/file2.txt".
Change the Username field to "anonymous".
Change the Password field to "anonymous@test.com".
First, create a CSV file with the logins and passwords. You can just do this in a text editor, and the format should be:
username,passwordusername2,password2username3,password3
Save that file in the bin directory where your JMeter installation lives. For this example, I saved the file as “logins.csv”.
Now go into JMeter and find the HTTP request step that you want to modify to use the values in this CSV file. Right click on it and go to Add > Config Element > CSV Data Set Config. This adds the CSV Data Set Config as a child of the HTTP request.