This STAMP public presentation by Benoit Baudry (INRIA) is part of the Open Cloud Forum by OW2 in parallel with Cloud computing world expo, IoT world and Solutions Datacenter Management 2017.

1. Software Testing Amplification
Benoit Baudry, INRIA
22 March 2017

2. DevOps
● Software development practices
– To reduce time between releases
– To favor feedback from production into development
● Emphasizes automation
– Testing, build, deployment, configuration, monitoring

3. Test automation
● Test execution
– Unit, UI, performance
● Quality metrics
– Static metrics, test coverage
● Improve quality
– Reduce regressions, detect bugs early

4. Test automation
● Test development
– Hard, time consuming
– Unit tests have limited coverage and some are flaky
● Configuration testing
– Little support for automation
– Very limited sampling among all configurations

5. Test amplification
Automatically transform existing
test assets in order to detect
regression bugs before production
and drive down the cost of software
testing.

6. Example
6
public void testSearch() {
final ArrayStack<E> stack = makeObject();
stack.push(((E)("First Item")));
stack.push(((E)("Second Item")));
assertEquals("Top item is 'Second Item'", 1, stack.search("Second Item"));
assertEquals("Next Item is 'First Item'", 2, stack.search("First Item"));
assertEquals("Cannot find 'Missing Item'", -1, stack.search("Missing Item"));}

7. Example
7
public void testSearch() {
final ArrayStack<E> stack = makeObject();
stack.push(((E)("First Item")));
stack.push(((E)("Second Item")));
assertEquals("Top item is 'Second Item'", 1, stack.search("Second Item"));
assertEquals("Next Item is 'First Item'", 2, stack.search("First Item"));
assertEquals("Cannot find 'Missing Item'", -1, stack.search("Missing Item"));}
public void testSearch_cf5() {
final ArrayStack<E> stack = makeObject();
stack.push(((E)("First Item")));
stack.push(((E)("Second Item")));
E o_5 = stack.peek();
assertFalse(((java.lang.String)o_5).isEmpty());
E o_6 = stack.pop();
assertFalse(((java.lang.String)o_6).isEmpty());
Object o_7_0 = stack.search("Second Item");
Object o_9_0 = stack.search("First Item");
assertEquals("Cannot find 'Missing Item'", -1, stack.search("Missing Item"));
assertEquals(((java.lang.String)o_5).length(), 11);
assertEquals(((java.lang.String)o_6).length(), 11);}
new calls and asserts
extracted from asserts
new asserts

8. Focus on regressions
● Amplified tests
– Embedd regression oriented
oracles
– Detect behavioral differences
between two versions of a program

9. 9
STAMP in DevOps
● Focus on regression bugs in the continuous integration phase
● Reduce configuration bugs before application deployment
● Identify more operation bugs in edge cases with semantic
logging

10. STAMP objectives
● Reduce time for manual writing of test
cases, while increasing coverage
● Test the application in multiple
deployment configurations
● Analyze program behavior to re-inject
production test cases in continuous
testing

11. 11
DSpot: JUnit Test Suites
Amplification
Amplified Test
Suite
Original Test Suite
for Program P
Update in the Program
Under Test

12. 12
P
TS
P'
TS+
se arch -b ase d
input sp ace
amp lifica tion
obse rva tion
sp ace
amp lifica tion
ru n and look
for differe nce s
TS++
DSpot: JUnit Test Suites
Amplification
github.com/STAMP-project/dspot

13. 13

14. 14
Descartes Engine for PIT
● Mutation Testing
– Inject faults into a program to determine if the test cases can
detect them.
● PIT: mutation testing system for Java
– Many fine-grained fault models
– But does not scale to large code bases
● Descartes
– STAMP’s adaption of PIT with extreme mutation operators
– Empty the body of methods github.com/STAMP-project/pitest-descartes

15. Example
15
public void setValue(int x) {
if(x<10){
a:=x;
}
}
public void setValue(int x) {
}
●
Original method
●
Mutant

16. 16
STAMP Use Cases
ProActive Workflows and
Scheduling
ATOS FIWARE Smart City
Ecosystem
TelluCloud e-health
XWiki hybrid Open
Source business/project
OW2 Software
Quality Platform

17. 17
● Project Long Name: Software Testing AMPlification
– A consortium of nine European members
● The STAMP project has received funding from the
EU H2020 research and innovation programme
– Start: December 1, 2016
– End: November 31, 2019
– EU program: Horizon 2020 ICT-10-2016-RIA
STAMP at a Glance

18. 18
STAMP Useful Links
Web Site
www.stamp-project.eu
Twitter
@stamp_project
SlideShare
www.slideshare.net/
stamp-project
Web Site
www.stamp-project.eu
Forge
gitlab.ow2.org/stamp
github.com/STAMP-project