Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to To Mock or Not To Mock
Similar to To Mock or Not To Mock (20)
More from Delft University of Technology
More from Delft University of Technology (6)
Recently uploaded
Recently uploaded (20)
To Mock or Not To Mock
- 1. To Mock Or Not To Mock? An Empirical Study on Mocking Practices Davide Spadini, Mauricio Aniche, Magiel Bruntink, Alberto Bacchelli @DavideSpadini
- 2. How do I test this?
- 3. With dependencies Without dependencies
- 4. Mock Unit under testUnit under test
- 9. • Faster • Easier Without dependencies
- 10. Without dependencies You lose in reality, after all, it is just a simulation!
- 11. To Mock Not to Mock So, what should we do?
- 12. RQ1: What test dependencies do developers mock? RQ2: Why do developers decide to (not) mock specific dependencies? RQ3: Which are the main challenges experienced with testing using mocks? Research Questions
- 15. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies Alura TA P3 Mock P1 P2 1,333 dependencies VRaptor TA P3 Mock P1 P2 21,768 dependencies Spring Framework TA P1 TA P1 TA P1 Methodology
- 16. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674 dependencies TA P1 Tn Pn … … Sampling P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Data collection TA P1 TA P1 TA P1 Methodology
- 17. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674 dependencies TA P1 Tn Pn … … Sampling Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Data analysisData collection Manual analysis TA P1 TA P1 TA P1 Methodology
- 18. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674 dependencies TA P1 Tn Pn … … Sampling Commit Comments Commit Comments Commit Comments 132 categories Commit Comments7 Categories Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Discussion Data analysisData collection Manual analysis Categorisation TA P1 TA P1 TA P1 Methodology
- 19. RQ1 RQ1: What test dependencies do developers mock?
- 20. RQ1: What test dependencies do developers mock? RQ1 0% 25% 50% 75% 100% Database Web service External Dependencies Domain object Java libraries Test support 94%93%64%32%31%28% 6%7% 36% 68%69% 72% Mocked Not mocked
- 21. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674 dependencies TA P1 Tn Pn … … Sampling Commit Comments Commit Comments Commit Comments 132 categories Commit Comments7 Categories Interview Guideline 3 developers Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Discussion Data analysis Interviews & Validation Data collection Manual analysis Categorisation TA P1 TA P1 TA P1 Methodology
- 22. RQ2 RQ2: Why do developers decide to (not) mock a dependency?
- 23. RQ2 RQ2: Why do developers decide to (not) mock a dependency? Not when the focus of the test is the integration Vehicle Car Plane Boat Mock interfaces rather than a specific implementation Concrete implementation is not easy
- 24. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674 dependencies TA P1 Tn Pn … … Sampling Commit Comments Commit Comments Commit Comments 132 categories Commit Comments7 Categories Interview Guideline 3 developers Commit Comments Review Comment Commit Comments Interview Transcript Commit Comments Commit Comments Survey (105 respondents) Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Discussion Affinity Diagram Data analysis Interviews & Validation Data collection Manual analysis Categorisation TA P1 TA P1 TA P1 Methodology
- 25. RQ3 RQ3:Which are the main challenges experienced with testing using mocks?
- 26. RQ3 RQ3:Which are the main challenges experienced with testing using mocks? Legacy systems Dealing with coupling Production quality code vs mocks
- 27. Discussion Mockito developer • He agreed on all the findings • How Mockito can help? • What Mockito is supposed to do more? Open questions • Software quality vs mocks • Are slow tests more mocked? • How faster are tests with mocks?