8. QA Myths
You only need to unit test. TDD testing is sufficient.
You can reuse unit tests to build a regression test suite.
We no longer need testers, as Developer have adequate coding skills.
Unit test remove the need of manual testing.
User Acceptance testing is no longer needed.
Automation in so early phases is impossible.
TDD/BDD is applicable on every project.
Developers and testers are like water and oil.
12. JUnit
@Test Annotation
JUnitCore
Exception Test
@Ignore
Timeout Test
TestSuite
Before and After Annotations
@BeforeClass and @AfterClass
Parameterized Test
13. TestNG
Annotations
Exception Test
Ignore Test
Timing Test
TestSuite
Parameterized Test
Test Dependency
Group Test and Dependency
17. 1. Write Story
Create New Maven project using MyNumber class used in previous
examples.
Add Jbehave plugin in IntelliJ
Add dependencies of Jbehave and Junit
Add a new story under test resources folder
18. 2. Map Steps to Java
Create a POJO and implement the Given, When, Then Steps
19. 3. Configure Stories
Implement a TestRunner class by extending class JUnitStory.
Override the configuration and stepsFactory methods.
Ensure that story name would be same as test name with _ updated for
each Camel Case.
This is because of the configuration used for loading of story, i.e.,
MostUsefulConfiguration
20. 4. Run the Story
Run the story same as Junit test
23. JUnit-enabled Embeddables
JUnitStory: provides a one-to-one mapping with the textual story.
JUnitStories: provides a many-to-one mapping with the textual story
paths
24. Story Syntax
JBehave Syntax
Narrative:
Lifecycle:
Before:
After
Scenario
Gherkin Syntax
Narrative:
Background:
Given:
Scenario/Scenario Outline:
Gherkin's Background element corresponds to the JBehave Lifecycle Before
element.
JBehave also supports a Lifecycle After element which is not currently available in
Gherkin.
26. Step Annotations
@Given
@When
@Then
@Alias
@Aliases
@Pending
Dry Run Mode
Check if any steps are pending but without actually executing any of the steps
31. Prioritizing Steps
For cases in which multiple candidates can match the same textual step,
the first candidate that matches is used to create an executable step.
To prioritize one candidate over another simply set a non-zero positive
priority.
32. Story Path’s
Class-path – Arrange in any folder hierarchy within project.
URL-Loader (Running Remote Stories) – Specify the path from a URL
Google-docs (Running Remote Stories) – No more supported, due to
change in google authorization, but can be customized, if needed for a
project.
34. Customizing Reports
Specify the FTL File
Define a custom Template that will use the FTL file
Use this reporter in TestRunner
35. UI Testing
Use Serenity Stories
Specify properties in serenity.properties or from maven command line
options
Specify “aggregate” goal to aggregate reports.
37. CI Setup - Jenkins
Download Jekins from https://jenkins.io/download/
Install Jenkins.
Editor's Notes
TDD cycle defines
Write a test
Make it run.
Change code to make it right i.e. Refactor.
Repeat process.
TDD Vs. Traditional Testing
ATDD also known as Behavioral Driven Development (BDD).
1) Shifting from thinking in “tests” to thinking in “behavior”
2) Collaboration between Business stakeholders, Business Analysts, QA Team and developers
3) Ubiquitous language, it is easy to describe
4) Driven by Business Value
5) Extends Test Driven Development (TDD) by utilizing natural language that non technical stakeholders can understand
6) BDD frameworks such as Cucumber or JBehave are an enabler, acting a “bridge” between Business & Technical Language
Example of e-commerce site :-
Let’s assume there is a requirement from a client for an E-Commerce website to increase the sales of the product with implementing some new features on the website. The only challenge of the development team is to convert the client idea in to something that actually delivers the benefits to client.
The original idea is awesome. But the only challenge here is that the person who is developing the idea is not the same person who has this idea. If the person who has the idea happens to be a talented software developer, then we might be in luck: the idea could be turned into working software without ever needing to be explained to anyone else. Now the idea needs to be communicated and has to travel from Business Owners(Client) to the development teams or many other people.
Most software projects involve teams of several people working collaboratively together, so high-quality communication is critical to their success. As you probably know, good communication isn’t just about eloquently describing your ideas to others; you also need to solicit feedback to ensure you’ve been understood correctly. This is why agile software teams have learned to work in small increments, using the software that’s built incrementally as the feedback that says to the stakeholders “Is this what you mean?”
What’s changed in this new test-first approach? First, testing has been automated and moved in front of coding. This means that code is developed to make the test pass. Secondly, these tests act as the requirements, removing one layer of documentation in the process, and limiting the risk of tests and code being built to a different understanding – pretty cool! Finally, we allow the developer only to write code until the test passes, then refactor that code to find a more elegant solution while still passing the code.
This test-first approach became increasingly popular and was coined as test driven development (TDD), but businesses quickly realized it didn’t give them the visibility and coverage they needed for the most important business cases in their systems. So, a variant of TDD was born called behavior driven development (BDD), focusing on the behavior of the system rather than its technical specifications. Now, organizations could push features quickly with quality AND confidence that they met end user expectations.
https://github.com/junit-team/junit4/wiki/Download-and-Install
Under IntelliJ
Create New Java project and add class Calculator
To add external Jars, click on File >> Project Structure >> Libraries or from File >> Project Structure >> Module >> Dependencies
@Test Annotation
To Generate Test -- Navigate -> Test or press alt+enter on class Name
Put assert statements for all the test methods and run by right clicking on the class
To run the same outside IDE, we need to create a Test Runner.
JUnitCore
Create class RunTestStandalone and execute classes using JUnitCore.runClasses
Can even specify multiple classes using runClasses(test1.class, test2.class, ...).
Export project as jar using following steps:-
File>>Project Structure>>Artifacts>>+>>Add main class and other options
Build >> Build Artifacts
Run using java –jar option
Exception Test
Add one more test named DivTest
Test will fail when no exception specified
@Ignore Annotation
Replace @Test with @Ignore and the test will not be executed
Or put @Ignore just above the @Test annotation
@Test(timeout = msec)
Check TimeoutTest.java
Test Suite
RunAllTest – Specify all classes in SuiteClasses and RunWith
Before and @After
Create new project JUnitTest2 and add class MyNumber, which has no static methods, so need to create object to invoke methods
Create new Test folder and mark it as source folder and add MyNumberTest class. Create Objects of MyNumber in @Before Method
@BeforeClass and @AfterClass
Parameterized Test
Annotate the test class with @RunWith(Parameterized.class)
Create a public static method annotated with @Parameters that returns a list (Iterable<Object>) as test data set.
Create a public constructor that takes its input from the @Parameters method to setup the test fixtures defined as instance variables. The constructor will be run before EACH test.
Create your tests case(s) using the instance variables as the source of the test data.
The output trace suggests that @Parameters method is run once. For EACH test, the constructor is run first, followed by @Before, @Test and @After methods.
Under Eclipse :-
Create a new Eclipse Java project called "JUnitTest".
Create a new class called "Calculator" under "src" folder, with the above program code.
Create a new folder called "test" for storing test scripts ⇒ Right-click on the project ⇒ New ⇒ Folder ⇒ In folder name, enter "test". Make "test" a source folder by right-click on "test" ⇒ Build Path ⇒ Use as source folder.
Create the first test case called "AddSubTest" ⇒ Right-click on folder "test" ⇒ New ⇒ Other ⇒ Java ⇒ JUnit ⇒ JUnit Test Case ⇒ New JUnit 4 test ⇒ In Name, enter "AddSubTest". Enter the following codes:
http://testng.org/doc/download.html --- Whole process defined how to setup in Eclipse, come by default with IntelliJ (Add path of lib folder)
Annotations
Test, BeforeClass, AfterClass, BeforeMethod, AfterMethod
Create class MyNumber same as in Junit example
Add class MyNumberTestNGTest with all annotated methods
Run the test
the @BeforeClass annotated method is run ONCE for one-time setup; the @AfterClass is run ONCE for one-time tear down. The @BeforeMethod and @AfterMethod (called @Beforeand @After in JUnit 4) are run before and after EACH @Test.
Exception Test
Mark the test method that is expected to throw an exception with @Test(expectedExceptions = ExceptionName.class)
Ignore Test
To ignore a test, mark it with annotation @Test(enabled=false)
Timing Test
To set a timeout (milliseconds) for a test, use annotation @Test(timeOut = msec)
TestSuite
Create an xml with tag names suite >> test >> classes >> class
The root tag is <suite>.
The <suite> tag can contain one or more <test> tags.
The <test> tag can contain one or more <classes> tags.
The <classes> tag can contain one or more <method> tags.
To execute suite in IntelliJ, EditConfiguration >> + >> TestNG >> TestKind(Suite) >> Specify xml file path
Parameterized Test
Via @Parameters in the test file and XML Description File <parameter> Tag
Check class TestNGParameterizedTest1
@DataProvider
Check class TestNGParameterizedTest2
Test Dependency
@Test(dependsOnMethods={"method1","method2"})
TestNGDependsTest class
Group Test and Dependency
Each test method can be assigned to one or more groups. We can select one or more groups to test via XML description file
Check class TestNGGroupTest and testnggroup.xml
Dependency on Groups
Instead of specifying dependency on individual method names as in the previous section, we can place related method (e.g., init methods) in groups, and specifying dependency on groups of methods
Check class TestNGGroupDependsTest
1) A build tool or a project management tool ?
2) In addition to providing buid capabilities, Maven can also run reports, generate a web site, execute unit testing and much more. The concept of testing is built right into the Maven lifecycle.
3) While testing is built into Maven, it limits itself to two stages of testing within the build lifecycle: unit testing, and integration testing
4) Unit testing is run after compilation but before packaging, therefore it is run on nearly every build. This indicates more accurately that the purpose of tests in this stage is those to be run on every build. They should pass before the build can complete and the artifact can be used.
5) Maven Eclipse plugin :-
http://download.eclipse.org/technology/m2e/releases/1.4/1.4.0.20130601-0317
Create empty maven projects and add the same files of Junit and TestNG projects and execute using mvn test
Story Name should be my_number_test.story
Refer project jbehaveExample1
public class MyNumberTest extends JUnitStory { @Override public Configuration configuration(){ return new MostUsefulConfiguration(). useStoryLoader(new LoadFromClasspath(this.getClass())). useStoryReporterBuilder(new StoryReporterBuilder().withDefaultFormats().withFormats(Format.CONSOLE,Format.TXT, Format.HTML)); } @Override public InjectableStepsFactory stepsFactory() { return new InstanceStepsFactory(configuration(),new MyNumberTestSteps()); }}
Right click the test and execute
JBehaveExample2 :- Implement storyPaths method by using StoryFinder().findPaths
We can have a single test runner that will execute all the stories.
Test runner can have different name from the story files.
Check :- http://jbehave.org/reference/stable/developing-stories.html
Install Eclipse Plugin from ---- http://jbehave.org/reference/eclipse/updates/
https://github.com/RSingh1983/JBehaveTraining
Jbehave Syntax ---- jbehaveExample3
Gherkin Syntax ---- jbehaveExample4 --- need to add explicitly gherkingParser in the Test runner.
Exercise ---- Covert to example table using Scenario and Scenario outline
Alias and Aliases ---- jbehaveExample5
The @Pending annotation allows steps developers to mark any step method as pending
-- Add @Pending to subtract_numbers method
jbehaveExample6 ---- If want to fail test on pending steps
.usePendingStepStrategy(new FailingUponPendingStep()) --- Use if want to fail the test, if there are pending steps
doDryRun(true) ---- just to check whether any steps are pending or not, no actual execution of Steps happen, so no print statements
useStepMonitor(new PrintStreamStepMonitor()) --- for verbose printing of each step
The @BeforeScenario and @AfterScenario annotations allow the corresponding methods to be executed before and after each scenario.
jbehaveExample7 --- has details for all types of scenarios.
--- the ScenarioType can be specified to execute only upon that type, e.g. for scenarios parametrised by examples
uponType=ScenarioType.EXAMPLE
@AfterScenario allows the setting of an optional Outcome value, which specifies whether the method should be executed depending on the outcome of the scenario
uponOutcome = AfterScenario.Outcome.SUCCESS --- only for After Scenario
The @BeforeStory and @AfterStory annotations allow the corresponding methods to be executed before and after each story
jbehaveExample8
@BeforeStory(uponGivenStory=true)
@AfterStory(uponGivenStory=true)
These 2 will execute only for Given Stories ---
Background:GivenStories:addNumber.story
The @BeforeStories and @AfterStories annotations allow the corresponding methods to be executed before and after a collection of stories
jbehaveExample9
JBehave supports multiple mechanisms for parameter injection:-
Ordered Parameters
This is the default behaviour. The arguments extracted from the step candidate are simply matched following natural order to the parameters in the annotated Java method.
The names of the parameters are not actually relevant, and there is no need for them to match the names of the argument capturers, although it is good practice to keep them matching.
Annotated Named Parameters
If we want to have named parameters, one mechanism is to use annotations
jbehaveExample10 --- validate the same in Given method
jbehaveExample11 --- Enable priority field with non-zero value
Classpath different folders ---- jbehaveExample12
URLPath – jbehaveExample13
ConsoleOutput: a text-based console output
IdeOnlyConsoleOutput: a text-based console output, which only reports events when running in IDEs (Eclipse and IDEA supported).
TxtOutput: a text-based file output
HtmlOutput: an HTML file output
HtmlTemplateOutput: an HTML template-based file output. By default, a Freemarker-based template processor is used but a different implementation based on a different templating system can be provided.
XmlOutput: an XML file output
PostStoryStatisticsCollector: collects statistics and stores them as properties after story is run
DelegatingStoryReporter: used by the StoryReporterBuilder to delegate to any number of reporters as a proxy.