SlideShare a Scribd company logo
1 of 20
Iosif Itkin
CEO
Exactpro, LSEG
Machine Learning
in Software
Testing
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com2
Machine Learning in
Software Testing
Open Access Quality Assurance & Related Software Development for Financial Markets
Tel: +7 495 640 2460, +1 415 830 38 49
www.exactpro.com
Iosif Itkin, Co-CEO and Co-Founder, Exactpro
London Stock Exchange Group
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com3
Advances in Computer Science
Cat on a
Screen
Bug in the
Code
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com4
Technology Trends
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com5
Guess Who
Relentlessly
Tortures the
Software
Prevents from
being Released
into the Wild
Breaks It
and
Criticises
it
Subject to
Ruthless Soak and
Negative Testing
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com6
Global Trends
Job Loss?
Death?
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com7
Contents
1) Software development life cycle
2) Key enablers for machine learning in
testing
3) An approach to improve the test librarie
4) Challenges with real-time systems
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com8
Software Development Life Cycle
Requirements
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com9
Software Development Life Cycle
Requirements
Design & Code
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com10
Software Development Life Cycle
Requirements
Software
Bugs
Design & Code
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com11
Software Development Life Cycle
Requirements
Design & Code Software
Bugs
Magic
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com12
Software Testing Factory
Requiremen
ts
& Code
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com13
Big Data in Testing
Identify requirements and
areas most prone to defects
in the next release of
software
Predict the
probability of a
defect to be re-
opened
Highlight the
area of the code
where the bug is
most likely
located
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com14
Dynamic Testing & Learning
Non-
Functional
Testing
Tools
Passive
Testing
Capability
Build-in
Testability in
the System
Machine-Readable
Specifications
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com15
Testing Algo Trading Systems
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com16
Machine Learning
Defects Database
Code Coverage
Mutation Testing
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com17
Real-Time Systems
Run All the Functional Tests
Concurrently
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com18
Risk Controls
Google DeepMind published a paper
claiming that it is possible to
implement "a button" to
interrupt undesired behaviour
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com19
Risk Controls
Will we have enough
time to press the
button once?
Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com20
Thank You
Innovation
Excellence
Partnership
Integrity

More Related Content

Viewers also liked

Artificial intelligence in qa
Artificial intelligence in qaArtificial intelligence in qa
Artificial intelligence in qa
Taras Lytvyn
 
TMPA-2014: Exactpro Pep Talk
TMPA-2014: Exactpro Pep TalkTMPA-2014: Exactpro Pep Talk
TMPA-2014: Exactpro Pep Talk
Iosif Itkin
 
The next generation of testing tools for exchanges
The next generation of testing tools for exchangesThe next generation of testing tools for exchanges
The next generation of testing tools for exchanges
Iosif Itkin
 
High Frequency Trading Infrastructure and Quality Assurance
High Frequency Trading Infrastructure and Quality AssuranceHigh Frequency Trading Infrastructure and Quality Assurance
High Frequency Trading Infrastructure and Quality Assurance
Iosif Itkin
 
SEC Regulation SCI Automation Review Compliance
SEC Regulation SCI Automation Review ComplianceSEC Regulation SCI Automation Review Compliance
SEC Regulation SCI Automation Review Compliance
Iosif Itkin
 
Trading Clearing Systems Test Automation
Trading Clearing Systems Test AutomationTrading Clearing Systems Test Automation
Trading Clearing Systems Test Automation
Iosif Itkin
 

Viewers also liked (15)

The Simple Matter of Project Management
The Simple Matter of  Project Management The Simple Matter of  Project Management
The Simple Matter of Project Management
 
Cost of Quality How to Save Money
Cost of Quality How to Save MoneyCost of Quality How to Save Money
Cost of Quality How to Save Money
 
Verification of Financial Models
Verification of Financial ModelsVerification of Financial Models
Verification of Financial Models
 
Artificial intelligence in qa
Artificial intelligence in qaArtificial intelligence in qa
Artificial intelligence in qa
 
Trading Systems: Testing at the Confluence of FT and NFT
Trading Systems: Testing at the Confluence of FT and NFTTrading Systems: Testing at the Confluence of FT and NFT
Trading Systems: Testing at the Confluence of FT and NFT
 
EXTENT October 2011 - Test Automation for Trading Systems
EXTENT October 2011 - Test Automation for Trading SystemsEXTENT October 2011 - Test Automation for Trading Systems
EXTENT October 2011 - Test Automation for Trading Systems
 
TMPA-2014: Exactpro Pep Talk
TMPA-2014: Exactpro Pep TalkTMPA-2014: Exactpro Pep Talk
TMPA-2014: Exactpro Pep Talk
 
EXTENT Trading Test Tools Evolution Theory
EXTENT Trading Test Tools Evolution TheoryEXTENT Trading Test Tools Evolution Theory
EXTENT Trading Test Tools Evolution Theory
 
Верификация технологических платформ Лондонской фондовой биржи
Верификация технологических платформ Лондонской фондовой биржиВерификация технологических платформ Лондонской фондовой биржи
Верификация технологических платформ Лондонской фондовой биржи
 
Innovation in Quality Assurance – What is the Impact on Trading technology?
Innovation in Quality Assurance –  What is the Impact on Trading technology?Innovation in Quality Assurance –  What is the Impact on Trading technology?
Innovation in Quality Assurance – What is the Impact on Trading technology?
 
The next generation of testing tools for exchanges
The next generation of testing tools for exchangesThe next generation of testing tools for exchanges
The next generation of testing tools for exchanges
 
High Frequency Trading Infrastructure and Quality Assurance
High Frequency Trading Infrastructure and Quality AssuranceHigh Frequency Trading Infrastructure and Quality Assurance
High Frequency Trading Infrastructure and Quality Assurance
 
SEC Regulation SCI Automation Review Compliance
SEC Regulation SCI Automation Review ComplianceSEC Regulation SCI Automation Review Compliance
SEC Regulation SCI Automation Review Compliance
 
Trading Clearing Systems Test Automation
Trading Clearing Systems Test AutomationTrading Clearing Systems Test Automation
Trading Clearing Systems Test Automation
 
Exactpro: Non-functional testing approach
Exactpro: Non-functional testing approachExactpro: Non-functional testing approach
Exactpro: Non-functional testing approach
 

More from Iosif Itkin

More from Iosif Itkin (20)

Foundations of Software Testing Lecture 4
Foundations of Software Testing Lecture 4Foundations of Software Testing Lecture 4
Foundations of Software Testing Lecture 4
 
QA Financial Forum London 2021 - Automation in Software Testing. Humans and C...
QA Financial Forum London 2021 - Automation in Software Testing. Humans and C...QA Financial Forum London 2021 - Automation in Software Testing. Humans and C...
QA Financial Forum London 2021 - Automation in Software Testing. Humans and C...
 
Exactpro FinTech Webinar - Global Exchanges Test Oracles
Exactpro FinTech Webinar - Global Exchanges Test OraclesExactpro FinTech Webinar - Global Exchanges Test Oracles
Exactpro FinTech Webinar - Global Exchanges Test Oracles
 
Exactpro FinTech Webinar - Global Exchanges FIX Protocol
Exactpro FinTech Webinar - Global Exchanges FIX ProtocolExactpro FinTech Webinar - Global Exchanges FIX Protocol
Exactpro FinTech Webinar - Global Exchanges FIX Protocol
 
Operational Resilience in Financial Market Infrastructures
Operational Resilience in Financial Market InfrastructuresOperational Resilience in Financial Market Infrastructures
Operational Resilience in Financial Market Infrastructures
 
20 Simple Questions from Exactpro for Your Enjoyment This Holiday Season
20 Simple Questions from Exactpro for Your Enjoyment This Holiday Season20 Simple Questions from Exactpro for Your Enjoyment This Holiday Season
20 Simple Questions from Exactpro for Your Enjoyment This Holiday Season
 
Testing the Intelligence of your AI
Testing the Intelligence of your AITesting the Intelligence of your AI
Testing the Intelligence of your AI
 
EXTENT 2019: Exactpro Quality Assurance for Financial Market Infrastructures
EXTENT 2019: Exactpro Quality Assurance for Financial Market InfrastructuresEXTENT 2019: Exactpro Quality Assurance for Financial Market Infrastructures
EXTENT 2019: Exactpro Quality Assurance for Financial Market Infrastructures
 
ClearTH Test Automation Framework: Case Study in IRS & CDS Swaps Lifecycle Mo...
ClearTH Test Automation Framework: Case Study in IRS & CDS Swaps Lifecycle Mo...ClearTH Test Automation Framework: Case Study in IRS & CDS Swaps Lifecycle Mo...
ClearTH Test Automation Framework: Case Study in IRS & CDS Swaps Lifecycle Mo...
 
EXTENT Talks 2019 Tbilisi: Failover and Recovery Test Automation - Ivan Shamrai
EXTENT Talks 2019 Tbilisi: Failover and Recovery Test Automation - Ivan ShamraiEXTENT Talks 2019 Tbilisi: Failover and Recovery Test Automation - Ivan Shamrai
EXTENT Talks 2019 Tbilisi: Failover and Recovery Test Automation - Ivan Shamrai
 
EXTENT Talks QA Community Tbilisi 20 April 2019 - Conference Open
EXTENT Talks QA Community Tbilisi 20 April 2019 - Conference OpenEXTENT Talks QA Community Tbilisi 20 April 2019 - Conference Open
EXTENT Talks QA Community Tbilisi 20 April 2019 - Conference Open
 
User-Assisted Log Analysis for Quality Control of Distributed Fintech Applica...
User-Assisted Log Analysis for Quality Control of Distributed Fintech Applica...User-Assisted Log Analysis for Quality Control of Distributed Fintech Applica...
User-Assisted Log Analysis for Quality Control of Distributed Fintech Applica...
 
QAFF Chicago 2019 - Complex Post-Trade Systems, Requirements Traceability and...
QAFF Chicago 2019 - Complex Post-Trade Systems, Requirements Traceability and...QAFF Chicago 2019 - Complex Post-Trade Systems, Requirements Traceability and...
QAFF Chicago 2019 - Complex Post-Trade Systems, Requirements Traceability and...
 
QA Community Saratov: Past, Present, Future (2019-02-08)
QA Community Saratov: Past, Present, Future (2019-02-08)QA Community Saratov: Past, Present, Future (2019-02-08)
QA Community Saratov: Past, Present, Future (2019-02-08)
 
Machine Learning and RoboCop Testing
Machine Learning and RoboCop TestingMachine Learning and RoboCop Testing
Machine Learning and RoboCop Testing
 
Behaviour Driven Development: Oltre i limiti del possibile
Behaviour Driven Development: Oltre i limiti del possibileBehaviour Driven Development: Oltre i limiti del possibile
Behaviour Driven Development: Oltre i limiti del possibile
 
2018 - Exactpro Year in Review
2018 - Exactpro Year in Review2018 - Exactpro Year in Review
2018 - Exactpro Year in Review
 
Exactpro Discussion about Joy and Strategy
Exactpro Discussion about Joy and StrategyExactpro Discussion about Joy and Strategy
Exactpro Discussion about Joy and Strategy
 
FIX EMEA Conference 2018 - Post Trade Software Testing Challenges
FIX EMEA Conference 2018 - Post Trade Software Testing ChallengesFIX EMEA Conference 2018 - Post Trade Software Testing Challenges
FIX EMEA Conference 2018 - Post Trade Software Testing Challenges
 
BDD. The Outer Limits. Iosif Itkin at Youcon (in Russian)
BDD. The Outer Limits. Iosif Itkin at Youcon (in Russian)BDD. The Outer Limits. Iosif Itkin at Youcon (in Russian)
BDD. The Outer Limits. Iosif Itkin at Youcon (in Russian)
 

Recently uploaded

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Recently uploaded (20)

[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 

EXTENT-2016: Machine Learning and Software Testing

  • 1. Iosif Itkin CEO Exactpro, LSEG Machine Learning in Software Testing
  • 2. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com2 Machine Learning in Software Testing Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 2460, +1 415 830 38 49 www.exactpro.com Iosif Itkin, Co-CEO and Co-Founder, Exactpro London Stock Exchange Group
  • 3. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com3 Advances in Computer Science Cat on a Screen Bug in the Code
  • 4. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com4 Technology Trends
  • 5. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com5 Guess Who Relentlessly Tortures the Software Prevents from being Released into the Wild Breaks It and Criticises it Subject to Ruthless Soak and Negative Testing
  • 6. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com6 Global Trends Job Loss? Death?
  • 7. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com7 Contents 1) Software development life cycle 2) Key enablers for machine learning in testing 3) An approach to improve the test librarie 4) Challenges with real-time systems
  • 8. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com8 Software Development Life Cycle Requirements
  • 9. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com9 Software Development Life Cycle Requirements Design & Code
  • 10. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com10 Software Development Life Cycle Requirements Software Bugs Design & Code
  • 11. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com11 Software Development Life Cycle Requirements Design & Code Software Bugs Magic
  • 12. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com12 Software Testing Factory Requiremen ts & Code
  • 13. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com13 Big Data in Testing Identify requirements and areas most prone to defects in the next release of software Predict the probability of a defect to be re- opened Highlight the area of the code where the bug is most likely located
  • 14. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com14 Dynamic Testing & Learning Non- Functional Testing Tools Passive Testing Capability Build-in Testability in the System Machine-Readable Specifications
  • 15. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com15 Testing Algo Trading Systems
  • 16. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com16 Machine Learning Defects Database Code Coverage Mutation Testing
  • 17. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com17 Real-Time Systems Run All the Functional Tests Concurrently
  • 18. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com18 Risk Controls Google DeepMind published a paper claiming that it is possible to implement "a button" to interrupt undesired behaviour
  • 19. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com19 Risk Controls Will we have enough time to press the button once?
  • 20. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com20 Thank You Innovation Excellence Partnership Integrity

Editor's Notes

  1. <date/time>
  2. <date/time>
  3. <date/time>
  4. <date/time>
  5. <date/time>
  6. <date/time>