Potential of AI (Generative AI) in Business: Learnings and Insights
OSS/BSS Testing and AMS
1. OSS/BSS
Testing And AMS
By Ahmed Adel Abdel Gawad
2. OSS (Operations Support Systems)/
BSS (Business Support Systems)
• OSS (Operations Support Systems) track
network inventory, assets and provisioning of
services, while BSS (Business Support Systems)
deal with customer relationship management
(CRM) and processes such as taking
orders, processing bills, and collecting
payments.
4. OSS/BSS landscape
• The OSS/BSS landscape can be broadly divided
into the following modules:
•Mediation Systems
•Billing Systems
•CRM - Customer Relationship Management
•E-Business & Web Self-Care
•Data Warehouse - Business Intelligence &
Analytics
•Provisioning Systems
•Service Assurance & Network Care
•Fraud Management
5. OSS/BSS landscape
• Business (Testing) Prospective
– Customer Life Cycle
• Customer Life Cycle and dismissal
• E-Business & Web Self-Care
• Customer Relationship Management
• Loyalty Systems
• Billing Systems
– Service Life Cycle
• Service Life Cycle
• Service Management
6. OSS/BSS landscape
• Technical (AMS) Prospective
– FrontEnd Systems
• Web Services
• Application Servers
• CRM
• ERP
– BackEnd Systems
• Database
• Service Assurance & Network Care
• Provisioning Systems
• Billing Systems
• Warehouse
• Business Intelligence & Analytics
• Payment GW
– Mediation Systems
• Provisioning Systems (ASAP)
• Fusion MW …
7. Target Approach
• Introducing (TaaS) for OSS/BSS Business and
Systems.
• Leverage testing team to serve business
instead of customer.
• Building testing repository for all business
related to Telco. Industry.
• Automat Testing Process for regression and
production support.
• Provide AMS for OSS/BSS Environments
8. Introducing (TaaS) for OSS/BSS Business
and Systems.
Clients are looking for greater flexibility and value
• Greater resource flexibility
• Faster time to value
• Lower cost of entry
As-a-Service
Managed ― Consumption Driven Models
Services ― Unit Pricing
Staff ― Resource Flexibility
Augmentation ― Lower Risk
― Lower Cost
Testing
Software
People, process, tools, accelerators, technology delivered
in a consumable “as-a-service” model
3/17/2013 Testing Catalog 8
9. Testing Services Market
Why Testing-as-a-Service?
Client Challenges Business Value
Scope of applications are organically defined as Reduce system outage risk caused by functional and
market conditions change performance defects
• Standard contracting models do not provide cost No upfront investment or recurring/maintenance fees
effectiveness in such scenarios
Forecast the consumption of testing services and
There are much more frequent peaks and valleys control testing expenses accordingly
in testing demand
Clients are looking for their Testing providers to:
• Testing resources are still available and being
paid for in valleys of idle time • Build the tests, automate and execute them (or have
them ready to be executed by the client)
Capital expense on software tools and tool
infrastructure and ongoing costs for support and • Use industry leading software tools like
maintenance ALM/Quality Center and Performance Center
Smaller projects often don’t have budget for • Use a professional certified testing team and test
engaging full time testing resources center of excellence
• Use a formal Quality Management methodology
with pre-built tests/scenarios, frameworks and
accelerators
3/17/2013 9
10. Testing-as-a-Service
Leading the evolution of consumption driven testing service models
Functional Quality Performance Security
Application Fundamentals
• Functional Testing Service • Performance Testing Service • Security Testing Service
• Test Planning • Test Planning • Application Code Scanning
• Test Case Development • Test Scripting • Web Penetration Testing
• Test Case Automation • Load and Stress Test Execution • Fortify
• Test Case Execution (manual and • Test Reporting (SLAs, baseline, etc.) • QAInspect / WebInspect
automated) • Load Controllers/Generators • Per Application / Site Pricing
• Test Automation Framework • Virtual User Licenses
• ALM/Quality Center/BPT/QTP • LoadRunner
• FTU – Functional Test Unit Pricing • PTU – Performance Test Unit Pricing
FTaaS – Functional PTaaS – Performance STaaS – Security
Testing-as-a-Service Testing-as-a-Service Testing-as-a-Service
3/17/2013 10
11. Testing-as-a-Service
Benefits of the flexible as-a-service model
Functional Quality Performance Security
FTaaS PTaaS STaaS
• Flexibility to consume functional • Identify and eliminate performance • Identification and removal of
testing resources as needed bottlenecks security vulnerabilities
• Access to testing tools, automation • Ensure IT systems can maximize their • Flexible, output-based pricing
frameworks and accelerators capacity model by application
scanned, website assessed
• Reduced test cycle time with test • Performance test more applications
automation for less • Removal of false positives
• Higher test coverage, fewer defects • Reduce the risk that a performance • Remediation of security
issue will impact business operations vulnerabilities
• Lower cost to test applications
• Minimize your investment in • Minimize investment in application
• Minimize your investment in test
performance testing security testing
automation
Output Based Consumption Driven
3/17/2013 11
12. Testing-as-a-Service
Same capabilities, same deliverables, flexible packaging and pricing
Staff Augmentation Managed Services Testing-as-a-Service
Service Features • System Integration Testing • System Integration Testing • System Integration Testing
• Systems Testing • Systems Testing • Systems Testing
• Service Testing • Service Testing • Service Testing
• Acceptance Testing • Acceptance Testing • Acceptance Testing
• Regression Testing • Regression Testing • Regression Testing
• Test Automation • Test Automation • Test Automation
• Performance Testing • Performance Testing • Performance Testing
• Security Testing • Security Testing • Security Testing
Packaging • Custom client based • Custom client based • Leveraged factory solution
solutions solutions with accelerators • Global availability
• Global availability • Global availability • FTU / PTU Volume
• Onshore / Best shore options • Onshore / Best shore options • Term Options
Pricing / Contracting • Software License Fees • Software License Fees • Unit-based Pricing: FTU, PTU
• FTE-based Pricing • Custom Pricing • Includes Software Tools and
• Time and Materials • Fixed Price or T&M Licenses
Contracting (T&M) • Managed Outcome • Demand Driven, Output-
• Headcount Driven based
3/17/2013 12
14. Testing-as-a-Service
FTaaS – Functional Testing-as-a-Service
Service Description Service Features
Functional Testing-as-a-Service delivers all the • Leveraged test factory service
people, process, tools and accelerators to deliver • Unified functional testing (system, integration, regression)
system, integration, and regression testing with test
automation using a consumption based service model. • Test case development and execution
TaaS from India Delivery Center offers: • Manual and automated testing
• FTU Price: USD TBD • Test data preparation
• Min. Monthly FTU: TBD • Test automation scripting
• Term Agreement: TBD • Test Automation Framework
• Reusable automated test packs (SAP, Oracle, etc)
• Testing and defect management and reporting
Client Benefits
• ALM/Quality Center, Unified Functional Testing
• Flexibility to consume testing resources as needed (UFT/QTP), Business Process Testing (BPT), Sprinter tools
• Supports small projects with small testing budgets
• Reduced test cycle time with test automation “Test automation has reduced our regression testing effort by more than 80%
while improving test coverage by as much as 40%”
• Higher test coverage, fewer defects
• Lower cost to test applications New York Financial Institution
3/17/2013 14
15. Functional Testing-as-a-Service
FTaaS functional test unit consumption values
Area Tasks FTUs Unit of Measure
Develop a new test plan 170 Per Test Plan
Test Planning
Update an existing test plan 70 Per Test Plan Update
Requirements Knowledge transfer for one use case / user story / test set 15 Per Use Case/User Story/Test Set
Develop a new test case 0.5 Per Test Step
Manual Testing Modify an existing test case 0.5 Per Test Step
Execute a manual test case 3 Per Test Case
Automation of a manual test case 1 Per Test Step Scripted
Modify an existing automated test script 1 Per Test Step Scripted
Automated Testing
Automated test case execution by 0.1 Per Automated Test Case
Automated test case execution by Client 500 Per 50,000 Scripts Executed
Define POC scope, select test cases, setup environment 135 Per POC
Test Automation Develop automated test scripts 3 Per Test Step Scripted
POC
(5 Test Scenarios) Test data preparation 30 Per POC
Demonstration and proposal for full implementation 135 Per POC
Test data preparation for one test case or test script 0.3 Per Test Case
Data Preparation
Automation of test data generation for one test step 1 Per Test Step
Quality Center license 45 Per License / Month
Software Licenses
QuickTest Pro concurrent license 32 Per License / Month
for Instance
Quality Center license + BPT license 73 Per License / Month
3/17/2013 15
16. Functional Testing-as-a-Service
Example Client Scenario
Client Scenario Client Cost per Test Cycle (Execution Only)
Web Application – 2,000 manual test cases Manual execution of web application test cases
• One-liners, no details Scope: 2,000 test cases
• Duplication of steps Execution: 1.5 hours/test case
• Completely dependent on application/business • Preparation, Data generation
knowledge for proper execution • Execution
No standard test repository, Excel-based which • Defect logging
requires manual updates every run
Cost per resource: ~$81/hour
No reusability of test assets
TOTAL cost per test cycle: ~$245K
Insufficient time to always execute 2,000 test
cases, so test coverage is compromised
Manual test data generation for each test case, 4
regression execution cycles/year
3/17/2013 16
17. Functional Testing-as-a-Service
Example Client Scenario with FTaaS
FTaaS Scenario Client Cost per Test Cycle (Execution Only)
Price/FTU: USD $9.50 Manual execution of web application test cases
Manual test execution (800 Test Cases) Scope: 2,000 test cases
• FTUs: 2,270 Execution: 1.5 hours/test case
• TOTAL cost per test cycle: ~$21,565 • Preparation, Data generation
Automated test execution (1,200 Test Cases) • Execution
• FTUs: 2,710 • Defect logging
• TOTAL cost per test cycle: ~$25,745 Cost per resource: ~$81/hour
Onsite travel (2 FTEs) TOTAL cost per test cycle: ~$245K
• FTUs: 2,000
• Travel cost/regression cycle: ~$19,000 FTaaS Delivers:
• Cycle time reduction over 80% with automation
Total FTU: 6,980 per test cycle
• Cost reduction over 70% with optimized resource usage
TOTAL cost per test cycle: ~$66K
• All 2,000 test cases executed; planned testing coverage achieved
73% • Standard test management tool with version control
Savings • Standard, consistent process and metrics
• Full reusability through Advanced Test Automation Framework
cost reduction of $179K per test cycle FTaaS
• Shorter startup/preparation/execution time with automation
3/17/2013 17
18. Linking testing
services to processes
• The workflow shows
the customer and
supplier testing
related processes in a
project and links them
the service catalog
items required
• Testing services for
each project are
identified during
project planning
• The Testing Service
agreement enables
flexible testing service
provision
3/17/2013 18
19. Customer responsibilities
Category Client Responsibility Client Capability Requirement
Test Planning Provide Project Plan input, Review Test Plan Client must be able provide project plans and schedules with testing task
definition, start/end dates and dependencies;
Client has resourced the project management function
Test Planning Provide Project Plan input, Review Test Plan Client must be able to do change control of project documentation
Requirements Provide documentation of application Client must able to provide application architecture, functionality, interfaces,
architecture, functionality, interfaces, technologies
technologies
Requirements Provide documented use case/user story Client must be able to provide functional requirements in the form of use cases/
user stories and non-functional requirements in the form of statements of need
Manual Testing Review test results and defect reports Client must provide the test environment
Client must be able to provide system to be tested installed and ready for test in
the test environment
Client must be able to access Quality Center to view test results and defect reports
Manual Testing Review test case test steps Client is willing to participate in the review of test cases, ideally using the QC tool
Manual Testing Review test case test steps Client is willing to participate in the review of test cases, ideally using the QC tool
Data Preparation Agree test data sourcing policy; provide test Client must establish the test data sourcing policy at the start of the service and be
data if implied by the policy; review supplier able to ful fill their responsibilities implied by the policy
provided test dat if implied by policy
Automation Proof of Agree scope of automation Client must be able to understand the benefits of automation and general
Concept concepts of automation
Automation Proof of Agree test data sourcing policy; provide test Client must establish the test data sourcing policy at the start of the service and be
Concept data if implied by the policy; review supplier able to ful fill their responsibilities implied by the policy
provided test dat if implied by policy
Automation Proof of Review demo and proposal Client must be able to attend automation demo and review automation proposal
Concept
Automated Scripts Review test results and defects and manage Client must be able to access Quality Center to view test results and defect reports
resolution of defects with respective
development team
Automated Scripts Initiate test execution, review test results and Client must be able to access Quality Center, and initiate the execution
defect and manage resolution of defects
3/17/2013 19
20. Leverage testing team / Environments to
serve business instead of customer.
• Utilize team knowledge in the industry to maximize the quality level.
• Cloud environments enable dynamic provisioning for multiple environments while
utilizing a shared pool of resources
