소프트웨어 QA 및 테스팅 트렌드

© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
소프트웨어
QA 및 테스팅 트렌드
Key Findings from World Quality Report 2014-15
김기배 차장 (kibae@hp.com)
Business Development Manager
HP Software Asia Pacific and Japan

목차
 World Quality Report 개요
 소프트웨어 QA 및 테스팅 트렌드
 Recommendation #1
 Recommendation #2
 Summary

World Quality Report
Published since 2009, the World Quality Report (WQR) is
recognized as industry’s largest research study and most
comprehensive source of market analysis and commentary on
the state of enterprise application quality and testing practices
1,543respondents
(from organizations with over 1,000 employees only)

World Quality Report 2014-15: The Research
Company Size
# of
respondents
1,000 – 4,999 407
5,000 – 10.000 543
10,000 + 593

World Quality Report 2014-15: The Research

소프트웨어QA 및 테스팅트렌드

Current Trends in QA and Testing
QA and Testing Budgets Continue to Rise, driven by Digital
Transformation initiatives
Budget
Centralization
SMAC Technologies
Agile & DevOps
Automation

Budget : QA and Testing Budgets Continue to Rise
 모든 업계에 걸쳐 QA 예산의 지속적인 증가가 나타나고 있으며 ,
High-Tech 업계의 경우 IT 예산의 40%를 QA에 배정하는 기업이 전체 응답자의 24%로 조사됨
 신규 기술을 요구하는 시장의 압력과 상대적으로 짧은 딜리버리 일정이 QA 예산 증가로 나타남

 신규 개발 업무에 대한 QA 및 테스팅 예산 배정이 꾸준히 증가하고 있으며, 2014년엔 QA 예산의
약 52%가 신규 개발 업무에 배정되었음
 유지보수 업무에 대한 지속적인 QA 및 테스팅 방안에 대한 대책 마련 필요

 업무 복잡도의 증가로 현업을 포함한 이해관계자와 QA
조직의 협업이 요구되어 Onshore 형태의 조직 구성이
높은 효율성을 보임
(Offshore Testing의 비중이 28%  21%로 감소)
 신규 기술의 도입으로 인해 특정 기술에 대해 전문성을
가진 QA 및 테스트 인력 구성이 요구됨

QA 및 테스팅 활동에 대한 개선이 필요
전체 응답자의 64%

Budget
Many businesses prefer to organize QA & Testing functions with a
combination of centralized and decentralized approaches
Centralization
SMAC Technologies
Agile & DevOps
Automation

Centralization : Centralization with Decentralized Approach
 중앙 집중 형태의 (Centralized) 테스트 조직으로 구성이 변화되어 왔으며, 조직의 성숙도가
높아지고 시간이 흐르면서 하이브리드 형태의 조직 구성이 선호되고 있음

 Testing Center of Excellence (TCoE)
운영이 지속적으로 증가하고 있음
 업계 및 업무에 특화된 전문성
 도구 및 프로세스의 표준화
 지속적인 경험 축적 및 개선
 TCoE 운영을 통해 테스트 조직에 대한
ROI 평가가 수치에 기반하여 이루어짐

 모든 조직 혹은 업무가 크라우드 소싱 기반 테스트에 적합하지는 않으나, 다양한 지역, 문화권,
환경에 대한 사용자 경험 측면에서의 피드백을 받을 수 있는 장점을 가지고 조금씩 확산 중

아직도 많은 조직에서 애플리케이션
개발 주기의 후반에 테스팅 및 QA
활동이 이루어지고 있음

Budget
Centralization
Security, performance and customer experience testing represent
increasingly important focus areas for QA & Testing
SMAC Technologies
Agile & DevOps
Automation

SMAC Technologies : Increases in focusing Customer Experiences

 모바일 테스팅에 대한 경험이 축적됨에 따라 기존 대비 모바일 테스팅에 어려움을 겪던 부분에
대한 이슈들이 해소되고 있는 추세임

 빅데이터 및 분석 기술을 이용한
애플리케이션 테스팅 관련하여 새로운
이슈들이 발생하고 있음
 관련 지식 부족
 테스팅 프레임워크/도구의 부재
 수집한 데이터에 대한 검증이 어려움

 소셜 미디어의 활성화와 함께 QA 및 테스팅 팀은 애플리케이션에 대한 시장의 피드백을 수집하는
채널로 소셜 미디어를 활용하고 있음

모바일, 클라우드와 같은 새로운 플랫폼의 도입으로 인해 보안 테스트에 대한
중요도가 높아지고 있으며 기능과 성능에 대한 검증도 지속적으로 이루어지고 있음

Budget
Centralization
SMAC Technologies
Organizations require fast and responsive Testing aligned with
agile development methods including focus on DevOps
Agile & DevOps
Automation

Agile & DevOps : Testing aligned with Agile Development Methods
 애자일 개발 방법론이 점점 대부분의 조직에서 표준 개발 방식으로 자리잡고 있는 반면 애자일
테스팅 체계는 아직 정립되어 있지 않음

애자일 개발 방식에 맞는 테스팅
접근 법과 전문 인력의 부재로 인해
많은 조직이 애자일에 대한 대응을
어려워 하고 있음

Budget
Centralization
SMAC Technologies
Organizations require fast and responsive Testing aligned with
agile development methods including focus on DevOps
Agile & DevOps
Many organizations’ maturity in the field of test environment,
data management and test automation is increasing
Automation

Automation : Organizations’ Test Automation Maturity is Increasing
 많은 조직이 테스트 환경 준비 및 관리에 대한 어려움을 지속적으로 겪고 있음
 클라우드의 도입으로 인해 테스트 환경을 클라우드로 전환하는 조직이 늘고 있으며 (약 32%),
이를 통해 테스트 비용 절감 효과를 보고 있음

Automation : Organizations’ Test Automation Maturity is Increasing
 변화하는 애플리케이션 환경을 지원하는 다양한 테스트 자동화 도구의 등장과 조직의 테스트
자동화 성숙도 향상으로 인해 테스트 자동화는 지속적으로 고려되고 활용되고 있음

Recommendation#1
Shift-Left your Tests

Shift-Left Testing with Virtualization

How Long do you Wait?
Wait time
Before Service
Virtualization
On-demand (no wait) 0%
Seconds to minutes 0%
30 minutes to 2 hours 0%
4 to 7 hours 2%
1 day 1%
2 days 1%
3 days 9%
4 days 3%
1 week 8%
2 weeks 15%
3 weeks 27%
1 month 14%
2 months 10%
3 months 5%
4 to 6 months 3%
Never (no access ever) 2%
Source: voke Market SnapshotTM Report: Service Virtualization – January 2015
81% 84%
QA work delayed while waiting
32 days
Average wait time
76%Wait at least 2weeks or more
on systems
Development work delayed
while waiting

Without virtualization, testing is incomplete
DATA CENTER
Local Virtual Users
• Load generators
• Local switch
Application
under test
Hi-Speed LAN
Hi-Speed WiFi
Tablets
Smartphones
LOCAL TEST ENVIRONMENT
Branch office
Residential
Mobile
anywhere
Road warrior
Mobile
workforce
SITES
DEVICES
FIRE
WALL
NETWORK
APPS
Business
systems
Online B2C Mobile
Corporate
Mobile
Consumer
Mobile
Consumer
Reality is rarely local. Why should your test be?

What is Lifecycle Virtualization?
Mainframe
Storage
DB servers
Web services
App
servers
Web
servers
Load
balancers
Mobile components
Major ISP
3rd party
cloud services
Content
delivery
networks
Mobile carriers
& ISPs
Browsers &
devices
Customers
Distributed
services
Employees
User Virtualization
UV
UV
UV
Service Virtualization
SV
SV
SV SV SV
SV
SV
SV
Network Virtualization
NV
NV
NV
NV
NV
NV
NV
NV
NV
NV
NV
NV
NV
NV
Data Virtualization
DV
DVDV
DV
DV
DV
DV
DV
DV
DV
Understanding the complexity of composite applications, and challenges

Service Virtualization
Application
Backend
Mainframe
CRM
3rd Party
Payment
REST
SV
Virtual
Claim
Service
Simulated
Transactions
Development/Design
Documentation

Performance Insights
Network Virtualization
Application
Backend
Mainframe
CRM
3rd Party
Payment
SV
Virtual
MF
Virtual
CRM
Virtual
Pay
Mobile App
Desktop App
Web App
NV
Network
Condit.
WiFi
3G4G
VPN
2.5G
21
3
100K Users
Load Generators
Virtual
users
Performance
Testing Tool

A Real-Life Scenario
Customer
login
Add to
cart
Checkout Shipping
detail
Delivery
tracking
Shopping app
(web or mobile)
Purchase
received
A typical online purchase has predictable landmarks in its process
But… the road from beginning to end is hardly
a direct route
Tracking the course of an online purchase

Shopping app
(web or mobile)
Database
activity
Security
validation
Inventory
query
Database
activity
3rd party ship
service
Logistical
services
Acc’t
info
3rd party
database
3rd party
reporting
Purchase
receivedAdd to
cart
Shipping
detail
Delivery
tracking
Customer
login
Security
validation
3rd party
payment
services
Account
authorization
Checkout
But… the road from beginning to end is hardly
a direct route

Shopping app
(web or mobile)
Database
activity
Security
validation
Inventory
query
Database
activity
3rd party ship
service
Logistical
services
Acc’t
info
3rd party
database
3rd party
reporting
Purchase
receivedAdd to
cart
Shipping
detail
Delivery
tracking
Customer
login
Security
validation
3rd party
payment
services
Account
authorization
Checkout
Network activity and external services
are completely integral to this
“simple” transaction
N N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
S
S S
S
S
S
S
S
S
S
S

Wait for SAP
availibility
Coordinate
remote LG for the
test
Pilot release
modules 1 and 2
Field test
Lifecycle Virtualization delivers velocity
Dev Module 1
Dev Module 2
Test 1 / 2 Perf Test 1 / 2 Release
Wait for Module 1 to be
complete
Wait for Module 1 and 2
to be complete
A typical application delivery schedule involves a lot of delays…
using Module 1 virtual svcs
with
real world conditions
Monitor User Experience
Virtualization enables faster releases and
higher quality
Accelerating the Process

Lifecycle Virtualization Results
400+% ROI50%
reduction in application
transaction response time
10X
improvements in mobile
transaction times
90
50-90%
reduction in wait time
5X
decreased effort to test
Less than
30%
faster application delivery
45%
increase in test coverage and
cycles in same time
EFFICIENCYPERFORMANCE VALUE
day break even
60%
cost savings due to
faster simulation of
services
Client Results are Unbeatable

Recommendation#2
MeasureReal User Experiences

Deliver amazing user experiences
MODERN APPLICATION DEVELOPMENT
Reduce costs
Increase customer attraction/retention
Increase the value of your brand
Get to market faster
The new style of IT
A data explosion in 2015…
* Source: “Enterprise Mobile Facts You Need to Know in 2015” by App Data Room.
http://appdataroom.com/enterprise-mobile-facts-need-know-2015/
Mobile
Cloud
45%
of mobile enterprise app initiatives
will be delayed or go over budget
due to infrastructure difficulties
25%
IT software budget dedicated to
mobile app development,
deployment and management
by 2017
35%
of large enterprises will use
mobile app development platforms
1 in 5
of these will be specifically
for enterprise use
381 million tablets
will be sold worldwide
Dev Ops
Agile

MODERN APPLICATION DEVELOPMENT
The new style of IT
Networks
Data center
moves
ERP
E2E testing
Agile
Composite apps
Security Services
Lost brand
reputation
Lost customers
Lost revenue
Lost competitive
advantage
Reduce costs
Increase customer attraction/retention
Increase the value of your brand
Get to market faster
Complicated, unavailable
or expensive testing
Deliver amazing user experiences
Poor quality
release
Poor
performance
Delayed release
schedule

Challenges in Testing Mobile

Database
activity
Security
validation
Inventory
query
Database
activity
3rd party ship
service
Logistical
services
Acc’t
info
3rd party
database
3rd party
reporting
Security
validation
3rd party
payment services
Account
authorization
Shopping app
(web or mobile)
Purchase
receivedAdd to
cart
Shipping
detail
Delivery
tracking
Customer
login
Checkout
Welcome to the Jungle
How does it look
on the screen?
What’s happening
with CPU / MEM?
OS’s?
Browsers?
What
devices?
Do I have
the right
transactions?
Where are my users
coming from?
What network
connection?
What
happens
offline or
unavailable?
Was the experience
within brand?

Social media for the feedback?

Measure Real User Experiences
Devices and
Operating Systems
Application Stability
Crash Analytics
User Flows
Resource
Consumption

Measure Focus Improve
The winning strategy for a 5-star app!

What customers are saying about HP AppPulse Mobile
“HP AppPulse Mobile
has already given us
valuable information in
QA environments. The
tool UI was self-
explanatory with
necessary graphs
and statistics.”
IT specialist, Standard Bank of South Africa
TVID: 369-806-BA5
“Using the FunDex scoring system in the new HP
AppPulse Mobile will allow our team to quickly
get to problematic areas and improve the
adoption of our app.”
Executive, Globe Testing
TVID: 50D-DOB-434
“86% of surveyed IT organizations would recommend HP
AppPulse Mobile because it captures application crash
details along with the mobile app user action that caused it.”
Survey of 7 users of HP software application performance management
TVID: C2C-C86-A0A
“HP AppPulse Mobile assisted us to identify performance
concerns early in development, saving us costs, and is now
part of our build process to improve overall quality.”
IT manager, large enterprise banking company
TVID: 50D-DOB-434

Summary
Budget
Centralization
SMAC Technologies
Agile & DevOps
Automation
Shift-Left Testing
Real UX Measurement
Deliver Application Faster
without Compromising Quality

56
ThankYou
김기배 차장 (kibae@hp.com)
Business Development Manager
HP Software Asia Pacific and Japan

소프트웨어 QA 및 테스팅 트렌드

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to 소프트웨어 QA 및 테스팅 트렌드

Similar to 소프트웨어 QA 및 테스팅 트렌드 (20)

Recently uploaded

Recently uploaded (20)

소프트웨어 QA 및 테스팅 트렌드