IT Operations people have several distinct challenges than software developers. Ops teams must balance large workloads and emergency tasks with assuring a stable infrastructure. Their workflow is continuous, and many times doesn't work well with timeboxes. Kanban is widely used to address the needs of IT Operations.
Cybersecurity Awareness Training Presentation v2024.03
KANBAN FOR IT OPS (DRAGOS DUMITRIU) - LKCE13
1. Kanban in IT Operations
Focus on delivering value and success will follow.
Dragos Dumitriu
Lean Kanban Inc.
2.
3. In IT From The Bottom
I am not the smartest or most talented person in the world, but I succeeded because I
keep going, and going, and going. - Sylvester Stallone
4. Keep Going at IT
I am not the smartest or most talented person in the world, but I succeeded because I
keep going, and going, and going. - Sylvester Stallone
5. Keep Going at IT
I am not the smartest or most talented person in the world, but I succeeded because I
keep going, and going, and going. - Sylvester Stallone
6. Made IT
I am not the smartest or most talented person in the world, but I succeeded because I
keep going, and going, and going. - Sylvester Stallone
7. Is That IT?
Deloitte’s Center for the Edge studies: the life expectancy of a firm in the Fortune 500
has declined from around 75 years half a century ago to less than 15 years and is
heading towards 5 years.
The pointlessness that people see in their jobs is an accurate reflection of the
deteriorating condition of the firms they work for.
When those doing the work are dispirited, it is inevitable that customers too will be
frustrated and that the firm will not prosper.
These problems cannot be solved by job redesign or clearer career paths. Instead, the
organizations must seek change with a focus on delighting the customer through
continuous innovation and that can only be delivered by engaged and motivated
employees.
Let’s do something about IT!
8. “Everything that can be counted does not necessarily count; everything that
counts cannot necessarily be counted.“ – Einstein
10. Where We Started
Triage
Ready
(5N, 4W)
Active
(3)
Validate
Resolved Monitor
(3)
Close
Major Incident
Normal Incidents
Explicit Criteria
Priority
Problem
M Crit
BU Crit
Not Crit
Weekend
… MI Review
Trend Analysis
11. Where We Started
Triage
Ready
(5N, 4W)
Active
(3)
Validate
Resolved Monitor
(3)
Close
Major Incident
Normal Incidents
Explicit Criteria
Priority
Problem
M Crit
BU Crit
Not Crit
Weekend
… MI Review
Trend Analysis
12. How We Treated Problems
Triage
Ready
(5N, 4W)
Active
(3)
Validate
Resolved Monitor
(3)
Close
Major Incident
Normal Incidents
Explicit Criteria
Priority
Problem
M Crit
BU Crit
Not Crit
Weekend
… MI Review
Trend Analysis
13. How We Treated Problems
Triage
Ready
(5N, 4W)
Active
(3)
Validate
Resolved Monitor
(3)
Close
Major Incident
Normal Incidents
Explicit Criteria
Priority
Problem
M Crit
BU Crit
Not Crit
Weekend
… MI Review
Trend Analysis
14. How We Treated Problems
Triage
Ready
(5N, 4W)
Active
(3)
Validate
Resolved Monitor
(3)
Close
Major Incident
Normal Incidents
Explicit Criteria
Priority
Problem
M Crit
BU Crit
Not Crit
Weekend
… MI Review
Trend Analysis
15. How Problems Treated Us
Triage
Ready
(5N, 4W)
Active
(3)
Validate
Resolved Monitor
(3)
Close
Major Incident
Normal Incidents
Explicit Criteria
Priority
Problem
M Crit
BU Crit
Not Crit
Weekend
… MI Review
Trend Analysis
16. M
Evolutionary Change
Triage
Ready
(BC)
Active
(BC)
Validate
Resolved Monitor
(BC)
Close
Major Incident
Normal Incidents
Tier 1 – 40% (75%)
Tier 2 – 45% (20%)
Tier 3 – 15% ( 5%)
D:C 400%
(110%)
Msg & Comms
Tier 2 – 50% A, TT1
Tier 3 – 60% A, TT2
Infrastructure
Tier 2 – 10%
Tier 3 – 20%
Priority
Explicit Criteria
Problem
M Crit
BU Crit
Not Crit
Weekend
Asset Prot
Business Apps
Tier 2 – 10%
Tier 3 – 10%
Asset Protection
Tier 2 – 30% A, TT1
Tier 3 – 10%
… MI Review
Trend Analysis
17. M
Evolutionary Metrics
Location
Country
City
Support volume
Average support incidents received per month
% incidents/requests submitted via phone
% incidents/requests submitted via email
Basic Metrics
Philippines
Manila
4942
15%
80%
% incidents/requests submitted via webchat (Lync)
5%
Service provided
Hours of support (eg 8 x 5, 24 x 7)
Language(s) supported
% of incidents/requests resolved by help desk
24x7
English
73%
% of incidents/requests assigned to local office support
12%
% of incidents/requests assigned to other support teams
15%
$
7.71
$
Average GHD cost/per employee/per month
6.04
Average tickets/per person/per month
380
Customer Satisfaction
92% of 100%
SLA
Average cost per incident:
Help desk Team Structure
# Total FTEs (US)
US contractors
Manila
# primarily focused on direct end user support
0.5
1
13
13
# primarily focused on management/supervision (US)
Tools
What tool is used for incident management
including assigning incidents to other support
groups?
What tool is used as a knowledge base?
What tool is used for self-service portal (if
available)?
0.5
# primarily focused on other supporting roles (Manila)
Please provide details of SLAs that apply to
help desk
1
Costs per Month (in thousands)
Total cost of team
Payroll costs
Other people related costs
Facility/technology costs
Other costs
$
38,083.00
$
33,749.00
$
$
3,751.00
$
583.00
Any other important tools used?
P1 - 4 Hours
P2 - 8 Hours
P3 - 24 Hours
P4 - 36 Hours
P5 - 72 Hours
SCSM
Sharepoint
SCSM will be used for Self-Help
Portal in Q4
Lync, Exchange Management
Console, Active Directory,
Alerts Tool, HR Reports
(Headcount, Terminations,
etc), SCOM, SharePoint,
Microsoft Office, Vovici
(customer satisfaction tool),
Asset Management Tool,
Inventory Tool (laptop
compliance).
Comments
Any other comments on requirements,
current services, customer satisfaction etc.?
Customer Satisfaction goal
must exceed 80% (actual is
92%).
18. M
Boring and Important: Budget
Fully
# of
Loaded
Vacation
hours per
Hourly
Hours
year
Rate
$ 75.39
1040
N/A
$ 26.00
2080
$ 22.71
520
0
$ 14.76
2080
120
$ 10.94
2080
120
$ 10.94
2080
120
$ 10.94
2080
120
$ 10.94
2080
120
$ 8.34
2080
120
Sick
Hours
N/A
0
40
40
40
40
40
40
# of
Training
Hours Per
Hours
Year
N/A
N/A
0
60
60
60
60
60
60
1040
2080
520
1860
1860
1860
1860
1860
1860
Number in
Australia - 1300
China - Toll Free (800) Low Volume
Malaysia - Toll Free
Singapore - Toll Free - Low
Belguim - National
Denmark - National
Finland - Helsinki
France - National
Germany - Munich
Netherlands - National
Norway - National
Spain - National
Swedent - National (10)
Switzerland - Zurich
United Kingdom - National 0870
Total Monthly
FY11 Cost
FY12 Cost
Annual Savings
FY12 Extra Minutes Overage
Total
Sub Total
$
$
$
$
$
$
$
$
$
78,405.60
54,080.00
11,809.20
27,453.60
20,348.40
20,348.40
20,348.40
20,348.40
15,512.40
Ringing to Plan
USA
USA
USA
USA
USA
USA
USA
USA
USA
USA
USA
USA
USA
USA
USA
250
125
250
250
250
250
250
250
250
250
250
250
250
250
250
Monthly Price
$
59.00
$
59.00
$
39.00
$
59.00
$
19.00
$
19.00
$
19.00
$
19.00
$
19.00
$
19.00
$
19.00
$
19.00
$
19.00
$
19.00
$
19.00
$
425.00
$
1,700.00
$
5,100.00
$
$
Prepaid Annual
$
44.00
$
44.00
$
34.00
$
44.00
$
14.00
$
14.00
$
14.00
$
14.00
$
14.00
$
14.00
$
14.00
$
14.00
$
14.00
$
14.00
$
14.00
$
320.00
$
1,280.00
$
3,840.00
$
1,260.00
Additional Minutes
$
0.27
$
0.47
$
0.15
$
0.20
$
0.60
$
0.60
$
0.60
$
0.60
$
0.60
$
0.60
$
0.60
$
0.60
$
0.60
$
0.60
$
0.60
2,400.00
7,500.00
N/A
N/A
N/A
Th-Sa 10p-10a
Th-Sa 10a-10p
M-Th 7p-5a
Su-W 9p-9a
Tu-Fr 11a-9p
W-Sa 10p-9a
Sunday
1
1
1
Monday
3
3
2
Tuesday
4
4
3
Wednesday
4
4
3
Thursday
4
3
3
Friday
2
2
2
Saturday # of Shifts/Week
1
19
1
18
1
15
Total
Cost per week Cost per Year
$399.00
$20,748.00
$60.30
$3,135.60
$50.25
$2,613.00
$509.55
$26,496.60
FY11 with 2 Gs
SEAAC
EALA
AMER
Sunday
1
1
1
Monday
4
4
2
Tuesday
5
5
3
Wednesday
5
5
3
Thursday
5
4
3
Friday
3
3
2
Saturday # of Shifts/Week
1
24
1
23
1
15
Total
Cost per week Cost per Year
$504.00
$26,208.00
$77.05
$4,006.60
$50.25
$2,613.00
$631.30
$32,827.60
FY12 with 2 Fs
SEAAC
EALA
AMER
Shift
FY 11
# of 1st Shifts
# of 2nd Shifts
# of 3rd Shifts
Sunday
1
1
1
Monday
5
4
3
Tuesday
6
5
4
Wednesday
6
5
4
Thursday
6
4
4
Friday
4
3
3
Saturday # of Shifts/Week
1
29
1
23
1
20
Total
Cost per week Cost per Year
$609.00
$31,668.00
$77.05
$4,006.60
$67.00
$3,484.00
$753.05
$39,158.60
Night
Transportat
Night
Meal
Differential
ion
Differential $ Allowance
Hours
Allowance
N/A
N/A
N/A
1236
0
1236
1236
0
1664
N/A
N/A
N/A
$1,824.34
$0.00
$1,352.18
$1,352.18
$0.00
$1,387.78
N/A
N/A
N/A
$544.19
$544.19
$725.58
$544.19
$725.58
$544.19
N/A
N/A
N/A
$634.88
$0.00
$725.58
$634.88
$0.00
$634.88
Sub Total
$
$
$
$3,003.41
$544.19
$2,803.35
$2,531.25
$725.58
$2,566.85
Monthly
Total
Yearly Total
$
$
$
$
$
$
$
$
$
78,405.60
54,080.00
11,809.20
30,457.01
20,892.59
23,151.75
22,879.65
21,073.98
18,079.25
$
$
$
$
$
$
$
$
$
6,533.80
4,506.67
984.10
2,538.08
1,741.05
1,929.31
1,906.64
1,756.17
1,506.60
19. Similar Results and Approach
M
180
160
140
120
Other
TTR
Sev 1
TTR
100
80
Cost
(x 100)
60
40
20
Productivity
0
FY04 Q4
FY05 Q1
FY05 Q2
FY05 Q3
FY05 Q4
FY06 Q1
FY06 Q2
20. The short story
Make work, workflow, and queue/wait times visible (Kanban)
Cross train people and focus on moving knowledge (first)
and work (second) from T3 to lower tiers
Put the pain where it belongs; resulted in jaw dropping reduction of
failure demand within weeks.
MI Reviews and Trend Analysis are tied to Service Team
rewards
80-300% boost to tiers capacity over 2 years, lower cost, higher
productivity
Establish Service Lines run by a Service Manager (ITIL v3)
Enterprise NOT Stakeholder prioritization based on
capacity, eliminate failure demand, shorten cycle time.
Automation, identification of training needs for IT and for customers
Established reputation for building the right thing, right
IT Org wins CIO’ Magazine Awards two years in a row.
21.
22. Conclusion
It’s not luck.
Making a person work and profiting from that work is
different.
It is not important to optimize locally, it is essential to
optimize flow through the organization from demand to
service consumption.
Reduce WIP to achieve
shorter cycle times,
increased output
reduced costs