Measures for assurance - Introduction and user group experience webinar Monday 29 April 2019 APM Assurance Specific Interest Group Presented by: Tim Podesta and Matt Bruce-Kingsmill The link to the write up page and resources of this webinar: https://www.apm.org.uk/news/measures-for-assurance-introduction-and-user-group-experience-webinar/

1. Measures for Assuring Projects
Webinar
Tim Podesta
MAPM, F.I Mech E.
Assurance SIG
Matt Bruce-Kingsmill
ChPP, FAPM MCMI.
Senior Project & Programme Manager Leidos

2. Measures for Assuring Projects
Introduction
Context
Creation
Example - Leidos
Q&A

3. 18 Oct 06
Three Lines of Defence Model for Assurance

4. Measures for Assuring Projects
Introduction
Context
Creation
Example - Leidos
Q&A

6. Measures for Assuring Projects : 10x10 Matrix

7. Category Acceptable (practices plus achievement)
Reactive
Poor (minimal practices/control)
score 7 if approaching good, otherwise score 6 score 5 if approaching acceptable, 4 if corrective
actions are being implemented but have not
taken effect yet, otherwise 3
Risks &
Opportunities
Up to date risk registers in place identifying
actions with monthly review cycle
Risk registers exist but may not be up to date
and risks not well articulated
Risk owners identified some coordinated action
and sharing of information
Not all risks allocated owners
In house core of expertise formally trained in
basic risk management skills
Variable availability of staff, limited or no training
Measures for Assuring Projects : Risk Extract

8. Measures for Assuring Projects
Introduction
Context
Creation
Example - Leidos
Q&A

9. LEIDOS / SCOTTISH GOVERNMENT CASE STUDY
APM MEASURES FOR ASSURING PROJECTS
Matt Bruce-Kingsmill (Leidos)
Ken Goodlad (Scottish Government)
Colin Napier (Leidos)
Andrew Fahey (Leidos)

10. Programme Leadership Team
Matt Bruce-Kingsmill Ken Goodlad Colin Napier
MiCase Delivery Manager MiCase SRO/Sponsor MiCase Technical
Assurance Advisor
Andrew Fahey
MiCase Development
Manager

11. Leidos - Critical Solutions for a Variety of Missions
A growing company delivering innovative technology and solutions focused on
safeguarding critical capabilities and transformation in frontline services, our work in the
United Kingdom includes addressing some of the most complex problems in defence,
healthcare, government, safety and security, and transportation.
Examples include managing and modernising critical infrastructures, applications and
business processes for clients such as the Scottish Government, Ministry of Justice,
NATS, National Health Service, NATO and the U.K. Ministry of Defence's £6.7 billion LCST
programme.
The success of this work in the U.K. and beyond shows that we are a trusted partner in the
region and that our people can deliver innovative solutions to solve the most challenging
problems.
https://youtu.be/2EO_WDJTPz4

12.  Customised software development to enable transformational business
change to how Scottish Government handle correspondence.
What is the MiCase Programme?

13. ➢ Governance requirement of funding award to produce and follow an SG Integrated
Assurance Plan.
➢ However the template process provided added very little value to the programme
delivery team, and we wanted to incorporate an assurance activity that would help our
delivery of the programme on a day to day basis.
Why we looked at the Toolkit

14. APM Assurance Toolkit Indicators
Assurance
Criterion
Best of breed (best practices
plus demonstrated
achievement)
Good (practices plus
achievement) proactive
Score 9 if approaching best of
breed, otherwise score 8.
Acceptable (practices plus
achievement) reactive
Score 7 if approaching good,
otherwise score 6.
Poor (minimal
practices/control)
Score 5 if approaching
acceptable, 4 if corrective
actions are being
implemented but have not
taken effect yet, otherwise 3.
Red
Score 2 if approaching poor, 1
if corrective actions are being
implemented but have not
taken effect yet, otherwise
score 0.
Clear scope aligned with
business need, deliverables,
schedule, acceptance
process and benefits
realisation.
Clear scope aligned with
business need, deliverables,
schedule, outline acceptance
process and benefits
realisation.
Scope aligned with business
need, deliverables, schedule,
benefits and acceptance
criteria are articulated but lack
clarity.
Scope lacks sufficient
definition for acceptance
criteria and for confident
implementation or benefit
realisation.
Scope does not contain
acceptance criteria or
sufficient detail for
implementation.
Clear sponsor and customer
obligations.
Clear high level sponsor and
customer obligations – some
ambiguity in detail.
Sponsor and customer
obligations apparent, but lack
clarity.
Sponsor and customer
obligations are vague and
benefits are not fully
articulated.
Sponsor and customer
obligations are not stated;
benefits are not articulated.
Recognised standard of
change control.
Change control process
essentially sound but may be
slow or fail to fully
communicate consequences
of change.
Change control in place, but
with some clear
shortcomings.
Informal or ineffective change
control.
No change control process.
Red, amber, green or blue
status and score for each
criterion
Risk and opportunity
management built in to day-to-
day business.
Up-to-date risk registers in
place, identifying actions, with
monthly review cycle.
Risk registers exist, but may
not be up-todate and risks not
well articulated.
No risk register or plan to
address risks.
Project management is risk All project team members Risk owners identified with Not all risks allocated owners. Risk management not
Client and
Scope
Risks and
Opportunities
Delivery Assurance Template

15. APM Assurance Toolkit Report
MiCase Pathway to Live
Ken Goodlad
Matt Bruce-Kingsmill
Avtar Virdee
Score Meets Indicator(s) Evidence Seen/Comments
1a
1b
1c
1
2a
2b
2c
2
3a
3b
3
Project/Programme:
Sponsor:
Delivery Manager:
Technical Architect:
Report Date:
Assessment Undertaken by:
Client and Scope
Risks and
Opportunities
Planning and
Scheduling
APM Assurance Toolkit Report
MiCase Pathway to Live
Ken Goodlad
Matt Bruce-Kingsmill
Avtar Virdee
Score Meets Indicator(s) Evidence Seen/Comments
1a 7
Scope aligned with business need, deliverables, schedule,
benefits and acceptance criteria are articulated but lack
clarity.
1b
1c
1
2a
2b
2c
2
3a
3b
3
Project/Programme:
Sponsor:
Delivery Manager:
Technical Architect:
Report Date:
Assessment Undertaken by:
Client and Scope
Risks and
Opportunities
Planning and
Scheduling

17. Report Date
1.ClientandScope
2.RisksandOpportunities
3.PlanningandScheduling
4.OrganisationalCapabilityandCulture
5.SupplyChain
6.Solution
7.Finance
8.SocialResponsibilityandSustainability
9.Performance
10.Governance
OverallAssuranceScore
11-Apr-2018 7.3 6 8 8 8 7 8 3 8 9 7.3
23-Mar-2018 7.3 3 7 8 8 8 9 3 7 9 6.9
12-Mar-2018 8 6 8 7 6 7 8 5 8 9 7.2
05-Feb-2018 8 6 8 7 6 6 9 5 8 8 7
15-Jan-2018 8 6 8 7 6 7 9 5 8 8 7.1
04-Dec-2017 7.7 5 8 7 6 7 8 5 7 7 6.8
06-Nov-2017 8 5 7 7 6 7 8 5 7 7 6.8
Latest Self Assessment Summary Status (From APM Assurance Toolkit)
Procedures have been standardised, documented, and communicated through
training and it has been mandated that the processes are to be followed.
Some significant deviations may not have been detected by the project team
and minor problems are occurring, but there is little risk of the project failing.
Documents are, at a minimum:
• agreed with the client/stakeholders;
• shared with and accessible to the project team;
• fit for purpose i.e. have a level of detail and structure that is in proportion to
the project.
N.B. Report dated 23/3/2018 represents scores given by Keith Nicholson as part of
independent assurance review and issue underpinning amber risk and opportunity
score has already been corrected.
Delivery Assurance Delivery
Assurance RAG
Report Date
1.ClientandScope
2.RisksandOpportunities
3.PlanningandScheduling
4.OrganisationalCapabilityandCulture
5.SupplyChain
6.Solution
7.Finance
8.SocialResponsibilityandSustainability
9.Performance
10.Governance
OverallAssuranceScore
14-Nov-2018 8 6 8 9 7 8 8 6 8 10 7.7
08-Oct-2018 8 6 8 9 7 7 8 6 8 10 7.7
04-Sep-2018 7.7 6 8 8 7 7 8 6 8 10 7.6
13-Aug-2018 7.7 6 8 8 5 7 6 6 8 10 7.2
29-Jun-2018 7.3 6 8 8 8 7 9 4 8 10 7.5
05-Jun-2018 7.3 6 8 8 8 7 9 4 8 10 7.5
09-May-2018 7.3 6 8 8 8 7 9 4 8 10 7.5
11-Apr-2018 7.3 6 8 8 8 7 8 3 8 9 7.3
23-Mar-2018 7.3 3 7 8 8 8 9 3 7 9 6.9
12-Mar-2018 8 6 8 7 6 7 8 5 8 9 7.2
05-Feb-2018 8 6 8 7 6 6 9 5 8 8 7
15-Jan-2018 8 6 8 7 6 7 9 5 8 8 7.1

18. Can be used across multiple project, programme and portfolio
layers

19.  assurance framework be to extended to cover technical
 Agile programme
 Been involved for a number of years
Technical Assurance

20. Tha sinn a’ lorg dòighean nas fhearr airson bathar-bog a leasachadh is sinne ris an aon rud is a’
cuideachadh daoine eile ’ga dhèanamh. Ri linn na h-obrach seo, tha sinn a-nis a’ cur luach sna
rudan a leanas:
 Daoine is dàimh seach pròiseasan is innealan
 Bathar-bog a tha ag obair seach docamaideadh mionaideach
 Co-obrachadh le custamairean seach rèiteachadh chùmhnantan
 Freagairt a thoirt air atharrachadh seach plana a leantainn
Ged a tha luach sna rudan air an taobh deas, tha sin ri ràdh gun cuir sinn luach nas motha sna
rudan air an taobh chlì.
https://agilemanifesto.org/
Aithris-rùn air leasachadh bathair-bhog sgiobalta

21. We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
 Individuals and interactions over processes and tools
 Working software over comprehensive documentation
 Customer collaboration over contract negotiation
 Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
https://agilemanifesto.org/
Manifesto for Agile Software Development

22.  Prioritises working software
 Welcome fluid requirements
 Frequent delivery of working software
 Continuous attention to technical excellence and good design
enhances agility
 At regular intervals, the team reflects on how to become more
effective, then tunes and adjusts its behavior accordingly
 Motivated individuals
 Face to face communication
Agile Manifesto Principles (abridged)

23. Basic Indicators

24.  Scope
 Risks and Opportunities
 Planning and Scheduling
 Solution
 Deployment
 Documentation
Added Relevant Criterion

25. Example: Scope

26. Example: Documentation

27. Report Date
1.Scope
2.RisksandOpportunities
3.PlanningandScheduling
4.Solution
5.Deployment
6.Documentation
OverallAssuranceScore
12-Apr-2018 7.3 6 7.5 6.3 8 2.3 6.2
03-Apr-2018 6 6 7 5.3 6 3.3 5.6
23-Mar-2018 8 5.5 7 8.8 7 2 6.4
13-Mar-2018 5 4 7 6 6 4 5.3
15-Feb-2018 5.3 3 7 5.5 5 4 5
29-Jan-2018 6.3 3.5 7 4.8 7 4.3 5.5
18-Jan-2018 6.7 5 7.5 5.8 8 5 6.3
13-Dec-2017 6.3 4.5 7.5 5.5 8 5 6.1
Latest Self Assessment Summary Status (From Leidos Agile
Technical Assurance Toolkit)
Procedures have been standardised, documented, and communicated
through training and it has been mandated that the processes are to
be followed. Some significant deviations may not have been detected
by the development team and minor problems are occurring, but there
is little risk of the project failing.
N.B:
1. As part of the continuous review of process Leidos felt that a
weakness in the Technical Assurance Toolkit report was recording
documents that were not due for completion as 10. The toolkit criteria
has been altered to mark documentation in that state as 5. As a result
scores in previous assessments have been retrospectively adjusted to
reflect this change of approach.
2. The report dated 23/3/2018 represents scores given by Keith
Nicholson as part of independent assurance review and issue
underpinning amber risk and opportunity score has already been
corrected.
Technical Assurance Technical
Assurance RAG
Report Date
1.Scope
2.RisksandOpportunities
3.PlanningandScheduling
4.Solution
5.Deployment
6.Documentation
OverallAssuranceScore
25-Sep-2018 8.3 7 8.5 7.5 8 5.3 7.4
04-Sep-2018 8.3 7 8.5 7.5 7 5.3 7.3
02-Aug-2018 8 7 8.5 7 6 5.3 7
18-Jun-2018 7.7 6.5 8.5 7 6 5.3 6.8
29-May-2018 7.7 6.5 7.5 6.8 7 2 6.3
07-May-2018 7.7 5.5 7.5 6.3 7 1.7 6
12-Apr-2018 7.3 6 7.5 6.3 8 2.3 6.2
03-Apr-2018 6 6 7 5.3 6 3.3 5.6
23-Mar-2018 8 5.5 7 8.8 7 2 6.4
13-Mar-2018 5 4 7 6 6 4 5.3
15-Feb-2018 5.3 3 7 5.5 5 4 5
29-Jan-2018 6.3 3.5 7 4.8 7 4.3 5.5

28.  Protected time focused on wider delivery picture
 Provide a repeatable & light touch approach for self assessment
 Could be tailored with relative ease
 Easy to rollup to different management layers
 Consistent and comparable regardless of who is undertaking the
review
Benefits of using the Toolkit

29. Questions ?

30. Measures for Assuring Projects
Webinar
Tim Podesta
MAPM, F.I Mech E.
Assurance SIG
Matt Bruce-Kingsmill
ChPP, FAPM MCMI
Senior Project & Programme Manager Leidos

31. This presentation was delivered
at an APM webinar
To find out more about upcoming webinars please
visit our website www.apm.org.uk/events