1.
The Value of Competency-based Medical EducationAcross the Continuum Eric S. Holmboe

2.
Conflicts of Interest• Employed by the American Board of Internal Medicine• Serve on the boards of the National Board of Medical Examiners and Medbiquitous (both non-profit)• Receive royalties from Mosby-Elsevier for a textbook on assessment 2

3.
Objectives for Today• Discuss the roles of assessment in a competency-based medical program• Develop approaches to help local educators effectively use performance-based assessment methods and tools• Discuss how performance-based assessment can be integrated into an education and assessment system for Memorial University. 3

4.
Nostalgialitis Imperfecta Syndrome characterized by the following signs and symptoms: – “When I was an student…<insert superlative>” – “Medicine was so much better 25 years ago” • Reality: Not really… – “Younger physicians today are less professional, skilled, etc. because of <insert favorite complaint>”

5.
Current Model of Training and Practice Retirement Competent

6.
Change in Performance Over Time Lower Performance All OutcomesChoudhry NK, Ann Intern Med, 2005;142:260-73

7.
With your immediate neighbors, discuss whatcompetency-based education and training means to you?

8.
Competency-Based Medical Education Is an outcomes-based approach to the design, implementation, assessment and evaluation of a medical education program using an organizing framework of competencies the unit of progression is mastery of specific knowledge, skills and attitudesFrank, JR, Snell LS, ten Cate O, et. al. Competency-based medicaleducation: theory to practice. Med Teach. 2010; 32: 638–645

9.
So What are the Outcomes and Who Determines Them? The Profession? The Public? Policy Makers?

10.
Determining Outcomes: Perspectives The Profession? – The “core” of a discipline? – Competence in the Can MEDS roles? Safe, effective, patient-centered care? The Public? – Trust that a doctor can do certain things? Policy Makers? – Meeting the needs of the complex and aging Canadian health population?

11.
Traditional versus Competency-based: Start with System NeedsFrenk J, et al. Health professionals for a new century: transformingeducation to strengthen health systems in an interdependent world.Lancet. 2010 11

12.
U.S. Institute of Medicine Competency Framework IOM, 2003

13.
Implications of CBME Curriculum and assessment follows from the competencies and outcomes, not vice versa Requires: – Definition of milestones of competency • What does competency look like? – Robust assessment methods, tools & systems

14.
Educational ProgramVariable Structure/Process Competency-basedDriving force: Content-knowledge Outcome-knowledge curriculum acquisition applicationDriving force: process Teacher LearnerPath of learning Hierarchical Non-hierarchical (Teacher→student) (Teacher↔student)Responsibility: content Teacher Student and TeacherGoal of educ. encounter Knowledge acquisition Knowledge applicationTypical assessment tool Single subject measure Multiple objective measuresAssessment tool Proxy Authentic (mimics real tasks of profession)Setting for evaluation Removed (gestalt) Direct observationEvaluation Norm-referenced Criterion-referencedTiming of assessment Emphasis on summative Emphasis on formativeProgram completion Fixed time Variable timeCarraccio, et al. 2002.

15.
Definitions and Frameworks: The Need for a Shared Understanding

16.
Competency / Competencies An observable ability of a health professional – Reflects a spectrum – Integrates multiple components such as knowledge, skills, values, and attitudes – Multiple competencies can be combined – Measureable with respect to a defined outcome© 2009 Royal College and The International CBME Collaborators

17.
Competent  Possessing the required abilities at a specified stage of medical education  Is always qualified by a frame of reference  …Dyscompetence  …Supracompetence© 2009 Royal College and The International CBME Collaborators

18.
Frameworks Webster’s New Collegiate Dictionary: – A skeletal or structural frame – A basic structure (as of ideas) – Frame of reference

19.
CanMEDS Framework

20.
KSA FrameworkK = KnowledgeS = Skill Information gathering skills – Interview, physical examination, communication Ability to use knowledge and information – Problem solving, clinical judgment Management skills – Diagnosis, treatment, patient education, counseling, procedural skillsA = Attitudes Professionalism, humanism

21.
RIME Model Developed at USUHS – Lou Pangaro and Gordon Noel in the 1980s for use in third year medical student clerkships in internal medicine “Synthetic” Model – Reporter – Interpreter – Manager – Educator

22.
Dreyfus Developmental Model• Novice – Don’t know what they don’t know• Advanced Beginner – Know what they don’t know• Competent – Able to perform the tasks and roles of the discipline – restricted breath and depth• Proficient – Consistent and efficient in performance of the tasks and roles of the discipline - know what they know and don’t know• Expert – In depth knowledge concerning the discipline – often rule based – know what they know• Master – Expert who relishes the unknown, or the situation that breaks the rules – who the experts go to for help – don’t know what they know 1 as presented by Leach, D., modified by Nasca, T.J. American Board of Internal Medicine Summer Retreat, August, 1999

23.
Competence Learning Performance is exhausting Sense of what’s important may be lacking Volume of various aspects can be overwhelming Competent model of decision making: – “is a detached, deliberative, and sometimes agonizing selection among alternatives” Dreyfus and Dreyfus, 1986

24.
Assessment Across the Continuum:Challenges and Principles

25.
Assessment Strategies in Undergraduate and Postgraduate Training

26.
Milestones Definition A significant point in development Merriam-Webster A scheduled event signifying the completion of a major deliverable or a set of related deliverables. mariosalaexandrou.com

27.
Milestones and Trajectories Milestones should enable the trainee, program and the regulatory bodies to know an individuals trajectory of competency acquisition. The focus is developmental

28.
Approaches to Developing Milestones 1 Discrete – Defining different behaviors in a domain of competence at each stage of training Continuous – A similar ability modified at each stage of training to reflect increasing complexity or sophistication – Parameters: • Setting, players, complexity, supervision 1 From Jason Frank, RCPSC 28

29.
Approaches to Developing Milestones 1 Narrative – Detailed descriptions of stages of development of competency by domain – Short essays and vignettes that describe a “competence story”  EPAs(entrustable professional activity) 1 From Jason Frank, RCPSC 29

30.
Patient CareACGME Developmental Milestones Approximate AssessmentCompetency Informing Time Frame Methods/Tools ACGME Competencies Trainee to Achieve StageClinical skills Historical Data Gathering Standardized and 1. Acquire accurate and relevant history 6 months patient reasoning from the patient in an efficiently Direct customized, prioritized, and Observation Manages hypothesis driven fashion Simulation patients using 2. Seek and obtain appropriate, verified, 9 months clinical skills of and prioritized data from secondary interviewing and sources (e.g. family, records, physical pharmacy) examination 3. Obtain relevant historical subtleties 18 months that inform and prioritize both differential diagnoses and diagnostic plans, including sensitive, complicated, and detailed information that may not often be volunteered by the patient Sub- competency

31.
Communicator Milestones Project Stage 1 Stage 2 Stage 3 Stage 4Greets the Demonstrates Demonstrates Demonstrates Demonstratespatient in a rule-based proficiency; proficiency; proficiency & way; some adapts to efficiency; Simulation or adaptation; many Across role play routine contexts; spectrum of clinical complex practice clinical An Introduction to CBME – Frank, Snell, Harris, Holmboe 2012 31

32.
Patient CareThe resident is demonstrating satisfactory development of the knowledge, skill and attitudes needed to advance in training.He/she is demonstrating a learning trajectory that anticipates the achievement of competency for unsupervised practice that includes the delivery of safe, timely, equitable patient-centered care. ____ Yes ____ No ____ Marginal 32

33.
Milestones Benefits Provide the learner with a clear path of progression. There are no surprises. Allow for rich formative feedback. Learners know where they are and where they need to go. Define specific behaviors that can focus assessment.

34.
Milestones Challenge Synthesizing milestones into larger global representations of competency that reflect those activities that define the profession. These activities have been described as entrustable professional activities or EPAs.

35.
Entrustable Professional Activities  EPAs represent the routine professional-life activities of physicians based on their specialty and subspecialty  The concept of “entrustable” means: – ‘‘a practitioner has demonstrated the necessary knowledge, skills and attitudes to be trusted to independently perform this activity.’’11 Ten Cate O, Scheele F. Competency-based postgraduatetraining: can we bridge the gap between theory andclinical practice? Acad Med. 2007; 82(6):542–547.

36.
Entrustable Trainee Activities ETAs, or entrustable resident or student activities, can help to define important benchmarks in a trainee’s development ETAs in a training program may mean: – A trainee has demonstrated the necessary knowledge, skills and attitudes to be trusted to perform this activity without constant or direct supervision.

37.
Why “ETAs” to Assess Competence? Sampling of events that: - are critical moments in medical training - inform developmental progression - faculty and leaders already implicitly assess - are manageable for busy training programs – are logical of assessment for stakeholders Supported by generalizability theory – 8-12 focused assessments can potentially allow a generalized statement of competency

38.
Synthesize to Physician trusted to meet Analyze to Educate and the health care needs of Understand Evaluate the population Competency Milestones LANDMARK in EPA in Training Practice Medical MK1 Knowledge MK2 Lead a resident Lead a health care team care team Patient Care PC1 PC2Professionalism Prof1 Prof2 Care for clinic patients with Practice Interpersonal ISC1 distance Skills independently ISC2 supervisionSystems-based SBP1 Practice SBP2 Complete an audit Lead Quality of a panel of ImprovementPractice-based PBLI1 clinic patients initiative learning PBLI2 Shared Mental Models and Frameworks

39.
Entrustment Trajectories

40.
Entrustments in Newfoundland With a neighbor(s), discuss an entrustment you make either with medical students or post-graduate trainees How do you arrive at this entrustment judgment?

41.
Questions?Contact Informationeholmboe@abim.org