Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Professionalism assessment

Related Books

Free with a 30 day trial from Scribd

See all

Professionalism assessment

  1. 1. Assessment of ProfessionalismDr Dalal ALQahtani Session starts at 1.00 P.M.
  2. 2. Individual Expectations
  3. 3. Icebreaker Exercise What constitutes having/being professional? One word
  4. 4. AimExplore the features of professionalism and its assessment in health care profession
  5. 5. Objectives1. Define professionalism and its domains2. Recognize the trouble with assessing Professionalism3. List and explain some tools used in professionalism assessment4. Articulate the reasons behind the need of assessing professionalism
  6. 6. Professionalism is generally defined as the body of qualities or features characteristic of a profession Oxford English Dictionaries, 1993
  7. 7. These qualities include a high degree of skill and knowledge that is applied to the practice of work
  8. 8. The Accreditation Council on Graduate Medical Education; 2009
  9. 9. Patient Care Systems-based Medical Practice Knowledge Practice-based Learning andProfessionalism Improvement Interpersonal and Communication Skills
  10. 10. Demonstrate respect, compassion, and integrity; aresponsiveness to the needs of patients and societythat supersedes self-interest; accountability topatients, society, and the profession; and acommitment to excellence and on-going professionaldevelopment.
  11. 11. Demonstrate a commitment to ethical principlespertaining to provision or withholding of clinicalcare, confidentiality of patient information, informedconsent, and business practices
  12. 12. Demonstrate sensitivity and responsiveness to adiverse patient population, including, but not limitedto diversity in gender, age, culture, race, religion, anddisabilities
  13. 13. American Board of Internal Medicine. American professionalismin medicine: issues andopportunities. Definition and objectives.Available at: 15, 2003.
  14. 14. It is very important competenceThat means we need to Teach and Assess
  15. 15. Complex constructSecond order competence Context variable Attitude and behavior
  16. 16. Complex construct Attributes How to measure ?
  17. 17. Complex construct Attributes How to measure ?
  18. 18. Complex constructIt encompasses a variety ofskills, knowledge and other attributes andhave complex relationships to observedbehavior
  19. 19. The Elements of ProfessionalismBy the American Board of Internal Medicine But are these the only attributes of professionalism? NO
  20. 20. The Authors’ Classification of Themesand Subthemes, Arising From Definitionsor Interpretations of Professionalism Wilkinson et al;2009
  21. 21. So… We need toUnderstand the various dimensions/attributesof professionalism Because
  22. 22. Without a validated definition of this construct, assessment ofprofessionalism within medical education will be compromisedIt guides the selection of the educational material and provides thecriteria for any assessment system
  23. 23. Complex constructSecond order competence
  24. 24. Second order competence‘can be expressed only via the performance of othercompetences’ Verkerk MA et al;2007
  25. 25. Second order competence
  26. 26. Complex constructSecond order competence Context variable
  27. 27. Context variableVaries between contexts and individuals andincludes both normative and ideological aspectsIt should reflect both local and national contexts van Mook WNKA et al; 2009
  28. 28. Context variableProfessionalism, and the literature supporting it todate, has arisen from Anglo-Saxon countries.Caution should be used when transferring ideas toother contexts and cultures. International Ottawa Conference (Miami 2010)
  29. 29. Context variableProfessionalism is intrinsically related to the socialresponsibility of the medical profession.Thus, definition should reflect societal and healthcare changes, and this is an important responsibilityof the profession and its educational institutions tothe public. International Ottawa Conference (Miami 2010)
  30. 30. Context variable
  31. 31. Context variable A cross-sectional designThe findings point to a low perception among theparticipating nurses regarding their professionalism
  32. 32. Context variableWorkplacePersonal background of the nurses, which includesthe personal interest in the nursing professionFamilySociety
  33. 33. 10 Focus group What do you mean by professionalism?To what extent have you developed these qualities?
  34. 34.  Admitted were deficient in the acquisition ofprofessional values A Professionalism not taught or assessed Very few teachers as positive role models
  35. 35. WHY ? Negative role modeling by the faculty Deficiencies in the curriculum Limited interaction with health team Absence of feedback
  36. 36. Students Quote I think 40–60% of the students enter medical college with good professional qualities . . . unfortunately, they gradually lose their qualities because of the influence of teachers and senior students . . . they end up in behaving badly and aggressively . . . (FG-05)
  37. 37. Students Quote Feedback is totally missing in the system . . . Either from teacher to the student . . . or from student to the teacher . . . even when feedback is given to the faculty, they may not implement, . . . they may even victimize the students . . . it may be a good idea to take feedback anonymously . . . (FG- 10)
  38. 38. Context variable Ginsburg et al conducted interviews with 30 clinicians after the clinicians had watched five videotaped scenarios of professionally challenging situations They were asked what they thought students should and should not do in these situations, and they were also asked what they would do themselves !The authors found little agreement between clinicians. Ethical principles such ashonesty were defined differently across clinicians and within clinicians acrossdifferent scenariosSuggesting that dishonest behavior could be interpreted as unprofessional or notunprofessional depending on the context Ginsburg S et al; 2004
  39. 39. Context variable Ginsburg et al conducted interviews with 30 clinicians after the clinicians had watched five Society videotaped scenarios of professionally challenging situations They were asked what they thought students Between individuals should and should not do in these situations, and they were also asked what they would do themselves ! Within individualThe authors found little agreement between clinicians. Ethical principles such ashonesty were defined differently across clinicians and within clinicians acrossdifferent scenariosSuggesting that dishonest behavior could be interpreted as unprofessional or notunprofessional depending on the context Ginsburg S et al; 2004
  40. 40. Complex constructSecond order competence Context variable Attitude and behavior
  41. 41. ProfessionalismAttitude Behavior Mismatch
  42. 42. Mismatch Attitude BehaviorUnprofessional attitude professional behavior FakingProfessional attitude Unprofessional behavior Social pressure to behave in a particular way
  43. 43. MismatchAttitude Behavior As a result, we must be carful of making assumptions about students’ professionalism on the basis of observed behavior alone
  44. 44. MismatchAttitude Behavior Therefore , when devising methods of assessment; knowledge of the reasoning behind the action is also required Attitude
  45. 45. An attitude can be defined as “a favorable or unfavorable evaluate reaction toward something or someone, exhibited in one’s beliefs, feelings, or intended behavior Myers DG. Social Psychology. Boston: McGraw- Hill College; 1999.
  46. 46. Summative assessmentFormative assessmentConversation help students to reflectcritically on their behaviorsHelp students to develop their professionalbehaviors in the future
  47. 47. MismatchAttitude Behavior Cognitive dissonance theory suggests “when we behave in a manner contrary to our attitudes, we experience dissonance, which we seek to reduce, either by changing our attitudes to match our behaviors or vice versa.” Festinger L. A Theory of Cognitive Dissonance. Stanford, CA: StanfordUniversity Press; 1957.
  48. 48. Complex constructSecond order competence Context variable Attitude and behavior HOW?
  49. 49. Both are important
  50. 50. Professionalism assessmentWritten assessmentCompetency-based assessmentPerformance-based assessmentPortfolio
  51. 51. Written assessment Selected • MCQ response • Questionnaire Constructed • Essays response • Short answer questions
  52. 52. Written assessment Ideal to assess • Knowledge of the judicial, legislative and administrative processes and ethical principle • Reflective ability of medical students and junior doctors
  53. 53. Written assessmentQuestionnaires Questionnaires are often based on vignettes or clinical scenarios and may involve the description of critical incidents They show validity and reliability Boenink AD et al; 2005
  54. 54. Written assessmentCritical incident report This method asks the doctor to reflect on a critical incident he or she has experienced or witnessed
  55. 55. Written assessmentCritical incident report • It can encourage reflection and attention to elements of professionalism • But it is dependent on the type of incident to determine which aspect of professionalism is being assessed
  56. 56. Competency-based assessment It take place in controlled representations of professional practice such as standard patient encounters and objective structured, clinical examinations (OSCEs)
  57. 57. Competency-based assessment Useful component of a systematic approach to assessing professionalism, especially in the earlier stages of the curriculum
  58. 58. Competency-based assessment These have the advantage over written assessment in that they can be used to assess the ‘Showshow’ level of Miller’s pyramid
  59. 59. Competency-based assessment OSCE OSCEs are also seen to be fair, as each student carries out a standardized procedure during the assessment and have high degrees of reliability because each student is assessed by many different examiners and several cases
  60. 60. Competency-based assessment OSCE BUT • An artificial environment and therefore may not reflect actual day-to-day clinical performance •OSCEs are complex to organize
  61. 61. Competency-based assessment OSCE BUT There are problems with the interpretation of student behaviors by differing assessors, even when calibrated and well trained Mazor KM;2007
  62. 62. Performance-based assessment Are those that take place within the natural clinical setting and include work-based systems with direct observation of the student
  63. 63. Performance-based assessment • It measures the upper end, i.e. the ‘show’s how’ or even ‘does’, of Miller’s pyramid •Authentic •It enable professionalism to be assessed as a second- order competence
  64. 64. Performance-based assessment BUT It is time-consuming and requires well-trained observers and accurate criteria to work well
  65. 65. Performance-based assessment 360 feedback Also known as multi-rater feedback or multi source feedback (MSF) Assessment by faculty, nursing or other members of staff and by patients
  66. 66. Performance-based assessment P-MEX The Professionalism Mini-Evaluation Exercise (P-MEX) Modifications of mini-CEX
  67. 67. This tool is used to assess a 15- to 30-minute observedsnapshot of a doctor/patient interaction that is conductedwithin actual patient-care settings using real patients andthat has a structured marking sheet that covers predefinedgeneric areas
  68. 68. Assess four discrete areas : doctor–patient relationship skills,reflective skills, time management, and interprofessionalrelationship skills
  70. 70. PortfolioPortfolios provide evidenceof competence and progressionand may stimulate reflection
  71. 71. PortfolioSuitable for both formative and summativeassessment of complex and multifacetedskills and competences of professionalism
  72. 72. Portfolio
  73. 73. Wilkinson et al;2009 Review of Tools That Assess Elements of Professionalism
  74. 74. Wilkinson et al;2009 BlueprintingFor example : The blueprint demonstrates that direct observations (through the mini-CEX and P-MEX) and collated views (through MSF and patients’ opinions) are crucial elements because they capture many aspects in reliable, valid, and feasible ways
  75. 75. Wilkinson et al;2009 GapsAttributes that would not be well assessed using the current methods
  76. 76. 360 FEEDBACK P-MEX Portfolios Critical incidents OSCE Physician /patient assessments
  77. 77. Criteria For Evaluation Of Instruments To Assess ProfessionalismWhat is the sample size, location, and demographics?How are data recorded?How is it scored?Is it reliable?Is it valid?Were standards set/classifications made?What is the feasibility? Length, Cost?Do the data derived from the tool seem amenable tochange?Does the tool educate users about the construct beingassessed?What components of professionalism is this toolmeasuring?
  78. 78. How should it be?Less than half the articles retained by Jha etal. demonstrated reliability or validity.Many of the problems of reliability derivefrom the fact that the assessment tools havebeen developed for different purposes andin different circumstances, perhapsreflecting the varying contexts ofprofessionalism ! Jha et al;2007
  79. 79. How should it be? NO single method of assessment has yet emerged that is reliable and valid !Triangulation of multiple assessmentsby multiple assessors over time
  80. 80. Why professionalism assessment isimportant?
  81. 81. Ronald and Edward;2002
  82. 82. ProfessionalismIssues
  83. 83. Is professionalism astate or a trait? 1
  84. 84. Do you think the results ofprofessionalism assessmentcould change the attitude ? 2
  85. 85. Many medical educators hope that by constantly monitoring students’ professional behaviors, the students will eventually come to internalize appropriate attitudes Charlotte et al;2007Reflecting on the writings of Immanuel Kant, Sherman puts it another way:“decorum can, in some cases, change inner states” Sherman; 2005
  86. 86. What if someone ‘fails’ inprofessionalism, what are thepenalties of ‘failing’? Whatremedial measures do we have?Would medical schoolfail a student purelybased on professionalissues? 3
  87. 87. OtherIssuesDerived from literature
  88. 88.  Are the assessment tools looking for a pattern of behaviors, or a single behavior or a single incident? Are there tools for students to evaluate faculty professionalism? Is the purpose of the instrument to identify professionalism or unprofessional behaviors?• Are we assessing professionalism or personality?• What is the public’s view of professionalism?
  89. 89. Professionalism is a concept that varies acrosscultural contexts
  90. 90. Elements of professionalism are vast andinclude: individual(attributes, characteristics, attitudes, behaviours, identities), interpersonal (relations, groupdynamics, etc) and societal(economic, political, etc).
  91. 91. There is need to develop concrete andoperationalizable definitions, and from themeffective teaching methods and defensibleassessment approaches are designed
  92. 92. A true evaluation of professionalism mustfocus on the reasons for a behavior, ratherthan just the behavior itself
  93. 93. Triangulation of multiple kinds of measures, bymultiple observers, synthesized over time withdata gathered in multiple, complex andchallenging contexts is likely to be appropriateat all levels of analysis
  94. 94. While summative assessment isimportant, formative methods shouldpredominate
  95. 95. No single method exists for the reliable and validevaluation of professional behavior
  96. 96. The overall assessment program is more important thanthe individual tools. The best programs use a variety oftools in a safe climate, provide rich feedback, anonymity(when appropriate) and follow-up of behaviour changeover time
  97. 97. It may be more important to increase the depth andquality of reliability and validity of a program existingmeasures in various contexts than to continue todevelop new measures for single contexts
  99. 99. References• Shaw D. Ethics, professionalism and fitness to practise: three concepts, not one. Br Dent J 2009: 207: 59–62.• Irvine D. The performance of doctors. I: professionalism and self-regulation in a changing world. BMJ 1997: 314: 1540–1542.• van Mook WNKA, de Grave WS, Wass V, et al. Professionalism: evolution of the concept. Eur J Intern Med 2009: 20: e81–e84.• American Board of Internal Medicine. American professionalismin medicine: issues and opportunities. Definition and objectives.Available at: AccessedOctober 15, 2003.• Messick S. The interplay of evidence and consequences in the vali-dation of performance assessments. Educ Res 1994: 23: 13–23• Verkerk MA, De Bree MJ, Mourits MJE. Reflective professional-ism: interpreting CanMEDS’ ‘‘professionalism’’. J Med Ethics• 2007: 33: 663–666• Boenink AD, Jonge P, Small K, Oderwald A, Tilburg W. The effects of teaching medical professionalism by means of vignettes: an exploratory study. Med Teach 2005: 27: 429–432.• Mazor KM, Zanetti ML, Alper EJ, et al. Assessing professionalism in the context of an objective structured clinical examination: an in-depth study of the rating process. Med Educ 2007: 41: 331–340.
  100. 100. References•International Ottawa Conference (Miami 2010) ;Professionalism Theme WorkingGroup (IOc-PwG) ; Post-Conference Draft Recommendations and Revisions v6•Wilkinson TJ, Wade WB, Knock LD. A blueprint to assess professionalism: resultsof a systematic review. Acad Med. 2009 May;84(5):551–8.•Zijlstra-Shaw S, Robinson PG, Roberts T. Assessing professionalism within dentaleducation; the need for a definition. Eur J Dent Educ. 2012 Feb;16(1):e128–136.•Cruess R, McIlroy JH, Cruess S, Ginsburg S, Steinert Y. The Professionalism Mini-evaluation Exercise: a preliminary investigation. Acad Med. 2006 Oct;81(10Suppl):S74–78.•Rees CE, Knight LV. The trouble with assessing students’ professionalism:theoretical insights from sociocognitive psychology. Acad Med. 2007Jan;82(1):46–50.Ginsburg S, Regehr G, Lingard L. Basing the evaluation of professionalism onobservable behaviors: a cautionary tale. Acad Med. 2004; 79:S1–S4.
  • starlettetolver

    Jul. 24, 2021
  • ManjushaLitake

    Feb. 28, 2021

    Jul. 10, 2020
  • FirasTariqismaeel

    Dec. 27, 2017
  • ranjana96

    May. 19, 2017


Total views


On Slideshare


From embeds


Number of embeds