Learn how to consider Artificial Intelligence as augmentation, to enhance your work. In this presentation we cover augmentation, cyborgs and critically appraise examples of #AI in #MedEd. We then discuss faculty development and can #AI be an #instructionaldesinger.

1. Cyborgs and Augmentation:
Unpacking AI in Health Care
and Medical Education
Janet Corral, PhD
Bruyère Research Institute, University of Ottawa
COO, Enlighten Strategies

2. What do you most hope
to get out of today’s
session?

3. To impact learning outcomes, we must apply
critical appraisal of what AI is and isn’t able to do
The law of shiny objects will lead education astray
AI offers great opportunity – yet we need to
unpack what our relationship is to AI as
augmented cyborgs
Take-home Messages

4. Part I: Cyborgs
Artificial
Intelligence as a
Cognitive Partner

5. What is a cyborg?
Cyborgs are spread
across the brain & non-
biological circuitry
- Andy Clark (2003)

6. Cyborg Examples
Computes math
problems so humans
do not need to
calculate all the
Stores phone
numbers so humans
remember contacts
by name
Measures water and
coffee so humans do
other morning
activities

7. Opportunity International: AI for Farmers
Case Study
Bhahati, a farmer, received
immediate answers from All
N’Gizi AI App.
Transcript: https://otter.ai/u/QWyuJgWWQc-x1_TKOlsNC66NmIU?message=dae07cf3-c892-4fec-b098-
580ecd16419b&tab=chat&utm_content=wrapup_v1&utm_source=va_chat
https://opportunity.org/

8. Cyborg Examples in Medical Education

9. Where do we allow cyborgs in med ed?

12. Your turn!
Break into a small group to answer:
Give 1 example of where you are a
cyborg. Discuss how good are you
at being a cyborg!

13. Debrief as a Large Group

14. Part II:
Augmentation

16. How well does augmentation work?
Three corollaries:
Charles P. Friedman, A “Fundamental Theorem” of Biomedical Informatics, Journal of the American Medical Informatics Association, Volume 16,
Issue 2, March 2009, Pages 169–170, https://doi.org/10.1197/jamia.M3092
1. This is more
about people
than technology
2. Resources must be
informative in addition to
being correct
3. Augmentation can fail.
We need to understand the
interaction between the
person and the resource

17. Quick Service Restaurants (QSR):
Case Study
11% drop in employees from 2018 to 2020
Turnover is real!
Management: 70.6% turnover in 2020
Frontline staff: 169.9% turnover in 2020
Source: https://www.intel.com/content/www/us/en/customer-spotlight/stories/lees-famous-recipe-chicken-
customer-story.html
Problems for Lee’s Famous Chicken:
- Chuck Doran, Franchise Owner
“The customers don’t care that
you’re short handed. I want my
food, and I want it in an acceptable
time frame, and I want it to be right.”

18. What can AI do well?

21. ChatGPT in Medical Education
Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, Chartash D (2023). How Does ChatGPT Perform on the United States
Medical Licensing Examination? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Med
Educ 2023;9:e45312 doi: 10.2196/45312
“By performing at a greater than 60% threshold on the NBME-Free-Step-
1 data set, we show that the model achieves the equivalent of a passing
score for a third-year medical student. ”
“Of the 4 data sets, AMBOSS-Step1, AMBOSS-Step2, NBME-Free-Step1,
and NBME-Free-Step2, ChatGPT achieved accuracies of 44% (44/100), 42%
(42/100), 64.4% (56/87), and 57.8% (59/102), respectively. ”

23. model-based interpretation
mathematical propagation of influence
identification of interactions between features
use of prior knowledge for transparent models

25. What can’t AI do?

26. Knowledge
Cutoff
• Where the LLM knows things about the world
only up to a certain moment in time, when it was
trained, or when text from the Internet was last
downloaded for the LLM's training.

27. Knowledge Cutoff – Examples for AI trained in 2022
Answer LLM 1:
“Avatar The Way of Water”
Answer from LLM 2:
I'm unable to provide real-time information or data
beyond my last knowledge update in January
2022. To find out the top-grossing movie of 2022,
I recommend checking a reliable source such as a
movie industry website, box office reporting
service, or a popular movie database like IMDb or
Box Office Mojo. They regularly update their
databases with the latest box office figures and
Prompt to LLM:
“What is the top grossing
movie of 2022?”

28. Hallucinations
When AI just makes something up!
https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-sanctions.html

29. - George Box, statistician (1919-2013)
“All models are wrong, but some are useful.”

30. Your turn!
Break into a small group to
answer:
What metrics would you use to
critically appraise AI?

31. Debrief as a Large Group

32. Part III:
Using AI

33. Job losses, AI-enabled advantage
• 59% more layoffs in tech in 2023 compared to 2022 (Layoffs.fyi; April 17, 2024)
AI employment gaps
Not Using AI Using AI
Gaps likely to
emerge between
AI-enabled workers
and those not
using AI

34. Amit Rawal, Global Product Leader for AI/ML Decision
Science & Analytics, Apple
“Being scared about something and not doing anything about it
doesn't really help…There will be people who will be best-in-class at
prompt engineering. You don't need a degree for that, but the value
they'll be able to get out of the same tool that everyone has access
to will make them seem like a genius.”

35. Clinician of the Future 2023: Education Edition survey
Conducted & Self-reported by Elsevier
Full report: https://tinyurl.com/CotFe2023
n=2,212 students from 91 countries April and May 2023

36. Year 1 MD Students: Using AI for Research Data
Before Class:
1. Download data from
Kaggle
2. Set up logins for: a)
ChatGPT and b) Claude
In Class, assemble into teams to
conduct a series of activities and
experiments with ChatGPT and
Claude.
All-class debrief:
1. What worked?
2. What didn’t?
3. Would you use AI as
your research assistant?

37. Academic AI
Tools and resources for faster research
• elicit.org
• SCITE.ai
• ResearchRabbit
• Consensus
• LitMaps
• Jasper

38. Medical Education Examples
Mentions do not imply endorsement
• Clinical Keys by Elsevier
• Chatbots for faculty development
• Write Step 1 questions, assessment questions
• First draft of a written assignment
• Provide feedback
• E.g. Night on Call - see Dr Tavinder Ark
Boscardin, C. K., Gin, B., Golde, P. B., & Hauer, K. E. (2024). ChatGPT and generative artificial intelligence for medical education: potential impact and opportunity. Academic Medicine, 99(1), 22-27.
Corral, J. (2021). Artificially intelligent chatbots for health professions education. In Digital Innovations in Healthcare Education and Training (pp. 127-135). Academic Press.
Gin, B. C., Ten Cate, O., O’Sullivan, P. S., & Boscardin, C. (2024). Assessing supervisor versus trainee viewpoints of entrustment through cognitive and affective lenses: an artificial intelligence investigation of bias
Tolsgaard, M. G., Boscardin, C. K., Park, Y. S., Cuddy, M. M., & Sebok-Syer, S. S. (2020). The role of data science and machine learning in Health Professions Education: practical applications, theoretical contribut
Gordon, M., Daniel, M., Ajiboye, A., Uraiby, H., Xu, N. Y., Bartlett, R., ... & Thammasitboon, S. (2024). A scoping review of artificial intelligence in medical education: BEME Guide No. 84. Medical Teacher, 1-25.

39. Can AI be used for curriculum mapping?

40. Your turn!
Break into a small group to
answer:
What activities might work for
your learners to explore the
possibilities of AI?

41. Debrief as a Large Group

42. Part IV:
AI as my med ed
cognitive partner

43. Where do I as an educator add value?
https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/

44. Can AI be my instructional designer?
� Step 1: ChatGPT, write the
content for the APHMG
competency “Compare and
contrast the organization of the
nuclear and mitochondrial
genome, including approximate
number of nuclear and
mitochondrial genes, how
nuclear DNA is packaged into
chromatin and the process of
replication.”
� Step 2: ChatGPT, turn
the content into a script
� Step 3: InVideo.AI, turn
the script into a video

45. Your turn!
Ask ChatGPT or Claude or Co-Pilot
to be the best instructional
designer and write:
1. A lesson plan (you pick the
topic) with 3 learning objectives
2. Write 3 multiple choice
questions based on those
learning objectives

46. Debrief as a Large Group