Presented at LILAC 2018

1. Bridging information literacy gaps using
a customised information literacy
framework developed for medical
undergraduates
LILAC Conference, 2018
(April 4-6 2018)
Rebecca David, Senior Assistant Manager
Caroline Pang, Director

2. Overview
1. Our Student Services
2. Observations
3. IL Intervention
a) Customised Information Literacy
framework for LKCMedicine Students (Year
1 -5)
4. IL Intervention Tool
a) PICO & Search Strategy Worksheet
b) Year 4 Scholarly Briefing Session
5. Data Collection & Analysis
6. Conclusion
2

3. Our Student
Services
Student Consultation Sessions
Background
▪ In 2016, our Year 4 students were
the pioneer batch to pursue their
Scholarly Project (6 weeks)
▪ Goal: Assist students to manage
their literature search, writing
styles and citations.
Setup
▪ Developed Medical Library blog
- Service Information
- Student Booking Forms
▪ Email Replies 3
Our Student Services

4. Student Consultation Sessions
Process
4
Our Student Services
Typically
30 minutes
Discussion
Teaching
Student
follow-up
Medical
librarian
Medical
student
OBSERVATIONS

5. Observations
5
Student
consultation
sessions
(Observations)
Strengths
IL skill
gapsTime
• scoping their search terms
• using MeSH browser
• developing their PICO question
• conducting advanced search on
database
• importing citations from database to
EndNote
• Instead of 30 minutes,
each session lasted
45minutes to 1 hour
• describe their research topic clearly
• breakdown their research topic and
identify the keywords
• present at least 2-3 articles using
Google or Google Scholar as a
discussion tool
(2) using databases to craft search
strategies and find relevant articles
2 critical IL gaps
(1) using the PICO (Patient, Intervention,
Comparison and Outcome) framework
to scope their search

6. Process
6
Our Student Services
Typically 30
minutes
Discussion
Teaching
Student
follow-up
Medical
librarian
Medical
student
OBSERVATIONS
(1) using the PICO (Patient, Intervention, Comparison and
Outcome) framework to scope their search
(2) using databases to craft search strategies and find
relevant articles
2 critical IL gaps
Developed a
customised IL
framework
for
LKCMedicine
students
Benchmarks (selected):
ACRL IL standards for Higher Education LKCMedicine curriculum
IL practices from top 3 medical schools (World) Gold standards framework Library
Intervention

7. Association of College & Research Libraries (ACRL) Standards1
The medical library framework anchors on the ‘Information Literacy
Competency Standards for Higher Education’ developed and reviewed by
the Association of College & Research Libraries (ACRL). The five standards
developed by the ACRL, 2000 cover:
 Standard 1: The information literate student determines the nature and
extent of the information needed.
 Standard 2: The information literate student accesses needed
information effectively and efficiently.
 Standard 3: The information literate student evaluates information and
its sources critically and incorporates selected information into his or
her knowledge base and value system.
 Standard 4: The information literate student, individually or as a
member of a group, uses information effectively to accomplish a
specific purpose.
 Standard 5: The information literate student understands many of the
economic, legal, and social issues surrounding the use of information
and accesses and uses information ethically and legally.
The medical library learning objective and outcomes framework for
medical students’ aligns to Standard 1, 2, 3, 4 and 5. The medical library’s
learning objectives and outcomes are tailored to meet information literacy
skills required at a ‘foundational level’ using ‘teachable moments’ during
their course of study.
LKC, School of Medicine2
Curriculum Standards
(Aligned to the Imperial
College London – Faculty of
Medicine, MBBS programme)
 Years 1 and 2: Integrated
Science in a Medical
Context
 Year 3: Core Clinical
Medicine in Practice
 Year 4: Medicine for All
Stages of Life & in
Diverse Settings
 Year 5: Preparing for
Practice as a Doctor
Top 3 Medical Schools in
the world (2016)
Medical/ Health Care Library
Standards
Harvard University, US
• Mobile Resources for
Medicine3
• EBM – Evidence Based
Medicine4 (Information
resources)
• EBM Search Tips5
University of Oxford, UK
• Research Skills – Health
Care Libraries6
• Medical Literature search
skills7
University of Cambridge, UK
• Literature Search – Services
Standards8
The gold standards
framework Library9
 Literature review
 Grey Literature
 Citations
 Medical Terminologies at
NLM10
Benchmarks used to develop our customised Information Literacy
framework for LKCMedicine students
1. (2018). Alair.ala.org. Retrieved 19 March 2018, from https://alair.ala.org/bitstream/handle/11213/7668/ACRL%20Information%20Literacy%20Competency%20Standards%20for%20Higher%20Education.pdf?sequence=1&isAllowed=y
2. Curriculum. (2018). Lkcmedicine.ntu.edu.sg. Retrieved 19 March 2018, from http://www.lkcmedicine.ntu.edu.sg/Programmes/Intro-to-MBBS-Programme/Pages/Curriculum.asp
3. Research Guides: Mobile Resources for Medicine: Mobile Apps/Websites. (2018). Guides.library.harvard.edu. Retrieved 19 March 2018, from https://guides.library.harvard.edu/hms/mobileApps
4. Research Guides: EBM - Evidence-based Medicine: Finding Evidence - EBM Resources. (2018). Guides.library.harvard.edu. Retrieved 19 March 2018, from https://guides.library.harvard.edu/hms/ebm
5. Research Guides: EBM - Evidence-based Medicine: Finding Evidence - Search Tips. (2018). Guides.library.harvard.edu.Retrieved 19 March 2018, from https://guides.library.harvard.edu/hms/ebm/SearchTips
6. Oxford LibGuides: Medicine: Medicine. (2018). Libguides.bodleian.ox.ac.uk. Retrieved 19 March 2018, from https://libguides.bodleian.ox.ac.uk/healthcarelibraries
7. Oxford LibGuides: Medical Sciences : Research Resources: Literature searching. (2018). Libguides.bodleian.ox.ac.uk. Retrieved 19 March 2018, from https://libguides.bodleian.ox.ac.uk/medsci/literaturesearching
8. Support, R., Service, L., Us, A., Library, A., Team, T., & Guide, L. et al. (2018). Literature Search - Service Standards - Medical Library. Medical Library. Retrieved 19 March 2018, from https://library.medschl.cam.ac.uk/research-support/literature-search-service/ls-service-standards/
9. Gold Standard Framework - Library. (2018). Goldstandardsframework.org.uk. Retrieved19 March 2018, from http://www.goldstandardsframework.org.uk/library-4
10. Medical Terminologies at NLM. (2018). Nlm.nih.gov. Retrieved 19 March 2018, from https://www.nlm.nih.gov/medical-terms.html

8. “
Research shows that when learners
lack essential frameworks, they
experience difficulties in narrowing,
focusing or refining their topics
[3,4,5].
8
There is evidence from the
literature that a year-by-year
progressive learning framework,
with laddered instruction that
offers close integration with
undergraduate medical curricula
is effective [1,2,4].
1. Brasley, S. S. (2008). Effective Librarian and Discipline Faculty Collaboration Models for Integrating Information Literacy into the Fabric of an Academic Institution. New Directions For Teaching And Learning, (114), 71-88.
2. Haraldstad, AM. (2002). Information literacy-curriculum integration with medical school’s syllabus. Liber Quarterly: The J of Euro Res Libr, 12(2): 192-198. Retrieved 1 February 2017, from http://liber.library.uu.nl/index.php/lq/article/view/7682/7718.
3. Kuhlthau, C. C. (1993a), Seeking meaning: A process approach to library and Information services. Norwood, NJ: Ablex
4. MacEachern, M., Townsend, W., Young, K., & Rana, G. (2012). Librarian Integration in a four-year medical school curriculum: a timeline. Med Reference Services Quarterly, 31(1),105-114. doi.org/10.1080/02763869.2012.641856
5. Rempel, H. G., & Davidson, J. (2008), Providing Information Literacy Instruction to Graduate Students through Literature Review Workshops. Issues In Science And Technology Librarianship, 53. Retrieved 1 February 2017, from http://www.istl.org/08-
winter/refereed2.html.

9. 9Fig 1: Customised Information Literacy Framework for LKCMedicine (Year 1 to 5), Learning Progression of Search, Writing and Citation Skills, Medical Library) ©
Customised Information Literacy Framework for LKCMedicine Students (Year 1 to 5)
Year 1
Year 2
Year 3
Year 4
Scholarly Project
Year 5
Foundation level Advanced level
IL Intervention
Implementation
(2016-2017)

10. 1
2Teaching students at a year 4
scholarly project briefing session
using advanced search techniques
on databases, writing and
citation styles
Year 4 scholarly briefing session
Using a PICO and search strategy
worksheet for students to
complete before attending a
consultation
PICO and search strategy worksheet
IL Intervention tools
Topics
Covered
Literature
Search
Literature
Review
Citation
Management
Using
Turnitin
Booking of
Student
Consultation
Sessions
Q&A
Implementation
(2017)

11. Data Collection & Analysis
Fig 1: Student response rate for Student Consultation Sessions, 2016
37 16
Fig 2: Student response rate for Student Consultation Sessions, 2017
69.23%
30.77%
24
54
Student Response Rate AY2016-17/2017-18
(Med Lib – STUDENT CONSULTATION SESSIONS)
n=53 n=78

12. Feedback from students AY2016-17
(Med Lib – STUDENT CONSULTATION SESSIONS)
Data Collection & Analysis
Table 1: Outline of participants’ feedback received for qualitative data
• Voluntary qualitative feedback was sent by Year 4
students via e-mail to our Medical Library account.
• 4 students expressed positive feedback based on
the “recommendations for project and citation tips”,
followed by 3 students expressed the student
consultation session were “helpful” to them.
• 1 student expressed positive feedback on the “time
spent for discussion” and “the medical library
offering help to students”.
• 1 student explained the consultation session was
“useful” and returned for two more consultations.

13. Feedback from students AY2016-17
(Med Lib – STUDENT CONSULTATION SESSIONS)
Data Collection & Analysis
Follow up consultations via E-mail for
non-respondents
• Students (n=37) who did not respond
to the consultation service were sent
follow up e-mails.
o MeSH terms
o Recommended databases
o Recommended article(s)
o Suggested use of the PICO
method
o Citation management tips.
• Students were given further
elaboration via mass e-mail on how to
format their citations.
• 2 students who did not attend the
consultation sessions gave positive
feedback on the follow up e-mails.

14. Scholarly Project Survey AY2017-18
(Med Lib – STUDENT CONSULTATION SESSIONS)
Q: Do you feel the library’s student consultation session services should continue for the following year?
Adapted from: LKCMedicine Student Voice, Educational Quality Assurance, Scholarly Project Survey, AY2017-18, 3
October 2017
Data Collection & Analysis
93.9% Agreed
Overall, students agreed the library
should continue conducting student
consultation sessions for the
following year

15. Scholarly Project Survey AY2017-18 (Med Lib – BRIEFING)
Q: How do you find the library’s Year 4 scholarly project briefing session?
Adapted from: LKCMedicine Student Voice, Educational Quality Assurance, Scholarly Project Survey, AY2017-18,
3 October 2017
Data Collection & Analysis
60%
Overall satisfactory rate
• Almost half (48.1%) of the cohort
(n=52) expressed that they were
satisfied with the briefing session
• 26.9% of students however felt
they were neither satisfied nor
dissatisfied (basically neutral)

16. Scholarly Project Survey AY2017-18 (Med Lib - SKILLS)
Q: The skills I have gained while working on my Scholarly Project will be useful to me in the future.
Adapted from: LKCMedicine Student Voice, Educational Quality Assurance, Scholarly Project Survey, AY2017-
18, 3 October 2017
Data Collection & Analysis
Overall, students agreed that the
skills gained was useful to them
from the Scholarly Project
98.3% Agreed

17. Perspectives of acquired skills
Writing
Skills
Benefits of
writing skills
“writing a narrative paper…the experience of
writing a narrative paper is different from
writing a scientific paper as well”
“Report writing - I learnt that every
aim or every piece of information I
included in the results must be
addressed in the discussion and
conclusion.”
“Problem solving writing”
“I have learnt a different approach to writing
up my exploratory report.”
“Writing an actual paper. I appreciate the
rigour better.”
“Report writing was also significantly different from that in M1 &
2, in a sense that it required a more meticulous approach at
answering the aim of the question to deliver a comprehensive
answer. The process of writing the report would help towards
further report writings for future research projects.”
“…writing a report…” (13 participants)
“…Basic understanding of report writing”
“…Not only am I more acquainted with the whole process of
writing a research paper, I am also able to read them better.”
“…being able to write research papers…This was valuable experience
in learning the dos and donts of writing a paper. I also learnt the
process of submitting an abstract to a conference.”
Using
EndNote
Benefits of
EndNote
“…to manage
citations…”
Research Skills
Learning to Search
for Articles
“…literature
search…” (3
participants)
“finding
information…
”
“reading and
analyzing
research
papers, which
can guide
evidence-
based
medicine skills
in designing
research and
data analysis
which could
be applied to
clinical
research as
well.”
“…searching
for
(information
in) scientific
papers."
Benefits of managing data
Statistics
“…statistics…(8
participants)
“…Basic
understanding of
statistics…”
“…doing statistics on my
own research help me
retain the skills better.”
“Statistical analysis
because research and
reading papers will be a
part of any clinician's
work.”
“SPSS - i am able to analyze
data with greater
confidence.”
“Learning how to process
statistical data and how to
interpret it.”
“…and how to perform
statistical analysis …”
Data
Collection
“data collection
really helped me to
learn how to use
TTSH systems
well…”
Data
Analysis
“…data
analysis
- helped
me
underst
and a bit
more
about
statistics
Benefits of
Literature
review
“…doing
literature
review…” (11
participants)
“Sifting
through
countless
reviews and
journals to find
the most up to
date sources
for knowledge
or clinical
application
purposes…”
“…important
skill that is used
in any form of
research.”
“…impt to do evidence based
medicine”
“Literature
review, to sieve
through vast
amounts of
information
available
online…”
Interpreting
Systematic
reviews
“Process of
conducting a
systematic
review and
understandin
g how
evidence is
synthesized
may be
helpful in
interpreting
studies.”
Formulating
research
questions
“…crafting
research
questions
will help
me in my
own
research in
the
future.”
“…importan
ce of
forming a
good
research
question
and
designing
study.”
Experience
on research
process
“Understandi
ng the
research
process “
“The rigour and
process of
research is
universally
applicable in
whatever fields
we choose in
future. I can
appreciate what
goes into
conducting
research.”
“Research skills”
“How to plan and
design research proj.”
“Basic understanding of
the planning and
execution of a clinical
research”
“designing research…skills in
future if we join a residency.”
Experience in
project
management
“I learnt
how to
start up a
project and
it also
sparked my
interest in
the
research
aspect.”
Q: Please explain which skills and why.

18. Conclusion
18
▪ Student consultation sessions was first conducted in 2016 and
continued in 2017 to help Year 4 students manage their literature
search, writing styles & citation management. This service will continue
in 2018.
▪ In 2017, survey questions in the Scholarly Project Survey AY2017-18
(Med Lib) received an overall positive feedback from students about
their literature search, data and citation management.
▪ In 2018, the team will further enhance the Year 4 scholarly briefing
session using a new intervention tool – “Tips Sheet”
▪ Evidence suggests that the most effective method of developing
information literate students is to embed or integrate information literacy
throughout the curriculum (Brown & Nelson, 2003; Samson, 2010; Tuttle
et al., 2009).
▪ Requires collaborative efforts of educators, librarians, administrators,
and the institution in which they all work.
▪ By doing this, the future direction provides medical librarians an
opportunity to complement efficient literature search with the
application of formal rules of evidence and develop an expert team of
future healthcare professionals (medical students) from LKCMedicine.

19. Q & A
19

20. Thank you very much
for your time
20
You can contact us at medlib@ntu.edu.sg