1. EFFECTIVE AND EFFICIENT COLLECTION OF DATA FOR PHASE III OUTCOMES
USING AN ELECTRONIC PROGRAM MANAGEMENT SYSTEM
Michelle Rickard; Rochelle L. Garcia, MD; Joseph York, PhD; University of Washington, Seattle WA
UW Medicine
SCHOOL OF MEDICINE
OBJECTIVES
METHODS
RESULTS
CONCLUSIONS
REFERENCES
SYSTEM DIAGRAM
DATABASE DIAGRAM
Data Collection
•Score Reports: USMLE Step Exam and in-service exam
scores included in semi-annual evaluations (medical
knowledge); comprehensive information used to determine
trends and deficiencies for program development and
improvement.
•Compliance Data: Completion of institutional requirements
(e.g. HIPAA) included in semi-annual evaluations to document
timeliness (professionalism) and participation in hospital
management events (systems-based practice); significantly
reduces compliance tracking of 100+ requirements for 38
residents and fellows.
Digital System
•Residents’ IWeb: Secured training resource for residents,
faculty and staff that is timely and accurate with decreased
costs for paper document production and storage.
•Resident Portfolios: Residents have access to digital
technology and secured server resources to compile portfolios.
•Digital Recruiting: Accessing ERAS through a secured
portal, interviewers participate in a paperless system; “export”
and “save to .pdf file” functions gather data and produce reports
used by the recruiting committee. This has reduced our paper
usage by 90%.
•Digital Archive: Selected training file documents and other
program files are digitized and stored electronically. This had
reduced storage needs by 50%.
1. Hernandez MJ. Database Design for Mere Mortals: A Hands-On Guide to Relational Database Design. Boston:
Addison-Wesley, 1997
2. Duff P, SnyderM. Statistics for the Residency Review Committee: A clear windows approach. Obstet Gynecol
1997; 89:1031-4.
3. Civetta JM, Morejbn OV, Kirton OC, et al. Beyond requirements. Residency management through the Internet.
Arch Surg 2001; 136:412-17.
4. Tabuenca A, Catalano R, Gollin G, Shieck J. An Internet-based residency assessment application that fulfills the
Outcome Project’s requirements. Current Surg 2003; 60:89-93.
5. Stromski CJ, Jeffers T, Bean E. Procedure documentation in Emergency Medicine residencies: A time of
change. Acad Emer Med 2005; 12:375-6.
6. Greengard S. Virtual Paper Cuts. Workforce 2000; 79:16-18.
7. Tittel E, James SN. HTML 4 for Dummies. Forest City, CA: IDG Books Worldwide, 1998
To develop an electronic program management system to
effectively and efficiently collect competency measurement data
for Phase III Outcomes by:
1. Developing a relational database application within an
overall electronic management system to collect
competency data to meet Phase III requirements, while
2. Maximizing the use of common office management tools
including software applications and web-based
technologies, and
3. Improving the overall program administrative management.
Our program has implemented an administrative management
system that utilizes an Access-based relational database of
residency program data along with common office computer
technologies. Data collected in a relational database has the
power to be queried in an endless combination of information
sets to provide objective outcome reports to meet Phase III
requirements. Information is digitized to facilitate efficient data
collection, transfer and storage. In addition to meeting our
primary objectives, the system also streamlines our
administrative operations, providing for a more effective
allocation of resources.
From the implementation of Phase II when programs were
instructed to obtain accurate resident performance data, we
have implemented successive improvement to our system
using our current technology and not allocating resources to
external management applications. We chose to maximize the
use of existing applications, increase our knowledge base and
utilize our in-house IT resources.
Effective Data Collection: A well-designed relational
database offers two key advantages: (1) data consistency
and accuracy and (2) easy data retreival.1
We have developed a relational database1
using MS Access
centered around a central trainee information table and
satellite task-specific tables. The central table represents
data items that have a one-to-one relationship. The satellite
tables have a one-to-many relationship to the main trainee
table. Rules of normalization have been implemented to
insure database integrity and validity.
The satellite configuration allows for the easy addition of
new tables. This added flexibility allows our program to
adjust to new and changing collection requirements.
Efficient Data Collection: Many
medical specialties are examining how
digital technologies can improve
processes for Outcomes Project
measurement.2-5
Digital technology can
streamline information management
processes. It is interactive and
accessible for users at multiple training
locations. The information is easily
updated for all users at one time. It can
link information sources from global
sources. And it can significantly reduce
costs.6
The key element of our management
system is digitizing data to a common file
format, e.g. .txt, .pdf, .html. Digital files
allow for the electronic transfer of
information between many applications
without the loss of data integrity.
Hard-copy items are scanned and saved
as .pdf files for documents and .jpg or .gif
files for images. With increasing
frequency, documents are transferred
electronically as .txt or .pdf files from an
application into our system.
Our two main output formats are
informational documents posted as .html
files to our two websites7
and records
archived as .pdf files.
System Components
Hardware
Dell Personal Computer:
Intel®
Pentium®
2.80 GHz, 1.0 G RAM
OS: Microsoft Windows XP
Professional 2002
17” Dell Flat Screen Monitor
HP LaserJet 3015 All-in-One Desktop
FAX/Copier/Scanner/Printer
Network Unsecured/Secured Server
Space
Software Applications
ACGME ADS - accreditation database
ERAS®
2007 - resident recruiting
UW Catalyst®
WebQ - HTML Forms
Wizard
Adobe®
Photoshop 7.0 (graphic)
SeaMonkey 1.1 - Email interface
Microsoft Office Suite©
2003
Professional including:
Word (word processing)
Access (database)
Excel (spreadsheet)
Publisher (publishing)
PowerPoint (presentations)
Adobe®
Acrobat 6.0
Reader 7.0.9
Sonic RecordNow! Plus (CD burner)
Internet Explorer - Internet browser
A common concern expressed by program directors about the
Outcomes Project is how the information collected will be
managed. No matter how much data are collected, there is little
value if it is not easily accessible for interpretation.
We have used readily available electronic tools to collect the data
and maximize administrative management of our program. This
process allows us to keep pace with the explosion of data needs
required by the ACGME and other regulatory agencies. These
common tools can be adapted to the individual needs of the
program.
Each program needs to address the common concerns of a
“paperless” system - data security, equipment conversion costs,
users’ needs and concerns, and additional training.3
As we enter Phase III of the Outcomes Project, the electronic
management of program information will be an effective and
efficient method for not only collecting data but producing the
information needed to identify program improvement
opportunities.
© 2007, Michelle Rickard, University of Washington; Marvin R. Dunn
Poster Session, Award Winner, ACGME Annual Meeting, March 2007