Presentation given at the meeting of the TEMPUS TRUST project at the University of Coimbra, Portugal, January 25, 2012. The TEMPUS TRUST project aims to support the modernization of Ukrainian higher education by introducing a common quality assurance framework to enable mutual understanding and trust between higher education institutions, national and international quality assurance actors and the society in general.

1. Quality Assurance in
Higher Education:
from
concepts
to practice
Tempus Trust, Coimbra, Portugal
January 23 - February 3, 2012

2. Presentation given at the meeting of the TEMPUS TRUST project
at the University of Coimbra, Portugal, January 25, 2012. The
TEMPUS TRUST project aims to support the modernization of
Ukrainian higher education by introducing a common quality
assurance framework to enable mutual understanding and trust
between higher education institutions, national and international
quality assurance actors and the society in general.

3. 1. concepts
2. from industry to education
3. recent developments
4. practice
5. conclusions

4. how
to
assess
what
to
assess
how
to
assess
1. concepts
2. from industry to education
3. recent developments
4. practice
5. conclusions

5. 1. concepts
WHAT IS HIGHER EDUCATION?
HIGHER
EDUCATION
producing
qualified
human
resources
for
the
labor
market
educa7ng
qualified
researchers
and
cultural
agents
providing
higher
level
teaching
in
all
fields
of
knowledge
extending
educa7on
beyond
secondary
educa7on
(Adapted from Barnett, R. (1992). Improving Higher Education: Total Quality Care, Buckingham, SRHE&OU)
Some representative models of higher education:
• Humboldt’s Model (the research-intensive university, 1800s)
• Gibbons’ Model (mode 2 knowledge production, 1994)

6. 1. concepts
WHAT IS QUALITY?
a concept born in industry
THE
QUALITY
MOVEMENT
IN
INDUSTRY
Before
1900
Quality
as
an
integral
element
of
the
craA
1900-­‐1920
Quality
control
by
foreman
1920-­‐1940
Inspec7on-­‐based
quality-­‐control
1940-­‐1960
Sta7s7cal
process
control
1960-­‐1980
Quality
assurance
(quality
department)
1980-­‐1990
Total
quality
management
(TQM)
1990-­‐Present
Culture
of
con7nuous
improvement,
organiza7on-­‐wide
TQM
(Adapted from Sallis, E. (1996). Total Quality Management in Education, 2nd Ed. London: Kogan Page)

7. 1. concepts
THE HIERARCHY OF QUALTY MANAGEMENT IN INDUSTRY
QUALITY
MANAGEMENT
Total
Quality
Management
(TQM)
Excellence
of
all
managerial,
opera7onal
and
administra7ve
processes
*
Culture
of
con7nuous
improvement
in
all
aspects
of
the
business
*
Understanding
that
quality
improvement
results
in
cost
advantages
and
beTer
profit
*
More
intense
rela7onships
with
customers
and
suppliers*
Involvement
of
all
personnel:
the
culture
of
quality
*
Market-­‐oriented
organiza7onal
prac7ces
*
Quality
Assurance
Use
of
sta7s7cal
process
control
Emphasis
on
preven7on
External
accredita7on
Delegated
involvement
Audit
of
quality
systems
Cause
and
effects
analysis
Quality
Control
Concerned
with
product
tes7ng
Responsibility
with
supervisors
Limited
quality
criteria
Some
self-­‐inspec7on
Paper
based
system
Inspec7on
Post
produc7on
review
Re-­‐working
Rejec7on
Control
of
workforce
Limited
to
physical
products
(Adapted from Dale, B.G. and Plunkett, J.J. (1990). Managing Quality, Hertfordshire: Philip Allan / *Economist (1994), Pocket MBA, London: Profile Books)

8. 1. concepts
2. from industry to education
3. recent developments
4. practice
5. conclusions

9. 2. from industry to education
FROM INDUSTRY TO EDUCATION - ISO 9000
European Quality Award (EQA), 1992
European Foundation for Quality Management (EFQM)
people satisfaction of
management collaborators
(9%) (9%)
policy & results of the
leadership processes satisfaction
strategy whole activity
(10%) (14%) of students (15%)
(8%) (20%)
resources impact on
(9%) society
(6%)
the
ability
to
mobilize
everyone,
building
change
and
ins7lling
a
culture
and
values

10. 2. from industry to education
FROM INDUSTRY TO EDUCATION
BALDRIGE CRITERIA (USA)

11. 2. from industry to education
WHAT’S QUALTY IN EDUCATION?
HARVEY
&
KNIGHT
(1996)
Quality
as
excep7onal
tradi7onal
no7on
of
quality
–
quality
as
exclusivity
excellence
–
eli7sm:
input/output,
centers
of
excellence
fulfilling
minimum
standards
–
pass/fail,
rankings,
accredita7on
Quality
as
perfec7on
or
zero
defects,
conformance
to
specifica7on,
right
first
7me
consistency
quality
culture,
involving
everybody
in
the
organiza7on
Quality
as
fitness
for
customer
specifica7on
&
requirements
purpose
the
ins7tu7on
fulfilling
its
mission
quality
assurance
customer
(students,
employers)
sa7sfac7on
Quality
as
value
for
money
performance
indicators,
efficiency
&
effec7veness,
market
view
customer
charters
and
league
tables,
rankings,
accountability
Quality
as
transforma7on
enhancing
the
par7cipant
adding
value
to
the
par7cipant
empowering
the
par7cipant
(Adapted from Harvey, L. and Knight, P. T. (1996). Transforming Higher Education, Buckingham, SRHE&OU)

12. 2. from industry to education
ABET Engineering Criteria 2000 - Matrix for Implementation Assessment (USA)
outcomes assessment
educational objectives
constituents processes
system
results

13. 2. from industry to education
ENQA
Quality Assurance across Europe
ENQA is the umbrella organization for the
accreditation of quality assurance agencies
in the European Higher Education Area.
Full membership of ENQA expresses
recognition that an agency complies with
the European Standards and Guidelines for
quality assurance in higher education.
Compliance with these standards is
confirmed every five years through
an independent review.

14. 1. concepts
2. from industry to education
3. recent developments
4. practice
5. conclusions

15. 3. recent developments
USA vs EUROPE
Accountability vs Accreditation & Ranking
USA
• Loss
of
trust
on
the
accredita7on
system
• demands
of
public
accountability
• shiA
from
quality
improvement
to
accountability
Europe
• declining
trust
in
the
ability
of
higher
educa7on
ins7tu7ons
to
ensure
quality
• movement
towards
accredita7on
(The
Netherlands,
Flanders,
Portugal,
Denmark,
Finland)
• prospect
of
a
mul7-­‐dimensional
ranking
system
for
universi7es
(Adapted from Amaral, A., Tavares, O., and Cardoso, S. (2011). Regaining Trust. Is it possible? 6th European Quality Assurance Forum)

16. 3. recent developments
OECD
Assessment of Higher Education Learning Outcomes (AHELO)

17. 3. recent developments
OECD
Assessment of Higher Education Learning Outcomes (AHELO)

18. 3. recent developments
OECD
Assessment of Higher Education Learning Outcomes (AHELO)

19. 3. recent developments
QUALITY ENHANCEMENT (QE)
Strongly promoted by some institutions,
namely the QAA (Quality Assurance Agency
for Higher Education), in the UK.
Enhancement - the process of taking deliberate
steps at institutional level to improve the
quality of learning opportunities
A wide variety of perceptions and
conceptions of quality enhancement
The process of quality management must
secure academic standards before it can
confidently turn to quality enhancement
(QAA, HEA, HEFCE (2011). Quality enhancement and assurance - a changing
picture? Retrieved January 21, 2012, from http://www.qaa.ac.uk/Partners/education/
Pages/Quality-enhancement-and-assurance-a-changing-picture.aspx)

20. 3. recent developments
QUALITY ENHANCEMENT (QE)
QUALITY
ASSURANCE
(QA)
QUALITY
ENHANCEMENT
(QE)
Focus
on
teaching
Focus
on
learning
Teaching
as
individual
performance
Learning
as
social
prac7ce
Focus
on
monitoring
/
judgment
Focus
on
personal
and
professional
development
Inflexible,
non-­‐nego7able,
approach
Flexible,
context-­‐sensi7ve,
approach
based
on
based
on
standards
building
professional
knowledge
LiTle
acknowledgement
of
the
links
Explora7on
of
the
links
between
teaching
and
between
teaching
and
research
research,
through
reflec7on
on
prac7ce
Teachers
as
individual
prac77oners
Teachers
as
collaborators
across
disciplines
Emphasis
on
documenta7on
Emphasis
on
discussion
(Adapted from Swinglehurst, D., Russell, J., & Greenhalgh, T. (2008). Peer observation of teaching in the online
environment: an action research approach. Journal of Computer Assisted Learning, 24(5), 383-393).
attractive concept but doubtful application

21. 1. concepts
2. from industry to education
3. recent developments
4. practice
5. conclusions

22. 4. practice
AGENCY FOR ASSESSEMENT & ACCREDITATION
IN HIGHER EDUCATION (A3ES), PORTUGAL
self-­‐
assessment
self-assessment
report A3ES
ONLINE
external
PORTAL
visit
external committee
report & recommendation (*)
(*) The external
committee report, once
approved by the A3ES,
is sent to the Institution
for consideration and
A3ES
• approval
possible contestation,
before the final
decision
• approval with conditions
• disapproval
decision of the A3ES
is made public

23. 4. practice
A3ES – VISITING COMMITTEE
TYPICAL
CONSTITUTION
OF
VISITING
COMMITTEE
senior
full
professor
(president)
senior
full
professor
senior
full
professor
or
top
expert
from
another
country

24. 4. practice
A3ES – ASSESSMENT PARAMETERS
1.
mission
&
objecSves
clarity
of
formula7on
coherence
dissemina7on
2.
internal
organizaSon
3.
material
resources
&
4.
staff
&
QA
mechanisms
partnerships
teaching
staff
internal
organiza7on
material
resources
non-­‐teaching
staff
QA
mechanisms
partnerships
5.
students
6.
processes
7.
results
3-­‐year
trends
study
plan
academic
results
teaching
&
learning
syllabus
R&D
and
arts
results
environment
teaching
methods
other
results

25. 4. practice
A3ES – ASSESSMENT PARAMETERS (CONTINUED)
legal
conformity
access
condi7ons
designa7on
&
study
plan
profile
of
coordinator
internships
internships
in-­‐service
training

26. 4. practice
A3ES – THE EXTERNAL VISIT
PARTS
OF
THE
EXTERNAL
VISIT
mee7ng
with
leaders
of
the
ins7tu7on
mee7ng
with
self-­‐assessment
team
mee7ng
with
course
and
department
coordinators
mee7ng
with
teaching
staff
mee7ng
with
non-­‐teaching
staff
mee7ng
with
students
mee7ng
with
graduates
lunch
with
representa7ves
of
the
other
stakeholders
visit
of
installa7ons
&
inspec7on
of
materials
produced
mee7ng
of
the
visi7ng
commiTee
oral
presenta7on
to
the
leaders
of
the
ins7tu7on

27. 4. practice
The visit gives a world of information that is
not contained in the self-assessment report
Some of it confirms the report, but
provides a much richer picture
Some of it contradicts the report, and
thus contributes to a richer picture
It also gives access to tacit knowledge about the
course and the institution, that no one from the
institution would have been able to make explicit
All this information is valuable to improve the results of
the visit, but also to refine the whole assessment process

28. 1. concepts
2. from industry to education
3. recent developments
4. practice
5. conclusions

29. 5. conclusions
When introducing a quality assurance framework to
enable mutual national and international
understanding, it may be useful to reflect upon:
• the key concepts: What is higher education? What is
quality? What is quality in higher education? How do they
relate to strategy, leadership, information architecture?
• how does the assessment of quality in industry inspire
the assessment of quality in higher education?
• what are the current European frameworks
and the world’s recent developments?
• what is the nature and value of practice, as it is experienced
when visiting the tangible reality of universities?

30. THE Quality Assurance in
Higher Education:
END
from
concepts
to practice
Slides available at:
http://www.slideshare.net/adfigueiredo
Tempus Trust, Coimbra, Portugal
January 23-29, 2012