Technologies, like augmented reality (AR), have the ability to catch children’s imagination and to promote their attention. Children with ASD usually have difficulty to recognize facial expressions and to understand associated emotions. We propose to design and develop an innovative GameBook to assist children with ASD to recognize and acquire emotions by engaging their attention and motivation. The GameBook will contain a story that describe some scenarios and real world situations which will conduct the children to become involved on fictional contents associated with emotions.

1. An
Augmented
Reality
GameBook
for
Children
with
Au8sm
Spectrum
Disorder
(ASD)
Jorge
Brandão
University
of
Minho/Algoritmi
R&D
Centre,
Guimarães,
Portugal
Pedro
Cunha
University
of
Minho/Algoritmi
R&D
Centre,
Guimarães,
Portugal
José
Vasconcelos
University
of
Minho/Algoritmi
R&D
Centre,
Guimarães,
Portugal
Vítor
Carvalho
University
of
Minho/Algoritmi
R&D
Centre,
Guimarães,
Portugal
IPCA-­‐EST,
Barcelos,
Portugal
Filomena
Soares
University
of
Minho/Algoritmi
R&D
Centre,
Guimarães,
Portugal
InternaKonal
conference
focused
on
e-­‐learning
in
the
workplace
June
10-­‐12,
2015,
at
Columbia
University
in
New
York.

2. • WHAT
IS
AUTISM
• ABSTRACT
• ABOUT
AUTISM
• INTRODUTION
• THE
GAMEBOOK
• CHARACTERS
• SCENARIOS
• IMPLEMENTATION
• CONCLUSIONS
• GAME
DEMONSTRATION
SUMMARY

3. • Technologies,
like
augmented
reality
(AR),
have
the
ability
to
catch
children’s
imaginaKon
and
to
promote
their
aYenKon.
• Children
with
ASD
usually
have
difficulty
to
recognize
facial
expressions
and
to
understand
associated
emoKons.
• We
propose
to
design
and
develop
an
innovaKve
GameBook
to
assist
children
with
ASD
to
recognize
and
acquire
emoKons
by
engaging
their
aYenKon
and
moKvaKon.
• The
GameBook
will
contain
a
story
that
describe
some
scenarios
and
real
world
situaKons
which
will
conduct
the
children
to
become
involved
on
ficKonal
contents
associated
with
emoKons.
ABSTRACT

4. ABSTRACT
Play
with
AR
Avatar
Play
Quiz
Games
Play
Memory
Games
The
child
will
have
to
interact
with
these
scenarios,
by
playing
with
one
3D
AR
avatar
with
different
facial
expressions
In
this
GameBook
the
child
will
have
to
idenKfy
the
correct
emoKonal
face
to
the
situaKon
in
a
Quiz
Game.
This
GameBook
the
child
will
have
to
idenKfy
the
correct
emoKonal
face
to
the
situaKon
in
a
Quiz
Game.
x
x

5. AUTISM
SPECTRUM
DISORDER
IN
USA
Source:
Center
for
disease
Control
and
PrevenKon
-­‐
hYp://www.cdc.gov/ncbddd/auKsm/data.html
“The
AuKsm
Society
esKmates
that
the
United
States
is
facing
almost
$90
billion
annually
in
costs
for
au8sm.
(This
figure
includes
research,
insurance
costs
and
non-­‐covered
expenses,
Medicaid
waivers
for
auKsm,
educaKonal
spending,
h o u s i n g ,
t r a n s p o r t a K o n ,
employment,
related
therapeuKc
services
and
caregiver
costs.)”
hYp://www.auKsm-­‐
society.org/about-­‐auKsm.

6. hWp://thautcast.com/drupal5/category/tags/centers-­‐disease-­‐control
HIGH
FUNCTIONING
LOW
FUNCTIONING

7. 1
out
of
every
54
boys
in
the
United
States
has
an
au8sm
spectrum
disorder
hYp://thautcast.com/drupal5/category/tags/centers-­‐disease-­‐control

8. 1
out
of
every
252
girls
in
the
United
States
has
an
au8sm
spectrum
disorder
hYp://thautcast.com/drupal5/category/tags/centers-­‐disease-­‐control

9. hYp://thautcast.com/drupal5/category/tags/centers-­‐disease-­‐control

10. INTRODUTION
• Technology
is
ubiquitous
and
can
be
a
safe
and
good
way
to
moKvate
and
engage
children
in
interacKve
learning
acKviKes
in
order
to
promote
their
cogniKve
and
social
skills
• It
is
being
used
increasingly
on
a
variety
of
pedagogical
contexts,
both
as
assisKve
technologies
and
as
tools
for
helping
us
to
understand
user’s
moKvaKon
• It
has
the
potenKal
to
provide
individually
tailored
intervenKons
that
are
suitable
for
a
wide
variety
of
abiliKes
•
Technological
intervenKons
allow
its
use
at
different
speeds
and
locaKons,
and
never
lose
paKence
with
the
frequent
repeKKon
that
many
people
with
ASD
has

11. INTRODUCTION
• ASD
children
struggle
with
significant
relaKonships
and
behavioural
challenges
that
in
most
cases
have
serious
implicaKons
for
inclusion
social
adulthood
• One
of
the
most
promising
applicaKons
in
the
intervenKon
process
of
ASD
children
came
from
developing
tools
for
promoKng
children’s
social
and
communicaKon
skills
• This
offers
the
opportunity
to
reduce
some
of
the
crippling
personnel
costs
associated
with
auKsm
care

12. INTRODUTION
• Children
with
ASD
usually
have
difficulty
to
recognize
facial
expressions
and
to
understand
associated
emoKons,
to
imitate
or
use
emoKonal
expressions,
to
understand
and
control
their
own
emoKons,
or
to
interpret
emoKons
or
empathy
with
others
• Several
studies
show
that
the
majority
of
people
with
ASD
exhibit
a
natural
affinity
with
technology
and
a
posiKve
ahtude
towards
computer-­‐based
training

13. INTRODUTION
• Technologies,
like
augmented
reality
(AR),
have
the
ability
to
catch
the
children’s
imagina8on
and
to
promote
their
aWen8on,
because
they
can
experiment
ar8ficial,
safe
and
fascina8ng
environments.
• AR
is
an
opKmal
interface
technology
and
a
helpful
tool
to
support
auKsKc
children
capabiliKes.
With
AR
it
is
possible
to
create
more
aYracKve
and
interacKve
interfaces
that
can
be
manipulated
by
hand,
without
using
convenKonal
peripherals
such
as
the
keyboard
and
the
mouse.
• This
characterisKc
offers
greater
interacKon
between
the
child
and
the
object,
raising
their
interest
and
their
curiosity,
providing
greater
adherence
to
the
task/acKvity.

14. THE
GAMEBOOK
Given
the
potenKal
benefits
of
AR
we
aim
to
design
a
tool
based
on
AR
in
order
to
improve
capabiliKes
of
ASD
children
Our
goal
is
to
create
an
innovaKve
serious
game
based
on
a
book
style
with
interacKon
inputs
to
assist
children
with
social
interacKon
and
communicaKon
skills.
We
aim
to
promote
children
with
ASD
recognizing
and
acquiring
emoKons
by
engaging
their
aYenKon
and
moKvaKon,
increasing
their
competence
on
this
handicap.

15. THE
GAMEBOOK
The
GameBook
presents
the
story
of
Tobias’s
adventure
during
a
visit
to
a
zoo
park.
The
player
can
read
the
story
and
interact
with
it.
The
story
will
describe
scenarios
and
Tobias’
interacKons
with
animals
and
real
world
situaKons
which
will
conduct
the
children
to
become
involved
on
ficKonal
contents
associated
with
emoKons.

16. THE
GAMEBOOK
We
believe
that
the
GameBook
will
put
the
ability
of
the
child
to
interact
with
the
story
and
to
promote
his/her
imagina8on
and
engaging.
hYp://www.tobiasadventures.in/thezoo
The
GameBook
can
be
played
on
any
mobile
technology,
such
as
a
tablet,
smartphone
or
laptop,
with
either
an
external
web
camera
or
an
inbuilt
camera.
The
GameBook
is
available
on:

17. GAMEBOOK
CHARACTERS
• The
character
Tobias
was
designed
to
capKvate
the
children
aYenKon
and
to
empathize
with
them.
• On
his
design
was
pretended
to
promote
a
face
with
characterisKcs
that
allows
the
players
to
feel
some
aYracKon
with
him.
• The
expressions
are
designed
by
special
details
in
the
avatar,
essenKally,
with
his
lips
and
eyes.
• It
was
given
a
cartoonish
look
to
Tobias,
which
was
deemed
by
the
group
members
as
potenKally
more
familiar
and
more
fun
to
the
children
than
a
photo-­‐realisKc

18. GAMEBOOK
SCENARIOS
The
game
includes
a
total
of
5
different
scenarios,
designed
to
involve
the
child
on
real
life
situa8ons.
The
scenarios,
all
in
the
zoo,
will
promote
the
child’s
contact
with
different
environments
and
will
create
some
emoKonal
reacKon
on
Tobias.
ANIVERSARY
GO
TO
THE
ZOO
WITH
ELEPHANT
WITH
LION
WITH
BIRD

19. IMPLEMENTATION
OF
THE
GAMEBOOK
To
create
this
serious
game,
the
following
technologies
were
used:
The
3D
flip
book
uses
the
HTML5/CSS3
and
JavaScript/Jquery
web
languages.
To
develop
the
AR
feature
it
was
selected
the
ArUco®
(Augmented
Reality
library
from
the
University
of
Cordoba).
This
opKon
was
held
on
the
easiness
of
use,
reliability
and
cross-­‐
plakorm
compaKbility.

20. CONCLUSIONS
AND
FUTURE
WORK
• The
aim
of
this
project
was
to
make
available
a
cost
effec8ve
tool
implemented
in
a
plagorm
usually
pleasant
to
ASD
children.
• In
order
to
test
the
GameBook,
an
exploratory
study
in
a
classroom
context
with
ASD
children
will
be
performed.
• In
the
near
future,
a
system
that
will
allow
monitoring
the
child’s
results
and
these
outcomes
will
be
available
on
the
website
of
the
game.
• Furthermore,
new
stories,
as
for
example,
Tobias
adventures
in
the
Sea
or
in
the
Space,
with
new
characters,
scenarios
and
situa8ons
can
be
easily
included.
• It
is
the
authors’
belief
that
the
GameBook
will
increase
ASD
children
engagement
in
the
learning
ac8vi8es
promo8ng
their
memory,
cogni8ve
and
social
skills
and
allowing
them
to
reinforce
facial
expression
recogni8on.

21. GAME
DEMONSTRATION

22. QUESTIONS?
Jorge
Brandão
University
of
Minho/Algoritmi
R&D
Centre,
Guimarães,
Portugal
E-­‐mail:
j.almeidabrandao@gmail.com
Pedro
Cunha
University
of
Minho/Algoritmi
R&D
Centre,
Guimarães,
Portugal
E-­‐mail:
pjgscunha@gmail.com
@gmail.com
José
Vasconcelos
University
of
Minho/Algoritmi
R&D
Centre,
Guimarães,
Portugal
E-­‐mail:
vasconcelos.jose@gmail.com
Vítor
Carvalho
University
of
Minho/Algoritmi
R&D
Centre,
Guimarães,
Portugal
IPCA-­‐EST,
Barcelos,
Portugal
E-­‐mail:
vcarvalho@ipca.pt
Filomena
Soares
University
of
Minho/Algoritmi
R&D
Centre,
Guimarães,
Portugal
E-­‐mail:
fsoares@dei.uminho.pt