Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Comparing interactive online and
face-to-face repertory grid interviews
in terms of construct quality and
practicability
M...
ca.$45%$of$*me$spent&
on&grid&administra.on&
movie&pilot&&
300&par.cipants&
h"p://www.intanges.com/relaunch/index.php?id=8&L=1=
Tourism&in&EU&&
1,000&par.cipants&
(& )&
...&
So>ware&for&automated&administra.on&
...&large&scale&via&the&web&
Research&on&automated&grid&
administra.on&is&sparse&
•  Research&by&Gaines&&&Shaw&in&the&80s&(1980;&e.g.&
PEGASUS),&but&no...
Current&situa.on&
Automated&grid&administra.on&is$
used$by&commercial&companies&
and&par.ally&in&research,&
but&there&is&v...
Research&ques.ons&
• How&can&we&do&“good”&online&
repertory&grid&administra.on?&
• Is&it&possible&to&get&results&
comparab...
Online&grid&situa.on&
No&chance&to&ask&for&clarifica.on&
Quality&of&verbal&labels&crucial&to&
infer&meaning&
Here:&Quality&...
Content&analysis&of&constructs&
nonV&
bipolarity& repeated&
constructs&
poor&
verbaliza.ons&
mortality&
Virtual&
interviewer&as&
harness&
unintended&
ra.n...
Pretest&&
Iden.fy&main&usability&issues&
to&consider&in&so>ware&
design&
The&gridhub&so>ware&
Study&Design&
•  n&=&80&
•  Two&groups&(faceVtoVface&&&online)&
•  31&+&49&subjects&
&
•  Average&age:&24,1&years&
•  Aver...
1.&Understandability&(of&verbaliza.on)&
Is&the&seman.c&verbaliza.on&of&the&
constructs&clear&to&understand&for&a&third&
pe...
2.&OneVdimensionality&
Has&the&construct&just&one&dimension&or&
does&one&of&the&poles&include&more&than&
one&aspect?&
Exam...
3.&Appropriate&level&of&abstrac.on&(with&
respect&to&aim&of&study)=
&
&Highly&abstract&
Very&concrete&
$
„I&like&it&–&I&do...
4.&Overall&ra.ng&of&construct&quality&
“Intui.ve”&score&by&expert&raters&with&
experience&in&grid&studies&using&content&
a...
Dimensionality:&Is&the&construct&oneVdimensional?&
***&***&
Δ=33,5%&
***&
Δ=29,3%&Δ=27,0%&Δ=31,8%&
***&
Understandability:&How&understandable&is&the&
construct&for&the&rater&on&a&scale&from&1&to&5?&
*&
*&
d=.28&
*&
d=.26&
d=.2...
Level&of&Abstrac.on:&Is&the&level&of&
abstrac.on&in&the&medium&range?&
How&is&the&overall&quality&of&construct&
verbaliza.on?&
*&
*&
*&
*&
d=.46&
d=.33&
d=.42&
d=.28&
Results&in&a&nutshell&
•  Online&grid&performs&significantly&worse&
on&oneVdimensionality&and&overall&quality&
•  No&signifi...
What’s&next?&
•  Which&design&elements&help&to&…&
&…&generate&oneVdimensionality?&
&…&systema.cally&steer&the&abstrac.on&l...
Gaines,&B.&R.,&&&Shaw,&M.&L.&G.&(1980).&New&direc.ons&in&the&analysis&
and&interac.ve&elicita.on&of&personal&construct&sys...
Thanks!
Comparing interactive online and face-to-face repertory grid interviews in terms of construct quality and practicability
Upcoming SlideShare
Loading in …5
×

0

Share

Download to read offline

Comparing interactive online and face-to-face repertory grid interviews in terms of construct quality and practicability

Download to read offline

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

Comparing interactive online and face-to-face repertory grid interviews in terms of construct quality and practicability

  1. 1. Comparing interactive online and face-to-face repertory grid interviews in terms of construct quality and practicability Matthias Kaulartz & Mark Heckmann University of Bremen, Germany EPCA, Brno, June 21, 2014
  2. 2. ca.$45%$of$*me$spent& on&grid&administra.on&
  3. 3. movie&pilot&& 300&par.cipants& h"p://www.intanges.com/relaunch/index.php?id=8&L=1= Tourism&in&EU&& 1,000&par.cipants&
  4. 4. (& )& ...& So>ware&for&automated&administra.on& ...&large&scale&via&the&web&
  5. 5. Research&on&automated&grid& administra.on&is&sparse& •  Research&by&Gaines&&&Shaw&in&the&80s&(1980;&e.g.& PEGASUS),&but&no&systema.c&research&on& outcome&quality& •  Van&de&Kerkhof&et&al.&(2009,&p.&430):& – “Quite&some&constructs&men.oned&were& unclear&to&the&analysts&of&the&research&team.& Many&were&nonVbipolar&....&Moreover,&in&a& faceVtoVface&interview&the&interviewer&can& check&if&two&constructs&are&similar&to&the& respondent.”&
  6. 6. Current&situa.on& Automated&grid&administra.on&is$ used$by&commercial&companies& and&par.ally&in&research,& but&there&is&virtually&no$research$ on&outcome&quality&of&online&grid& administra.on&
  7. 7. Research&ques.ons& • How&can&we&do&“good”&online& repertory&grid&administra.on?& • Is&it&possible&to&get&results& comparable&to&faceVtoVface& interviews?&
  8. 8. Online&grid&situa.on& No&chance&to&ask&for&clarifica.on& Quality&of&verbal&labels&crucial&to& infer&meaning& Here:&Quality&of&constructs&refers& to&quality&of&verbaliza.on&
  9. 9. Content&analysis&of&constructs&
  10. 10. nonV& bipolarity& repeated& constructs& poor& verbaliza.ons& mortality& Virtual& interviewer&as& harness& unintended& ra.ng& behaviour&
  11. 11. Pretest&& Iden.fy&main&usability&issues& to&consider&in&so>ware& design&
  12. 12. The&gridhub&so>ware&
  13. 13. Study&Design& •  n&=&80& •  Two&groups&(faceVtoVface&&&online)& •  31&+&49&subjects& & •  Average&age:&24,1&years& •  Average&number&of&shoes:&21,6& •  !&516&constructs&in&total& •  Three&expert&raters&rate&construct&on&four& dimensions&of&construct&“quality”&
  14. 14. 1.&Understandability&(of&verbaliza.on)& Is&the&seman.c&verbaliza.on&of&the& constructs&clear&to&understand&for&a&third& person?& & Example:& Good:&“This&shoe&makes&me&feel&comfortable&–& makes&me&feel&uncomfortable”& Bad:&“This&shoe&is&so&‘shoeVish’&–&this&one&isn’t”&
  15. 15. 2.&OneVdimensionality& Has&the&construct&just&one&dimension&or& does&one&of&the&poles&include&more&than& one&aspect?& Example:& Good:&& &“This&shoe&is&for&daily&use&–&only&for& & &&&special&occasions”& Bad:&& &“Feels&comfortable,&looks&good&and&is& & &&&cheap&–&…”&
  16. 16. 3.&Appropriate&level&of&abstrac.on&(with& respect&to&aim&of&study)= & &Highly&abstract& Very&concrete& $ „I&like&it&–&I&don‘t&like&it“& $ „Shoe&–&Gumboot“& & “Makes&me&friendly&and& selfVconfident&V&…”&
  17. 17. 4.&Overall&ra.ng&of&construct&quality& “Intui.ve”&score&by&expert&raters&with& experience&in&grid&studies&using&content& analysis& Examples:& Good:&&“Makes&me&feel&happyVgoVlucky&–&& & &Makes&me&feel&weighted”& Bad:&&“High&or&low&heels&–&Medium&height& heels”&
  18. 18. Dimensionality:&Is&the&construct&oneVdimensional?& ***&***& Δ=33,5%& ***& Δ=29,3%&Δ=27,0%&Δ=31,8%& ***&
  19. 19. Understandability:&How&understandable&is&the& construct&for&the&rater&on&a&scale&from&1&to&5?& *& *& d=.28& *& d=.26& d=.27& Reverse&effect&(!)&
  20. 20. Level&of&Abstrac.on:&Is&the&level&of& abstrac.on&in&the&medium&range?&
  21. 21. How&is&the&overall&quality&of&construct& verbaliza.on?& *& *& *& *& d=.46& d=.33& d=.42& d=.28&
  22. 22. Results&in&a&nutshell& •  Online&grid&performs&significantly&worse& on&oneVdimensionality&and&overall&quality& •  No&significant&differences&for&abstrac.on& •  Reverse&effect&for&understandability&(?)& !&While&faceVtoVface&interviews&achieve& bever&results,&online&administra.on&is&able& to&generate&quality&verbaliza.on&of& constructs&
  23. 23. What’s&next?& •  Which&design&elements&help&to&…& &…&generate&oneVdimensionality?& &…&systema.cally&steer&the&abstrac.on&level?& •  Differences&with&regard&to&other&grid& measures&of&interest&(e.g.&complexity)& •  How&can&we&make&online&grid& administra.on&(more)&fun?&
  24. 24. Gaines,&B.&R.,&&&Shaw,&M.&L.&G.&(1980).&New&direc.ons&in&the&analysis& and&interac.ve&elicita.on&of&personal&construct&systems.& Interna?onal=Journal=of=ManCMachine=Studies,&13(1),&81–116.& Honey,&P.&(1979).&The&repertory&grid&in&ac.on:&How&to&use&it&to& conduct&an&awtude&survey.&Industrial=and=Commercial=Training,& 11(11),&452–459.&doi:10.1108/eb003756.& Russell,&C.&G.,&&&Cox,&D.&N.&(2003).&A&computerised&adapta.on&of&the& repertory&grid&methodology&as&a&useful&tool&to&elicit&older& consumers’&percep.ons&of&foods.&Food=Quality=and=Preference,& 14(8),&681–691.& Van&de&Kerkhof,&M.,&Cuppen,&E.,&&&Hisschemoller,&M.&(2009).&The& repertory&grid&to&unfold&conflic.ng&posi.ons:&The&case&of&a& stakeholder&dialogue&on&prospects&for&hydrogen.&Technological= Forecas?ng=and=Social=Change,&76(3),&422–432.& & References&
  25. 25. Thanks!

Views

Total views

955

On Slideshare

0

From embeds

0

Number of embeds

18

Actions

Downloads

7

Shares

0

Comments

0

Likes

0

×