SlideShare a Scribd company logo
1 of 28
General Online Research Conference 2015 – GOR15
18-20 March 2015, Cologne University of Applied Sciences, Germany
Holger Lütters – Hochschule für Technik und Wirtschaft Berlin
Suggested citation: Lütters, Holger (2015): New survey recruiting strategies: Online Panel vs. Mobile Advertising. General Online Research (GOR) Conference, Cologne.
Data Report available under http://goo.gl/K6xH4t
Contact: holger.luetters@htw-berlin.de @luetters
New survey recruiting strategies:
Online Panel vs. Mobile Advertising
This work is licensed under a Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/)
1
Study set-up and field report
Empirical study in comparison of two market
research recruiting approaches:
Online Panel vs. Mobile Ad
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 3
• Questionnaire development
• Online Panel
• Mobile Ad
• Questionnaire Technology
Berlin based market researchers teamed up for the research project
Berlin Market
Research Initiative Field report
• 3029 contacts
• 1732 complete answers
• 8 minutes questionnaire with
questfox® using Gyroscale®
• Random distribution of
demographics to determine
differences in sampling
• Quota on devices
• Survey in July 2014
by
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 4
Two recruiting approaches competing with one identical
questionnaire
GapFish
• (Classic) Online Panel
www.entscheiderclub.de
• 110.000 participants
• Germany, Switzerland, Austria
• Advertising sampling approach
• Global reach
(200 million potential users)
• All languages possible (need of
developed mobile advertising
market structures)
Invitation on third
party ad platform
Generic invitation
Real-time screening
Re-Direct to third
party survey
(Non-)Monetary
Incentive
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 5
Methodological question: What do participants in studies of
(mobile) market research surveys expect from research?
Methodological issues:
• Effects of different sources on
the data?
• Effects on non-reactive data
(response time, willingness to
participate, etc.)?
• Effects of different methods
used on different devices (AHP,
PSM, GyroScale)
General interest in
• Expectations of users towards
mobile surveys
• Decision-making factors of
survey participation
• Expectations of compensation
for participation in (mobile)
surveys
Online Panel vs. Mobile Ad Recruiting
Sociodemographics, Devices and Non-Reactive Data
2
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 7
Target per approach n=800 spread across four device categories
with a quota of 200 each
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 8
The two approaches reached a similar, structurally representative
regional population pattern without quota setting
Same questiontype for all device categories; Launch from Berlin
Online Panel GapFish (n=888) Mobile Ad Dalia Research (n=811); Data BRD DESTATIS 2012
10,7%
12,5%
7,8%
4,7%
0,5%
2,4%
7,2%
2,0%
5,2%
22,9%
3,6%
1,6%
6,2%
3,5%
5,4%
3,5%
9,9%
16,2%
7,2%
2,5%
1,0%
2,6%
5,7%
2,7%
9,7%
21,9%
4,7%
1,2%
4,9%
2,1%
3,5%
2,8%
13,1%
15,5%
4,2%
3,0%
0,8%
2,2%
7,5%
2,0%
9,7%
21,8%
5,0%
1,2%
5,0%
2,8%
3,5%
2,7%
Baden-Württemberg
Bayern
Berlin
Brandenburg
Bremen
Hamburg
Hessen
Mecklenburg-Vorpommern
Niedersachsen
Nordrhein-Westfalen
Rheinland-Pfalz
Saarland
Sachsen
Sachsen-Anhalt
Schleswig-Holstein
Thüringen
Online Panel
Mobile Ad
BRD
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 9
Mobile Ad approach is significantly less representative for
sociodemographics, reaching younger and more male participants
Age Distribution Gender
Base: Online Panel GapFish (n=888); Mobile Ad Dalia Research (n=811); no quota settings
Female
Male
Female
Male
Female
Male
Mobile
Ad
Online
Panel
Average Age
Mobile
Ad
Online
Panel
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 10
Both approaches are not representative in terms of education.
Mobile Ad approach achieves higher educational levels
Higher Education
A-Level
vocational school
comprehensive school
secondary school
Without formal degree
No answer
Base: Online Panel GapFish (n=888); Mobile Ad Dalia Research (n=811); no quota in question
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 11
“Couch surfing” as a standard pattern in mobile research
4 out of 5 participants in the study are mobile …but at home
At Home
Work / School / University
Public Place
On the road
None of these places
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 12
Technographics: Mobile Ad participants have a slightly more modern
equipment, use a broader variety of browsers
but do not seem to be more tech savvy in general
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 13
Mobile participants tend to speed through a questionnaire
Panel participants take much more time on a mobile device
Base: Online Panel GapFish (n=888); Mobile Ad Dalia Research (n=811); Information in seconds for complete interviews
Speeder
Mobile Ad
Online Panel
Response time
in seconds
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 14
The two approaches show different behavioral patterns
Criteria Online Panel Mobile Ad
Contacts 1912 1032
Complete Interviews 888 811
Average Completion Rate 79,5% 82,9%
Quota reach Complete Incomplete
Duration 7 days 13 days
Response Time Completes 497 seconds 441 seconds
Response Time including drop-outs 296 seconds 383 seconds
Overall time used in seconds 565.952 395.256
Drop-out pattern Decision in the first minute
(independent from device used)
Slightly higher drop-out during
the entire interview (independent
from device used)
Very good and good consistency AHP (CRH< 0,2) 29,5% 29,2%
Inacceptable consistency AHP (CRH> 0,5) 24,0% 29,0%
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 15
Conclusion non-reactive data: Strong differences in response
behavior: Mobile participants respond faster (even on the desktop)
Online Panel
● Short overall project time
● Intensive use of panelists
● Slightly more drop-outs
● Exact quota reach
● Longer interview duration
● Smartphone interview much
longer
Mobile Ad
● Double overall project time
● Used less participants
● Slightly less drop-outs
● Quota not reached for classic
devices
● Speeding behavior: All
interviews are answered
quicker on all kinds of devices
3
Expectations of participants when taking
surveys
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 17
The online panelists have much more experience with surveys of
market research compared to the mobile participants
Basis: Online Panel GapFish (n=888); Mobile Panel Dalia Research(n=811) und n=31 professionelle Marktforscher
Mittelwerte einer Skala von 0 (überhaupt keine Erfahrung bis 10 extrem viel Erfahrung)
Mean:
Online Panel 6,63
Mobile Ad 3,72
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 18
¾ of the respondents prefer the (classic) online survey
Participants of the mobile recruitment are less enthusiastic about
online surveys (and could live without any kind of survey)
Basis: Online Panel GapFish (n=888); Mobile Panel Dalia Research (n=811) und n=31 professionelle Marktforscher
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 19
Fair compensation for entertaining – but short - interviews are
demanded. The Mobile Ad group has lower or no expectations
Basis: Online Panel GapFish (n=888); Mobile Panel Dalia Research (n=811) und n=31 professionelle Marktforscher
Mittelwerte einer Skala von 0 (trifft überhaupt nicht zu bis 10 trifft voll und ganz zu)
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 20
Compensation and duration are the most important factors
influencing the willingness to participate in a survey
Questions asked in pairwise comparisons with Analytic Hierarchy Process
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 21
The majority of online panelists expect the same compensation for
a participation in mobile market research
Frage nur an n=879 Teilnehmer Online Panel GapFish und n=30 Teilnehmer MaFo Profis
Frage nach Kompensation/Kosten im Vergleich zu klassischer Online-Befragung
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 22
The mobile participants expect a form of compensation derived from
the context of their recruitment into the study
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 23
Price Sensitivity Meter to measure a fair compensation for
participating in a mobile study
Imagine the following invitation to a survey:
Topic: Shopping; Duration: 4 minutes; participation only to mobile device
(smartphone / tablet)
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 24
Participants express high hopes for the incentive of a 4 minute
mobile interview which is not backed-up by the willingness to pay
of Market Research Professionals
Online Panel: 3,56 €
Mobile Ad: 4,52 €
MR Professionals: 0,94 €
Optimal Price Point (VanWestendorp)
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 25
The identical survey is evaluated very differently by the different
approaches
Base: Online Panel GapFish (n=888); Mobile Ad Dalia Research(n=811)
Measurement Net Promoter Score 0=would not recommend at all .. 10 totally would recommend
NPS: 0-6 Detractor; 7-8 Neutral; 9-10 Promotor
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 26
Mobile participants are shockingly honest about lying
... who is less honest?
Basis: Online Panel GapFish (n=888); Mobile Panel Dalia Research(n=811) und n=31 professionelle
Marktforscher; Mittelwerte einer Skala von 0-10 unter Anwendung NPS: 0-6 gelogen; 7-8 neutral; 9-10
total wahrheitsgemäß)
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 27
Mobile research is happening anyway.
We better find ways to manage it appropriately
Mobile Ad approach works
• Dalia Research developed a new
technology which offers more
options to the researcher
• The approach can be as
appropriate as it can be wrong
for a specific topic
• The reach of fresh and
unexperienced target groups is
an interesting option
• Complicated approaches with
special methodologies might
not work (due to the screen size
and due to the user)
Things to consider when using the
Mobile Ad recruiting approach
• Mobile takes more time in
creation, testing and surveying
• Might be more expensive
(overall)
• Mobile users have different
expectations which are not
necessarily the same as before
• Researchers should not lower
their standards in order to
please the new target groups
Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 28
Author:
Holger Lütters – HTW Berlin
Contact information
Prof. Dr. Holger Lütters
Hochschule für Technik und Wirtschaft Berlin
Fachbereich 3 Wirtschafts- und Rechtswissenschaften
Treskowallee 8
D 10318 Berlin -Germany
Raum TA VG 733
Holger.Luetters@htw-berlin.de
www.luetters.com
Twitter @luetters
xing.com/profile/Holger_Luetters
de.linkedin.com/in/luetters/
The project was one of the interactions during the HTW
Berlin / HWR Berlin Project InnoRadar.
InnoRadar is supported by the Berlin Institute of Applied
Research (IFAF) www.ifaf-berlin.de.
Thank you for your interest in the study
Data live report available under http://goo.gl/K6xH4t
Thanks to the supporters of the study:
www.GapFish.com
www.DaliaResearch.com
www.pangealabs.com
Gyroscale® is a development of pangea labs GmbH Switzerland and
is available in the SaaS survey tool questfox® www.questfox.com

More Related Content

Viewers also liked

170530 mobilitätskultur münchner kreis print
170530 mobilitätskultur münchner kreis print170530 mobilitätskultur münchner kreis print
170530 mobilitätskultur münchner kreis printKlaus Markus Hofmann
 
Digitale Ökosysteme: Digitalisierung in Unternehmen
Digitale Ökosysteme: Digitalisierung in UnternehmenDigitale Ökosysteme: Digitalisierung in Unternehmen
Digitale Ökosysteme: Digitalisierung in UnternehmenMe & Company GmbH
 
Social Media Recruiting for Market Research panelbiz vs adidas Facebook HTW B...
Social Media Recruiting for Market Research panelbiz vs adidas Facebook HTW B...Social Media Recruiting for Market Research panelbiz vs adidas Facebook HTW B...
Social Media Recruiting for Market Research panelbiz vs adidas Facebook HTW B...Holger Lütters
 
Zukunft der mobilen Marktforschung GapFish vs Dalia Research; Research & Resu...
Zukunft der mobilen Marktforschung GapFish vs Dalia Research; Research & Resu...Zukunft der mobilen Marktforschung GapFish vs Dalia Research; Research & Resu...
Zukunft der mobilen Marktforschung GapFish vs Dalia Research; Research & Resu...Holger Lütters
 
Device agnostic research sex drugs politics
Device agnostic research sex drugs politicsDevice agnostic research sex drugs politics
Device agnostic research sex drugs politicsHolger Lütters
 
E-Government in Deutschland 2016
E-Government in Deutschland 2016E-Government in Deutschland 2016
E-Government in Deutschland 2016Wolfgang Ksoll
 
Face the facebook: Social Media Market Research
Face the facebook: Social Media Market Research Face the facebook: Social Media Market Research
Face the facebook: Social Media Market Research Holger Lütters
 
Avatars in Market Research
Avatars in Market ResearchAvatars in Market Research
Avatars in Market ResearchHolger Lütters
 
StartingUpBerlin 2013 - das VBKI Gründerbarometer
StartingUpBerlin 2013 - das VBKI GründerbarometerStartingUpBerlin 2013 - das VBKI Gründerbarometer
StartingUpBerlin 2013 - das VBKI Gründerbarometerdeutsche-startups.de
 
Social Network Analysis Holger Luetters HTW Berlin
Social Network Analysis Holger Luetters HTW BerlinSocial Network Analysis Holger Luetters HTW Berlin
Social Network Analysis Holger Luetters HTW BerlinHolger Lütters
 
Open Data - Wertschöpfung im digitalen Zeitalter
Open Data - Wertschöpfung im digitalen ZeitalterOpen Data - Wertschöpfung im digitalen Zeitalter
Open Data - Wertschöpfung im digitalen ZeitalterWolfgang Ksoll
 

Viewers also liked (11)

170530 mobilitätskultur münchner kreis print
170530 mobilitätskultur münchner kreis print170530 mobilitätskultur münchner kreis print
170530 mobilitätskultur münchner kreis print
 
Digitale Ökosysteme: Digitalisierung in Unternehmen
Digitale Ökosysteme: Digitalisierung in UnternehmenDigitale Ökosysteme: Digitalisierung in Unternehmen
Digitale Ökosysteme: Digitalisierung in Unternehmen
 
Social Media Recruiting for Market Research panelbiz vs adidas Facebook HTW B...
Social Media Recruiting for Market Research panelbiz vs adidas Facebook HTW B...Social Media Recruiting for Market Research panelbiz vs adidas Facebook HTW B...
Social Media Recruiting for Market Research panelbiz vs adidas Facebook HTW B...
 
Zukunft der mobilen Marktforschung GapFish vs Dalia Research; Research & Resu...
Zukunft der mobilen Marktforschung GapFish vs Dalia Research; Research & Resu...Zukunft der mobilen Marktforschung GapFish vs Dalia Research; Research & Resu...
Zukunft der mobilen Marktforschung GapFish vs Dalia Research; Research & Resu...
 
Device agnostic research sex drugs politics
Device agnostic research sex drugs politicsDevice agnostic research sex drugs politics
Device agnostic research sex drugs politics
 
E-Government in Deutschland 2016
E-Government in Deutschland 2016E-Government in Deutschland 2016
E-Government in Deutschland 2016
 
Face the facebook: Social Media Market Research
Face the facebook: Social Media Market Research Face the facebook: Social Media Market Research
Face the facebook: Social Media Market Research
 
Avatars in Market Research
Avatars in Market ResearchAvatars in Market Research
Avatars in Market Research
 
StartingUpBerlin 2013 - das VBKI Gründerbarometer
StartingUpBerlin 2013 - das VBKI GründerbarometerStartingUpBerlin 2013 - das VBKI Gründerbarometer
StartingUpBerlin 2013 - das VBKI Gründerbarometer
 
Social Network Analysis Holger Luetters HTW Berlin
Social Network Analysis Holger Luetters HTW BerlinSocial Network Analysis Holger Luetters HTW Berlin
Social Network Analysis Holger Luetters HTW Berlin
 
Open Data - Wertschöpfung im digitalen Zeitalter
Open Data - Wertschöpfung im digitalen ZeitalterOpen Data - Wertschöpfung im digitalen Zeitalter
Open Data - Wertschöpfung im digitalen Zeitalter
 

Similar to Online Panel vs. Mobile Advertising Recruiting in Market Research Study

Moving answers with the gyroscale
Moving answers with the gyroscaleMoving answers with the gyroscale
Moving answers with the gyroscaleHolger Lütters
 
E-Society 2016 - The GOV Indicator - Mark Verhijde
E-Society 2016 - The GOV Indicator - Mark VerhijdeE-Society 2016 - The GOV Indicator - Mark Verhijde
E-Society 2016 - The GOV Indicator - Mark VerhijdeMark Verhijde
 
Evaluating two different mobile research ad-hoc and personal application
Evaluating two different mobile research   ad-hoc and personal applicationEvaluating two different mobile research   ad-hoc and personal application
Evaluating two different mobile research ad-hoc and personal applicationMerlien Institute
 
Effectively aligning m-Government strategies to your organisational objective...
Effectively aligning m-Government strategies to your organisational objective...Effectively aligning m-Government strategies to your organisational objective...
Effectively aligning m-Government strategies to your organisational objective...Reading Room
 
Research in a data-rich world, or faster, cheaper and closer-to- the-moment r...
Research in a data-rich world, or faster, cheaper and closer-to- the-moment r...Research in a data-rich world, or faster, cheaper and closer-to- the-moment r...
Research in a data-rich world, or faster, cheaper and closer-to- the-moment r...Инга Кныш
 
Enlarge citadel v0.1
Enlarge citadel v0.1Enlarge citadel v0.1
Enlarge citadel v0.1Citadelh2020
 
Tns - Mobile measurability unlocks spending final
Tns - Mobile measurability unlocks spending finalTns - Mobile measurability unlocks spending final
Tns - Mobile measurability unlocks spending finalGabriella Bergaglio
 
Digital Vision for CALP
Digital Vision for CALPDigital Vision for CALP
Digital Vision for CALPtaipida
 
Ileger: a web based application for participative elections
Ileger: a web based application for participative elections Ileger: a web based application for participative elections
Ileger: a web based application for participative elections Luis Borges Gouveia
 
Evitant l'Hora Punta, la recompensa a una bona conducta
Evitant l'Hora Punta, la recompensa a una bona conductaEvitant l'Hora Punta, la recompensa a una bona conducta
Evitant l'Hora Punta, la recompensa a una bona conductaAMTU
 
Strategies for product improvement: Tracing the attitude of real estate mobil...
Strategies for product improvement: Tracing the attitude of real estate mobil...Strategies for product improvement: Tracing the attitude of real estate mobil...
Strategies for product improvement: Tracing the attitude of real estate mobil...inventionjournals
 
Market and Social Research Part 5
Market and Social Research Part 5Market and Social Research Part 5
Market and Social Research Part 5bestsliders
 
Digitally-Enabled Market Research Techniques
Digitally-Enabled Market Research TechniquesDigitally-Enabled Market Research Techniques
Digitally-Enabled Market Research TechniquesHawkPartners
 
24 millica begovic undp 2013
24 millica begovic undp 201324 millica begovic undp 2013
24 millica begovic undp 2013UNDPhr
 
Policy Compass - WeGO Workshop Presentation
Policy Compass - WeGO Workshop PresentationPolicy Compass - WeGO Workshop Presentation
Policy Compass - WeGO Workshop PresentationPolicyCompass
 
Colloquium_Bart_Lenderink_20012015
Colloquium_Bart_Lenderink_20012015Colloquium_Bart_Lenderink_20012015
Colloquium_Bart_Lenderink_20012015Bart Lenderink
 
Case Study: TULI Programme, Finland
Case Study: TULI Programme, FinlandCase Study: TULI Programme, Finland
Case Study: TULI Programme, FinlandDUCO
 

Similar to Online Panel vs. Mobile Advertising Recruiting in Market Research Study (20)

Moving answers with the gyroscale
Moving answers with the gyroscaleMoving answers with the gyroscale
Moving answers with the gyroscale
 
Co-creation of a tourist experience enhanced by technology, in the context of...
Co-creation of a tourist experience enhanced by technology, in the context of...Co-creation of a tourist experience enhanced by technology, in the context of...
Co-creation of a tourist experience enhanced by technology, in the context of...
 
E-Society 2016 - The GOV Indicator - Mark Verhijde
E-Society 2016 - The GOV Indicator - Mark VerhijdeE-Society 2016 - The GOV Indicator - Mark Verhijde
E-Society 2016 - The GOV Indicator - Mark Verhijde
 
Evaluating two different mobile research ad-hoc and personal application
Evaluating two different mobile research   ad-hoc and personal applicationEvaluating two different mobile research   ad-hoc and personal application
Evaluating two different mobile research ad-hoc and personal application
 
Effectively aligning m-Government strategies to your organisational objective...
Effectively aligning m-Government strategies to your organisational objective...Effectively aligning m-Government strategies to your organisational objective...
Effectively aligning m-Government strategies to your organisational objective...
 
Research in a data-rich world, or faster, cheaper and closer-to- the-moment r...
Research in a data-rich world, or faster, cheaper and closer-to- the-moment r...Research in a data-rich world, or faster, cheaper and closer-to- the-moment r...
Research in a data-rich world, or faster, cheaper and closer-to- the-moment r...
 
Enlarge citadel v0.1
Enlarge citadel v0.1Enlarge citadel v0.1
Enlarge citadel v0.1
 
Tns - Mobile measurability unlocks spending final
Tns - Mobile measurability unlocks spending finalTns - Mobile measurability unlocks spending final
Tns - Mobile measurability unlocks spending final
 
Digital Vision for CALP
Digital Vision for CALPDigital Vision for CALP
Digital Vision for CALP
 
Ileger: a web based application for participative elections
Ileger: a web based application for participative elections Ileger: a web based application for participative elections
Ileger: a web based application for participative elections
 
Evitant l'Hora Punta, la recompensa a una bona conducta
Evitant l'Hora Punta, la recompensa a una bona conductaEvitant l'Hora Punta, la recompensa a una bona conducta
Evitant l'Hora Punta, la recompensa a una bona conducta
 
Strategies for product improvement: Tracing the attitude of real estate mobil...
Strategies for product improvement: Tracing the attitude of real estate mobil...Strategies for product improvement: Tracing the attitude of real estate mobil...
Strategies for product improvement: Tracing the attitude of real estate mobil...
 
Market and Social Research Part 5
Market and Social Research Part 5Market and Social Research Part 5
Market and Social Research Part 5
 
Mobility intro2014
Mobility intro2014Mobility intro2014
Mobility intro2014
 
Digitally-Enabled Market Research Techniques
Digitally-Enabled Market Research TechniquesDigitally-Enabled Market Research Techniques
Digitally-Enabled Market Research Techniques
 
24 millica begovic undp 2013
24 millica begovic undp 201324 millica begovic undp 2013
24 millica begovic undp 2013
 
Policy Compass - WeGO Workshop Presentation
Policy Compass - WeGO Workshop PresentationPolicy Compass - WeGO Workshop Presentation
Policy Compass - WeGO Workshop Presentation
 
Colloquium_Bart_Lenderink_20012015
Colloquium_Bart_Lenderink_20012015Colloquium_Bart_Lenderink_20012015
Colloquium_Bart_Lenderink_20012015
 
Case Study: TULI Programme, Finland
Case Study: TULI Programme, FinlandCase Study: TULI Programme, Finland
Case Study: TULI Programme, Finland
 
European Network of Living Labs
European Network of Living LabsEuropean Network of Living Labs
European Network of Living Labs
 

Recently uploaded

Machine learning workshop, CZU Prague 2024
Machine learning workshop, CZU Prague 2024Machine learning workshop, CZU Prague 2024
Machine learning workshop, CZU Prague 2024Gokulks007
 
The Real Story Of Project Manager/Scrum Master From Where It Came?!
The Real Story Of Project Manager/Scrum Master From Where It Came?!The Real Story Of Project Manager/Scrum Master From Where It Came?!
The Real Story Of Project Manager/Scrum Master From Where It Came?!Loay Mohamed Ibrahim Aly
 
Dynamics of Professional Presentationpdf
Dynamics of Professional PresentationpdfDynamics of Professional Presentationpdf
Dynamics of Professional Presentationpdfravleel42
 
Burning Issue presentation of Zhazgul N. , Cycle 54
Burning Issue presentation of Zhazgul N. , Cycle 54Burning Issue presentation of Zhazgul N. , Cycle 54
Burning Issue presentation of Zhazgul N. , Cycle 54ZhazgulNurdinova
 
Communication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxCommunication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxkb31670
 
Juan Pablo Sugiura - eCommerce Day Bolivia 2024
Juan Pablo Sugiura - eCommerce Day Bolivia 2024Juan Pablo Sugiura - eCommerce Day Bolivia 2024
Juan Pablo Sugiura - eCommerce Day Bolivia 2024eCommerce Institute
 
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8Access Innovations, Inc.
 
Communication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxCommunication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxkb31670
 

Recently uploaded (8)

Machine learning workshop, CZU Prague 2024
Machine learning workshop, CZU Prague 2024Machine learning workshop, CZU Prague 2024
Machine learning workshop, CZU Prague 2024
 
The Real Story Of Project Manager/Scrum Master From Where It Came?!
The Real Story Of Project Manager/Scrum Master From Where It Came?!The Real Story Of Project Manager/Scrum Master From Where It Came?!
The Real Story Of Project Manager/Scrum Master From Where It Came?!
 
Dynamics of Professional Presentationpdf
Dynamics of Professional PresentationpdfDynamics of Professional Presentationpdf
Dynamics of Professional Presentationpdf
 
Burning Issue presentation of Zhazgul N. , Cycle 54
Burning Issue presentation of Zhazgul N. , Cycle 54Burning Issue presentation of Zhazgul N. , Cycle 54
Burning Issue presentation of Zhazgul N. , Cycle 54
 
Communication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxCommunication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptx
 
Juan Pablo Sugiura - eCommerce Day Bolivia 2024
Juan Pablo Sugiura - eCommerce Day Bolivia 2024Juan Pablo Sugiura - eCommerce Day Bolivia 2024
Juan Pablo Sugiura - eCommerce Day Bolivia 2024
 
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8
 
Communication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxCommunication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptx
 

Online Panel vs. Mobile Advertising Recruiting in Market Research Study

  • 1. General Online Research Conference 2015 – GOR15 18-20 March 2015, Cologne University of Applied Sciences, Germany Holger Lütters – Hochschule für Technik und Wirtschaft Berlin Suggested citation: Lütters, Holger (2015): New survey recruiting strategies: Online Panel vs. Mobile Advertising. General Online Research (GOR) Conference, Cologne. Data Report available under http://goo.gl/K6xH4t Contact: holger.luetters@htw-berlin.de @luetters New survey recruiting strategies: Online Panel vs. Mobile Advertising This work is licensed under a Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/)
  • 2. 1 Study set-up and field report Empirical study in comparison of two market research recruiting approaches: Online Panel vs. Mobile Ad
  • 3. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 3 • Questionnaire development • Online Panel • Mobile Ad • Questionnaire Technology Berlin based market researchers teamed up for the research project Berlin Market Research Initiative Field report • 3029 contacts • 1732 complete answers • 8 minutes questionnaire with questfox® using Gyroscale® • Random distribution of demographics to determine differences in sampling • Quota on devices • Survey in July 2014 by
  • 4. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 4 Two recruiting approaches competing with one identical questionnaire GapFish • (Classic) Online Panel www.entscheiderclub.de • 110.000 participants • Germany, Switzerland, Austria • Advertising sampling approach • Global reach (200 million potential users) • All languages possible (need of developed mobile advertising market structures) Invitation on third party ad platform Generic invitation Real-time screening Re-Direct to third party survey (Non-)Monetary Incentive
  • 5. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 5 Methodological question: What do participants in studies of (mobile) market research surveys expect from research? Methodological issues: • Effects of different sources on the data? • Effects on non-reactive data (response time, willingness to participate, etc.)? • Effects of different methods used on different devices (AHP, PSM, GyroScale) General interest in • Expectations of users towards mobile surveys • Decision-making factors of survey participation • Expectations of compensation for participation in (mobile) surveys
  • 6. Online Panel vs. Mobile Ad Recruiting Sociodemographics, Devices and Non-Reactive Data 2
  • 7. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 7 Target per approach n=800 spread across four device categories with a quota of 200 each
  • 8. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 8 The two approaches reached a similar, structurally representative regional population pattern without quota setting Same questiontype for all device categories; Launch from Berlin Online Panel GapFish (n=888) Mobile Ad Dalia Research (n=811); Data BRD DESTATIS 2012 10,7% 12,5% 7,8% 4,7% 0,5% 2,4% 7,2% 2,0% 5,2% 22,9% 3,6% 1,6% 6,2% 3,5% 5,4% 3,5% 9,9% 16,2% 7,2% 2,5% 1,0% 2,6% 5,7% 2,7% 9,7% 21,9% 4,7% 1,2% 4,9% 2,1% 3,5% 2,8% 13,1% 15,5% 4,2% 3,0% 0,8% 2,2% 7,5% 2,0% 9,7% 21,8% 5,0% 1,2% 5,0% 2,8% 3,5% 2,7% Baden-Württemberg Bayern Berlin Brandenburg Bremen Hamburg Hessen Mecklenburg-Vorpommern Niedersachsen Nordrhein-Westfalen Rheinland-Pfalz Saarland Sachsen Sachsen-Anhalt Schleswig-Holstein Thüringen Online Panel Mobile Ad BRD
  • 9. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 9 Mobile Ad approach is significantly less representative for sociodemographics, reaching younger and more male participants Age Distribution Gender Base: Online Panel GapFish (n=888); Mobile Ad Dalia Research (n=811); no quota settings Female Male Female Male Female Male Mobile Ad Online Panel Average Age Mobile Ad Online Panel
  • 10. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 10 Both approaches are not representative in terms of education. Mobile Ad approach achieves higher educational levels Higher Education A-Level vocational school comprehensive school secondary school Without formal degree No answer Base: Online Panel GapFish (n=888); Mobile Ad Dalia Research (n=811); no quota in question
  • 11. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 11 “Couch surfing” as a standard pattern in mobile research 4 out of 5 participants in the study are mobile …but at home At Home Work / School / University Public Place On the road None of these places
  • 12. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 12 Technographics: Mobile Ad participants have a slightly more modern equipment, use a broader variety of browsers but do not seem to be more tech savvy in general
  • 13. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 13 Mobile participants tend to speed through a questionnaire Panel participants take much more time on a mobile device Base: Online Panel GapFish (n=888); Mobile Ad Dalia Research (n=811); Information in seconds for complete interviews Speeder Mobile Ad Online Panel Response time in seconds
  • 14. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 14 The two approaches show different behavioral patterns Criteria Online Panel Mobile Ad Contacts 1912 1032 Complete Interviews 888 811 Average Completion Rate 79,5% 82,9% Quota reach Complete Incomplete Duration 7 days 13 days Response Time Completes 497 seconds 441 seconds Response Time including drop-outs 296 seconds 383 seconds Overall time used in seconds 565.952 395.256 Drop-out pattern Decision in the first minute (independent from device used) Slightly higher drop-out during the entire interview (independent from device used) Very good and good consistency AHP (CRH< 0,2) 29,5% 29,2% Inacceptable consistency AHP (CRH> 0,5) 24,0% 29,0%
  • 15. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 15 Conclusion non-reactive data: Strong differences in response behavior: Mobile participants respond faster (even on the desktop) Online Panel ● Short overall project time ● Intensive use of panelists ● Slightly more drop-outs ● Exact quota reach ● Longer interview duration ● Smartphone interview much longer Mobile Ad ● Double overall project time ● Used less participants ● Slightly less drop-outs ● Quota not reached for classic devices ● Speeding behavior: All interviews are answered quicker on all kinds of devices
  • 16. 3 Expectations of participants when taking surveys
  • 17. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 17 The online panelists have much more experience with surveys of market research compared to the mobile participants Basis: Online Panel GapFish (n=888); Mobile Panel Dalia Research(n=811) und n=31 professionelle Marktforscher Mittelwerte einer Skala von 0 (überhaupt keine Erfahrung bis 10 extrem viel Erfahrung) Mean: Online Panel 6,63 Mobile Ad 3,72
  • 18. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 18 ¾ of the respondents prefer the (classic) online survey Participants of the mobile recruitment are less enthusiastic about online surveys (and could live without any kind of survey) Basis: Online Panel GapFish (n=888); Mobile Panel Dalia Research (n=811) und n=31 professionelle Marktforscher
  • 19. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 19 Fair compensation for entertaining – but short - interviews are demanded. The Mobile Ad group has lower or no expectations Basis: Online Panel GapFish (n=888); Mobile Panel Dalia Research (n=811) und n=31 professionelle Marktforscher Mittelwerte einer Skala von 0 (trifft überhaupt nicht zu bis 10 trifft voll und ganz zu)
  • 20. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 20 Compensation and duration are the most important factors influencing the willingness to participate in a survey Questions asked in pairwise comparisons with Analytic Hierarchy Process
  • 21. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 21 The majority of online panelists expect the same compensation for a participation in mobile market research Frage nur an n=879 Teilnehmer Online Panel GapFish und n=30 Teilnehmer MaFo Profis Frage nach Kompensation/Kosten im Vergleich zu klassischer Online-Befragung
  • 22. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 22 The mobile participants expect a form of compensation derived from the context of their recruitment into the study
  • 23. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 23 Price Sensitivity Meter to measure a fair compensation for participating in a mobile study Imagine the following invitation to a survey: Topic: Shopping; Duration: 4 minutes; participation only to mobile device (smartphone / tablet)
  • 24. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 24 Participants express high hopes for the incentive of a 4 minute mobile interview which is not backed-up by the willingness to pay of Market Research Professionals Online Panel: 3,56 € Mobile Ad: 4,52 € MR Professionals: 0,94 € Optimal Price Point (VanWestendorp)
  • 25. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 25 The identical survey is evaluated very differently by the different approaches Base: Online Panel GapFish (n=888); Mobile Ad Dalia Research(n=811) Measurement Net Promoter Score 0=would not recommend at all .. 10 totally would recommend NPS: 0-6 Detractor; 7-8 Neutral; 9-10 Promotor
  • 26. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 26 Mobile participants are shockingly honest about lying ... who is less honest? Basis: Online Panel GapFish (n=888); Mobile Panel Dalia Research(n=811) und n=31 professionelle Marktforscher; Mittelwerte einer Skala von 0-10 unter Anwendung NPS: 0-6 gelogen; 7-8 neutral; 9-10 total wahrheitsgemäß)
  • 27. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 27 Mobile research is happening anyway. We better find ways to manage it appropriately Mobile Ad approach works • Dalia Research developed a new technology which offers more options to the researcher • The approach can be as appropriate as it can be wrong for a specific topic • The reach of fresh and unexperienced target groups is an interesting option • Complicated approaches with special methodologies might not work (due to the screen size and due to the user) Things to consider when using the Mobile Ad recruiting approach • Mobile takes more time in creation, testing and surveying • Might be more expensive (overall) • Mobile users have different expectations which are not necessarily the same as before • Researchers should not lower their standards in order to please the new target groups
  • 28. Holger Lütters · New survey recruiting strategies · #GOR15 · Cologne 20th March 2015 · @luetters 28 Author: Holger Lütters – HTW Berlin Contact information Prof. Dr. Holger Lütters Hochschule für Technik und Wirtschaft Berlin Fachbereich 3 Wirtschafts- und Rechtswissenschaften Treskowallee 8 D 10318 Berlin -Germany Raum TA VG 733 Holger.Luetters@htw-berlin.de www.luetters.com Twitter @luetters xing.com/profile/Holger_Luetters de.linkedin.com/in/luetters/ The project was one of the interactions during the HTW Berlin / HWR Berlin Project InnoRadar. InnoRadar is supported by the Berlin Institute of Applied Research (IFAF) www.ifaf-berlin.de. Thank you for your interest in the study Data live report available under http://goo.gl/K6xH4t Thanks to the supporters of the study: www.GapFish.com www.DaliaResearch.com www.pangealabs.com Gyroscale® is a development of pangea labs GmbH Switzerland and is available in the SaaS survey tool questfox® www.questfox.com

Editor's Notes

  1. 1
  2. Sampling over advertising platforms is a technical challenge Dalia Research offers a thrilling approach (even compared to Google)
  3. First presentation of the use of the Gyroscale: Idea and first study In this session the results of the first test with the Gyroscale are presented
  4. The Mobile Ad is generally a little more mobile (but also much at home)
  5. Mobile Ad participants are using more recent tech equipment and less Apple devices
  6. Online Panel 6,63 Mobile Ad 3,72
  7. während die Mehrheit der Profis keine erhöhten Kosten durch mobile Marktforschung kommen sieht