2. Slide 1.2
FREE BOOK: Download it!
http://www.b2binternational.com/b2b-blog/free-ebook-
questionnaire-design/
Also see http://pareonline.net/pdf/v10n12.pdf
3. Slide 1.3
Session Contents
How to Make Your Questionnaire Great !!!!
Who Should you Send it To?
What Type of Questions Are There?
4. Slide 1.4
Foundation
Definition of Questionnaires
Techniques of data collection in which each person
is asked to respond to the same set of questions in
a predetermined order
Adapted from deVaus (2002)
5. Slide 1.5
Types of questionnaires
Types of questionnaire
Saunders et al. (2009)
Figure 11.1 Types of questionnaire
6. Slide 1.6
Should I Choose On-line or Face to Face?
Consider
Characteristics of the respondents and access
Respondents answers not being contaminated or distorted
Size of sample required for analysis
Type and number of questions required
Available resources including use of computer software
7. Slide 1.7
How to get a Good Response Rate
http://www.cartoonstock.com/directory/q/questionnaire.asp
8. Slide 1.8
How to Encourage a Good Response Rate
Careful design of the questionnaire
Clear & pleasing layout
Clear statement of the purpose of the questionnaire
Clear questions
Pilot testing
Careful planning and execution
9. Slide 1.9
Design of the Questionnaire
Cover Page / Letter
Directions (What to do)
Page Design
Order of Questions
Grouping of Questions
Navigational Path
Survey Length
10. Slide 1.10
Cover Page
Good quality paper:
Official letterhead / logo (obtain permission):
Clear Title:
Date:
Greeting:
1st key message: Purpose
2nd key message: Value your response & ‘x’ mins
3rd key message: Confidentiality
4th key message: Results
Contact point for return / queries
11. Slide 1.11
First Set of Questions:
Apply to everyone
Easy to answer in a few seconds
Easy to read, understand & respond to (CLOSED)
Interesting
Connect to the purpose of the survey
12. Slide 1.12
Question Groupings
Group by content, user can focus & organise thoughts
Group by type of question (e.g. all rating questions
together)
Colour to establish groupings
Objectionable questions at the end
13. Slide 1.13
Question Layout
Short and easy to answer
Avoid double barreled questions
Dark print for questions & light print for answer options
Consistent in layout
E.g. Scales go the same way
E.g. The Phrasing of the questions is consistent
People like putting ‘X’ in boxes
14. Slide 1.14
Navigational Path
If everyone does not need to answer all questions make it clear where they
should carry on.
E.g. Do you use a Mac computer at work?
(check one)
Yes Skip to question 9
No
15. Slide 1.15
Types of Question
http://www.cartoonstock.com/directory/q/questionnaire.asp
16. Slide 1.16
Types of Questions
Classification Information that can be used to group
respondents to see how they differ one
from the other - such as age, gender,
social class, location of household, type
of house, family composition.
Behavioural Factual information on what the
respondent is, does or owns. Also the
frequency with which certain actions are
carried out.
Attitudinal What people think of something. Their
image and ratings of things. Why they do
things.
17. Slide 1.17
Classification Questions
Gender.
Female
Male
•Household status.
- Head of household ( )
- Housewife ( )
- Other adult ( )
•Marital status. This is usually asked by simply saying "Are you ....."
- Single ( )
- Married ( )
- Widowed ( )
- Divorced ( )
- Separated ( )
18. Slide 1.18
Examples of Attitudinal questions:
• What do you think of ........?
• Why do you ........?
• Do you agree of disagree ........?
• How do you rate ........?
• Which is best (or worst) for ........?
19. Slide 1.19
Examples of Behavioural questions:
• Have you ever ........?
• Do you ever ........?
• Who do you know ........?
• When did you last ........?
• Which do you do most often ........?
• Who does it ........?
• How many ........?
• Do you have ........?
• In what way do you do it ........?
• In the future will you ........?
20. Slide 1.20
Examples of question types (1)
Open questions
6 Please list up to three things you like about your job
1…………………………………………
2…………………………………………
3…………………………………………
Useful for Attitudes
Saunders et al. (2009)
21. Slide 1.21
Examples of question types (2)
List questions
7 What is your home city?
Please tick the appropriate box
Dalian 大连 Shanghai 上海
Chongqing 重庆 Beijing 北京
Chengdu 成都 Hong Kong 香港
Hangzhou 苏杭 Guangzhou 广州
Changsha 长沙
Nanjing 南京
Useful for Classification & also Behaviours
22. Slide 1.22
Examples of question types (3)
Category questions
8 How often do you visit the shopping centre?
Interviewer: listen to the respondent’s answer and tick as
appropriate
First visit
Once a week
Less than fortnightly to once a month
2 or more times a week
Less than once a week to fortnightly
Less often
Saunders et al. (2009)
23. Slide 1.23
Examples of question types (4)
Ranking questions
9 Please number each of the factors listed below in order
of importance to you in choosing a new car. Number the
most important 1, the next 2 and so on. If a factor has no
importance at all, please leave blank.
Factor Importance
Carbon dioxide emissions [ ]
Boot size [ ]
Depreciation [ ]
Price [ ]
Adapted from Saunders et al. (2009)
24. Slide 1.24
Examples of question types (5)
Rating questions
10 For the following statement please tick the box that
matches your view most closely
Agree Tend to agree Tend to disagree Disagree
I feel employees’
views have
influenced the
decisions taken
by management
Saunders et al. (2009)
25. Slide 1.25
Rating Categories
Agreement:
Strongly agree / agree / neither agree nor disagree / disagree / strongly
disagree
Amount:
Far too much / too much / about right / too little / Far too little
Frequency:
Nearly all the time / frequently / sometimes / rarely / practically never
Likelihood:
Very / good / reasonable / slightly / not at all
26. Slide 1.26
Examples of question types (6)
Quantity questions
14 What is your year of birth?
(For example, for 1988 write: )
Saunders et al. (2009)
1
1
9
9 8 8
27. Slide 1.27
CLASSIFICATION : SEX / AGE / SALARY ETC.
OPEN WHAT DO YOU ENJOY …
LIST TICK WHICH ARE RELEVANT
RANKING LIST FROM MOST IMPORTANT
TO LEAST
RATING STRONGLY AGREE TO STRONG
DISAGREE
QUANTITY
28. Slide 1.28
Who Should I Send it To? : Selecting Samples
http://www.cartoonstock.com/directory/q/questionnaire.asp
30. Slide 1.30
The importance of response rate
Key considerations
Non- respondents and analysis of refusals
Obtaining a representative sample
Calculating the active response rate
Estimating response rate and sample size
31. Slide 1.31
The need to sample
Sampling- a valid alternative to a census when
A survey of the entire population is impracticable
Budget constraints restrict data collection
Time constraints restrict data collection
Results from data collection are needed quickly
30+ in each category is a useful rule of thumb
32. Slide 1.32
Sample size
Choice of sample size is influenced by
Confidence needed in the data
Margin of error that can be tolerated
Types of analyses to be undertaken
Size of the sample population and distribution
33. Slide 1.33
Overview of sampling techniques
Sampling techniques
Source: Saunders et al. (2009)
Figure 7.2 Sampling techniques
34. Slide 1.34
Probability sampling
The four stage process
1. Identify ALL the possible cases from research objectives
2. Decide on a suitable sample size (larger the sample to
lower the chance of error)
3. Select the appropriate technique and the sample
4. Check that the sample is representative (if 60% of the
sample are x then 60% population are x)
35. Slide 1.35
Simple random: Next 10 people in the room
Systematic: People whose student number ends in ‘3’
Quota sampling : Random but 80% Female in BFSU
Snowball sampling: Subject 1 identifies ‘3’ other people
Self-selection sampling: “we are interviewing on ... Come along...”
Convenience sampling: Grab & Go
Sampling techniques
36. Slide 1.36
Sampling Summary:
Choice of sampling techniques depends upon the
research question(s) and their objectives
Factors affecting sample size include:
- confidence needed in the findings
- accuracy required
- likely categories for analysis
37. Slide 1.37
End of the sampling detour !!!
All choices depend on the ability to gain access to
organisations
38. Slide 1.38
Designing individual questions (2)
Right level of detail?
Do they have the right knowledge?
Have you avoided jargon?
Could your question cause offence?
Could your question be shorter?
Are you asking more than one question?
Does your question imply the right answer?
Is your question likely to embarrass the respondent?
39. Slide 1.39
Summary:
Data validity and reliability and response rate depend on
design, structure and rigorous pilot testing
Wording and order of questions and question types are
important considerations
Closed questions should be pre-coded to facilitate data
input and analysis
40. Slide 1.40
Summary:
Important design features are a clear layout, a logical
order and flow of questions and easily completed
responses
Questionnaires should be carefully introduced and pilot
tested prior to administration
Administration needs to be appropriate to the type of
questionnaire
42. Slide 1.42
Quantitative data analysis
Key points
Data must be analysed to produce information
Computer software analysis is normally used for this
process (Microsoft Excel, SPSS etc.)
Present, explore, describe & examine relationships
43. Slide 1.43
Examples of basic chart
Pie chart
Saunders et al. (2009)
Figure 12.8 Pie chart
44. Slide 1.44
More advanced work requires Statistical analysis
Establishing the statistical relationship between two
variables (e.g. If I am in this group I am have a %
probability of doing X).
If you need to do this then see:
http://www.statsoff.com/textbook
http://oli.web.cmu.edu/openlearning/forstudents/
freecourses/statistics
45. Slide 1.45
Quantitative data analysis: Main Concerns
Preparing, inputting and checking data
Choosing the most appropriate statistics to
describe the data
Choosing the most appropriate statistics to
examine data relationships and trends
46. Slide 1.46
Type of Data: category data
Example: Number of cars hatchback / saloon /
estate
Can’t measure it, just simply count occurrences
Focus on one discrete variable (i.e. Hatchback)
Dichotomous data (e.g. either Male or Female)
Ranked data (how strongly you agree with
statement X)
47. Slide 1.47
Type of Data: numerical data
Example: height of students
Quantifiable data that can be measured
Interval data e.g. Degrees Celsius [zero
degrees is not actually ZERO]
Ratio (calculate the difference) data e.g.
Profits up 34% for a year
48. Slide 1.48
Type of Data: continuous data
Example: height of students
Can be any value [within a range]
49. Slide 1.49
Level of Precision
Category Numerical Continuous
LESS MORE
Precise data can be grouped to make it less precise
(e.g. Mark of 85% grouped into a ‘Very Good’ category but
Not the other way round)
50. Slide 1.50
Exploring Data: Tukey’s (1977) exploratory data
analysis approach focus on tables & diagrams
Great Tables & Diagrams Need:
Clear & Distinctive Title
Clearly stated units of measurement
Clearly stated source of data
Abbreviations explained in notes
Size of the sample is stated “n = 43”
Column / Row / Axis Labels
Dense shading for smaller areas
Logical Sequence of columns & rows
51. Slide 1.51
Exploratory Analysis: Individual unit of data
Highest and lowest values
Trends over time
Proportions (relative size)
Distributions (number in a group)
Sparrow (1989)
52. Slide 1.52
What Do You Want To Show?
Highest / Lowest: Bar Chart / Histogram for Categories
You can reordered it for Non-continuous data
53. Slide 1.53
What Do You Want To Show?
Frequency: Again a Histogram / Bar Chart (reorder it
to make it clearer)
Perhaps a pictogram
57. Slide 1.57
Normal Distribution
Sample of 100+ people should
produce a normal curve.
Standard deviation shows how wide
the spread of results are.
Low standard deviation shows a narrow range of values
High standard deviation shows a wide range of values
58. Slide 1.58
How to calculate it:
Consider a population consisting of the following eight values:
2, 4, 4, 4, 5, 5, 7, 9
Calculate the Mean (2, 4, 4, 4, 5, 5, 7, 9) / 8 = 5
Calculate the difference between each individual data point and
the mean. Then square each one
Calculate the average of these values (i.e. 32 / 8 = 4)
Find the sqaure root of this number (square root of 4 is 2)
http://www.statsoff.com/textbook
http://oli.web.cmu.edu/openlearning/forstudents/freecourses/statistics
59. Slide 1.59
Exploring and presenting data (4)
Comparing variables to show
Specific values and independence
Highest and lowest values
Proportions
Trends and conjunctions
60. Slide 1.60
Exploring and presenting data (5)
Comparing variables to show
Totals
Proportions and totals
Distribution of values
Relationship between cases for variables
61. Slide 1.61
Describing data using statistics (1)
Statistics to describe a variable focus on
two aspects
The central tendency
The dispersion
62. Slide 1.62
Describing data using statistics (2)
Describing the central tendency
To represent the value occurring most frequently
To represent the middle value
To include all data values
63. Slide 1.63
Describing data using statistics (3)
Describing the dispersion
To state the difference between values
To describe and compare the extent by which values
differ from the mean
64. Slide 1.64
Examining relationships, differences and
trends
Using statistics to
Test for significant relationships and differences
Assess the strength of relationship
Examine trends
65. Slide 1.65
Summary:
Data for quantitative analysis can be collected and
then coded at different scales of measurement
Data type constrains the presentation, summary and
analysis techniques that can be used
Data are entered for computer analysis as a matrix
and recorded using numerical codes
Codes should be entered for all data values
Existing coding schemes enable comparisons
66. Slide 1.66
Summary:
Data must be checked for errors
Initial analysis should use both tables and diagrams
Subsequent analyses involve describing data and
exploring relationships by using statistics
Longitudinal data may necessitate different
statistical techniques