In this lecture you will learn about the importance of research questions, how they related to research problems, the properties of good research questions, and the differences between quantitative and qualitative research questions.

1. ICT
Research Questions

2. ICT
Agenda
 Why research questions?
 Relevance
 Rigor
 How to formulate one?
 Design and creation
 Positivist
 Interpretive

3. ICT
Why research questions?
Discussion
(Write down your research question!)

4. ICT
The thinking man
You
Other researchers
Your research
Practice
Rigor
Relevance

5. ICT
Practical vs. Research Problems
Practical
Problem
Research
Question
Research
Problem
Research
Answer
motivates
definesfinds
helps to solve
Applied
Research
Purpose
Aim
Objective

6. ICT
Conceptual Problems
Conceptual
Problem
Research
Question
Research
Problem
Research
Answer
motivates
definesfinds
helps to solve
Pure/Basic
Research

7. ICT
Practical vs. Research Problems
Condition Cost
Practical Problem An undesirable
situation in the real
world
Unhappiness, pain,
material cost, social
cost, etc.
Research Problem Not knowing or not
understanding
something
Cost of not knowing
(and thereby not
being able to solve
the practical
problem).

8. ICT
Examples
Practical problem Research question
An aging population leads to more
fall injuries. These injuries are costly
in terms of mental, social and
economical costs.
• What do we know about the causes
of falls among seniors?
• What do we know about the role of
IT in preventing falls among
seniors?
• What type of IT systems are best in
preventing falls among seniors?

9. ICT
The "So what?" Test
 Topic: I am studying X
 Question: Because I want to answer Y
 Significance: In order to help solve real world problem Z
 From "So what?"
 So what if problem Z is not solved?
 To "OMG! Tell me, what do we do about that?"
 Problem Z is important also for me! Please solve it!
 Discussion: How relevant are your research questions?

10. ICT
Examples of the "So what?" test
 Topic: I am studying falls among seniors.
 Question: Because I want to find out what causes these falls.
 Significance: In order to create IT-based solutions that can reduce
falls among seniors.
 Topic: I am studying X
 Question: Because I want to answer Y
 Significance: In order to help solve real world problem Z

11. ICT
Agenda
 Why research questions?
 Relevance
 Rigor
 How to formulate one?

12. ICT
How does a rigorous research
question look like?
Discussion

13. ICT
Non-rigorous research questions
 Research questions that are not asked.
 Research questions that are not answered.
 Research questions that have Yes/No answers.
 Research questions whose answers don't produce any
new knowledge.
 Research questions that don't have "symmetry of
outcome".
 Research questions that are too broad.
 Research questions that are too narrow.
 Research questions that include presumptions.

14. ICT
Rigorous RQs should
 Be focused,
 Identify the right paradigm,
 Positivist, interpretive, critical.
 Drive the right research design,
 Case study, design and creation, survey,….
 Drive the management of the research project,
 Needed time,
 Needed resources,
 Inherent dependencies on others.

15. ICT
RQs should be focused
Research Area
Research Topic
Research Question
Problem
space
dimension

16. ICT
How to focus? Qualitative example.
Coherence
 Use of active verb:
 Understanding, Exploring,
Interpreting, Constructing,
Explaining, etc.
 Use of relevant nouns:
 Experiences, Feelings, Views,
Perspectives, Knowledge, etc.
 Indication of methodology:
 Grounded theory, Action research,
Exploratory study, etc.
Structure
 What will be studied?
 Who will be studied
 When will they be studied
 Where will they be studied
 What will be studied
 How it will be studied
 Why it will be studied

17. ICT
Examples of focusing
Area Topic Research Question
Aging and IT Falls among seniors and
the use of IT
What do we know about the role
of IT in preventing falls among
seniors?
Empirical evidence of IT
and falls among elderly
What types of systems have
shown to actually reduce falls
among seniors?
Exercise-based
interventions for fall
prevention and the role of
IT
How can an IT system help
seniors to comply with preventive
exercise interventions?
What is the impact of using
system X to promote exercise Y
with N elderly living in nursing
home A in Trondheim?

18. ICT
Research question should drive
research design
Practical
Problem
Research
Question
Research
Problem
Research
Answer
motivates
definesfinds
helps to solve

19. ICT
The research design process
What is the impact of using
system X to promote
exercise Y with N elderly
living in nursing home A in
Trondheim?

20. ICT
Primary and secondary RQs
Primary
Research
Question
Primary
Research
Question
Secondary
RQ1
Secondary
RQ1
Secondary
RQ1
Secondary
RQ1
Secondary
RQ1
Secondary
RQ1
help answer help answer
Contributory research questions Ancillary research questions
Remember: There should always be a relation between primary and secondary RQ!

21. ICT
Focusing through designing:
Lean research and iterative RQs
Pre-pilot Pilot Main study
Secondary
Question 1
Secondary
Question 2
Secondary
Question 3
Main
Question

22. ICT
Secondary questions and iterations
Sample n = 100
e.g. questionnaire
Sub-sample n = 10
e.g. interviews
Sub-
sub-
sample
n = 1
e.g. case study
Progress of the
research
Third sample n = 100
e.g. questionnaire
Second sample n = 10
e.g. interviews
Initial sample
n = 1
e.g. case
study

23. ICT
Examples of primary and secondary RQs
Primary RQ Secondary RQs
What is the impact of using system X
to promote exercise Y with N seniors
living in nursing home A in
Trondheim?
• What is system X?
• What do seniors living in nursing
home A think system X is?
• What do seniors in nursing home
A think about a paper prototype of
system X?
• What is the effect of using system
X for 4 weeks to promote exercise
Y with N seniors…..?
• What is the effect… after one
year?

24. ICT
Check list
 Is my RQ something that I am curious about and that others might
care about? Does it present an issue on which I can take a stand?
 Does my RQ put a new spin on an old issue, or does it try to solve a
problem?
 Is my RQ too broad, too narrow, or OK?
 Is my RQ researchable…
 …within the time frame of the assignment?
 …given the resources available at my location?
 Is my RQ measurable? What type of information do I need? Can I find
actual data to support or contradict a position?
 What sources will have the type of information that I need to answer
my RQ (journals, books, internet resources, government documents,
interviews with people)?

25. ICT
Mr. Rigorosi and Mrs. Relevanté
Who wins the H2020 race?

26. ICT
Mr. Rigorosi and Mrs. Relevanté
Who wins the Nobel prize?

27. ICT
Agenda
 Why research questions?
 Relevance
 Rigor
 How to formulate one?
 Design and creation
 Positivist
 Interpretive

28. ICT
RQs should use the right wording
 "What is the effect of ….?"
 Requires a design based on Randomized Controlled Trials.
 "What is the impact/influence of ….?"
 Might require case study research.
 "What are the attitudes of … towards …?"
 Might require case study or ethnography.
 Etc.

29. ICT
RQ should use the right wording
 Positivist:
 "What is…..?"
 "How big….?"
 Interpretist:
 "How do….?"
 "Why do…..?"
 Critical:
 "Why is it….?"
 "Why not….?"
 Design and creation
 How good is….?
 How can we create…?

30. ICT
Design and creation RQs
 First ask yourself: What is the role of the IT system in my
research?
 Is it: The main focus of the research?
 Research domain: Computer science.
 Research questions: Addressing the system as a phenomenon.
 Is it: A vehicle for something else?
 Research domain: Information systems and applied CS.
 Research questions: Addressing the system in its context of use.
 Is it: A tangible outcome of a process, where the process is my
focus?
 Research domain: Software engineering.
 Research questions: Addressing the process and tools used during
the development of the system.

31. ICT
Design and creation RQs: Areas
How might a persuasive system based on reward theories look like?

32. ICT
SE research questions

33. ICT
Quantitative RQs
 Three Rules for Quantitative Research Questions
 They Begin with “How”, “What”, or “Why” and can NEVER be
answered by a simple Yes or No
 Specify the independent and dependent variables
 IF your questions deal with connections among multiple variables,
you will again - use relate or compare – just as you did in the
purpose statement
 General Descriptive Examples [using rules 1 and 2]:
 What factors positively impact employee turnover rates of hourly
workers?
 Why do Asian-American students outscore other student groups
on AP Calculus Exams?

34. ICT
RQs: Positivist or interpretive?
Research
Question
Research
Design
(Interpretive)
Hypothesis
Research
Design
(Positivist)
Quantitative research starts with
a hypothesis:
• RQ: What do I want to know?
• RH: What is my tentative
answer?
Avoid "Fishing expeditions"
Quantitative
Qualitative
Research
Propositions

35. ICT
RQs in qualitative research
 Generalizing Qs vs. particularizing Qs
 From "How do students use social media in their study?"
 To "How do 5. year CS students at NTNU use Facebook to coordinate
their project-based subjects?"
 Generalization important?  Consider quantitative!
 Instrumental Qs vs. realist Qs
 Do we need to reframe questions to collect objectively measurable data,
or do we collect what is there, even if subjective?
 Objective data mandatory?  Consider quantitative!
 Variance Qs vs. process Qs
 Variance Qs focus on differences and variations: "How much?", "To what
extent?", "Is there?" Consider quantitative!
 Process Qs focus on the meaning of events and the influence of the
physical and social context of use on these events.

36. ICT
RQs in qualitative research
 Descriptive RQs
 Ask about what actually happened.
 Interpretive RQs
 Ask about the meaning of what happened to those involved.
 Theoretical RQ
 Ask about why those things happened.
 Generalization RQ
 Normally not asked in QR (not big enough samples).
 Evaluation RQ
 Normally not asked in QR (imposes values on people).

37. ICT
PICO (Quant.) and SPIDER (Qual.)

38. ICT
RQs in qualitative research
 Will be revised during the research (very different than
quantitative research!)
 Beware of "smuggling unexamined assumptions"!
 Can be a statement rather than a question.

39. ICT
Qualitative RQs in context
Research Questions
Purposes
Conceptual
context
Methods Validity

40. ICT
FINER (Clinical)
 F:Feasible
 Adequate number of subjects
 Adequate technical expertise
 Affordable in time and money
 Manageable in scope
 I: Interesting
 Getting the answer intrigues investigator, peers and community
 N: Novel
 Confirms, refutes or extends previous findings
 E: Ethical
 Amenable to a study that institutional review board will approve
 R: Relevant
 To scientific knowledge
 To clinical and health policy
 To future research

41. ICT
Choosing the right questions

42. ICT
The wrong research question
leads to wasted time and effort!

43. ICT
Credits
This presentation is based on a number of publications from others. The
figure introduced in 5 is by (Booth, Colomb, and Williams 2003), so is the
"So what?" test in slide 9. The terms used in "How to focus?" slide are by
(Mantzoukas 2008). The figure in slide 18 and a lot of ideas in the slides
are by (Oates 2005). The discussion of primary and secondary questions
is adopted from (Andrews 2003). The research outputs/activities matrix is
by (March and Smith 1995). The table showing software research
questions is by (Shaw 2002). A lot of the material on qualitative data is
adopted from (Maxwell 1996) and (Agee 2009).... Slides on PICO and
SPIDER are from (Cooke, Smith, and Booth 2012). Some of the
discussion on choosing a good research question and the graph are from
(Alon 2009).

44. ICT
References
 Agee, Jane. 2009. “Developing Qualitative Research Questions: A Reflective Process.” International Journal of
Qualitative Studies in Education 22 (4): 431–47. doi:10.1080/09518390902736512.
 Alon, Uri. 2009. “How to Choose a Good Scientific Problem.” Molecular Cell 35 (6): 726–28.
doi:10.1016/j.molcel.2009.09.013.
 Andrews, Richard. 2003. Research Questions. London: Continum.
 Booth, Wayne C., Gregory G. Colomb, and Joseph M. Williams. 2003. The Craft of Research. 2nd ed. Chicago,
London: The University of Chicago Press.
http://www.press.uchicago.edu/ucp/books/book/chicago/C/bo5821939.html.
 Cooke, Alison, Debbie Smith, and Andrew Booth. 2012. “Beyond PICO: The SPIDER Tool for Qualitative Evidence
Synthesis.” Qualitative Health Research 22 (10): 1435–43. doi:10.1177/1049732312452938.
 Mantzoukas, Stefanos. 2008. “Facilitating Research Students in Formulating Qualitative Research Questions.” Nurse
Education Today 28 (3): 371–77. doi:10.1016/j.nedt.2007.06.012.
 March, Salvatore T., and Gerald F. Smith. 1995. “Design and Natural Science Research on Information Technology.”
Decision Support Systems 15 (4): 251–66. doi:10.1016/0167-9236(94)00041-2.
 Maxwell, Joseph A. 1996. Qualitative Research Design- An Interactive Approach. Thousand Oaks: SAGE
Publications.
 Oates, Briony J. 2005. Researching Information Systems and Computing. 1st ed. SAGE Publications.
 Shaw, Mary. 2002. “What Makes Good Research in Software Engineering?” International Journal on Software Tools
for Technology Transfer 4 (1). Springer-Verlag: 1–7. doi:10.1007/s10009-002-0083-4.