Full paper at http://scott.hale.us/pubs/?chi2017
The number and quality of user reviews greatly affects consumer purchasing decisions. While reviews in all languages are increasing, it is still often the case (especially for non-English speakers) that there are only a few reviews in a person’s first language. Using an online experiment, we examine the value that potential purchasers receive from interfaces showing additional reviews in a second language. The results paint a complicated picture with both positive and negative reactions to the inclusion of foreign-language reviews. Roughly 26–28\% of subjects clicked to see translations of the foreign-language content when given the opportunity, and those who did so were more likely to select the product with foreign-language reviews than those who did not.
Full paper at http://scott.hale.us/pubs/?chi2017
Foreign-language Reviews: Help or Hindrance? (Slides)
1. Foreign-language Reviews: Help or Hindrance?
Scott A. Hale1 Irene Eleta2
1Oxford Internet Institute, University of Oxford
@computermacgyve
http://scott.hale.us/
2
Barcelona Institute for Global Health (ISGlobal)
http://www.ieleta.com/
@ieleta
10 May 2017
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
2. Localness
Localness
One oft-cited benefit of user-generated content (and volunteered geographic
information) is localness.
Engages local content producers
“Mapping where it should be, which is the hands of local people who
know an area well” (Geographer M. Goodchild in New York Times, 17 Nov. 2009)
Novel research opportunities and applications
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
3. Localness
Localness
One oft-cited benefit of user-generated content (and volunteered geographic
information) is localness.
Engages local content producers
“Mapping where it should be, which is the hands of local people who
know an area well” (Geographer M. Goodchild in New York Times, 17 Nov. 2009)
Novel research opportunities and applications
Language is a large barrier
“Information in languages not native to a place tends to be produced and
sourced by non-locals.” (Sen et al., 2015)
Large differences in information between languages (Hecht & Gergle,
2010)
Low/moderate levels of multilingualism online (Hale, 2014a, 2014b)
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
4. Diversity of designs: Google Play
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
5. Diversity of designs: Google Play
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
6. Diversity of designs: TripAdvisor
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
7. Question
Multilingual design (e.g., showing reviews from multiple languages) may
encourage the spread of information between languages (most
expectedly by multilingual users), but
could multilingual design also confuse some users (most expectedly
monolingual users)?
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
8. Online reviews
Online reviews affect product sales (Korfiatis, Garc´ıa-Bariocanal, &
S´anchez-Alonso, 2012), but many factors involved, including:
the quality of reviews
trust associated with reviewers
quantity of (bad) reviews
temporal effects (Hu, Liu, & Zhang, 2008)
Quantity of reviews desired varies from platform to platform, but 6 reviews
has been suggested as generally sufficient (Park, Lee, & Han, 2007).
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
10. Approach
Three cycle tours in London
(randomly match photos, titles, and descriptions for each participant)
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
11. Approach
Three cycle tours in London
(randomly match photos, titles, and descriptions for each participant)
15 randomly selected 4 or 5 star reviews written on a large travel
website
Reviews edited to be generic; all reviews translated to Spanish
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
12. Multilingual interface design
Each tour was randomly assigned one of three conditions:
(en3) Three
English reviews
(en3es3) Three
English and three
Spanish reviews
No translation
button
With
translation
button
Pretranslated
(en6) Six English
reviews
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
13. Multilingual interface design
Each tour was randomly assigned one of three conditions:
(en3) Three English
reviews
(en3es3) Three
English and three
Spanish reviews
No translation
button
With
translation
button
Pretranslated
(en6) Six English
reviews
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
14. Multilingual interface design
Each tour was randomly assigned one of three conditions:
(en3) Three English
reviews
(en3es3) Three
English and three
Spanish reviews
No
translation
button
With
translation
button
Pretranslated
(en6) Six English
reviews
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
15. Multilingual interface design
Each tour was randomly assigned one of three conditions:
(en3) Three English
reviews
(en3es3) Three
English and three
Spanish reviews
No translation
button
With
translation
button
Pretranslated
(en6) Six English
reviews
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
16. Multilingual interface design
Each tour was randomly assigned one of three conditions:
(en3) Three English
reviews
(en3es3) Three
English and three
Spanish reviews
No translation
button
With
translation
button
Pretranslated
(en6) Six English
reviews
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
19. Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
20. Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
21. Results
(Study I: 480 participants from Mechanical Turk)
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
22. Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
23. Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
24. Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
en6
N=416
en3es3
N=416
en3
N=416
Condition
b. Individual likelihood of booking
25. Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
26. Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
Condition
en3es3: No translation
(N=198)
en3es3: Translation
(N=218)
b. Individual likelihood of booking
27. Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
28. Study II: Individual differences
What individual differences predict finding foreign-language reviews helpful?
? Big Five personality dimensions
? Openness
? Foreign experience
265 participants from Prolific.ac
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
29. Study II: Individual differences
Replicate key findings from Study I.
(Those who click to view translations are more likely to prefer the bilingual
tour.)
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
30. Study II: Individual differences
Replicate key findings from Study I.
(Those who click to view translations are more likely to prefer the bilingual
tour.)
What individual differences predict finding foreign-language reviews helpful?
X Big Five personality dimensions
X Openness
X Foreign experience
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
31. Study II: Individual differences
Replicate key findings from Study I.
(Those who click to view translations are more likely to prefer the bilingual
tour.)
What individual differences predict finding foreign-language reviews helpful?
X Big Five personality dimensions
X Openness
X Foreign experience
O Mean importance given to reviews in all three conditions was
significantly higher amongst subjects clicking to see translations.
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
32. Summary and next steps
Summary
No overall effect of foreign-language reviews
Very different behavior from people who (don’t) click to see translations
Personality/openness do not predict behavior;
people who highly value reviews?
Next steps
Beyond tours
Non-English speaking locations
Subjects with different language abilities
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
34. Applications: Platform design
Personalization.
Show reviews in multiple languages for people who engage with reviews
(e.g., click to see translations); hide foreign-language reviews for those who
do not.
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
35. Applications: Multiple Twitter accounts or one?
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
36. Applications: Multiple Twitter accounts or one?
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
37. Foreign-language Reviews: Help or Hindrance?
Scott A. Hale1 Irene Eleta2
1Oxford Internet Institute, University of Oxford
@computermacgyve
http://scott.hale.us/
2
Barcelona Institute for Global Health (ISGlobal)
http://www.ieleta.com/
@ieleta
10 May 2017
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
38. Hale, S. A. (2014a). Global Connectivity and Multilinguals in the Twitter
Network. In Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems (pp. 833–842). New York: ACM. Retrieved
from http://www.scotthale.net/pubs/?chi2014 doi:
10.1145/2556288.2557203
Hale, S. A. (2014b). Multilinguals and Wikipedia editing. In Proceedings of
the 6th annual acm web science conference. New York: ACM.
Retrieved from http://www.scotthale.net/pubs/?websci2014
Hecht, B., & Gergle, D. (2010). The Tower of Babel meets Web 2.0:
User-generated content and its applications in a multilingual context.
In Proceedings of the 28th international conference on human factors
in computing systems (pp. 291–300). New York, NY, USA: ACM.
Retrieved from http://doi.acm.org/10.1145/1753326.1753370
doi: 10.1145/1753326.1753370
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
39. Hu, N., Liu, L., & Zhang, J. J. (2008). Do online reviews affect product
sales? the role of reviewer characteristics and temporal effects.
Information Technology and Management, 9(3), 201–214. Retrieved
from http://dx.doi.org/10.1007/s10799-008-0041-2 doi:
10.1007/s10799-008-0041-2
Korfiatis, N., Garc´ıa-Bariocanal, E., & S´anchez-Alonso, S. (2012).
Evaluating content quality and helpfulness of online product reviews:
The interplay of review helpfulness vs. review content. Electronic
Commerce Research and Applications, 11(3), 205–217. Retrieved from
http://www.sciencedirect.com/science/article/pii/
S1567422311000639 doi: 10.1016/j.elerap.2011.10.003
Park, D.-H., Lee, J., & Han, I. (2007). The effect of on-line consumer
reviews on consumer purchasing intention: The moderating role of
involvement. International Journal of Electronic Commerce, 11(4),
125–148. doi: 10.2753/JEC1086-4415110405
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?
40. Sen, S., Giesel, M. E., Gold, R., Hillmann, B., Lesicko, M., Naden, S., . . .
Hecht, B. (2015). Turkers, scholars, ”arafat” and ”peace”: Cultural
communities and algorithmic gold standards. In Proceedings of the
18th acm conference on computer supported cooperative work &
social computing (pp. 826–838). New York, NY, USA: ACM.
Retrieved from http://doi.acm.org/10.1145/2675133.2675285
doi: 10.1145/2675133.2675285
Scott A. Hale & Irene Eleta Foreign-language Reviews: Help or Hindrance?