Reddit Usability Testing
•
1 like•3,080 views
Testing and Evaluation on Reddit User Interface Design
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to Reddit Usability Testing
Similar to Reddit Usability Testing (20)
Recently uploaded
Recently uploaded (20)
Reddit Usability Testing
- 1. Reddit Usability Test 560 Submitted by: Jessica Chambers, Brooke Hundley Sasha Campbell COM 560 XU Date: 12.6.13 method test results conclusion search reddit username password remember me reset password login MY SUBREDDITS FRONT • ALL • RANDOM PICS • FUNNY • GAMING • ASKREDDIT• WORLDNEWS • NEWS • VIDEOS • IAMA • TODAYILEARNED • WTF • AWW • TECHNOLOGY • MORE introduction want to join? login or register in seconds Englishappendices
- 2. 2 EXECUTIVE SUMMARY Reddit is a social news aggregation site that allows registered users to submit links and comments in a variety of categories (SubReddits). Registered users are also able to send messages and share articles with other users and can customize which SubReddit’s they are subscribed to. In order to test the functionally and user satisfaction of Reddit, a usability test was administered to 10 participants who had very little or no previous experience with Reddit. The selected participants fell within the primary and secondary age demographic for the website and were all active Internet users (PewResearchCenter). The usability test consisted of two questionnaires (pre-survey and post-surveys) and nine tasks that were timed. All tests were screen captured with Quicktime and were later analyzed. The data was broken down into three categories: number of errors (wrong clicks), time-per-task and whether the tasks were completed. Furthermore, the information was then compared with the questionnaire results to analyze and better understand the usability of Reddit. Primary findings: • Users were able to complete a majority of the tasks but did not find Reddit to be user friendly. • Tasks that were familiar to other websites (i.e. search bar, logging in, etc.) were more easily achieved than those specific to Reddit. • Users had trouble locating information and utilizing the main navigation menu. • Participants had difficulty customizing their Reddit subscriptions in the SubReddits section. • Only 40% of users said they would use Reddit again.
- 3. 3 INTRODUCTION Reddit is a social news and entertainment, bulletin-style site in which users submit content (primarily links or comments), and other users vote up or down to rank the post and establish its position on the home page. Reddit.com is ranked in the top 100 sites used globally (alexa.com) and in the top 50 used in the United States, and had more than 27 million users each month (quantcast.com). According to the PewResearchCenter’s July 2013 report (Appendix A), 6% of all adult online users visit reddit.com, and the top demographic of users on the site consists of 18-29 year olds, with an emphasis on male users. The site has a slogan of “the front page of the internet” and is often used as a news aggregator or a personal bookmark collection. Content is organized by a series of “subreddits,” otherwise known as categories of user interest. In order to submit content, users must have a registered account. Registered users can also post comments on any submission, and exchange information back and forth through these comments. The comments can then be voted up or down. Reddit has a series of customizable options including a personal front page in which users can choose which subreddits appear, as well as the ability to “friend” other redditors, giving the user quicker access to postings and comments on their friend list via these subscriptions. The community aspect of Reddit extends to a reward system in which redditors accumulate “karma” (points) for highly rated comments and links. The site also offers a subscription service known as Reddit Gold, which adds a number of features to the interface, including the ability to display more comments on a page, exclusive access to the private subreddit lounge, and notifications whenever the user’s username is mentioned in a comment. Reddit requires users to create an account in order to access many of the interactive features of the site, however, users who are not members still have access to the most
- 4. 4 informational features. Users who create an account have the ability to comment on articles/links, vote up or down on other users’ comments, share content via email or Reddit inboxes, and subscribe/unsubscribe to “subreddit” categories of interest. Without an account, visitors can still navigate to any posting, view the existing comments on any thread, and view the subreddit categories. A Reddit account unlocks features that allow users to participate in the Reddit community and leave more feedback. The interface relies solely on user-generated content, making members part of a collective community that allows Reddit to provide a thorough and engaging experience. The creation of a Reddit account does not require a valid email address, although one is suggested. Instead the user only needs to register a login name and password and answer a captcha element to verify the user is human. Once the account is created, users can participate in social interactions on the interface by submitting a “new link” or a “new text post.” Users can share content with other Reddit users or via email addresses, can save content of interest to a “saved” section of their account, and can connect with other users by leaving comments on threads or voting other comments up or down. These up or down votes are aggregated to move the posting’s rank higher or lower on the Reddit homepage. Reddit.com also provides incentive for their members to be active on the site. Users are given karma points for voting on links or comments, as well as leaving comments for others to view. Karma points are used to establish a member’s position in the community. Reddit houses a variety of features on their site, but for the purposes of this usability test we focused on specific tasks that were more integral to the average user’s needs. The search, commenting, voting, sharing, and customizing subreddit lists were tested. While these functions do not cover every capability of the site, they are representative of the purpose of the site as a
- 5. 5 whole. A typical user would frequently utilize these core functions in order to navigate the site, and the usability of these features would weigh heavily on recruiting and retaining users. METHODS Our objective in this usability test was to measure the navigability, efficiency, and user satisfaction with the Reddit browsing experience. The test was completed with ten participants consisting of six females, and four males between the ages of 22 and 45. The participants were chosen via a convenience sample of Elon University students. All but one participant fit into the largest demographic of Reddit users, which are men and women between the ages of 18-29. The remaining participant fit into the second highest demographic, that of men and women aged 30- 49. Although the study was limited through a convenience sample of university-educated individuals, the participants should be considered a relatively accurate representation of Reddit users. Our pre-survey results revealed that 90% of our participants typically browse the Internet for more than 3 hours a day, with the remaining 10% still browsing for 1-3 hours a day, thus our sample population should be considered power users. Although participants are advanced Internet consumers, 70% had no experience with Reddit.com, and there were no participants within the remaining 30% that ranked their Reddit experience level higher than a 2 on a scale from 0 to 5, with “0” representing no experience. Before our team could administer the usability test, a set of required materials were developed to ensure consistency in our results. A participant consent form was generated (Appendix B). A pre-survey and post-survey were created for participants to take before beginning the test and after exiting the test. Before we could start the test, 10 Reddit accounts with unique usernames and passwords were created. Quicktime’s screen recording was set up to capture each participant’s actions and testing time on every testing computer. The usability test
- 6. 6 was composed of three stages. After participants were read the script detailing the procedure, they signed a consent form and began the test. During the test, participants were first asked to take a survey. Once they finished the initial survey, participants worked through the set of tasks provided. Finally, participants took an exit survey regarding their experience. Administers created a reward system to entice volunteers to participate which included brownies, cookies and cupcakes. The aforementioned sustenance was given upon completion of the required tasks and subsequent exit survey. Alexa.com data states that people who access Reddit from school are greatly overrepresented relative to the general Internet population. Given this information researchers wanted to simulate a collegiate environment by holding the survey in a similar area. Therefore, researchers held the survey at Elon University, which allowed for a simulated real world exp- erience. Researchers administered the usability test to participants in groups of four in a leisure space in an Elon University Building. Each participant had a computer and chair with space between test takers to minimize distractions. The screen recording software was running when users arrived so as not to distract the participant. During the usability test, a set procedure for each of the ten participants was followed. One researcher read an instructional script to ensure that each participant had the same understanding of the rules and requirements of the usability test (Appendix C). The script details that researchers were focused on studying the interface and not the users, and asked the participants to honestly answer any questions about their experience. To best capture the typical browsing experience of users, participants were encouraged to complete the entire test without help, just as they would in a non-test-taking setting.
- 7. 7 The usability test began with a pre-survey of eight questions hosted on surveymonkey.com (Appendix D) that gauged our participants experience with Reddit.com, as well as their Internet consumption, and practice of sharing multimedia content. Out of 10 participants, 70% reported that they typically use the Internet to access news, half reported that they were “very likely” to like or dislike media on the Internet, and half rated that they were “very likely” to share media on the Internet. Once they completed the pre-survey, participants were asked to move onto a set of provided scenario tasks (Appendix E). The scenarios were designed to illustrate the effectiveness and navigability of the site’s browsing capabilities. The scenarios are representative of the context and situations in which participants would explore most of the available features on Reddit including signing in, searching for a topic, leaving comments, voting on content, sharing content, and subscribing and unsubscribing to content streams (subreddits). After completion of the scenario tasks, participants were asked to answer a post-survey (Appendix F) gauging their ease of use with the site and overall opinions to include future use and suggested design changes. Once all of the data was collected, the videos of our participants were recorded noting the tasks completed, errors, and time of each task (Appendix G). The researchers measured this information and cross-referenced it with the post-survey results in order to determine overall effectiveness and user satisfaction with the Reddit interface. RESULTS The results of the usability test were obtained from screen recordings and the participants’ post-survey responses (Appendix H). The results are assembled based on averages amongst all test takers however additional insights were gained from individual experiences and are mentioned in further detail throughout this results section. While reviewing the videos,
- 8. 8 results were organized into three different categories: error rate (number of incorrect clicks), time per task, and if the task was actually completed or not. Researchers established that the error rate would denote unnecessary clicks to propel the user forward in completing the task. However, error rate did not include clicks for viewing purposes given that the website’s main function is to click and view the content. Nine separate tasks were observed in total including: finding a specified (Thor) article, signing in to Reddit, adding a comment, rating/voting a comment up or down, going to a specified (Technology) SubReddit, saving a found article, unsubscribing from SubReddits, subscribing to SubReddits, and sharing the previously saved article. With 10 participants and nine categories, approximately 78% of the tasks were completed successfully. Only two participants were able to complete all nine of the tasks with the lowest success rate at five out of nine completed tasks. In the post survey, users were asked to rate if they agreed or disagreed with the question, “I was able to successfully complete all the tasks.” Participants responded, with 60% feeling they were able to complete all of the tasks. Actual task completion data revealed that some participants were able to quickly and efficiently accomplish the tasks, and that three tasks: finding an article on a specific topic, signing into Reddit, and saving an article, had a perfect success rate of 100% completion. This is to be expected since search bars and account logins are a part of most websites, and a familiar part of the web browsing experience. However, while all the participants were able to save an article, the time to complete this task ranged from 14 seconds to 1 minute and 50 seconds. Reddit’s interface provides links to content that is hosted on other sites, thus many participants failed to realize that they had to go back to Reddit to save. Overall, saving did not have a high number of wrong clicks, but we observed confusion on navigability given the
- 9. 9 cluttered interface design through viewing our screen recordings and time lapsed documentation. Despite a great deal of aptitude in completing three of the tasks, a hierarchy of tasks became evident as two tasks fell under a 50% successful completion rate among all participants. Rating a comment was only accomplished by 50% of the group and only 40% were able to go to the SubReddit category. Since the participants revealed at least 60% were ‘Likely’ or ‘Very Likely’ to like/dislike content online, it was surprising to see this had such a low success rate. Our observations and screen recordings revealed that some of the participants were unable to clearly locate the “Up” or “Down” buttons and could not complete this task. It was not as surprising to see users having difficulty accomplishing our task of going to the technology SubReddit. From any page, participants had two ways to go to this area; either by visiting the navigation that ran across the top of the page or utilizing a drop down menu on the top left of page. Of the four users who completed this task, three used the navigation menu at the top of the page. The six individuals who failed to compete this task did so with mixed experiences. Two of the users simply searched for ‘new technology’ or ‘iPhone 6’ in the search bar rather than trying to locate the SubReddit menu. Several individuals also went to other SubReddits instead. It was inconclusive just from watching the screen recordings if this was due to an inability to locate the specified technology SubReddit or simply choosing to disregard the instructions. SubReddit’s are the main navigation for the website and are difficult to notice. Overall, dealing with the SubReddit’s was one of the more difficult tasks given. Eight individuals were able to successfully unsubscribe from different SubReddit’s while only seven were able to subscribe. This is in part due to confusion on where users should go to change this information. Many of the errors occurred when users went to their username’s
- 10. 10 preference section, which doesn’t allow you to customize SubReddits. The navigation bar offers an edit button that will take a user to their personal subscription page. The navigation bar also features a drop down navigation area titled ‘My SubReddit’s’ with the last option listed as ‘Edit Subscriptions.’ Even once users arrived on the edit page, there was confusion on what should be done. Some participants went to specific SubReddits to try and unsubscribe/subscribe. While this is a possible solution, 30% of users were not able to locate subscribe and 20% could not locate unsubscribe. Two users had an extended subscribe time because they looked for specific topics they were interested in rather than subscribing to the suggested SubReddits on the edit page. When it came to our final tasks, 80% of the participants were able to locate their saved article and share it, yet it remained the second most difficult task according to the post-survey results. This task had the highest error rate and average of 1.25 incorrect clicks. It was clear that sharing the article was not actually the most difficult part of this portion of the test. Instead, participants had high difficulty in understanding where to recover their saved articles. One individual was able to share an article but their response was disregarded because they had been unable to locate their specific saved article to share. This task was the most time consuming with the fastest time taking 1 minute and 6 seconds. Overall, the average user took 1 minute and 45 seconds to complete the task. When completing the post-survey, participants were asked if they would use Reddit again, 60% of the individuals said they would not. The four individuals who said they would use Reddit stated they would utilize it to find funny pictures and look at specific topics/SubReddits. One individual commented they would not use it for, “anything serious.” It is unclear if that result would have been different if more individuals had been able to locate the technology SubReddit and explore non-entertainment content. As previously shown, most participants did
- 11. 11 not like the layout and navigation of the website. Furthermore, when asked what were the most challenging parts of the user experience with Reddit, 70% of participants wrote “not able to find” the required links to complete the tasks. Navigability and design were the top reasons why users said they would not use the site again. While the users were able to complete most Reddit tasks, overall the non-user friendly interface alienates a majority of its target audience. CONCLUSION Reddit’s current website has a large global audience and while current use of the site is widespread and the interface is deemed suitable, our study showed a significant opportunity for improvement. Researchers suggest that more thought should be given to navigation and design of the interface for ease of use. One of the complaints from 50% of users was the small, illegible font size. This is a simple fix that will give the interface and navigability a boost in usability. Furthermore, the visual aesthetics of the site should be addressed. The site could be greatly improved from the muted tones and cluttered, monotone text. Specifically, users had issues finding the ‘rate a comment’ button and therefore the design and labeling should be altered to make it more readily noticeable. Moreover, the articles that were archived were not marked appropriately which gave users difficulty in trying to comment or rate items. Given that articles of this nature are not interactive in terms of common user interaction they should be separated in some way. As a whole the interface is poorly designed and gave our participants a reason to never visit the site again. Therefore, Reddit should consider altering its interface for functionality and aesthetics to best serve the user and the site. The study conducted was informative, however in future testing we suggest the following edits and additions to gather more conclusive data. In future testing of Reddit usability, researchers suggest a larger quantity of participants, as well as a more diverse demographic from
- 12. 12 which a various skill level of previous use could be compared. Testing should be conducted in a more secluded area to simulate a personal computing environment. Furthermore to avoid distractions, standardized computer settings are advised given that our participants met with distractions due to the computers being personal in ownership and other programs signaling for attention. It is also highly advised to obtain a better plan regarding how to time the participants as they conduct both the tasks and survey. This piece of the usability test was met with some confusion for researchers to review because some tasks were abandoned or done out of order and thus it was hard to calculate accurate times. By implementing these suggestions, more conclusive data can be obtained to develop an accurate depiction of Reddit’s usability.
- 13. method test results conclusion Appendix (A-H) 560 Submitted by: Jessica Chambers, Brooke Hundley Sasha Campbell COM 560 XU Date: 12.6.13 search reddit username password remember me reset password login MY SUBREDDITS FRONT • ALL • RANDOM PICS • FUNNY • GAMING • ASKREDDIT• WORLDNEWS • NEWS • VIDEOS • IAMA • TODAYILEARNED • WTF • AWW • TECHNOLOGY • MORE introduction want to join? login or register in seconds Englishappendices
- 14. www.pewresearch.org JULY 3, 2013 6% of Online Adults are reddit Users Young men are especially likely to visit the “front page of the internet.” Maeve Duggan Research Assistant, Pew Internet Project Aaron Smith Senior Researcher, Pew Internet Project http://pewinternet.org/Reports/2013/reddit.aspx FOR FURTHER INFORMATION, CONTACT: Pew Research Center’s Internet & American Life Project 1615 L St., N.W., Suite 700 Washington, D.C. 20036 Media Inquiries: 202.419.4500
- 15. Six percent of online adults are reddit users,1 according to a nationally representative survey by the Pew Research Center’s Internet & American Life Project. This is our first survey that specifically examines reddit as a standalone platform, and it finds that young men are especially likely to visit the site. Some 15% of male internet users ages 18-29 say that they use reddit, compared with 5% of women in the same age range and 8% of men ages 30-49. Young males are especially likely to use reddit % of internet users in each age/gender grouping who use reddit Source: Pew Research Center’s Internet & American Life Project Spring Tracking Survey, April 17 – May 19, 2013. N=2,252 adults ages 18+. Interviews were conducted in English and Spanish and on landline and cell phones. The margin of error for results based on all internet users is +/- 2.5 percentage points. Self-described as the “front page of the internet,” reddit is a bulletin of user-submitted text, links, photos, and videos. The site hosts active sub-communities on a wide selection of topics, ranging from world news headlines, to animal GIFs, to fan forums and various niche topics. The popularity and prominence of material on the site is determined by voting from the reddit community. reddit gained widespread publicity when President Barack Obama took part in the site’s “Ask Me Anything” series in August 2012, and again in April 2013 when the site was used as a collaboration space for amateur sleuths attempting to identify the perpetrators of the Boston Marathon bombings. 1 In a separate survey in December 2012, we asked about the use of a number of other social media platforms. The proportion of online adults who use reddit is similar to the proportion that use Tumblr (also 6%), and around half the number that use Twitter (16%), Pinterest (15%), or Instagram (13%). Some 67% of online adults are Facebook users. For more information on these sites please read our report, The Demographics of Social Media Users -2012. 15% 8% 3% 3% 5% 5% 2% 1% 0% 10% 20% 30% 18-29 30-49 50-64 65+ Men Women
- 16. Overall, men are twice as likely as women to be reddit users, those under 50 are significantly more likely to use reddit than those 50 or older, and the site is much more common among urban and suburban residents than among those living in rural areas. Indeed, just 2% of internet users ages 50 and older— and 2% of rural residents—say they use the site. reddit usage by demographic group % of internet users in each group who use reddit % who use reddit All internet users (n=1,895) 6% a Men (n=874) 8 b b Women (n=1,021) 4 Race/ethnicity a White, Non-Hispanic (n=1,331) 5 b Black, Non-Hispanic (n=207) 4 c Hispanic (n=196) 11 ab Age a 18-29 (n=395) 11 bcd b 30-49 (n=542) 7 cd c 50-64 (n=553) 2 d 65+ (n=356) 2 Education attainment a No high school diploma (n=99) 9 b High school grad (n=473) 4 c Some College (n=517) 6 d College + (n=790) 7 b Household income a Less than $30,000/yr (n=417) 6 b $30,000-$49,999 (n=320) 6 c $50,000-$74,999 (n=279) 7 d $75,000+ (n=559) 6 Urbanity a Urban (n=649) 7 c b Suburban (n=893) 6 c c Rural (n=351) 2 Source: Pew Research Center’s Internet & American Life Project Spring Tracking Survey, April 17 – May 19, 2013. N=2,252 adults ages 18+. Interviews were conducted in English and Spanish and on landline and cell phones. The margin of error for results based on all internet users is +/- 2.5 percentage points. Note: Percentages marked with a superscript letter (e.g., a ) indicate a statistically significant difference between that row and the row designated by that superscript letter, among categories of each demographic characteristic (e.g. age).
- 17. More about reddit reddit features a user-voting system (called up- and down-voting) to rank posts, comments, and links within its sub-communities (known as subreddits). Subreddits and posts can range from the serious to the absurd, and from general topics to niche interests. The front page of reddit features popular submissions from across the site and typically rotates top posts regularly throughout the day. People who use reddit are called redditors. reddit is often pivotal in the spread of popular memes, like Ridiculously Photogenic Guy, Overly Attached Girlfriend, and Sudden Clarity Clarence. One of the most popular subreddits, IamA (“I am a”) features a popular question forum called Ask Me Anything (AMA). redditors with “something uncommon that plays a central role in your life” or who have experienced “a truly interesting and unique event” host a session with the prompt “I am a…..ask me anything.” Other redditors can then submit questions and comments. Examples vary from “IAmA prison guard at a maximum security prison, AMA” to “IAmA wildlife cameraman. AMA” and “IAmA Liberal Saudi Woman Living in Riyadh, Saudi Arabia. AMA!” Celebrities have also participated in AMA, including Stephen Colbert, Judy Blume, Paul Krugman, and Bill Gates. For a more in-depth overview of the site, visit Mashable’s “Reddit for Beginners” or poke around the site at reddit.com.
- 18. Survey Questions Spring 2013 Tracking Survey Final Topline 5/21/2013 Data for April 17-May 19, 2013 Princeton Survey Research Associates International for the Pew Research Center’s Internet & American Life Project Sample: n=2,252 national adults, age 18 and older, including 1,127 cell phone interviews Interviewing dates: 04.17.2013 – 05.19.2013 Margin of error is plus or minus 2.3 percentage points for results based on Total [n=2,252] Margin of error is plus or minus 2.5 percentage points for results based on all internet users [n=1,895] INTUSE Do you use the internet, at least occasionally? EMLOCCDo you send or receive email, at least occasionally? EMINUSEDo you use the internet or email, at least occasionally? INTMOBDo you access the internet on a cell phone, tablet or other mobile handheld device, at least occasionally?2 USES INTERNET DOES NOT USE INTERNET Current 85 15 December 2012 81 19 November 2012 85 15 September 2012 81 19 August 2012 85 15 April 2012 82 18 February 2012 80 20 INTUSE/EMLOCC/EMINUSE/INTMOB continued... 2 The definition of an internet user varies from survey to survey. In the current survey, half the sample was asked INTUSE/EMLOCC/INTMOB and half was asked EMINUSE/INTMOB. Current results are for both forms combined. Throughout the current topline, total internet users are defined as those who answered yes to any question INTUSE, EMLOCC, EMINUSE or INTMOB. From April 2012 thru December 2012, an internet user is someone who uses the internet at least occasionally, sends/receives email at least occasionally or accesses the internet a mobile device (three-part definition with question wording “Do you use the internet, at least occasionally?” OR “Do you send or receive email, at least occasionally?” OR “Do you access the internet on a cell phone, tablet or other mobile handheld device, at least occasionally?”). From January 2005 thru February 2012, an internet user is someone who uses the internet at least occasionally or sends/receives email at least occasionally (two-part definition with question wording “Do you use the internet, at least occasionally?” OR “Do you send or receive email, at least occasionally?”). Prior to January 2005, an internet user is someone who goes online to access the internet or to send and receive email (question wording “Do you ever go online to access the Internet or World Wide Web or to send and receive email?”).
- 19. INTUSE/EMLOCC/ EMINUSE/INTMOB continued... USES INTERNET DOES NOT USE INTERNET December 2011 82 18 August 2011 78 22 May 2011 78 22 January 2011 79 21 December 2010 77 23 November 2010 74 26 September 2010 74 26 May 2010 79 21 January 2010 75 25 December 2009 74 26 September 2009 77 23 April 2009 79 21 December 2008 74 26 November 2008 74 26 August 2008 75 25 July 2008 77 23 May 2008 73 27 April 2008 73 27 January 2008 70 30 December 2007 75 25 September 2007 73 27 February 2007 71 29 December 2006 70 30 November 2006 68 32 August 2006 70 30 April 2006 73 27 February 2006 73 27 December 2005 66 34 September 2005 72 28 June 2005 68 32 February 2005 67 33 January 2005 66 34 Nov 23-30, 2004 59 41 November 2004 61 39 July 2004 67 33 June 2004 63 37 March 2004 69 31 February 2004 63 37 November 2003 64 36 August 2003 63 37 June 2003 62 38 May 2003 63 37 March 3-11, 2003 62 38 February 2003 64 36 INTUSE/EMLOCC/EMINUSE/INTMOB continued...
- 20. INTUSE/EMLOCC/ EMINUSE/INTMOB continued... December 2002 57 43 November 2002 61 39 October 2002 59 41 September 2002 61 39 July 2002 59 41 March/May 2002 58 42 January 2002 61 39 December 2001 58 42 November 2001 58 42 October 2001 56 44 September 2001 55 45 August 2001 59 41 February 2001 53 47 December 2000 59 41 November 2000 53 47 October 2000 52 48 September 2000 50 50 August 2000 49 51 June 2000 47 53 May 2000 48 52 WEB1-ANext... Please tell me if you ever use the internet to do any of the following things. Do you ever use the internet to...[INSERT ITEM; RANDOMIZE]? Based on all internet users [N=1,895] TOTAL HAVE EVER DONE THIS ---------- DID YESTERDAY HAVE NOT DONE THIS DON’T KNOW REFUSED Use Reddit Current 6 n/a 90 4 *
- 21. Methods This report is based on the findings of a survey on Americans' use of the Internet. The results in this report are based on data from telephone interviews conducted by Princeton Survey Research Associates International from April 17 to May 19, 2013, among a sample of 2,252 adults, age 18 and older. Telephone interviews were conducted in English and Spanish by landline (1,125) and cell phone (1,127, including 571 without a landline phone). For results based on the total sample, one can say with 95% confidence that the error attributable to sampling is plus or minus 2.3 percentage points. For results based on Internet users3 (n=1,895), the margin of sampling error is plus or minus 2.5 percentage points. In addition to sampling error, question wording and practical difficulties in conducting telephone surveys may introduce some error or bias into the findings of opinion polls. A combination of landline and cellular random digit dial (RDD) samples was used to represent all adults in the United States who have access to either a landline or cellular telephone. Both samples were provided by Survey Sampling International, LLC (SSI) according to PSRAI specifications. Numbers for the landline sample were drawn with equal probabilities from active blocks (area code + exchange + two- digit block number) that contained three or more residential directory listings. The cellular sample was not list-assisted, but was drawn through a systematic sampling from dedicated wireless 100-blocks and shared service 100-blocks with no directory-listed landline numbers. New sample was released daily and was kept in the field for at least five days. The sample was released in replicates, which are representative subsamples of the larger population. This ensures that complete call procedures were followed for the entire sample. At least 7 attempts were made to complete an interview at a sampled telephone number. The calls were staggered over times of day and days of the week to maximize the chances of making contact with a potential respondent. Each number received at least one daytime call in an attempt to find someone available. For the landline sample, interviewers asked to speak with the youngest adult male or female currently at home based on a random rotation. If no male/female was available, interviewers asked to speak with the youngest adult of the other gender. For the cellular sample, interviews were conducted with the person who answered the phone. Interviewers verified that the person was an adult and in a safe place before administering the survey. Cellular sample respondents were offered a post-paid cash incentive for their participation. All interviews completed on any given day were considered to be the final sample for that day. Weighting is generally used in survey analysis to compensate for sample designs and patterns of non- response that might bias results. A two-stage weighting procedure was used to weight this dual-frame sample. The first-stage corrected for different probabilities of selection associated with the number of adults in each household and each respondent’s telephone usage patterns.4 This weighting also adjusts for the overlapping landline and cell sample frames and the relative sizes of each frame and each sample. 3 Internet user definition includes those who use the internet or email at least occasionally or access the internet on a mobile handheld device at least occasionally. 4 i.e., whether respondents have only a landline telephone, only a cell phone, or both kinds of telephone.
- 22. The second stage of weighting balances sample demographics to population parameters. The sample is balanced to match national population parameters for sex, age, education, race, Hispanic origin, region (U.S. Census definitions), population density, and telephone usage. The Hispanic origin was split out based on nativity; U.S born and non-U.S. born. The basic weighting parameters came from the US Census Bureau’s 2011 American Community Survey data. The population density parameter was derived from Census 2010 data. The telephone usage parameter came from an analysis of the January-June 2012 National Health Interview Survey. Following is the full disposition of all sampled telephone numbers: Sample Disposition Landline Cell 41,291 24,698 Total Numbers Dialed 1,755 411 Non-residential 1,516 88 Computer/Fax 12 ---- Cell phone 24,344 9,674 Other not working 2,038 226 Additional projected not working 11,626 14,299 Working numbers 28.2% 57.9% Working Rate 679 75 No Answer / Busy 3,442 3,668 Voice Mail 41 16 Other Non-Contact 7,464 10,540 Contacted numbers 64.2% 73.7% Contact Rate 450 1,537 Callback 5,786 7,097 Refusal 1,228 1,906 Cooperating numbers 16.5% 18.1% Cooperation Rate 45 68 Language Barrier ---- 684 Child's cell phone 1,183 1,154 Eligible numbers 96.3% 60.5% Eligibility Rate 58 27 Break-off 1,125 1,127 Completes 95.1% 97.7% Completion Rate 10.0% 13.0% Response Rate The disposition reports all of the sampled telephone numbers ever dialed from the original telephone number samples. The response rate estimates the fraction of all eligible respondents in the sample that were ultimately interviewed. At PSRAI it is calculated by taking the product of three component rates:
- 23. Contact rate – the proportion of working numbers where a request for interview was made Cooperation rate – the proportion of contacted numbers where a consent for interview was at least initially obtained, versus those refused Completion rate – the proportion of initially cooperating and eligible interviews that were completed Thus the response rate for the landline sample was 10 percent. The response rate for the cellular sample was 13 percent.
- 27. Appendix D: PreSurvey Survey was administered on SurveyMonkey.com. 1. What is your username? 2. How often do you typically browse the Internet? Never A few times a month A few times a week 13 hours a day More than 3 hours a day Always connected 3. How often do you typically share multimedia content? Never A few times a month A few times a week 13 hours a day More than 3 hours a day Always connected 4. How often do you typically use the Internet to access news? Never Every other month Every other week Every other day Everyday 5. Rate your level of experience with Reddit? No Experience Expert 0 1 2 3 4 5 6. Rate your likelihood to share images/videos/media? Not likely at all Very likely 1 2 3 4 5 7. Rate your likelihood to comment on media on the Internet? Not likely at all Very likely 1 2 3 4 5 8. Rate your likelihood to like or dislike media on the Internet? Not likely at all Very likely 1 2 3 4 5
- 29. Appendix F: PostSurvey Survey was administered on SurveyMonkey.com. Rate the following: 1. I was able to successfully complete all the tasks. Strongly Disagree Strongly Agree 1 2 3 4 5 2. I found the site easy to navigate. Strongly Disagree Strongly Agree 1 2 3 4 5 3. I was able to quickly find the topics and articles I was looking for. Strongly Disagree Strongly Agree 1 2 3 4 5 4. What tasks did you consider the most challenging part of your experience with Reddit today? 5. How would you use Reddit in the future? If you would not use again, respond “would not use” and list the reasons why. 6. If any, what changes would you like to see implemented in the site to increase the user experience? 7. What username did you use? 8. What is your age? 9. Select your gender: Male Female Other
- 30. Error Rate Find Thor Article Sign In Comment Rate Comment 1 0 0 0 X 2 0 0 0 0 3 3 0 X X 4 0 0 2 X 5 1 0 2 1 6 0 0 0 0 7 0 0 0 X 8 0 0 X 0 9 0 0 0 X 0 2 0 0 0 Time-‐Per-‐Task Find Thor Article Sign In Comment Rate Comment 1 :30 :28 :51 X 2 :44 :45 :10 :19 3 1:06 2 2:10 X 4 :47 :24 1:25 X 5 1:10 :31 1 X 6 :45 1:02 :32 :48 7 :43 :17 :30 X 8 :26 :18 X :55 9 :27 :31 :19 :07 0 :49 :42 :27 :23 APPENDIX G
- 31. Task Competion (Y/N) Find Thor Article Sign In Comment Rate Comment 1 YES YES YES NO 2 YES YES YES YES 3 YES YES NO NO 4 YES YES YES NO 5 YES YES YES NO 6 YES YES YES YES 7 YES YES YES NO 8 YES YES NO YES 9 YES YES YES YES 0 YES YES YES YES
- 32. Go to Technology SubReddit Save Unsubscribe Subscribe Share Article X 1 0 0 2 0 0 1 0 1 0 0 1 0 2 X 0 x x 2 1 X 0 0 1 0 1 0 X 1 X 1 3 X 1 0 0 0 0 1 X 0 X X 0 0 0 0 4 X Go to Technology SubReddit Save Unsubscribe Subscribe Share Article X 1:50 1:39 3:26 2:14 :18 :14 1:22 :30 1:06 :05 :20 :26 :43 1:43 X :24 X X 3:06 X 0.57 1:05 1:01 1:33 :37 1:06 1:17 :32 1:14 X 1:01 3:44 X X :33 :20 :47 :39 1:53 X 1:29 X X 1:18 X 1:14 :32 X X
- 33. Go to Technology SubReddit Save Unsubscribe Subscribe Share Article NO YES YES YES YES YES YES YES YES YES YES YES YES YES YES NO YES NO NO YES NO YES YES YES YES YES YES YES YES YES NO YES YES YES NO YES YES YES YES YES NO YES NO NO YES NO YES YES NO NO
- 35. 0% Expert (5) 6 Rate your likelihood to share images/videos/media? 0% Not at all likely (1) 20% 2 10% 3 20% 4 50% Very Likely (5) 7. Rate your likelihood to comment on media on the Internet? 30% Not at all likely (1) 0% 2 30% 3 20% 4 20% Very Likely (5) 8. Rate your likelihood to like or dislike media on the Internet? 10% Not at all likely (1) 20% 2 10% 3 10% 4 50% Very Likely (5) PostSurvey Responses 1. I was able to successfully complete all the tasks. 10% Strongly Disagree (1) 20% 2 10% 3 40% 4 20% Strongly Agree (5) 2. I found the site easy to navigate. 50% Strongly Disagree (1) 10% 2 40% 3 0% 4 0% Strongly Agree (5) 3. I was able to quickly find the topics and articles I was looking for. 0% Strongly Disagree (1)