This usability test is based on our roommate matching website--Roomie.

1. Last update: 2014.3.18
Zhao Wenbin Chen Cheng Wu Tianyi Xu Yihua Zhu Ruoyu Zuo Xingyuan
Roomie
Usability Test Report
1

2. Content
2

3. About Roomie
Roomie is an online platform that aims at helping university
students find suitable roommates in Hong Kong according to
their personality, life style and other daily issues.
3
Target population of Roomie
While the primary users of Roomie are students from Mainland
China, as most of them cannot apply for the university
dormitories, we also prefer to target graduates who determine
to work in Hong Kong but lack roommates to share an
apartment, as well as local students who prefer to live
separately from parents.

4. Goal of usability testing
Our goal for the usability testing was to collect both
quantitative and qualitative data of new users learning how to
use Roomie to search for potential roommates for the first
time. We hope that the results of the usability testing can be
used by us to understand the strengths and weaknesses of this
project and serve as a benchmark for future usability testing.
4

5. Overview
Usability testing occurred during the one-week span of March
9-16. We divided our testing into two rounds, the first round
on March 9 and the second round on March 16. After the first
round testing, we optimized our prototype according to
participants’ suggestions, then held the second round testing
one week after. Ten participants were recruited who had never
heard of our project before but yet had a need to look for
roommates, or had related roommate-finding experiences.
During the half-an-hour testing sessions, each participant was
asked to do the task of searching for suitable roommates on
Roomie. Time spent on the task and qualitative comments
were collected for each participant. Each session was recorded
using QuickTime to allow for later analysis.
5

6. Prototype Building
We used Axure to make the prototype of Roomie online
platform. Axure is a business drawing software for creating
mockups and wireframes of application user interfaces.
This easy to use software generates HTML prototypes and
functional specifications.
6

7. Participant Selection & Recruitment
Since our goal was to find the strengths and weaknesses of this
project, we recruited users who had never heard Roomie
before but had a need to look for roommates, or had related
roommate-finding experiences. They should be future
students, present students or graduates of a certain university
in Hong Kong. We recruited our participants through our social
networks, and offered them a small gift as an incentive for
participation.
7

8. Test Protocol
Each test session last for one hour, including 10 minutes for a
pre-test background interview and post-test debriefing
questions, and 20 minutes for the tasks.
We began each test session by asking them some general
questions about how they got along with their former
roommates, whether they had some good or bad experiences,
and what they cared the most when looking for a roommate.
Then we gave them two tasks, and asked them to complete
each one while talking aloud about their thought process and
any confusion they encountered.
8

9. Test Protocol
The two tasks we gave them were organized by the sequential
order. The summary of tasks is listed below.
Task Number Task Summary
1 Register for Roomie
2 Search for roommates on Roomie
9
After the participant completed the tasks, we conducted a
post-test brief, asking them about their interesting or confused
points about Roomie. We also asked them to score each page
according to their satisfaction (4-point Likert scale, from 1
=dissatisfied to 4=satisfied) and asked if they thought they
would be likely to use Roomie, based on their initial
impression.

10. Test Environment and Equipment
The tests were conducted in Learning Commons of Wu Ho Man
Yuen Building. We used QuickTime to record our test screens.
We each took turns moderating one session and observing one
session. We set up QuickTime on a Mac, and give this Mac to
participant to use during the test.
10
Data Collection
We collected both qualitative and quantitative data about the
participants’ behavior. Our quantitative data consisted of the
performance data of time spent on tasks for each participant,
and their satisfaction (4-point Likert scale) towards Roomie.
The qualitative data we collected were quotes, comments and
observations noted down by the observer during the test.

11. Score analysis ： Test 1*
3. 3
3. 6 3. 6
2. 5
3 3. 1
2. 8
0
0. 5
1
1. 5
2
2. 5
3
3. 5
4
登
陆
注
册
页
邮
箱
注
册
填
写
信
息
个
人
主
页
首
页他
人
的
个
人
主
页
加
好
友
Test 1均分
Test 1均分
11
*Test 1 stands for the first-round test. After the first round, we optimized our
prototype and conducted Test 2, which is the second–round test.

12. Score analysis: Test 2
2. 7
2. 8
2. 9
3
3. 1
3. 2
3. 3
3. 4
3. 5
登
陆
注
册
页
邮
箱
注
册
填
信
息
页
个
人
主
页
首
页
搜
索
结
果
页
他
人
的
个
人
主
页
感
兴
趣
留
言
页
Test 2均分
Test 2均分
12

13. Score analysis: Test 1 & Test 2
0
0. 5
1
1. 5
2
2. 5
3
3. 5
4
登
陆
注
册
页
邮
箱
注
册
填
写
信
息
个
人
主
页
首
页
他
人
的
个
人
主
页
t est 1均分
t est 2均分
13

14. Time spent on task: Test 1 & Test 2
Tim e spent on task
0
5
10
15
20
25
1 2 3 4 5
Minutes
Test 1
Test 2
14
From this chart we can see clearly that after optimizing the prototype,
participants spent less time on completing tasks than the first time. It
means that now our prototype is easier to understand, and the
searching process is clearer than before.

15. Findings
In the first-round test, we found 5 broken experiences and
figured out solutions for each broken experience. In the second-
round test, we found 8 problems that needed to be solved.
15
3.Results
Fixed 5 problems
from Test 1
Fixed 8 problems
from Test 2

16. 3 person points out that the picture on the first page is not
appropriate.
“The picture in registration page cannot clearly represent the
function of Roomie.” “The picture make the website look like a
social media.”
16
Broken experience 1

17. Solution 1
Changed to a more clear and high quality picture, cannot find a
suitable picture with Asian faces.
17

18. Broken experience 2
5 users mentioned that the design of user’s own information page
is not clean and clear.
“I find the table very stressful and the classification of information
is not clear. ”
18

19. Solution 2
Change table to listed information, add more suctions in user’s
own information page.
19

20. 3 person points out that
“The advanced search is confusing”, “ I do not understand the
relationship between apartment and roommate.”
20
Broken experience 3

21. Solution 3
Change boxes to listed options to make it easier to understand,
and also add missed buttons.
21

22. 4 users mentioned
that more personal
information such as
lifestyle is necessary.
“I hope to see more
personalized
information to know
what kind of person
the potential
roommate is.”
22
Broken experience 4

23. Solution 4
Provide choices related to lifestyle and apartment requirements .
23

24. 3 person points out that the whole process of registration and
search is not very smooth, some buttons are missing.
“I think there should be a next or finish button.”
“ I can’t go back to the homepage.”
24
Broken experience 5

25. Solution 5
Adding navigation at the top of each page, and change “submit”
to “finish”.
25

26. After we fixed problems from Test 1, we updated our
prototype from V1.0 to V2.0 and conducted Test 2.
26
Fixed 5 problems
from Test 1

27. In the “fill-in personal information” page, we want to collect
more information about the user. But 3 users complained that
they don’t want to fill in all the information.
“Some information I don’t want to share to others. It’s all
required? ”
27
Broken experience 1

28. We add “*” in front of those mandatory fields to distinguish
from optional ones, so as users can clearly figure out which
part they must fill in.
28
Solution 1

29. 2 users pointed out that they don’t want to type in the
descriptions of their characters, or it took some time to think
about how to describe themselves. So there is a high
possibility that they will leave in this stage.
“I don’t want to type, I want to choose.”
29
Broken experience 2

30. Besides words to describe the characteristic,we first provide
menu for users to choose their character type.
30
Solution 2

31. 31
2 person points out that they can not understand the word
‘ ’洋楼 and‘ ’唐楼 , and actually the type of the building is
not that important. Moreover, the type of the apartment is
something that they concern more.
“I don’t know how to choose, I have no idea what it is.”
Broken experience 3

32. To be more clear, we change “ ”唐楼 and “ ”洋楼 to “ 无电
”梯楼宇 and “ ”有电梯楼宇 . We also add Type of
Apartment to provide more details.
32
Solution 3

33. After registration process, user will landing to personal homepage,
but there is no instructions for what to do next.
“OK, I’ve registered. So what I goanna do next? ”
33
Broken experience 4

34. Solution 4
Besides words to describe the characteristic,we first provide
menu for users to choose their character type.
34

35. Recommendation function on the homepage is overlooked by
users.
“See I can search the website. If there are recommended
roommates for me, that would be great!”
35
Broken experience 5

36. Solution 5
We put recommendation part on the top of homepage to
reinforce its function, and put search bar on the right side. If users
want to search more information, they can click Advanced Search
and go to search page.
36

37. 37
2 person points out that when searching, they put gender in
priority.
“Where is the gender in the search box? I would definitely
want that option before all the other choices. ”
NO GENDER!
Broken experience 6

38. We rearrange the choices and add “ ”同一性别 so as it can
be seen more easily.
38
Solution 6

39. In the process of finding search function,
user click “people I interested in” below
head portrait instead of “search” button on
the header. However, “people I interested
in” is not the right search button. There is
some misunderstanding of “people I
interested in”.
“I’ll click this(‘people I interested in’) to see
who is my suitable roommate.
39
Broken experience 7

40. We build list page of users who I am interested in after click
“people I interested in ” button. Users can check and
manage all the list here.
40
Solution 7

41. 41
2 person points out that they
can’t understand what
grouping means when I want
to leave a note to other users.
“I don’t know what the
grouping here means, will it
appear every time I leave
message for someone? Will
other people see my grouping
information? Would it be
more appropriate if it shows
when I liked someone?”
Broken experience 8

42. We revise Grouping faction, add a drop list for users to select
and add group category.
42
Solution 8

43. Further demand of renting a house can not be fulfilled in our
website
“Besides finding a roommate, I want to find a flat. If there is only
information of roommates, I think that’s not enough.”
Solution
Unfortunately,we cannot solve this problem so far.
43
Broken experience 9

44. The tasks are about the basic functions of Roomie. Most users
successfully finished all the tasks. All of them experienced
some difficulties but all successfully figured out a way to
overcome them.
From the first-round test we found many weaknesses of
Roomie and optimized it according to participants’ suggestions.
And in the second-round test, though Roomie had been
optimized, participants still figured out some broken
experiences that need to be solved. It means that to satisfied
users’ real needs, we still have a lot to do.
44

45. • Roomie usability test video
• Prototype (in the folder “V3.0/ index.html”)
45

46. Thank you
46