We present GaussBits, which is a system of the passive magnetic tangible designs that enables 3D tangible interactions in the near-surface space of portable displays. When a thin magnetic sensor grid is attached to the back of the display, the 3D position and partial 3D orientation of the GaussBits can be resolved by the proposed bi-polar magnetic field tracking technique. This portable platform can therefore enrich tangible interactions by extending the design space to the near-surface space. Since non-ferrous materials, such as the user's hand, do not occlude the magnetic field, interaction designers can freely incorporate a magnetic unit into an appropriately shaped non-ferrous object to exploit the metaphors of the real-world tasks, and users can freely manipulate the GaussBits by hands or using other non-ferrous tools without causing interference. The presented example applications and the collected feedback from an explorative workshop revealed that this new approach is widely applicable.
Project page: http://graphics.csie.ntu.edu.tw/~howieliang/GaussBits.html
Rong-Hao Liang, Kai-Yin Cheng, Liwei Chan, Chuan-Xhyuan Peng, Mike Y. Chen, Rung-Huei Liang, De-Nian Yang, and Bing-Yu Chen,
"GaussBits: Magnetic Tangible Bits for Portable and Occlusion-Free Near-Surface Interactions", In Proceedings of ACM CHI 2013, pp.1391--1400.
Project pages:
GaussBits: http://graphics.csie.ntu.edu.tw/~howieliang/GaussBits.html
Project Gauss: Portable Object Tracking Using Magnetic Sensor Grid http://www.cmlab.csie.ntu.edu.tw/~howieliang/HCIProjects/projectGauss.html
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
ACM CHI 2013 - GaussBits: Magnetic Tangible Bits for Portable and Occlusion-Free Near-Surface Interactions
1. Rong-Hao Liang, Kai-Yin Cheng, Liwei Chan, Chuan-Xhyuan Peng,
MikeY. Chen, Rung-Huei Liang, De-NianYang, and Bing-Yu Chen
National Taiwan University,Academia Sinica, and National Taiwan University of Science and Technology
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
5. CapWidgets [Kratz et al. CHI EA’11]
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
6. CapWidgets [Kratz et al. CHI EA’11] TUIC [Yu et al. CHI’11]
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
7. CapWidgets [Kratz et al. CHI EA’11] TUIC [Yu et al. CHI’11]
Clip-on Gadgets [Yu et al. UIST’11]
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
8. CapWidgets [Kratz et al. CHI EA’11] TUIC [Yu et al. CHI’11]
Clip-on Gadgets [Yu et al. UIST’11]
CapStones and ZebraWidgets
[Chan et al. CHI’12]
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
9. Interaction design is
to 2D
Limitation:
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
12. their hands without dropping the object. If the user is grab-
bing the object for longer than 2.5s, it starts specific func-
tions such as record and play back.
While stereo IR cameras were useful in obtaining the accu-
rate position and orientation of the object using retro-
reflective tape, it was challenging to distinguish users’
hands from background or objects. We chose to use an ad-
ditional depth camera Microsoft Kinect to detect the user’s
hand pose with computer vision techniques built on top of
position and the diameter of the circle. We have n
much effort to optimally determine the focal plan
projected image - focusing the projectors roughl
middle of the interactive space is sufficient.
2
https://github.com/ofTheo/ofxKinect
Figure 8. Tracking and Projection System of
ZeroN.
Figure 10. As the user tilts the outer plastic lay
the system senses the orientation and updates t
projected images, while the spherical magn
stays in the same orientation.
331
ZeroN [Lee et al. UIST’11]
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
13. their hands without dropping the object. If the user is grab-
bing the object for longer than 2.5s, it starts specific func-
tions such as record and play back.
While stereo IR cameras were useful in obtaining the accu-
rate position and orientation of the object using retro-
reflective tape, it was challenging to distinguish users’
hands from background or objects. We chose to use an ad-
ditional depth camera Microsoft Kinect to detect the user’s
hand pose with computer vision techniques built on top of
position and the diameter of the circle. We have n
much effort to optimally determine the focal plan
projected image - focusing the projectors roughl
middle of the interactive space is sufficient.
2
https://github.com/ofTheo/ofxKinect
Figure 8. Tracking and Projection System of
ZeroN.
Figure 10. As the user tilts the outer plastic lay
the system senses the orientation and updates t
projected images, while the spherical magn
stays in the same orientation.
331
Daniel Avrahami1,2
, Jacob O. Wobbrock2
and Shahram Izadi3
1
Intel
2200 Mission College Blvd.
Santa Clara, CA 95054-1549
2
The Information School | DUB Group
University of Washington
Seattle, WA 98195-2840
3
Microsoft Research
7 J J Thomson Avenue
Cambridge, UK
ZeroN [Lee et al. UIST’11] Portico [Avrahami et al. UIST’11]
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
14. Problems: Sensitive to Occlusions
Hand Occlusion +
Material Occlusion
X
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
15. Hand Occlusion +
Material Occlusion
Problems: Sensitive to Occlusions
X
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
22. GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
The prototype Hall-Sensor Grid is only 2mm-thick
1. Magnetic field tracking is portable and accurate.
GaussSense [Liang et. al. UIST’12]
23. GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
The prototype Hall-Sensor Grid is only 2mm-thick
1. Magnetic field tracking is portable and accurate.
GaussSense [Liang et. al. UIST’12]
24. GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
Non-Touch Portable Displays can apply the sensing
25. GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
Non-Touch Portable Displays can apply the sensing
26. 2. Strong magnetic fields can be tracked above the display.
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
27. 2. Strong magnetic fields can be tracked above the display.
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
28. More DOF to interact with the display contents
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
29. GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
3. Magnetic fields penetrate through
non-ferrous materials. e.g. hands
30. The magnetics allow for free grasping
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
31. Allow for crafting magnetics in favorable form
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
32. Interact with the magnetics by non-ferrous instruments
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
33. Interact with the magnetics by non-ferrous instruments
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
34. Interact with the magnetics by non-ferrous instruments
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
35. Interact with the magnetics by non-ferrous instruments
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
36. Interact with the magnetics by non-ferrous instruments
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
37. Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
What we have done
39. Tiltable GaussBits
The magnetic field is Cylindrical Symmetric
acking
ency magnetic fields can pass through the
suitable for use in occlusion-free track-
ented a high-resolution 6-DOF (Degree-
ng method using an electromagnetic field,
anism depends on an external emitter that
integrate into mobile devices. Passive res-
cause the resonance of the sensing elec-
ay to provide ID, single axis position, and
tion. A ferromagnetic input device [12]
coils which allows to sense free-form fer-
However, both of these solutions are typ-
ortable use.
tability, magnetometers are used in many
can track the 3D position of a passive
dial detection range [9], but the orienta-
c field cannot be resolved. Although an
4
N
S
a
Figure 2. (a) Principle of Tilt
(b) cylindrical or a (c) ring m
shown in (d) and (e), tilt info
map of the dipole fields.
To resolve the tilt of a G
ily resolved in any direc
symmetric 3D magneti
quired, as shown in Fig.
magnetized cylindrical m
gested. If see-through c
R camera arrays at
bjects on the screen
ameras that are po-
se vision-based ap-
allow the interaction
pace above the dis-
problem because of
logy.
can pass through the
cclusion-free track-
on 6-DOF (Degree-
ectromagnetic field,
external emitter that
devices. Passive res-
of the sensing elec-
le axis position, and
ic input device [12]
sense free-form fer-
red and blue, respectively. The IMPLEMENTA
will provide details of the hardware and sensin
Types of GaussBits
GaussBits designs are of two types: Tiltable
Rollable GaussBits.
Tiltable GaussBits
N
S
3
3
ca
b
Figure 2. (a) Principle of Tiltable GaussBits, which can
(b) cylindrical or a (c) ring magnets. Based on the mag
shown in (d) and (e), tilt information can be extracted f
map of the dipole fields.
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
43. Rollable GaussBits
The bi-polar magnetic field is Not Cylindrical Symmetric
S
esign element for use with a mag-
ique. Based on the properties of a
ich is symmetrically and uniformly
s can provide their 3D position in-
orientation information through.
can be performed with the Gauss-
ilt, roll, and flip. The 3D transla-
e easily resolved by analyzing the
imum strength of the sensed mag-
cussed previously [18]. The present
g GaussBits to enable the resolution
odymium magnets are used in pro-
olar magnetic-field tracking sensor
a), is used here to visualize the N-
d intensity maps of the GaussBits in
NS
a
Figure 3. (a) Principle
(b) a single magnet or (c)
the magnetic field, roll in
position of the dipole. (e
The simplest way to r
rately is to lay down
the magnetization pa
or to attach another
GaussBit as shown i
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
44. Rollable GaussBits can support Flipping
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
45. Rollable GaussBits
The bi-polar magnetic fields are
Not Cylindrical Symmetric
Tiltable GaussBits
The magnetic fields are
Cylindrical Symmetric
We can NOT build a GaussBit which is
tiltable and rollable to support 6DOF operations30°
30°
c
db
e
ble GaussBits, which can be made by using
gnets. Based on the magnetic field images
mation can be extracted from the intensity
aussBit’s tilt information to be eas-
on without ambiguity, a cylindrical-
sBits
gns are of two types: Tiltable GaussBits and
Bits.
its
30°
30°
c
d
a
b
e
iple of Tiltable GaussBits, which can be made by using
a (c) ring magnets. Based on the magnetic field images
e), tilt information can be extracted from the intensity
Limitation
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
46. Ci
’
Ci
: by higher, dynamic threshold
: by lower threshold
Extract the connected components
I’
Oi’N
Oi
N
Oi’N
Oi’S
Results
Sensing algorithm
Tiltable
Rollable
Raw Image
I IN
IS
Thresholding
: centroid of Ci
’
Oi
’
: centroid of Ci
’
Oi
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
47. Types of GaussBits can be recognized by the shape of magnetic fields
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
48. Sensing algorithm
Roll direction
2D position (x,y)
Tilt direction
Hover height (z)
Max. Intensity
Tilt angle
Without Pre-Registering the Magnets With Registration
Rollable GaussBit Tiltable GaussBit Tiltable GaussBit
2D position +
Direction of Roll orTilt
3D position +
Direction and Angle of Tilt
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
49. 0° 30° 60°
10x4
H R
H
R
H T H
T
H
T
10x4 3x15 3x15x12 6x12.5+10x4
Hover
Roll
Tilt
GaussBit
Hall-sensor grid
b3x19x15
GaussClock GaussNavigator GaussPilot GaussCooker
Evaluation
17-44mm above the display
h
θ
H
R
6x12.5+10x4
Hover
Roll
Tilt
a
b c
ooker Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
50. h
θ
0° 30°15° 45°
0° 30° 60° 90°
0° 30°15° 45°
3x15x12
3x15
10x4
T
H R R
T H
T
H
T
10x4 3x15 3x15x12 6x12.5+10x4
Tilt
b c
d
e
HxORxIR: Height x Outer-Radius x Inner-Radius (Unit:mm)
3x19x15
GaussClock GaussNavigator GaussPilot GaussCooker
Figure 10. (a) Experimental apparatus. (b) Sensing distance of the sam-
ple GaussBits. Within this distance, the position of hovering and the
directions of roll or tilt can be resolved. (c) Definition of sensing dis-
tance h and tilt angle q. (d) Resulting magnetic field images of a Rollable
GaussBit sample and (e) two sample Tiltable GaussBits at a hover height
Evaluation
Track the tilt and roll well (10mm height)
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
52. c d
e f
a b
hg
Figure 13. Two methods identifying rollable magnets while determining
the orientation of roll. (a) Carving a gap in the side of a ring magnet
yields the magnetic field that is shown in (b). (c) The image of the mag-
netic field shows that the gap can be clearly used to provide the orien-
tation. (d) The pink area represents the ID design space. (e) Attaching
one or many cylindrical magnets to the side of a ring magnet yields the
magnetic field image in (f). (g) The orientation can be resolved and (h)
the pink area represents the ID design space.
Using Shape as ID
Providing the roll information as well, but no tilt
Subtraction
Addition
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
53. Available
Information/
Type of Tag
ID
LocationLocation OrOrAvailable
Information/
Type of Tag
ID 2D
(x,y)
Height
(z)
Rotation
(Roll)
NFC Tag v
TUIC-2D
TU
Rollable Ga b
Available
Information/
Type of Tag
ID
LocationLocation OOAvailable
Information/
Type of Tag
ID 2D
(x,y)
Height
(z)
Rotation
(Roll)
NFC Tag v
TUIC-2D
T
Rollablea b
Combine with other ID techniques
Need to identify the tag before each session of use
+ NFC tag + Capacitance tag
(TUIC-2D)
TUIC [Yu et al. CHI’11]
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
54. Explorative Workshop
To explore the applicability of GaussBits
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
55. 8 Senior Graduate Students from a Design Institute
2 Groups
2 Hours for Each Group
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
56. 1. Experience theTechnology
8 Senior Graduate Students from a Design Institute
2 Groups
2 Hours for Each Group
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
57. 1. Experience theTechnology 2. Brainstorm through prototyping
8 Senior Graduate Students from a Design Institute
2 Groups
2 Hours for Each Group
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
61. Real-World Simulation & Entertainment
Gardening by
Wielding an axe
Treasure hunting
using a metal detectorSculpting with a Knife
62. Real-World Simulation & Entertainment
Gardening by
Wielding an axe
Treasure hunting
using a metal detectorSculpting with a Knife
Holding a car for Extreme Racing
63. Real-World Simulation & Entertainment
Gardening by
Wielding an axe
Treasure hunting
using a metal detectorSculpting with a Knife
Holding a car for Extreme Racing
City map navigation with
a magnifying glass
64. Real-World Simulation & Entertainment
Fishing with a rod
with a magnet as the bait
Gardening by
Wielding an axe
Treasure hunting
using a metal detectorSculpting with a Knife
Holding a car for Extreme Racing
City map navigation with
a magnifying glass
66. Second Display as aToolglassEducation
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
67. Toolkits
Second Display as aToolglassEducation
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
Gardening Storytelling
68. The proposed wide range of applications demonstrates that
GaussBits are easily usable, and would be applied in interesting ways
by interaction designers when magnetic-field tracking becomes
available as a feature in commodity hardware.
Design > Sensing algorithm > Evaluation > Identification > Explorative Workshop
80. Rong-Hao Liang, Kai-Yin Cheng, Liwei Chan, Chuan-Xhyuan Peng,
MikeY. Chen, Rung-Huei Liang, De-NianYang, and Bing-Yu Chen
GaussBits:
Magnetic Tangible Bits for
Portable and Occlusion-Free Near-Surface Interactions
National Taiwan University,Academia Sinica, and National Taiwan University of Science and Technology
13年1月5日星期六
Thanks for your attention!