BeyondLab Tokyo #2 : Robotics in Softbank HQ “Coming Future of Robotics” Student Innovation through VR+Robotics Akihiko Shirai, Ph.D 2016/May/20

1. BeyondLab Tokyo #2 : Robotics
in Softbank HQ
“Coming Future of Robotics”
Student Innovation through VR+Robotics
Akihiko Shirai, Ph.D
Associate professor
Kanagawa Institute of Technology, JAPAN
http://www.shirai.la/project/expixel

3. Biography
1973 Born in Yokohama, Japan
1983 (10) 1st programming
1988 (15) Play Nintendo too much.
1990 (17) News paper boy and arcade game
1992-1998 Tokyo Polytechnic Institute
Photo Engineering, Virtual Reality in Lab
1998 Hired, Canon Inc. Printing company.
(Got a chance of planner in Namco)
2000 Moved to Video Game industry
Criterion Inc. Guildford U.K.
Real-time Rendering Engine for PS2
Meet up with Ph.D in Engineering
2001 Back to Academia
Tokyo Institute of Technology
Virtual Reality Entertainment Systems
for infancy. (Ph.D in Engineering)
2003-2004 Post-Doc in NHK ES.
2005-2007 Move to France
ENSAM Presence & Innovation Lab.
Prototype of VR Theme park project
2008-2009 Science Communicator
in National Science Museum
2010 Kanagawa Institute of Technology

4. [definition] Entertainment system is System
which affects to human amusement.
Video game, museum exhibition, entertainment VR, media arts, e-sports,
broadcasting, social network…
1. Device and/or System
2. Service and/or Contents
3. Evaluation methods
Research in Entertainment system

5. Tangible Playroom: Penguin Hockey (2001)

6. AceSpeeder2 + GPU Vision (2006)
https://www.youtube.com/watch?v=N4WtupBoRMI

7.  https://www.youtube.com/watch?time_continue=2&v=75jA7F8AXC4
“The song of Anagura”, Miraikan
常設展示『アナグラのうた 〜消えた博士と残された装置〜』

8. Manga Generator Project

9. Robo Remote, Yamazaki Lab in KAIT

10. Concept: Solving common issue in living room

11.  Pixel-Shader Technology that can show multiple images on one screen at same time.
 ExPixel
 Extended Pixel
 X-Pol (FPR for 3D)
 Extra Image…
What is ExPixel

12. Scritter to ExPixel: Scritter
NAGANO, K., UTSUGI, T., HIRANO, M., HAMADA, T., SHIRAI,A., AND NAKAJIMA, M. 2010.
A new ”multiplex content” displaying system compatible with current 3d projection technology.
In ACM SIGGRAPH 2010 Posters, ACM, New York,NY, USA, SIGGRAPH ’10, 79:1–79:1.
Multiplex by
two projectors.

13. Scritter to ExPixel: ScritterH
Wataru FUJIMURA, Yukua KOIDE, Robert SONGER, Takahiro HAYAKAWA, Akihiko SHIRAI, Kazuhisa YANAKA,
"2x3D: Real-Time Shader for Simultaneous 2D/3D Hybrid Theater",
ACM SIGGRAPH ASIA 2012 Emerging Technologies, Singapore, 2012
Hidden imagery
with two projectors.

14. ScritterH, Projector based hidden imagery
 Merge an arbitrary image A for
naked eyes and an image B that
you would like to hide to generate
a different image C and multiple-
project B and C.
 Image A visible to naked eyes while
image B with glasses.
* Figure from ScritterH project

15. “2x3D”, hybrid 2D+3D theater

16. 2-3 ハイブリッドシアター[2X3D]
Palarax!

17.  Naked eye: Left + Filter: Right.
 It is Implemented as a GPU based shader.
 Good for theater but not suitable for consumer usage.
2x3D: Hybrid 2D + 3D display
Wataru FUJIMURA, Yukua KOIDE, Robert SONGER, Takahiro HAYAKAWA, Akihiko SHIRAI, Kazuhisa YANAKA,
"2x3D: Real-Time Shader for Simultaneous 2D/3D Hybrid Theater",
ACM SIGGRAPH ASIA 2012 Emerging Technologies, Singapore, 2012

18. Line-by-line: Even odd selection
Only even lines Only odd lines

19.  Everybody can make multiplex hidden content with Unity 3D
Content Creation tool
1) Import Package 2) Set Images 3) Run!!

20. Multi-language presentation
Naked Channel (Janpanese)
Hidden Channel (English)

21. UbiCode
Naked Channel (Advertisement)
Hidden Channel (QR code)

22. POLAR: Physical Optical Layered AR
AR marker on the main content

23. POLAR: Physical Optical Layered AR
AR marker on
the hidden channel

25. Road Mapping of Multiplex-Hidden technologies
1st gen “Scritter”Multiplex
2nd gen“ScritterH Multiplex Hidden
3rd gen on LCD Projector
4th gen on Flat Panel
5th gen Public Media
2010 2012 2014 2016 2018 2020
A. Shirai 2015.8.10
Multilanguage Presentation, education application
▲ Medial appli.
▶ “PARAOKE”
(2012)
◀ 4K/240 inch large screen
(2010)
◀ “Scritter” by Koki Nagano (2010)
▲ “2x3D”, Hybrid theater （2012）▲ “UbiCode”
Interactivity for digital signage （2012）
▶ private on public signage
◀ 「ニコニコメガネ」
(2013)

27. Website: “blog.shirai.la” and @o_ob

28.  WiiRemote Programming
(2009)
Books
 The Future of Game Design
– Science in Entertainment systems (2013)

29. Come to see & join to Laval Virtual and IVRC
(1) What is IVRC?
(2)What is Laval Virtual?
(3)What is ReVolution?
Today’s conclusion

30. Sponsored by
Symposium On Education
Education Workshop:
Innovation in the Age of Virtual
Reality through Organizing
International Student Competition
国際学生コンテストを通した
VR世代のイノベーション
Akihiko SHIRAI, Ph.D
IVRC Executive Committee
Kanagawa Institute of Technology, Japan
白井暁彦 神奈川工科大学／IVRC実行委員会

31. Sponsored by
Agenda of this workshop
Innovation in the Age of Virtual Reality
through Organizing International Student Competition
11:00-11:30 [Part1]Free Play of past IVRC projects
11:30-11:50 [Part2] Opening Address by Prof. Susumu TACHI
11:50-12:00 coffee break
12:00-12:30 [Part3] Panel Talk
12:30-13:30 [Part4] Panel Discussion (Japanese/English trans)
13:30 Photo session
-14:00 [Part5]Demo and Discussion

32. Innovation in the Age of Virtual Reality through Organizing International
Student Competition
Education Workshop, Symposium on Education, SIGGRAPH ASIA 2015
Exhibition Hall No.2, Meeting Room 2A
11:00 – 14:00, Tuesday, November 3, 2015
http://ivrc.net

33. What is IVRC?
 International collegiate Virtual Reality Contest
establish in 1993 by Prof. Susumu Tachi
 Contest of interaction
and virtual reality, which
fuses art and technology
 Honors the greatest Virtual Reality and Interactive
works of students
 New education system through contest

34. What is IVRC?
 Organized by Virtual Reality Society of Japan
 Supported by Ministry of Economy, Trade and Industry and
Ministry of Education, Culture, Science and Technology
 Ambassade de France au japon, Service pour la Science et la
Technologie
 Held at Miraikan, National science museum of Emerging
Science and Innovation
Virtual Reality Society of Japan
http://www.vrsj.org/

35. IVRC: contest of the students, by the students, for the students
 IVRC is the acronym of “International collegiate Virtual Reality Contest”, which has been held since
1993 and is the contest of Interactive products using advanced technologies such as virtual reality and
robots. Our English name changed for internationalization in 2004, because we began to invite
awarded teams from the French VR contest “Laval Virtual”.
 When you hear the words such as “Virtual Reality” and “robot”, you should feel that special expensive
machines are necessary to realize them. However, there are many ideas which can be actualized only
with handmade machines. The main purpose of this contest is for participants to gain deeper
understanding of and familiarity with interactive technologies through activities such as planning
interactive systems, making devices themselves, and improving quality of their contents.
 IVRC has not only the function of contest but also the function of education. It aims to construct the
new education system of the 21st century to develop excellent human resources, who can think, learn,
and work by themselves.

36. Call for Submission
Deadline of submission
2nd stage, Demonstration
Final stage, Demonstration
SIGGRAPH/ Laval Virtual
Online submission
Improvement phase
1st
stage, Paper review
Development phase
IVRC Flow Chart
4 +1 projects
Annual Conference of VRSJ
DCEXPO: Digital Content Expo
April
Late June or Early July
September
October

37.  Semi final at the annual conference of the Virtual Reality Society
of Japan (9-11th September, Tokyo)
Demonstration in academic conference
 Grand final in Digital Contents EXPO 2015
22-25th October, Miraikan, Tokyo.
Demonstration in Public
 U-18: Youth section
 International Video Selection for the final stage

38. The First Presence at ACM SIGGRAPH 2002
 IVRC Committee helps the champion team submit to SIGGRAPH
as a reward to the winner.
 “Virtual Chambara”, SIGGRAPH2002
“Virtual Chambara”,
SIGGRAPH 2002

39. History of IVRC Presence at SIGGRAPH
 2003 THE DIMENSION BOOK
 2004 Fragra / Dis-Tansu
 2005 Straw-like user interface / Kobito: Virtual Brownies /
Ton2
 2006 bubble cosmos / INVISIBLE ~The Shadow Chaser~ /
Powder Screen: A Virtual Materializer …..

40. History of IVRC Presence at SIGGRAPH
 2007 BYU-BYU-View / CoGAME
 2008 Ants in the Pants / Landscape Bartender
 2009 Baby type robot “YOTARO” / Back to the mouth /
Funbrella: making rain fun
 2010 Haptic Canvas
 2011 Mommy Tummy
 2015 Childhood/ vibroSkate

41. International Cooperation with Laval Virtual
“The winner of the IVRC Award in France will be invited to the
Tokyo stage of the IVRC in Japan as a challenger. Similarly, the
winner of the Laval Virtual Award in Japan will be invited to the
final stage of Virtual Fantasy, category “Demos”, student
competition of Laval Virtual, in France as a challenger. Conditions of
invitation will be defined every year by each part of this agreement.”

43. Agreement on International Cooperation
at Laval Virtual 2004
IVRC and Laval Virtual will maintain an award partnership for
three years (2004-2007): the Laval Virtual Award at the IVRC
in Japan, the IVRC Award at “Laval Virtual’s Le Village de la
Creations” in France.
J.F. Fontaine
(Laval Virtual)
Prof. S. Tachi
(IVRC)

44. International Cooperation with ETC, CMU
“The best work certified by CMU among other works created as student
project will be invited for the final contest of IVRC. The invited work will be
assessed in the final contest in the same way as other selected works in the
preliminary contest of IVRC and will be entitled to receive each award of
IVRC.”

45. Partnership
in SIGGRAPH 2010
Don Marinelli
Executive producer of ETC

46. http://ivrc.net/
Carnegie Mellon University (CMU) Entertainment Technology
Center (ETC) and International collegiate Virtual Reality Contest
(IVRC), in recognition of the growing importance of the roles
Virtual Reality and Entertainment Technology play in the 21st
century, have come to the agreement:
Agreement on International Cooperation in Virtual
Reality and Entertainment Technology
Michelle Macau
(ETC) Ambassador
Agent for Donald Marinelli
Executive Producer of ETC
Prof. S. Tachi
(IVRC)

48. 1st
3D Craze
（the1950s)
Dawn of
VR
(the1960s)
1940 1960 1980 2000 20201920
Dawn of
3D
（the1920s)
2nd
3D Craze
（the1980s)
3rd
3D Craze
（the2010s)
2nd
VR Wave
(the2020s)
?
1st
VR Wave
(the1990s)
movie movie
Theme-park
computer
game
TV
movie movie

49. Virtual Reality in the 1990s
 1989 VPL Research Data Glove and Eyephone. “Virtual reality" was first used.
 1990 Santa Barbara Conference (The Engineering Foundation)
Virtually all the researchers were summoned.
 1991 ICAT (International Conference on Artificial Reality and Telexistence) was
held in Japan.
 1992 Committee on Virtual Reality was established in National Research Council.
Its report was published in 1995.
 1993 IVR（Industrial Virtual Realty) was held in Japan.
 1993 VRAIS (Virtual Reality Annual International Symposium) and Research
Frontiers in Virtual Reality were held. They gathered together to form IEEE
Virtual Reality Conference.
 1993 IVRC（International Virtual Realty Contest) was held.
 1996 VRSJ (Virtual Reality Society of Japan) was established.
 2001 IEEE Virtual Reality Conference was held in Japan.
 2008 National Academy of Engineering (NSF) announced 14 grand engineering
challenges for the 21st century on February 15, 2008. Enhance Virtual Reality is chosen
as one of the 14 challenges.

50. 23 years…
 More than 200 projects have been realized through 23
IVRCs including 21 projects selected as SIGGRAPH E-
Tech.
 Average 5 persons for each project mean over 1,000
persons have deeply experienced and understood VR!
 IVRC students become staffs, and staffs now become
world famous professors, designers, creators, and
engineers.

51. IVRC Alumni Members

52. 21 CENTURY'S GRAND
ENGINEERING CHALLENGES
 National Academy of Engineering announced 14 grand
engineering challenges for the 21st century on February 15,
2008. http://www.engineeringchallenges.org/
 The final choices fall into four themes that are essential for
humanity to flourish -- sustainability, health, reducing
vulnerability, and joy of living.
 Tremendous advances in quality of life have come from
improved technology in areas such as farming and
manufacturing. If we focus our effort on the important grand
challenges of our age, we can hugely improve the future.
 Enhance Virtual Reality is chosen one of the 14 challenges.

53. Second VR Wave（the 2020s ？)
VR Challenges
 Autostereoscopic Display
 Haptic VR
 Augmented Reality
 Telexistence
 Human Augmentation

54. 1st
3D Craze
（the1950s)
Dawn of
VR
(the1960s)
1940 1960 1980 2000 20201920
Dawn of
3D
（the1920s)
2nd
3D Craze
（the1980s)
3rd
3D Craze
（the2010s)
2nd
VR Wave
(the2020s)
?
1st
VR Wave
(the1990s)
movie movie
Theme-park
computer
game
TV
movie movie

56. IVRC Wishes to thank…

57. New comer via “International Video Session”
 The Maze Game(Puzzle, SAS VR, Oman)

58. Laval Virtual 2014 to IVRC 2014
Stimulus(ESIEA Ouest, Laval, France)
In Digital Contents Expo 2015
Collaboration with KAIT students

59. Sponsored by
IVRC2015 to Laval Virtual 2016
Laval Virtual 2016
23rd-27th March, 2016 in Laval France
[Invited Project]
Winner: “Nyokinyoki beans of the tree”
(Jack and bean stalk) Univ. Keio
<ReVolution>
Open competition in VR
11th Edition Theme “Real-Virtuality”
(submission deadline: End of Dec.)
http://www.laval-virtual.org/

60. Come to see & join to Laval Virtual and IVRC
(1) What is IVRC?
(2)What is Laval Virtual?
(3)What is ReVolution?
Today’s conclusion

61. Laval Virtual

62. Laval Virtual
Academic Conference “ACM VRIC”
Award, Competition, Public Day

63.  Keynotes
 Workshop + Aural Presentation
ACM VRIC: Virtual Reality International Conference
Dr. Pattie Maes, MIT Media LaboratoryPr Alain Berthoz, College de France
Virtual avatars and real brains: how do they interact? AR and VR for learning and collaboration
Dr. Oliver Bimber, Johannes Kepler Univ. Linz
Light-Field Technology: A Revolution to Imaging and Display
Dr. Carolina Cruz-Neira, Emerging Analytics Center, University of Arkansas at Little Rock
Beyond Fun and Games: VR as a Tool of the Trade
Christine Perey, AR for Enterprise Alliance
The Future of Work with Augmented Reality
Dr. Skip Rizzo
Director for Medical Virtual Reality Institute for Creative
Technologies - Research Professor USC Davis School of
Gerontology and USC Keck School of Medicine Department of
Psychiatry & Behavioral Sciences
Beyond Video Games:
A Virtual Reality Revolution in Behavioral Health

64.  Workshop + Aural Presentation
ACM VRIC: Virtual Reality International Conference

65. Special Session on Friday
14:00-14:20 Opening Address “Real-Virtuality”, Prof. Akihiko SHIRAI (KAIT)
14:20-14:30 “IVRC to Laval Virtual / Jack and the Beanstalk”, Mr. Shota SUGIMOTO, (Univ Keio)
14:30-14:50 “Embodied Media”, Prof. Kouta MINAMIZAWA (Univ keio)
14:50-15:00 “Metamorphosis Hand: Interactive Experience of Embodying Virtually Transformed Hands”, Nami
Ogawa (Univ of Tokyo)
15:00-15:30 “VR to Superhuman Sports” Prof. Masahiko INAMI (Univ of Tokyo)
15:30-16:00 Round table and debate, exchanges with the audience
ACM VRIC: Virtual Reality International Conference
http://superhuman-sports.org/

66. AR/VR Contents, and for the next year…
Thierry Frey, chair of AR/VR contents and VR Startup Contest. At Final day of Laval Virtual 2016

67. URANIOM in Laval

68. Student Competition

69. Student Competition

70. RIKKYO IKEBUKURURO HIGH SCHOOL
MATHEMATICAL SCIENCE CLUB
SOARING
BIKE
LAVAL VIRTUAL 2016
Virtual Fantasy DEMO Category

71. “The flying bike that everyone can enjoy.”
The concept of SOARING BIKE is...
The fascinating parts
・Doesn’t have a target of age group

72. All Hand made

73. How to experience SOARING BIKE

74. ・You could fly anywhere inside the stage.
Enjoy your flight.

75. Come to see & join to Laval Virtual and IVRC
(1) What is IVRC?
(2)What is Laval Virtual?
(3)What is ReVolution?
Today’s conclusion

76. Open Competition of VR demo
 Technical innovation / Interaction
 Academic + Public demo testing
 Theme “Real-Virtuality”
…Realizing Virtual, Bring real to virtual, Fantasy to be Real…
ReVolution

77. Glassless Augmented Display, KAIT

78. Gifts of Blizzard, Tama Univ.

79. "Real Virtuality"

80. We Wave II, Paris8

82.  Submission deadline: end of year (31/Dec/2016)
 Theme: Beyond visuals, Bring your body (temp.)
See you next year in Laval 