Yue Fei (uSens) Inside Out Tracking Case Study

•

4 likes•596 views

This presentation will include a demonstration of uSens' "Fingo" technology which includes tracking hands using computer vision. Fingo tracks the hand skeleton using infrared stereo cameras on a mobile head mounted display. This is one of the most natural use cases for gesture control in a virtual AR and VR environment. Augmented World Expo (AWE) is back for its seventh year in our largest conference and expo featuring technologies giving us superpowers: augmented reality (AR), virtual reality (VR) and wearable tech. Join over 4,000 attendees from all over the world including a mix of CEOs, CTOs, designers, developers, creative agencies, futurists, analysts, investors, and top press in a fantastic opportunity to learn, inspire, partner, and experience first hand the most exciting industry of our times. See more at http://AugmentedWorldExpo.com

Technology

Inside-out Tracking for VR and AR
Dr. Yue Fei, CTO & Co-founder, uSens Inc.

3D OS
3D Desktop
Room
VR
Table
VR
HMD
VR

Xerox PARC, Information Visualizer, 1991
Interactive 3D Desktops
Microsoft Research, TaskGallery, 2000
Sun Microsystems, Project Looking Glass, 2003 Infinite 3D, Cube, 2003

High accuracy but Low DOFs
CAVE systems Natural Interaction system
Panasonic Silicon Valley Labs 2010
Room scale VR
High DOFs -> Natural, Rich Interaction

ZSpace
High Accuracy VR Visualization & Interaction
Table based VR
Tavola, Panasonic Silicon Valley Labs 2012
Natural 3D Free Hand Interaction Immersive
(120 degrees max HFOV, 6DOF Head tracking)

3D OS
3D Desktop
Room
VR
Table
VR
HMD
VR
ubiquitous

Tracking Tracking Tracking
Richness Quality Interactivity

Mobile VR
with lightweight interaction
3D Free Hand Interaction
3D Head Tracking
AR
Full body tracking?

Mobile VR
Easy to setup
Low power consumption
CPU/GPU usage
Weight
Needs

uSens DEVNET starting soon!
• Devkits and SDKs for Unity3D, Java, and C++ Devs
• Interactive workshops and Devjams summer 2016
• Come to our tutorial session at 9:00 June 2 at AWE
• We’re excited to support the VR/AR community!

Thank You!
Contact:
Dr. Yue Fei, CTO & Co-Founder
info@usens.com
Q&A

More from AugmentedWorldExpo

A talk from the Training & Education Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Bill Meyer (Virtual Science Center): Turning Kids on to STEM by Letting Them Discover How VR and AR Tricks Our Brain How exactly does Virtual and Augmented Reality game human perception so things we know aren’t there seem to exist? Can simple and robust public learning experiences empower and delight people of all ages to experiment and discover the answers for themselves? The answer is yes. This talk will present examples and discuss how UX best practices can be applied to create effective inquiry-based learning experiences that unpack seemingly complex modern technologies. Through iterative design coupled with repeated visitor testing, we deconstructed the surprising perceptual science and technology behind VR and AR into 16 cool hands-on experiences that spark “aha” moments and minimize the need for wordy explanations. We keep concepts intuitive, playful, and fun to understand. Our traveling exhibition goal is to show visitors they're smarter than they suspect, inspire students to aim for STEM careers, and help teachers lead deeper NGSS and Common Core dives both in and out of the classroom. This 45 second video shows an example exhibit and part of our visitor testing process: https://youtu.be/Sgh0Sys57lY Without needing to learn the terms, visitors discover the secrets of how things like Motion Parallax, Stereo Vision, 3D Audio Spatialization, Outside-In, and Inside-Out Tracking work in concert to make virtual, augmented and mixed reality experiences possible. https://awexr.com

Bill Meyer (Virtual Science Center): Turning Kids on to STEM by Letting Them ...

AugmentedWorldExpo

A talk from the Gaming & Entertainment Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Nick Klingensmith (Microsoft): Increasing Immersion with the MR Lighting Tools To-date, most 3D graphics have been rendered on opaque displays. Now, not only are we putting holograms into the real world, but we're also doing it on additive displays! How do we render holograms that look good in this environment, and how do we make them look like they belong? The Mixed Reality Lighting Tools are a collection of easy Unity based solutions, from replicating your environment's lighting, to casting believable additive shadows. Come find out how easy it is to make your Mixed Reality application more immersive! https://awexr.com

Nick Klingensmith (Microsoft): Increasing Immersion with the MR Lighting Tools

AugmentedWorldExpo

A talk from the Creator Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Rokid: Design a seamless solution for AR glasses Qi Xiong | Rokid Paris Fan | Rokid Nan Wang | Rokid Gonglue Jiang | Rokid In this talk, we will discuss how Rokid creates a seamless AR experience from two aspects: ergonomic design(hardware) and systematic UI/UX design(software). The presentation will also share the insights of the entire product development cycle of Rokid Glass: from crafting product appearance with strict constraints, deploying user interfaces on a new and unique AR device, to the case studies and progressive design of real implementations. The talk offers a glimpse of the future AR design methodology by demonstrating the superpower enabled by design thinking. Rokid Glass is an AI-enabled, all-in-one AR glasses already in mass production, with seamless user experience which widely adopted by the industry. https://awexr.com

Rokid: Design a seamless solution for AR glasses

AugmentedWorldExpo

A talk from the Marking & Sales Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Tom De Vos | Fat Boy Dalia Lasaite | CGTrader Augmented Reality is having a profound impact on the way people shop online. By enabling consumers to interact with and try products before purchasing as they would in a brick and mortar store, Augmented Reality has been proven to increase sales conversions, reduce product returns and extend online engagement. This presentation will examine the latest technologies, trends and strategies for deploying AR for e-Commerce. https://awexr.com

How AR is Transforming the Online Retail Experience

AugmentedWorldExpo

A talk from the Enterprise Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Chloe Doyeon Kim (MAXST): The Current and Future State of MAXST Platform MAXST, a persistent challenger, is proudly specialized in Augmented Reality working with prominent Korean enterprises, such as Hyundai, Samsung Display, Samsung Electronics, and KT. MAXST’s cloud-based AR platform pursues innovation in terms of AR development tools, AR production costs, and profitability. We also believe Visual SLAM is critical basis for Future AR. We will reveal our stories and insight to the immersive world of today and tomorrow. https://awexr.com

Chloe Doyeon Kim (MAXST): The Current and Future State of MAXST Platform

AugmentedWorldExpo

A talk from the Start-up Pitch Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Owning Augmented Space Neil Mandt | The Layer Group Brian Wassom | Warner Norcross + Judd LLP "How can I own the AR 'air rights' around my physical property?" This may be the most commonly asked legal question around AR content. It is gaining new momentum and importance today, as location-based AR finally turns a profit, and as the quest for the Open AR Cloud takes shape. Finding the answers requires crossing the boundaries of multiple legal disciplines and practical concerns, and will not always be easy. This session will equip you to understand the legal risks and give you strategies to protect the integrity of both physical space and digital expression. https://awexr.com

AWE USA 2019: Owning Augmented Space

AugmentedWorldExpo

A talk from the XR Enablement Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Victor Sun (Shadow Creator): How to turn Mixed Reality “headset” into “glasses”? Microsoft has released HoloLens 2 on MWC. I think it is the benchmark for mixed reality devices, but it only reaches the pass mark. In order to be used by ordinary consumers, headsets must be turned into glasses to eliminate their fears of tech equipment. https://awexr.com

Victor Sun (Shadow Creator): How to turn Mixed Reality “headset” into “glasses”?

AugmentedWorldExpo

Chris Pickett (DigiLens): XR is Hard: Here’s Why

AugmentedWorldExpo

A talk from the XR Enablement Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Antti Sunnari (Dispelix Ltd): Full-color, single-waveguide near-eye displays for AR glasses and MR headsets For years, many have tried to develop a see-through, near-eye display technology that combines beautiful design, excellent image quality and scalable mass production with high yields. At Dispelix, we’ve figured it out. Dispelix is helping product companies to create beautiful AR glasses based on single-waveguide full-color displays with superior image quality and mass manufacturability. https://awexr.com

Antti Sunnari (Dispelix Ltd): Full-color, single-waveguide near-eye displays ...

AugmentedWorldExpo

A talk from the XR Enablement Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Jian Liang (HiScene): AR for Industry in China: From Concepts to Real Applications AI/AR industry has attracted attention never seen before of academia and industry, into which numerous talents and resources have been invested. However, academic achievements are not equal to products, which need to be adjusted and optimized in technology, engineering, product, etc. according to specific application scenarios. This talk will share with you some difficulties, misconceptions and experience in commercializing AR based on HiScene’s practice. https://awexr.com

Jian Liang (HiScene): AR for Industry in China: From Concepts to Real Applica...

AugmentedWorldExpo

A talk from the Gaming & Entertainment Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Augmented reality meets Television -Ellen DeGeneres' Game of Games AR App Daniel Leary | The Ellen DeGeneres Show Alexa Hook | The Ellen DeGeneres Show Paul Hake | HitPoint Inc. Ellen DeGeneres, America's most loved comedian and popular TV host, entered the augmented reality era with the "Game of Games" show. Over 900,000 app users get to tune in and play along with Ellen with AR by unlocking prizes that jump from the TV screen into viewers' phones. The app has hit 741,614 AR gift box markers scanned from NBC broadcasts of Game of Games, during the weekly Play Along Live game. At AWE, HitPoint Studios gives an insight into the creative and technical process of building the "Game of Games" Game app and how to connect traditional TV shows with AR technology. https://awexr.com

Augmented reality meets Television -Ellen DeGeneres' Game of Games AR App

AugmentedWorldExpo

Jennifer Cook (Dreamscape Immersive): Dreamscape Immersive Presentation

AugmentedWorldExpo

A talk from the Enterprise Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. 2 Partners sharing 1 vision for smart operators Mark Fleischer | AMA XPERTEYE INC. Peter Verstraeten | Proceedix The partnership between XpertEye and Proceedix offers a best-in-class solution to empower smart operators and technicians. The combination leverages both mobile technology and smart glasses for remote assistance and instruction and inspection execution. https://awexr.com

AWE USA 2019: 2 Partners sharing 1 vision for smart operators

AugmentedWorldExpo

Phil LaFond (Bosch Automotive Service Solutions Inc.): Bosch Technical Traini...

AugmentedWorldExpo

A talk from the Main Stage at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Scott Montgomerie (Scope AR): AR’s Influence on the Workforce of Tomorrow: Job Eliminator or Creator? As the speed of technology continues to accelerate automation in the manufacturing world, the inevitable question of whether or not the “human touch” will become obsolete is top of mind for workers. The fear that robots and smart technologies will take everyone's jobs is prevalent, but not necessarily true. AR has the power to be a job creator, not a job eliminator. Its ability to make anyone an instant expert can in fact increase job security by quickly helping workers become more proficient in tasks. Learn about real-world use cases where AR is making people better, and safer, at their jobs, and explore why enterprises who create a workplace that’s augmented, not automated, will be the leaders of tomorrow. https://awexr.com

Scott Montgomerie (Scope AR): AR’s Influence on the Workforce of Tomorrow: Jo...

AugmentedWorldExpo

A talk from the Main Stage at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Hugo Swart (Qualcomm, Inc.): How 5G, Distributed Processing and Technology Advancements are Accelerating the Timeline to a Ubiquitous XR World XR has the potential to become one of the world’s most disruptive computing platforms in the coming decade. Over the past year, we’ve seen significant strides in technology and with the introduction of new XR form factors such as standalone, tethered viewers and dual-mode HMDs – there has been an acceleration of XR opportunities, use cases and adoption. In this keynote presentation, Hugo Swart, Head of XR at Qualcomm Technologies, Inc. will speak to cutting-edge technological developments and how Qualcomm has met previous challenges, with a forward outlook on how strong ecosystem collaboration, the proliferation of 5G and AI will fast-track the industry timeline to an XR ubiquitous world. https://awexr.com

Hugo Swart (Qualcomm, Inc.): How 5G, Distributed Processing and Technology Ad...

AugmentedWorldExpo

A talk from the Creator Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Valentin Heun (PTC): One Reality - A 3rd generation AR User Interface Many of our technological innovations seem to expand the computational abilities without the human in the loop. Augmented Reality is one of the view technologies that promises technological advances that enable us humans a better connection with the ubiquitous technology around us. We are not there yet to make use of such a user interface. We currently have individual islands of interactivity via AR. This conference may have many spatial demos as well, however, a single ubiquitous layer that penetrates all screens, surfaces, machines, and environments are not available. This is what the PTC Reality Lab researches. The core of this research is the ability to use AR in a scalable distributed platform to bridge among technological domains to allow the weaving of multiple technologies into one user experience. The resulting user experience is called One Reality. The goal for One Reality is to provide one seamless, ubiquitous flow of user experience for all technologies that touch the user’s decision making. This talk will iterate on the different stages of Augmented Reality User Interfaces and identifies One Reality as the 3rd stage in the evolution of AR user interfaces. This visionary talk aims to inspire the audience to think beyond the known applications for Augmented Reality. https://awexr.com

Valentin Heun (PTC): One Reality - A 3rd generation AR User Interface

AugmentedWorldExpo

A talk from the Training & Education Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Anton Ebert (Siemens Healthcare): Challenges and approaches on integrating a cutting edge technology in modern healthcare businesses This session will provide insights and a basis for further discussions on how technologies like Augmented and Virtual Reality are being tested and explored in the healthcare sector. https://awexr.com

Anton Ebert (Siemens Healthcare): Challenges and approaches on integrating a ...

AugmentedWorldExpo

A talk from the Humanities Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Sarah Hill (Healium by StoryUP Studios): Healing Stories Powered by Biometrics Users aren't just watching your content, they're feeling it with their wearables. Brain patterns and heart rate are XR storytelling's new inputs. Peek inside the world's first VR & AR content channel powered by a brain-computer interface and heart rate to learn about a new study on XR's therapeutic impact on anxiety. https://awexr.com

Sarah Hill (Healium by StoryUP Studios): Healing Stories Powered by Biometrics

AugmentedWorldExpo

A talk from the Humanities Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Iva Leon (ARVR Women and Allies): Building community with social VR Social VR platforms like Facebook Spaces can help community builders and purpose-driven organizations innovate the way they engage their members and bring people together. Find out how ARVR Women uses social VR to elevate the visibility of their members and add value to the group. https://awexr.com

Iva Leon (ARVR Women and Allies): Building community with social VR

AugmentedWorldExpo

More from AugmentedWorldExpo (20)