A talk from the XR Enablement Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Bill Parker (Creative Microsystems): Augmented Applications for an AI Interface Creative Microsystem was among the first near-eye optics companies to successfully demonstrate a laser-to-waveguide hybrid display, of the kind most recently made known by HoloLens 2 (not affiliated). Bill Parker first saw fame when he invented the Plasma Lamp while at MIT. Today he wants to speak to AWE about an Artificial Intelligence interface for Augmented Reality applications. https://awexr.com

1. AR FOR AI INTERFACE
AWE 2019
Santa Clara, CA
May 29, 2019
“XR Enablement”
Bill Parker
Michael Strauss
When I had hair

2. 2
Highly challenging customer requirements force development of “beyond”
off-the-shelf AR systems
The use cases consist of extreme environments and highly demanding work
loads, such as with military, underwater, aviation and space applications.
Because of our unique approach, we developed the capabilities to develop
unique solutions
Our ultimate goal is to provide enhanced situational awareness and
augmented cognition through AI
Every customer is our partner in this hardware/software and AI interface
development process
Motivation – AR for AI Interface
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker Copyright © 2019 by Creative Micro

3. 3
Founded by MIT Media Lab graduates
20+ years proven integration of display
systems for commercial and military users
Design, Development and Production
under one roof
6 years building Customer Driven
“bespoke” AR Systems
Optical, Fabrication, Hardware,
Electronics, Software, and Human
Factor Teams
Flexibility to use any combination of
display types, processing platforms and
applications – ours or others
3 Color Single Substrate
Imageguide™
Custom built prototypes and
limited production
Imageguide™ systems
Custom AR/AI Interface – Design, Engineering and Production
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker Copyright © 2019 by Creative Micro

4. 4
Compact Low Light Imager
Low Profile/Low Latency, Multi-spectral,
Full Color Digital Night Vision
With Sensor Data Overlay
Imageguide™ DiV Goggle
Underwater AR
AR Display for Tactical Users
Low Profile AR Display and Human
Machine Interface for high stress
engagements
Wide-FOV Imageguide™ System
Low Profile 80° deg/eye (110° total) AR Display
Integrated with panoramic night vision goggles
AR Enabled
One Size Does Not Fit All – CMC AR/EV Project Examples
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker Copyright © 2019 by Creative Micro

5. 5
Real time sensors provide immediate intelligence in Context
Resources with ML capabilities can support and expand the user’s cognitive capabilities
AI systems can establish rapport with a user and learn their needs and preferences
The challenge is in interfacing efficiently with an “idiosyncratic” computational system
What does this mean?
Enabling the user to effectively communicate with a Contextual AI
Technology and processes designed to maximize the interaction regardless of the situation
Currently the approach is limited to definable use cases and using generic hardware
Augmented Artificial Intelligence Interface (AI2)
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker Copyright © 2019 by Creative Micro

6. 6
Custom AR Design Process
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker
Requirements
Analysis
Technology
Analysis
Customer’s
Vision / Goals
Quantitative and
Qualitative
Evaluation
Design
Refinement
Final
Delivery
Processing / Hardware
Design
Display
Design
Software / AI
Design
Human Factors
Design
Copyright © 2019 by Creative Micro

7. 7
Custom AR Designs
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker
Analysis
Design /
Integration
Testing Delivery
• Customer Vision
and Goals
• Requirements
• Technology
• Market
• Teammates
• Display / Optics
• Processor(s)
• Software
• AI Interface
• Human Factors
• Connectivity
• Sensors
• Size and Power
• Voice of the
Customer
• Subject Matter
Experts
• Consultants
• In Situ Testing
• Quantitative
Questionnaires
• System Delivery
• Support
• Next Generation
Requirements
Refine
Copyright © 2019 by Creative Micro

8. 8
Custom AR Design Use Case – Tactical Air Controller System
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker
Requirements
Analysis
Technology
Analysis
Customer’s
Vision / Goals
Evaluation
Design
Refinement
Final
Delivery
System
Design
Human
Factors
Copyright © 2019 by Creative Micro

9. 9
Firefighter SCBA mask with
Thermal Camera and data link (2014)
Thermal Camera with see-
through display (2012)
Image and data overlay for
simulation and training
Firefighter’s wireless mobile
headworn display for Naval Research
Lab Ship database and wireless
tracking to provide rectified images
in dense smoke (1993)
Thermal
Camera
AR System tracks firefighter’s location
Provides life support information
2-Way visual communications
Thermal imagery to see through smoke
Custom AR Design Use Case – EyeFire™ Wearable System for Firefighters
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker Copyright © 2019 by Creative Micro

10. 10
AR System tracks diver’s location
Provides life support information
Integrated dive computer display
Communications with other divers
Sonar imagery overlay
Copyright © 2019 by Creative Micro
Custom AR Design Use Case – DiV Wearable System for Divers
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker

11. 11
AI for Decision Cues - Memory and Cognition Support
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker
Real Time Situation
Information and
Decision Aides
“Reach Back” into
Historic Data
Edge, Local, and
Cloud Processing
Sensor Enhanced
Imagery
Positive ID
Amanda Smith
Last Spoke
March 21, 2018
Last Conversation
Trip to Alaska
Marketing Progress
Owes Report – Not Recv’d
Access
Text Messages
Emails
Voice Conversations
DETECT
Nervousness
Copyright © 2019 by Creative Micro

12. 12
AI for Decision Cues – Outdoor Inspection
Copyright © 2019 by Creative Micro
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker
WARNING
Linkage Failure
High Probability
Corrosion Detected
Hyperspectral Detection
Detected Similar Bridge
- March 3, 2015
Linkage Batch
A13-044X-1932
ACTION
Notified Admin
Report Filed
Real Time Sensor
Processing w/ AI
“Reach Back” into
Historic Data and
Trends
Cloud Data and
Processing
Sensor Enhanced
Imagery

13. 13
AI for Decision Cues – Future
PROCESS
IMPROVEMENT
Possible Improvement
Based on joint limit
Result
Less Pain
Higher Accuracy
New Process Available
ACKNOWLEDGE FOR
TRAINING
Real Time Sensor
Processing w/ AI
Cloud Data and
Processing
Use of
Unstructured /
Unrealized Data
Watches, Analyzes,
Provides
Improvements and
Training
Copyright © 2019 by Creative Micro
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker

14. 14
Future Connectivity Needs
Local (PAN)
5G enables Personal Area Network
Moves compute hardware off head (glasses miniaturization)
Local AI processes low latency needs (sensors, info, etc.)
On Site (RAN)
5G enables link to augmenting local area processing / data
Moves compute hardware off person (glasses miniaturization)
Enables high performance through dedicated hardware
“Local” datasets and compute resources
Cloud / Enterprise
Large dataset accessible AI and data
Critical, but restricted by high latency or low bandwidth
Copyright © 2019 by Creative Micro
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker
2.5 Quintillion Bytes of Data Produced per
Day!
- Domo, Inc. 2019

15. 15
Summary
AWE 2019 / Santa Clara
“XR Enablement” May 29
Bill Parker Copyright © 2019 by Creative Micro
There is a race to customize AI to integrate with existing AR systems
However, the argument can be made that off the shelf hardware has
limitations and needs as much customization as the AI
Without bespoke hardware a significant amount of implementations will not
provide maximum enhancement
Creative Micro develops proprietary displays, methods, and technology to
specifically support these and other emerging use cases
By converging AI with custom AR solutions, customers will be better able to
excel

16. QUESTIONS, COMMENTS OR FEEDBACK?
Bill Parker, CEO
bparker@creativemicro.com