Spatial Audio Lessons Learned from Unscramble the Oracle BOSE AR
3 Lessons from our BOSE AR Audio-First game. While photorealistic graphics grab the lion share of attention, deeply immersive experiences must address all of the senses. This talk covers a post mortem of an audio-only Unscramble the Oracle (BOSE AR) game using fully spatialized interactive audio scapes that require no vision. Come learn about the challenges we faced creating this first of a kind interactive walkable audiobook. Learn how storytelling, navigation, character interaction change when the player can only can hear. Together we look at how we succeeded and stumbled in creating an audio world to explore with native head, body, and interactions How we leveraged characters, specialize sound, and audio UX. What audio attributes increased spatial positioning, what reduced it, and how create audio highlight states. Additionally, it offers lessons from designing to the other senses hearing and touch including several techniques to cross talk and fool the senses. Pick up any smart phone and gestures to operate it are nearly universal. Put on a set of smart glasses or play a mobile AR game and they are hobbled by out dated UX/UI standards that interfere with the new opportunities on the emerging AR platforms for spacial computing. In contrast to the UX/UI guidelines for WIMP interfaces; precious little has been written about best practices for AR. With screenless information, spatial computing, floating menus, mid air gestures, restyled 3DOF and 6DOF controllers, new graphics, audio, and haptics AR offers novel interaction patterns that demand new interface language and standards. Come join us for a dive into the emerging UX / UI standards for AR and spatial computing with inspiring examples from Leap Motion, North, mixed reality, Magic Leap, as well as some from XEO's own laboratory and latest experiments. Takeaway Attendees will walk away with concrete examples and techniques for spatial audio design for AR as well as traditional games. Position is the power of immersive audio. Attendees will learn how precise location separates one audio source from another and the motion of these audio sources unlocks new story telling possibilities. Additionally how to create flexible layouts for the Real World in AR. The lessons in immersive audio also offers lessons for highly dynamic flat screen game worlds that are more than a backdrop for gameplay. Additionally, attendees will learn how to enhance button and object interaction in an audio only world. Intended Audience Audio engineers, game designers, and developers of all game platforms will walk away with insights in how leverage psychological properties of sound and how to use use spacialized audio to tell stories, crate interactive audio objects and menus, and interact with a game world that they can hear but not see. These lessons are also suitable for developers of flat screen games wishing to create more immersive experiences
Recommended
Recommended
More Related Content
More from Nicole Lazzaro
More from Nicole Lazzaro (20)
Recently uploaded
Recently uploaded (20)
Spatial Audio Lessons Learned from Unscramble the Oracle BOSE AR
- 1. Nicole Lazzaro Hear Me Feel Me Let Me Play: Lessons from Unscramble the Oracle a BOSE AR Audio-First AR Game GDC Summer SF/Online August 4, 2020 @NicoleLazzaro
- 5. ThankYou!UnscrambletheOracleTEAM! • DavidFox writing&Sfx ElectricEggplant.com • PatrickO’ShaughnesseyNarr.Engine PatchedReality.com • WonkubusWonkWalk,VO.&Art (MichaelCrabbs) MichaelCrabbs.com • Nicole Lazzaro GameDesign& UX XEODesign.com
- 6. AudioFirstChooseYourOwnAdventureYouPlayWalkingAroundtheBlock How We Designed an Audio First GameImage: Wonkubus
- 7. LessonsfromBeatles,SaturdayNightFever,AndStonehenge How We Designed an Audio First GameImage: Wonkubus
- 11. PushLimitsLearnAudioSpatializationForAllgames • AudioFirst • 1stperson • “PhoneinPocket” • FullyPlayable • Walk&LookAround • ExplorableAudioWorld • InteractiveAudioGameplay • Onlyplayableinspatialaudio • CoreLoopRequiredAudioSpatialization AudioGoals Game TechnologyDidWell 1. Headpose 2. Tapheadsettoselect 3. Positionaudio(ResonancePlugin) Challenge ✓Responsive ✓Accurate X 6DoF(only3DoF)
- 13. Stand 1
- 14. MaximizeAudioPositionDataAroundPlayer • ImmersiveAliveAudioWorld • Sound OutsideHead • NoHeadlock • PositionalClarity AudioGoals Stand
- 16. BeatlesAbbeyRoad1st toRecordinStereoandSynthesizerMore: https://www.pbs.org/newshour/arts/the-beatles-abbey-road-revolutionized-how-a-track-could-be-recorded-listen-closely-here Here Comes the Sun ©1969 EMI
- 25. HowBigisanAuxel?
- 30. SpatializationTips • RespondtoHeadMovementEsp.Rotation • TIMETODISCOVER • Length orLongEnoughLoops • Identifiable&Separation • MonoSourcesinGameWorldNoReflections • SharpAttack • PitchAboveandBelow AudioGoals Stand
- 31. Walk 2
- 34. MaximizeSenseofForwardMotionThroughScene • AudioOnlyStorytelling • AudioDepthPerception (howFarOutthere) • WalkAnywhereandStaywithinBounds • RespondtoPlayer’sWalkSpeed AudioGOALS Walk 3
- 41. SpatialAudioWalk–BelievableRealWorldOverlay 3.5DoF AsDesigned AsUsed KeepPlayersIn AudioWorld SeatBounce WalkAround theBlock Walk Wander
- 43. SpatializationTips • NarrativeAudioBeats • GroupAudioSourcesSidetoSide&3Depths • AuxelSpaceforSlow&FastMotion • ShowProgressPassByHeadw/in1Meter • FootstepsareRespectiveand SamePosition • ActionsLikePullSlicePizzaOffDoor AudioGOALS Walk 3
- 46. Mysterious
- 47. MaximizeChoiceFromAudioMenusinWorld • 4StateButtons • ChooseBetweenOptions • OptionsIntegratedintoWorld AudioGOALS Choose
- 57. SpatializationTips • 4-5 AuxelsHorizontallySpaced • BewareDeadZoneFront&Behind • Latency:Bluetooth,WiFi,HRTF • Frogs>Turnips • Character&StoryBeats=EasiertoRemember • BeginAudioMemorably • ReinforcewithIcons AudioGOALS Choose
- 60. MaximizeRotationMotionHandheld • RespondtoAuxels withPhone • MakeHandMotionAsNaturalAsHeadMotion • HapticstoenhanceAudioMenuUX AudioGOALS HoldIn Hand
- 63. SpatializationTips • AudioTracksPlayer’sPhoneHand • MinimizelookingleftHeadand MovingPhoneRight • ButtonStateswithHapticSignatures • HapticSurfaceandmediumMovePhone ThroughLikeWater,ForestBranches,Sand AudioGOALS HoldIn Hand
- 64. WrapUp 5
- 65. 3+ ChallengesforAudio-FirstGames Stand Walk Choose Hold&Haptics Spatial Audio>
- 68. AudioGOALS Stand Walk Choose HOLD SpatializationTips 1. RespondtoheadPoseandMovement 2. SpaceatLeast1Auxelapart 3. ContrastAudioPatternToSpatializeWell 4. NarrativeCuesExpandShortTermMemory
- 69. ThankYou!UnscrambletheOracleTEAM! • DavidFox writing&Sfx ElectricEggplant.com • PatrickO’ShaughnesseyNarr.Engine PatchedReality.com • WonkubusWonkWalk,VO.&Art (MichaelCrabbs) MichaelCrabbs.com • Nicole Lazzaro GameDesign& UX XEODesign.com
- 71. ContactMeForMore • Nicole@xeodesign.com • @NicoleLazzaro • XEODesign.com FreeDownload 4K2F.com Followthe WhiteRabbit TiltWorld.com Unscramble theOracle