Virtual reality (VR) and augmented reality (AR) are powerful tools for storytelling, but poor execution can negatively impact consumer reactions and engagement. This session guides you through the latest Unity tech and best practices for creating stunning high-end VR and mobile AR visuals. Speaker: Dan Miller – Unity Watch the session on YouTube: https://youtu.be/dvOZ7IL2iOI

2. GenerativeArt—MadewithUnity
XR Graphics in Unity
Maximizing your AR/VR experience’s potential
Dan Miller
Unity Technologies, XR Evangelist
@DanMillerDev

3. Agenda
3
Virtual Reality
- Stereo Rendering Considerations
- Custom Shader Modifications
- RenderScale and RenderViewportScale
- Scriptable Render pipelines for VR
Augmented Reality
- Occlusions
- Shadows
- Platforms Specific Rendering
- Demo

4. Rendering For Each Eye 👀
4
Stereo Rendering

5. Stereo Rendering
5
Single-pass instancingSingle-passMulti-pass
Two textures are packed into one
large texture (referred to as a
double-wide texture).
Only goes through the scene graph
once, so it’s much faster on the
CPU. However requires a lot of
extra GPU state changes to
accomplish this
Uses a single texture array
containing two textures storing
the rendering for each eye.
Involves the same amount of
GPU work, but fewer draw calls
and work for the CPU, so it’s
significantly more performant.
Requires going through the scene
graph multiple times, once for each
view you need

6. Custom Shader Changes 🔮
6
Vertex Shader Changes

7. struct appdata
{
float3 pos;
UNITY_VERTEX_INPUT_INSTANCE_ID; // uint instancedID : SV_InstanceID;
};
struct v2f
{
float4 vertex : SV_POSITION;
UNITY_VERTEX_OUTPUT_STEREO; // uint stereoEyeIndex : SV_RenderTargetIndex;
};
v2f vert (appdata v)
{
v2f o;
UNITY_SETUP_INSTANCE_ID(v); // unity_StereoEyeIndex = v.instanceID & 0x01;
UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o); // o.stereoEyeIndex = unity_stereoEyeIndex
o.vertex = UnityObjectToClipPos(v.vertex);
return o;
}
7
struct appdata
{
float3 pos;
UNITY_VERTEX_INPUT_INSTANCE_ID; // uint instancedID : SV_InstanceID;
};
struct v2f
{
float4 vertex : SV_POSITION;
UNITY_VERTEX_OUTPUT_STEREO; // uint stereoEyeIndex : SV_RenderTargetIndex;
};
v2f vert (appdata v)
{
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
return o;
}
struct appdata
{
float3 pos;
UNITY_VERTEX_INPUT_INSTANCE_ID; // uint instancedID : SV_InstanceID;
};
struct v2f
{
float4 vertex : SV_POSITION;
};
v2f vert (appdata v)
{
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
return o;
}
struct appdata
{
float3 pos;
};
struct v2f
{
float4 vertex : SV_POSITION;
};
v2f vert (appdata v)
{
v2f o;
o.vertex = UnityObjectToClipPos(v.vertex);
return o;
}
Vertex Shader Changes

8. 8
RenderScale vs RenderViewportScale

9. RenderScale vs RenderViewportScale
9
RenderViewportScaleRenderScale
Controls the actual size of eye textures as a
multiplier of the device's default resolution.
● value of 1.0 will use the default eye texture
resolution specified by the XR device.
● less than 1.0 will use lower resolution eye
textures
● greater than 1.0 will yield higher resolutions
Controls how much of the allocated eye
texture should be used for rendering.
Valid range is 0.0 to 1.0. This value can
be changed at runtime without
reallocating eye textures. Therefore it is
useful for dynamically adjusting eye
render resolution.
EyeTextureResolutionScale

10. Current Support and Features 😳 🛋🛋
10
HDRP for Virtual Reality

11. HDRP for VR
11
Supported in 2019.1+
Some HDRP features are not compatible
with VR. When you enable VR in your
Project, HDRP automatically disables the
features that it does not support
2019.1 — Single Pass Stereo
2019.2 — Single pass instanced

12. Not supported
● Multi-pass rendering
● Single-pass instancing
● Tile lighting
● Deferred rendering
● Volumetrics
● Render and viewport scale
● VFX Graph
HDRP for VR
12
In 2019.1 Supported
● Multi-pass rendering
● Single-pass instancing
● Tile lighting
● Deferred rendering
● Volumetrics
● Render and viewport scale
● VFX Graph
In 2019.2+
Supported In 2019.3+

13. Universal Render Pipeline for VR 😎
13
URP for Virtual Reality

14. Universal RP for VR
14
Supported in 2019.1+
● Better performance than legacy in most lighting setups
~25% increase on mobile hardware
● Has a new batcher that reduces setup cost for each
draw call
● Shades light in single pass decreasing the overdraw
and amount of drawcalls
● Only performs a depth prepass when using Cascade
shadows

15. Universal Render Pipeline
15
Current Limitations
● There are currently some issues with MSAA and PostProcessing in VR.
● We added single pass double wide as the fallback to SRP in 2019.1

16. Challenges and Solutions 🌅
16
Post-Processing Stack

17. Post processing in XR
17
Always consider what if any post-processing effect you need.
● Most temporal effects add blurriness and latency
● Simulating depth of field, a property of physical cameras, in a headset, which is simulating the
human eye, will most likely cause nausea.
● If you’re running into issues, try the different stereo rendering modes.
● For mobile Augmented Reality, effects will be processed on the entire camera feed, not just
digital objects.

18. Rendering techniques ✨ 👀
18
Augmented Reality

19. Things to keep in mind
● Digital and augmented objects are rendering on top of
the camera feed
● Occlusion and Shadows are not enabled by default and
require special shaders and consideration
● Your users will be using this in a variety of different
conditions and environments
● Universal Render Pipeline support is continuing to be
updated and improved
○ Only available in AR Foundation
Augmented Reality Rendering — Mobile
19

20. Things to keep in mind
● Additive displays — Blacks = Transparent
● Physically based rendering and self shadowing can create dark areas
● Consider leveraging unique platform features like eye tracking
● Universal Render Pipeline only supported in single pass and single pass instanced stereo rendering.
Augmented Reality Rendering — Head Mounted
20

21. Occlusion
21
http://bit.ly/MobileOcclusionShader
● Special shader
● Can be applied to planes or world mesh
● Used with feature points for quicker occlusion
● Built in support on specific platforms
Human Segmentation enabled in ARKit 3.0 with AR Foundation

22. Shadows
22
http://bit.ly/MobileARShadow
● Casting shadows on transparent geometry
● Consider size and scale - rescale size of rendered
content
● For static objects use blob shadows

23. Platform Specific Features — Mobile
23
Environmental Probes — ARKit
● Generate environment textures during an AR session
using camera imagery
● Plug that cubemap texturing into a reflection probe in
Unity to provide realistic image-based lighting for virtual
objects in AR
Environmental HDR — ARCore
● Directional light rotations to help with placing shadows in
the right direction
● Ambient Spherical Harmonics: helps model ambient
illumination from all directions
● HDR Cubemap: provides specular highlights and reflections

24. Tips and Tricks ✨
24
Demo

25. Demo Breakdown
25
Colored Materials Specular, Normal
Occlusion maps
Environmental
Probes
Light estimation Post Processing -
Color Grading

26. Want to learn more?
Project:
VR in Unity: A Beginner's Guide
Tutorial:
Configuring Unity for AR Development
Subscribe now for a 3-month free trial at unity.com/learn-
premium with code: UNITECPH19
*Learn Premium is included in Unity Plus and Pro licenses. Just sign in with your Unity ID.

27. Thank you.
Dan Miller — XR Evangelist
DanielMi@unity3d.com
@DanMillerDev