Unreal Summit 2016 Seoul, Lighting the Planetary World of Project A1

1. UNREAL SUMMIT 2016
Lighting the Planetary World of
Project A1
Hyunwoo Ki
Lead Graphics Programmer

2. UNREAL SUMMIT 2016
A1
• New IP of Nexon
– High-End PC / AAA-quality visuals
– MOBA / Space Opera
– UE4 + @@@
– In Development
• Announced our development last month

3. UNREAL SUMMIT 2016
A1
• Talk about character rendering at last NDC 2016 talk
• This talk presents techniques for lighting the world of A1
– Used a test scene
– Not the game world

4. UNREAL SUMMIT 2016
World of A1
• Spherical planet
• Real-time day and night cycle
• Partially environment destruction
– Trees, buildings, etc.
• Partially terrain modification
– Craters, explosion, etc.

5. UNREAL SUMMIT 2016
Challenges
• Spherical coordinates
• Longitudinal time variation
• Time of day lighting changes
• Dynamic
– Moving Sun
– Destruction
– Modification

6. UNREAL SUMMIT 2016
Our Approach
• “Fully Dynamic” if possible
– Shadow maps, SSAO, SSR, SSIS Screen Space Inner Shadows, etc.
• Partially Precomputation and Relighting
– Global illumination, sky lighting and reflection environment

7. UNREAL SUMMIT 2016
Changes for Planet: 1
• Vector towards the sky
– Vary according to latitude and longitude
– Standard world: just (0, 0, 1)
– Planetary world: normalize(WorldPosition)
• Assuming (0, 0, 0) is the center of the world
• We call this ‘Planet Normal’

8. UNREAL SUMMIT 2016
Changes for Planet: 2
• Longitudinal time variation
– Like GMT
float ComputeGMTFromWorldPosition(float3 WorldPosition)
{
float Longitude = atan2(WorldPosition.y, WorldPosition.x) / PI * 0.5f + 0.5f; // [0, 1]
float NormalizedGMT = frac(Frame.NormalizedDayTime – Longitude); // East to West
return abs(NormalizedGMT);
}
* Note: Frame.NormalizedDayTime = GMT+0 = [0. 1)

9. UNREAL SUMMIT 2016
Agenda
• Directional Light
• Global Illumination
• Sky Light
• Reflection Environment

10. UNREAL SUMMIT 2016
Directional Light

11. UNREAL SUMMIT 2016
Directional Light
• Symmetry of two directional lights
– Sun and Moon
• Movable mobility
– For dynamic scenes
– Real-time lighting
• Deferred rendering: opaque materials
• Forward+ rendering: transparent materials

12. UNREAL SUMMIT 2016
Two of Directional Lights
• Sun:
– Dominant light
• Moon:
– Night area
– Adding direct specular

13. UNREAL SUMMIT 2016
Two of Directional Lights
• Problem:
– Directional lights commonly affect all surfaces on the world
– Incorrect results
• Ex) Moonlight leaks in the daytime
• Slow (2X)
• Solution:
– Cull backside of the planet from the light
– Smoothly attenuate radiance at boundaries

14. UNREAL SUMMIT 2016
Time of Day Lighting
• Different time for each pixel
• Overriding light color by using a hand-painted texture in the shader
• Different methods for GI, sky light and reflection environment
– See further slides

15. UNREAL SUMMIT 2016
Sun Shadows
• Shadows have an important role to recognize time during game play
• Presented at my NDC 2016 talk
– Use UE4 implementation
• CSM + PCF
– Add improved PCSS
• To control shadow softness by time: only for 0 and 1 cascade splits
• Shadow normal offset: to remove Peter Panning
• Temporal reprojection: to reduce flickering due to slow moving

16. UNREAL SUMMIT 2016
Tighter Shadow Bounds
• For both quality and speed
• Setting tighter bounds
– Assuming very far objects on the view do not cast shadows
– Backside of the planet culling + planetary view frustum culling
– More than 1.5X faster rendering

17. UNREAL SUMMIT 2016
Planetary View Frustum Culling
• Limit the far plane as distance
between the camera and the center of the planet
– Assume that we can’t see backside of the planet
• Closer the camera, shorter the far plane
– A proportional expression between
the center of the planet and view frustum planes

18. UNREAL SUMMIT 2016
Before
Backside culling
Backside culling
+ View frustum culling

19. UNREAL SUMMIT 2016
Global Illumination

20. UNREAL SUMMIT 2016
Existing Solutions in UE4
• Lightmaps (X)
– Static, and high memory consumption
• LPV (X)
– Slow, and low quality
• DFGI (X)
– Slow, and not supporting skeletal meshes
• Indirect lighting cache
– Be possible!

21. UNREAL SUMMIT 2016
UE4 Indirect Lighting Cache
• SH irradiance volume
• Per-primitive caching
– 5x5 volume -> upload to the global volume texture atlas
– For movable components or preview / Update when the component is moved
• “Try to use volume ILC for all types of components in the scene”
– Including static components and terrain

22. UNREAL SUMMIT 2016
Lacks of Volume ILC
• Lighting discontinuity (a.k.a. seams)
• Low density at a large geometry
– Need size-dependent cache distribution
• High memory consumption and slow cache update
• High CPU costs
– Per-primitive computation on the render thread
– Need update cache if a primitive is moved or lighting is changed
• Unsuitable for our game 

23. UNREAL SUMMIT 2016
Need of a New Method
• Keep using SH irradiance volume
• More efficient data structure
• Seamless
• Faster update (or no cache update)
• Time of day lighting changes

24. UNREAL SUMMIT 2016
Related Work
• Far Cry series
• Assassin Creed series
• Quantum Break
• TC: The Division
• …

25. UNREAL SUMMIT 2016
Deferred Cubic Irradiance Caching
• ‘Deferred’:
– As post processing: avoiding overdraw
– Faster development iteration: quick recompile shaders
– But use forward rendering for transparency
• ‘Cubic’:
– Exploiting cubemaps: fit to GPUs
– Cache placement on the world: seamless
– Faster addressing: using planet normal = normalize(WorldPosition)

26. UNREAL SUMMIT 2016

27. UNREAL SUMMIT 2016

28. UNREAL SUMMIT 2016
Time of Day Lighting
• All day light
– Affect all day
• Time of day light
– Limited by time span
– 12 time spans: 0, 2, 4, …, 20, and 22 hour
• Twelve SH irradiance volumes
– Interpolate lighting from nearest 2 volumes
– No cache update

29. UNREAL SUMMIT 2016
Overview
• Offline
– Cache placement
– Photon emission
– Irradiance estimation
• Run-time
– Cubemap caching
– SH lighting

30. UNREAL SUMMIT 2016
Offline: Cache Placement
• Based on texels of the cubemap
– N x N x 6
– 50 cm space
– Finding Z by using ray casting

31. UNREAL SUMMIT 2016
Offline: Cache Placement
• 2.5D cache placement
– Above the surfaces
– No multi-layers or indoor in our game
– Incorrect results for flying characters
• Multiple layered cubemaps?

32. UNREAL SUMMIT 2016
Offline: Photon Emission
• UE4 Lightmass: a photon mapping based light builder
• Store ‘time of day light index’ for deposited photons
• No changes for remainders
class FIrradiancePhotonData
{
FVector4 PositionAndDirectContribution;
FVector4 SurfaceNormalAndIrradiance;
int32 TimeOfDayLightIndex;
};

33. UNREAL SUMMIT 2016
Offline: Irradiance Estimation
• Twelve SH irradiance volumes
– Sharing world position, bent normal and sky occlusion
– Difference irradiance by time spans
• Per-time span irradiance
– Indirect photon final gathering
• All day lights: always
• Time of day lights: filtering by its index
• Global sky occlusion and bent normal

34. UNREAL SUMMIT 2016

35. UNREAL SUMMIT 2016
Run-time: Cubemap Caching
• Once at loading time
• CPU irradiance cache -> GPU cubemaps
– half4 texture cube array
• 2nd Order SH: encoded on 3 textures (RGB)
• 12 time spans * 3 = 36 elements
• Misc.:
– Bent normal and sky occlusion: 1
– Average color and directional shadowing: 12 – for reflection environment relighting
• Average color is computed by integrating incident radiance from all directions

36. UNREAL SUMMIT 2016
Run-time: SH Lighting
• Every frame
• Texture addressing
– Coordinates: planet normal
– Array index: time of day light index + 12 * {0|1|2}
• SH2 diffuse lighting
– Interpolate radiance from nearest 2 time spans
• Total 6 times fetches of a texture cube array

37. UNREAL SUMMIT 2016
Run-time: SH Lighting

38. UNREAL SUMMIT 2016
Performance
• Light building
– Scene-dependent
– Costly 12 times final gathering but faster than lightmaps
• Rendering
– Scene-independent
• approximately 0.4 ms: GTX970 1080p / ignoring transparency
• Stable visuals
– No seams or flickering
– Consistent looks for characters and environment

39. UNREAL SUMMIT 2016
Sky Light

40. UNREAL SUMMIT 2016
Transform to Spherical World
• Sky vector
– Not (0, 0, 1)
– Planet normal = normalize(WorldPosition)
• Rotate planet normal to (0, 0, 1) basis
– Heavy ALU
float3 TransformVectorToLandsphereSpace(float3 InVector, float3 WorldPosition)
{
float3 PlanetNormal = normalize(WorldPosition);
float3 SkyUp = float3(0, 0, 1);
float3 RotationAxis = normalize(cross(PlanetNormal, SkyUp));
float RotationAngle = acos(dot(SkyUp, PlanetNormal));
float3x3 Rotator = RotationMatrixAxisAngle(RotationAxis, RotationAngle);
return mul(Rotator, InVector);
}

41. UNREAL SUMMIT 2016
Time of Day Lighting
• Multiple sky cubemaps generated by artists
• Interpolated lighting like GI

42. UNREAL SUMMIT 2016
Bent Normal and Sky Occlusion
• As large scale ambient occlusion
– RGB: bent normal
– A: sky occlusion
– One R8G8B8A8 cubemap (no time variation)
• Diffuse lighting
– Widen and soft: sqrt
• Specular lighting
– Narrow and sharp: Square
Sky Occlusion + Bent Normal

43. UNREAL SUMMIT 2016
Screen Space Inner Shadows
• For GI and sky lighting
• Better looks when a character is on shadowed surfaces
• SSR styled ray marching
– Tracing on scene depth
– Directionality rather than SSAO
– No precomputation or asset building
– See my NDC 2016 presentation

44. UNREAL SUMMIT 2016
Reflection Environment

45. UNREAL SUMMIT 2016
Capture Probe Placement
• Uniform distribution on the sphere as the base
– Using golden spiral
• Additional placement by artists

46. UNREAL SUMMIT 2016
Time of Day Lighting
• Use relighting instead of pre-capturing all day
– Capture the scene without lighting
– Relight probes with tweak
• Relighting
– Geometric properties
• Relighting position = world position + (reflection vector * capture radius * specular occlusion)
• Relighting normal = normalize(relighting position)
– Direct lighting: Sun and SH2 diffuse sky lighting
– Indirect lighting: deferred cubic irradiance caching
• RGB: irradiance = average color of GI
• A: directional shadowing = occlusion of light sources

47. UNREAL SUMMIT 2016

48. UNREAL SUMMIT 2016
Volumetric Lighting
• Average color of GI (irradiance) is
also used for volumetric lighting
• For each ray marching step (for high spec.)
or at the surface (for low spec.)

49. UNREAL SUMMIT 2016
Summary
• Spherical world of A1
– Partially dynamic
– Time of day lighting changes
• Different approaches
– Deferred cubic irradiance caching
– Twelve time spans
– Relighting

50. UNREAL SUMMIT 2016
Future Work
• Improved GI
– 2nd Order SH -> 3th Order SH
– Layered cubemaps
– Volumetric fog

51. UNREAL SUMMIT 2016
Future Work
• Environment destruction and modification
– Multiple versions of irradiance volumes
• Pre-build for destroyed scenes
• Run-time update of cubemaps
• Like Quantum Break did
– Real-time GI
• SSGI?
– Recapture reflection environment

52. UNREAL SUMMIT 2016
Thank you
WE ARE HIRING!