The rendering technology of 'lords of the fallen' philip hammer

The Rendering Technology
of 'Lords of the Fallen'
Philip Hammer (Deck13 Interactive)
@philiphammer0
phammer@deck13.com
GameConnection Europe 2014
October 29-31, Porte de Versailles, Paris

Agenda
● Introduction
● Current-Generation Deferred Rendering
● Deferred Transparent Lighting
● Transparent Shadowcasting
● Volumetric Lighting
● Q&A
The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014

Intro: Who is Deck13?
● Deck13 Interactive is one of the leading game developers in Germany
○ ~50 people in Frankfurt office + 6 people in Hamburg office
○ In-house multiplatform technology FLEDGE
(PS4, Xbox One, PC, PS3, Xbox 360, iOS)
○ Previously: Venetica, Jack Keane, Blood Knights, Tiger & Chicken, etc.

Intro: Who am I?
● Senior Engine/Graphics Programmer
○ Working since >8 Years @ Deck13 Interactive
○ Part of an amazing team of 9 full-time programmers
● Responsible for the overall rendering system, lighting, materials, etc.

Intro: What is ‘Lords of the Fallen’?
● Deck13’s largest title so far
● 3rd-person challenging Action-RPG
● Dark, demonic fantasy setting
● Platforms: PS4, Xbox One, PC (DX11)
● Release this week
○ First PS4 release from a german developer
○ Some reviews are already out

Current Generation Deferred Rendering

● We’re not doing super-duper funky stuff..
○ .. just combining some well-known techniques in the right way
● We’re not covering the basics of deferred rendering..
○ .. but our specific differences to the textbook
● Game-production started as a PS3/X360 title
○ Switched early in development after new platforms got introduced
○ All last-gen assets replaced
○ We had to find “next gen” for ourselves and adapt the tech

● Major advancements in FLEDGE
○ Switched from DX9 to DX11 only
○ Switched from Light-Prepass to more traditional Deferred Rendering
○ Support for deferred transparent object lighting (incl. particles)
○ Support for Volumetric Lighting
○ Support of parallax-corrected cubemap reflections [Lagarde12]
○ Massively multithreaded rendering for consoles
○ Massive utilization of hardware-instanced geometry
○ Global Illumination (using Geomerics Enlighten)
○ Advanced physics simulations for Clothing, GPU-Particles, etc.
(using NVIDIA PhysX)
○ Support a lot more objects, lights, shadows, decals, etc.
○ etc.

● Switch from Light-Prepass to Deferred Rendering
○ Second geometry-pass was too costly
○ Way more bandwidth and memory available in this HW generation
(+ more flexible ESRAM on XB1)
○ Challenge: Move a lot of rendering effects to deferred stages
(e.g. Fog, Reflections, etc.)

G-Buffer Layout
● G-Buffer should be as small as possible
○ Parameter encoding
■ 2-channel normals
■ 2-channel albedo + specular-color
(chroma subsampling, YCoCg, inspired by [Mavridis12])
○ Parameter packing
■ Pack single bits and use lower precision for some attributes
○ Share channels for mutually exclusive parameters
■ Material-ID/-Index for referencing per-material parameters
○ 128 bpp (31,64 MB @ 1080p) even fits fully into XB1 ESRAM

G-Buffer Layout
RED GREEN BLUE ALPHA
MRT0
RGBA8_UNORM
Encoded Normal Material-
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Irradiance Lightmap -
Depth/Stencil
D24S8
Depth Stencil

G-Buffer Layout
MRT0
RGBA8_UNORM
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Depth/Stencil
D24S8
Depth Stencil
● Colored Specular and Translucency are mutually exclusive!
○ Determined by Material-ID

G-Buffer Layout
MRT0
RGBA8_UNORM
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Depth/Stencil
D24S8
Depth Stencil
● 1 Bit Material-ID determines whether MRT1.a contains
translucency or specular chroma.
● 7 Bit Material-Index used to reference
Material Parameter Buffers (more on this later)

G-Buffer Layout
MRT0
RGBA8_UNORM
ID / Index
Specular
Exponent
MRT1
RGBA8_UNORM
Albedo
Luma
Albedo
Chroma
Spec.
Luma
Spec. Chroma /
Translucency
L-Buffer (Diffuse)
R11G11B10_F
Depth/Stencil
D24S8
Depth Stencil
● Stencil used for masking
○ 3 Bits for Decals, Reflections, Skin/SSSSS
○ 2 Bits double-sided Light-Volume Stenciling

G-Buffer Layout
● We needed more attributes for special effects
○ Tangents for smooth alpha-test
○ Per-object rimlights, emissive/glow & other FX
○ Motion-Vectors of moving/dynamic objects for motionblur
○ Distortion-Vectors for Post-Distort
● Not enough space in G-Buffer, but only needed for ~10% of objects
○ Render them in a separate pass and blend (glow) or
use immediately in subsequent pass (motion-/distortion-vectors)

Material Parameter Buffer
● G-Buffer stores surface-parameters ..
○ .. but some surface-parameters are not needed per-pixel
○ Store certain surface parameters indirectly to avoid
wasting precious G-Buffer space
● Introduce a “Material Parameter Buffer”
○ Generate a LUT which contains material parameters once per frame
○ Store the LUT index in the G-Buffer to gain access to the parameters later

Material Parameter Buffer
● “Sparse” parameters would unnecessarily bloat G-Buffer
○ We don’t want to afford more G-Buffer space
○ We have quite a lot artist-tweakable parameters
● Material-based Parameter examples
○ Reflection-Strength
○ Fresnel
○ Subsurface-scattering parameters
○ etc.

Material Parameter Buffer (Example)
● Example: Implementation of
“Approximating Translucency for a Fast, Cheap and
Convincing Subsurface Scattering Look” [Brisebois12]
in a deferred scenario without compromising the parameters
● Problem:
A lot of properties varying per material, but effect is applied during in
deferred light-pass
● Solution:
Store scalar translucency values per-pixel in G-Buffer and all other
properties in Material Parameter Buffer

Material-Index sssDistortion sssPower sssMaterialColor …
0 4.5 2.4 0.43, 0.39, 0.84 …
1 8.0 3.5 0.22, 0.51, 0.52 …
2 5.6 4.1 0.41, 0.43, 0.11 …
... ... ... ... ...
Parameters per Light
Parameters per Material
Parameters per Pixel
// backscattering, according to [Brisebois12]
float3 sssLightVector = L + N * sssDistortion ;
float sssDot = exp2 ( saturate (
dot ( V, -sssLightVector ) ) *
sssPower - sssPower ) * sssScale ;
float3 sssColor = ( sssDot *
sssMaterialColor * sssLightColor ) * translucency ;

Material-Index sssDistortion sssPower sssMaterialColor …
0 4.5 2.4 0.43, 0.39, 0.84 …
1 8.0 3.5 0.22, 0.51, 0.52 …
2 5.6 4.1 0.41, 0.43, 0.11 …
... ... ... ... ...
Find implementation details in
Appendix 1 or ask me in Q&A.

Deferred Transparent Lighting

Deferred Transparent Lighting
● Most engines use Forward Rendering for transparents in Deferred
Rendering scenarios
○ Typical drawbacks: light-count, shadowing, etc.
● We wanted a deferred solution
● And we ended up with two different kinds of it
○ Deferred Per-Pixel
■ “expensive” - used for selected objects
○ Deferred Per-Vertex
■ “cheap” - used mostly for particles

Deferred Per-Pixel Transparent Lighting
● Gives you one additional layer of lighting
● Straight-forward implementation
(with similarities to Light-Prepass Rendering)
○ 1. Render a second G-Buffer
■ Only with lit transparent objects
■ Advise the artists to flag only a few transparent objects as lit
○ 2. Render a second L-Buffer
■ Render all light volumes also on the second layer
○ 3. Blend the transparent objects onto final scene while
sampling the light of the second L-Buffer

Deferred Per-Pixel Transparent Lighting
● Caveats
○ You *can* easily destroy your performance ,,
■ .. but it won’t if used wisely
○ Memory consumption is higher, but not a big deal on current-gen console ..
■ .. and you can reuse rendertargets in different rendering stages
● Optimizations
○ Mask rendered pixels in stencil for culling
○ Rough Scene-AABB culling
■ Build AABB of all lit transparent objects once per frame
■ Check against light-volume AABBs
■ Omit a lot of lights cheaply

Deferred Particle Lighting
● “We want lit particles!”
○ Particles fitting better to the scene
○ Particles can be re-used under different lighting conditions

● Common type of particles: atmospheric, smoke, dust & fog effects
○ Usually transparent, dense ... and cause a lot of overdraw
○ Costs can easily explode when lighting every fragment
○ Non-directional, low-frequency information
○ Perfect for vertex-lighting
● Deferred vertex-lighting
○ No compromise in number of affecting lights and shadows
○ Fits perfectly into deferred pipeline

● Again three steps to success
○ Step 1: Render Vertex G-Buffer
■ Write particle vertex positions sequentially into a texture
○ Step 2: Render Vertex L-Buffer
■ Accumulate light volumes
○ Step 3: Particle Rendering
■ Sample L-Buffer and apply lighting
● Find implementation details in Appendix 2 and ask in Q&A.

● Great for many types of particles
○ Cheap and effective
○ Apply all light types and features like shadows, gobos, IES, etc.
● Inherits problems of all per-vertex techniques
○ Low spatial resolution
○ Works best with small particles
○ Very small and bright lights can lead to flickering if
isolated corners of particles getting suddenly lit
● Be wary of buffer overflow with many lit particles

Transparent Shadows

Transparent Shadows
● All lights can cast colored, dynamic, transparent shadows
● Simple and effective approach

Transparent Shadows
● All lights can cast colored, dynamic, transparent shadows
● Simple and effective approach
○ Render solid shadows as usual into shadowmap
○ Render/Blend transparent objects into shadowmap-sized RGBA8 buffer
○ Use shadowmap as depth-buffer
○ Sample transparent shadowbuffer with shadow UVs and multiply with light

Transparent Shadows
● Works also with volumetric lights, transparent particles, etc.
● Individual features really shine when everything works together

Volumetric Lighting

Volumetric Lighting
● One of the major effect for “Next Gen”-Look
● Raymarching-based Approach
○ Inspired by “Real-time Volumetric Lighting in participating media” [Toth09]
○ Raymarching in light’s view space while evaluating the shadowmap
○ Interleaved Sampling and Temporal Reprojection for Performance

Volumetric Lighting
● Multiple light-types and -features
○ Point, Spot, Box
○ Also evaluate shadowmap, projector-texture(s), transparent shadows, IES light-profiles,
animated noise volume-texture, etc.

Volumetric Lighting
● Optimizations
○ Interleaved Sampling
○ Low-resolution rendering with depth-aware upsampling
○ Temporal Reprojection

Volumetric Lighting
● Performance
Pass PC / DX11 (GTX
700 GPU)
PS4 / GNM
Accumulation 0.362 ms 0.161 ms
Gather 0.223 ms 0.375 ms
Upscale 0.127 ms 0.321 ms
= 0.712 = 0.857 ms

Volumetric Lighting
● Attention: Artists love it and therefore tend to overuse it
● See for the gritty details:
“Volumetric Lighting for Many Lights in Lords of the Fallen” by Benjamin
Glatzel

Thanks for listening!
@philiphammer0
phammer@deck13.com

@philiphammer0
phammer@deck13.com
Questions?

.. and thanks to the Deck13 development team, making all this possible.
.. and very special thanks to the Tech-Team which contributed a lot to this talk (alphabetical)
● Arturo Cepeda
● Holger Durach
● Michele Giacalone (@miccalone)
● Benjamin Glatzel (@begla)
● Robert Hallinger (@robih29)
● Philip Hammer (@philiphammer0)
● Thorsten Lange
● Dominik Lazarek (@Omme)
● David Reinig (@D13_Dreinig)

References
[Glatzel14] “Volumetric Lighting for Many Lights in Lords of the Fallen”, Digital Dragons 2014,
http://de.slideshare.net/BenjaminGlatzel/volumetric-lighting-for-many-lights-in-lords-of-the-fallen
[Mavridis12] "The Compact YCoCg Frame Buffer", Journal of Computer Graphics Techniques, Vol. 1, No. 1, 2012
[Lagarde12] “Local Image-based Lighting With Parallax-corrected Cubemap”, Siggraph, 2012
[Brisebois12] “Approximating Translucency for a Fast, Cheap and Convincing Subsurface Scattering Look”, GDC 2011
http://dice.se/wp-content/uploads/Colin_BarreBrisebois_Programming_ApproximatingTranslucency.pdf
[Persson07] “Deep Deferred Shading”, Blog Post 2007
http://www.humus.name/index.php?page=3D&ID=75
[Drobot12] "Lighting of Killzone: Shadow Fall"
http://de.slideshare.net/guerrillagames/lighting-of-killzone-shadow-fall
[Toth09] “Real-time Volumetric Lighting in Participating Media”, Eurographics 2009
http://sirkan.iit.bme.hu/~szirmay/lightshaft.pdf

Appendix 1 - Material Parameter Buffer
● Step 1/3: Buffer generation
○ Generate a buffer / LUT (array of structs) each frame on the CPU
■ Determine for each drawcall a set of material parameters
■ Smart value quantization and interleaving
○ LUT contains selected data of the currently rendered materials
■ DX11/GNM: structured buffer
■ DX9: multiple 1D textures

● Step 2/3: G-Buffer stage
○ Write the LUT-index to the G-Buffer
■ Pass the index as shader constant
(index is associated with the material during (1) )
○ can index up to 128 different materials per frame
● 7 bit, because 1 bit reserved for Material-ID

● Step 3/3: Lighting/Combine stage(s)
○ Bind the Material Parameter Buffer(s) to the shader
○ Sample/unpack the Material-Index from G-Buffer
○ Grab and use the parameter values when needed like
value = buffer[index].x

Pixel-Shader
// fragment shader for indexing deferred material-parameter Buffer
Buffer <float4 > parameterBuffer ;
uint materialIndex = unpackMaterialIndex(gbuffer0Value . a);
float4 matParamBuffer0 = parameterBuffer[materialIndex];
float4 matParamBuffer1 = parameterBuffer[materialIndex + 1];
float sssDistortion = matParamBuffer0 . x ;
// backscattering according to [Brisebois12]
float3 sssLightVector = L + N * sssDistortion ;
float sssDot = exp2 ( saturate ( dot ( V, -sssLightVector ) ) * sssPower - sssPower ) * sssScale ;
float3 sssColor = ( sssDot * sssMaterialColor * sssLightColor ) * backscattering_mask ;

Appendix 2 - Deferred Particle Lighting
● Step 1/3: Particle Vertex G-Buffer
○ Render a non-screenspace particle “G-Buffer”
■ Basically a sequential list
○ Render particle vertex-buffer as point-list
(D3D11_PRIMITIVE_TOPOLOGY_POINTLIST)
○ Rasterize viewspace-positions of particles subsequently into a buffer
■ Currently a 1024x512 RGBA16F texture for all visible lit particles
○ G-Buffer currently contains only vertex-positions
■ Could be extended with more surface attributes like
translucency or normals

● Step 1/3: Particle Vertex G-Buffer
// vertex-shader
// pos = SV_Position
float4 pos = 1.0 ;
// vertexId = SV_VertexID, instanceId = SV_InstanceID
// param_vertexcount = shader constant holding the currently rendered number of vertices
float vertexIndex = vertexId + ( instanceId * vertexCountPerInstance) + param_vertexcount ;
float rasterPosY = trunc ( vertexIndex / textureWidth ) ;
float rasterPosX = trunc ( vertexIndex - rasterPosY * textureWidth ) ;
pos . x = ( ( rasterPosX * 2.0 ) - textureWidth ) / textureWidth ;
pos . y = ( ( rasterPosY * 2.0 ) - textureHeight ) / textureHeight ;
pos . zw = 1.0 ;

● Step 2/3: Particle Vertex L-Buffer
○ Apply lighting similar to normal deferred lighting
○ Use fullscreen-quads instead of light-volumes, because the fragments
have no spatial relationship (non-screenspace fragment list)
■ Optimization: Stencil-Mask each fragment in Per-Vertex G-Buffer
○ Different light-types, shadows, projected textures, etc. “just works”
○ Very simple “lighting model”: just add up light colors
○ Shadowing: PCF with large kernel width reduces flickering
in shadow/light transitions

● Step 2/3: Particle Vertex L-Buffer
// pixel shader
// trivial implementation (different for each light type):
// - sample g-buffer containing the viewspace positions of particle vertices
// - compute light attenuation base on position and light parameters (...)
float3 positionVS = SAMPLE ( perVertexGBufferSampler, screenUV . xy ) . rgb ;
col0 . rgb = light_color . rgb * getLightAttenuation ( positionVS ) * getShadowing ( positionVS ) ;

● Step 3/3: Particle Rendering
○ Render particles as you would normally render them
○ Apply lighting
■ Compute UVs according to the rasterized position from Step 1
■ Sample the Particle L-Buffer in Vertex-Shader
■ Interpolate lighting to fragment shader and modulate with particle texture
○ Take care of exact rendering order
(must be the same as in Step 1/3)

● Step 3/3: Particle Rendering
// vertex-shader
// vertexId = SV_VertexID, instanceId = SV_InstanceID
// param_vertexcount = shader constant holding the currently rendered number of vertices
float vertexIndex = vertexId + ( instanceId * vertexCountPerInstance ) + param_vertexcount ;
float3 perVertexLightBufferUV = 0;
perVertexLightBufferUV . y = trunc ( vertexIndex / textureWidth ) ;
perVertexLightBufferUV . x = trunc ( vertexIndex - perVertexLightBufferUV . y * textureWidth ) ;
perVertexLightBufferUV . y = ( textureHeight - 1.0 ) - perVertexLightBufferUV . y ;
float4 lbufferValue = perVertexLightBufferTexture . Load ( (int3)perVertexLightBufferUV . xyz ) ;
// pixel-shader
col0 . rgb = particleTextureColor . rgb * f_in . lbufferValue . rgb ;

The rendering technology of 'lords of the fallen' philip hammer

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to The rendering technology of 'lords of the fallen' philip hammer

Similar to The rendering technology of 'lords of the fallen' philip hammer (20)

More from Mary Chan

More from Mary Chan (20)

Recently uploaded

Recently uploaded (20)

The rendering technology of 'lords of the fallen' philip hammer