Nubis: Realtime Volumetric Cloudscapes in a Nutshell
•
5 likes•9,665 views
Download the full presentation here: https://www.guerrilla-games.com/read/nubis-realtime-volumetric-cloudscapes-in-a-nutshell This talk is a distillation of all of the previous publications about our cloud system, Nubis. Its purpose is to make the concepts behind our approach more accessible and to give the reader some insight into our process at Guerrilla. This was originally presented at the 2018 Eurographics Conference in Delft, The Netherlands on April 19, 2018.
Recommended
Recommended
More Related Content
More from Guerrilla
More from Guerrilla (18)
Recently uploaded
Recently uploaded (20)
Nubis: Realtime Volumetric Cloudscapes in a Nutshell
- 2. NUBIS Realtime Volumetric Cloudscapes In A Nutshell Andrew Schneider - Principal FX Artist, Guerrilla
- 3. NUBIS | In a Nutshell Previous Publications: • SIGGRAPH 2015 Advances in Realtime Rendering • SIGGRAPH 2017 Advances in Realtime Rendering • GPU Pro 7 = 80min, 20+pages In a Nutshell: • Observation • Abstraction • Implementation What this Talk is NOT: • Implementation • Exactly what we shipped with Horizon: Zero Dawn [ hyperinks on the last slide ]
- 4. Portrait / Art from the archive of the National Meteorological Library And Archive [UK] Luke Howard, Meteorologist, 1802 Cumulus Stratus Cirrus NUBIS | The Modifications of Clouds
- 5. Photographs NUBIS | The Modifications of Clouds
- 6. Cloud Coverage Cloud Type NUBIS | Density Model
- 7. type 0 1 CumulusStratocumulusStratus NUBIS | Density Model height
- 8. StratusStratocumulusCumulus in-game render NUBIS | Density Model
- 9. Photographs NUBIS | Density Model
- 11. base_cloud_density = remap(low_freq_noise, high_freq_noise, 1.0, 0.0, 1.0) NUBIS | Density Model
- 12. in-game render NUBIS | Density Model
- 13. coverage 0 1 cloud_with_coverage = remap(base_cloud_density * height_gradient, cloud_coverage, 1.0, 0.0, 1.0) NUBIS | Density Model
- 14. in-game render NUBIS | Density Model
- 15. N S W E NUBIS | Authoring System
- 16. in-game render Cloud Map NUBIS | Authoring System
- 17. in-game render Cloud Map NUBIS | Authoring System
- 18. x x NUBIS | Lighting Model Photograph
- 19. NUBIS | Lighting Model Energy = Attenuation * Phase * InScattering
- 20. NUBIS | Lighting Model Energy = exp( -density_along_light_ray ) Beer-Lambert Law
- 21. NUBIS | Lighting Model Energy = 1Energy = exp( -density_along_ray) in-game render
- 22. NUBIS | Lighting Model HG( cos(θ), eccentricity) Henyey-Greenstein Phase Function
- 23. NUBIS | Lighting Model in-game render
- 24. NUBIS | Lighting Model 1 - exp( -density_sample)
- 25. NUBIS | Lighting Model in-game render
- 26. Energy = exp( -density_along_light_ray ) Beer-Lambert Law Absorption / Out-scattering Henyey-Greenstein Phase Function Scattering Phase HG( cos(θ), eccentricity) In-Scattering Probability Function Dark Edges 1 – exp( -density_sample)** NUBIS | Lighting Model in-game render
- 27. NUBIS | Renderer
- 28. NUBIS | Renderer in-game render
- 29. NUBIS | Renderer
- 30. NUBIS | Renderer
- 31. NUBIS | Performance Base Parallelization March Optimizations Exclude Hidden Pixels 60+ms 22ms 3.34ms 1.2ms in-game render
- 32. NUBIS | In a Nutshell Do I need to render this pixel? * 16 Step * 2, Sample Cheap Is result non-zero? Sample Expensive Is result non-zero? Step toward light, Sample Expensive, sum values * 6 Calculate Light Energy Attenuate by Alpha Add density to alpha Is Alpha >= 1? STOP! Is the result zero for several samples? NUBIS | In A Nutshell
- 33. • Thanks to: • Nathan Vos • Jan Bart van Beek • Marijn Giesbertz • Elco Vossers • Coen Klosters • Vlad Lopatin • Felix van den Bergh • Maarten van der Gaag • Hugh Malan • Giliam de Carpentier • Kevin Ortegren • Michal Valient • Michiel van der Leeuw • Hermen Hulst • Angie Smets • Kojima Productions • Personal Thanks: • Trevor Thomson • Matthew Wilson • Matthew Roach • Carl Ludwig • Hugo Ayala • Donald Sajda & Jerry Ford • Joe Pasquale, Malcom Kesson, Clarke Stallworth • Tim McLaughlin • Natalya Tatarchuk • SideFX Software • Colleagues around the Game and VFX industry • The community References: [Hamblyn, 2001] Richard Hamblyn, The Invention Of Clouds. New York: Picador Reprints., 2011. [Pretor-Pinney, 2007] Gavin Pretor-Pinney, The Cloudspotter’s Guide. London: Sceptre., 2007. [Schneider, 2015] A. Schneider. “The Real-Time Volumetric Cloudscapes Of Horizon: Zero Dawn”. ACM SIGGRAPH. Los Angeles, CA: ACM SIGGRAPH, 2015. Web. 26 Aug. 2015. [Clausse and Facy, 1961] R. Clausse and L. Facy, The Clouds. London: Evergreen Books LTD., 1961. [Wrenninge, 2013] M. Wrenninge, Production Volume Rendering: Design and Implementation. CRC Press, 2013. [Schneider 2016] Andrew Schneider, GPU Pro 7: Real Time Volumetric Cloudscapes. p.p. (97-128) CRC Press, 2016. [Schneider, 2017] A. Schneider. “Nubis: Authoring The Real-Time Volumetric Cloudscapes Of Horizon: Zero Dawn”. ACM SIGGRAPH. Los Angeles, CA: ACM SIGGRAPH, 2017. Web. 2017. . NUBIS | Thanks & References
Editor's Notes
- Welcome to the slides! How to use them: Start the presentation in presenter view. The bullet points indicate when you should press the arrow key or click to advance the presentation. This talk is a distillation of all of the previous publications about our cloud system, Nubis. Its purpose is to make the concepts behind our approach more accessible and to give the reader some insight into our process at Guerrilla. This was originally presented at the 2018 Eurographics Conference in Delft, The Netherlands on April 19, 2018.
- My name is Andrew Schneider and I’m from Guerrilla. Last year, we released the Open World Game, Horizon Zero Dawn for the PS4 and PS4Pro. Today I’m going to talk about Nubis, which is our solution to create and render evolving cloudscapes for gameplay and cinematics which, in addition to working with the day/night cycle, had to represent a wide variety of cloud types for specific weather scenarios while rendering in 2 milliseconds.
- I’ve shared a lot of this information before at SIGGRAPH in 2015 and 2017 and I also authored a chapter in GPU Pro 7 that got into the nuts and bolts of our approach. This totaled to about 80 minutes and 20+ pages of material. This talk is designed for people who are interested in clouds, and/or people with some knowledge of programming or computer graphics. We are going to approach the problem of realtime volumetric cloudscapes much the same way that we did at Guerrilla, by first observing things in the real world, then forming abstractions or algorithms to describe them and then at the end, we will connect these components together. It will get technical at times, but I promise that at the end I will sum it all up with one huge run-on sentence. This talk is NOT, however, a step by step implementation guide or even our most current approach to cloud rendering at Guerrilla. There is a lot that has been excluded from this talk or glossed over. That is because this is a talk about the theory behind our approach. But before we dive in, a little background on clouds is required. For millennia, clouds were regarded as mystical undefinable objects. It wasn’t until 1802 that a comprehensive classification system was devised to help us better understand them.
- One night in that year, a young man named Luke Howard gave a lecture called “On the modifications of clouds” In this lecture, he classified clouds by their altitude, basic physical characteristics and the conditions that gave rise to their development using latin nouns and adjectives. Here are some Renderings that Howard made to illustrate his points. Howard dubbed the stacked round clouds Cumulus, which means a heap or pile in Latin. The Long flat clouds became Stratus, which means layer or sheet in latin. The wispy stretched clouds that exist high in the atmosphere became Cirrus, which means hair or fibre in latin. Howard understood that clouds are a snapshot of a fluid dynamic process unfolding in the sky. Vapor rises and enters cold air, which causes it to condense on dust particles rendering it visible to us. He called this process Nubification…
- The amount of vapor, temperature changes over altitude, wind and turbulence all play a role in modeling clouds. Low emission into cool air produces the bandy stratus clouds that flow over the landscape and are generally more wispy. High emission corresponds to round billowing cumulus clouds that grow higher and higher until they flatten out on another layer of colder air.
- Lets distill all of this information down to two probabilities that tell us what type and how much of a cloud to draw at a given point. Cloud Coverage describes the presence of clouds. Cloud type describes the height of the cloud and whether it is billowy or wispy.
- If we were to describe the probability of change in density over height for a cloud we might envision a gradient like this. Since we know that cloud type is determined by rate of emission and temperature, and that each cloud type has a different height range we can order a set of these gradients according to cloud type in order to represent the height probabilities for each type of cloud. Type, can then serve as the value which blends between these three height functions. Lets look at this in practice.
- This is our cumulus Gradient Our Stratocumulus gradient And our stratus gradient. Now lets look at how we define the details that you see in this image.
- The detailed shapes on clouds can be classified into two categories: billows and wisps. Billows form when density increases in a given area and pushes water vapor outward and upward Wisps form when density decreases and vapor dissipates or curls around the wake created by turbulence forces.
- If we were to describe the curling and roiling shapes over 3 dimensional space, we might use 3d textures like Perlin noise, inverted Worley noise or, as we proposed in 2015, a combination of both depending on cloud type or height. Our approach, In principle, was to subtract the web like shapes of Worley noise from the low density regions of perlin noise in order to introduce round shapes there. This way we get the best of both worlds while only having to take one sample of a 3d texture, something that is very expensive.
- As detailed in in the 2015 course, we define a base cloud density as low frequency Perlin Worley and Worley noise an then eroding that shape using high frequency Worley noises to add detail. Using two noise samples allows us to only do the high frequency noise operations close by and improve performance. The erosion is done using a remapping function. What we call a remapping function is just something that takes a number in a given range and repositions it into a new range. This is the same thing as Maya’s SetRange and Houdini’s FitRange. We decided to use remapping functions when constructing our base cloud shape because of a really useful behavior: as opposed to multiplying the noises together, Remapping prevents a loss of too much density at the core of the base cloud shape. I’ll give an example…
- Here’s what our clouds look like WITHOUT using noise as a base probability for density. Our Low frequency noises really form the foundation of our clouds, as you can see. And the high frequency noises add detail without taking anything away at the center of the cloud. Its kind of like carving out a block of clay, except that all of the details are already stored in the clay and you are just revealing them as you carve. But in that analogy what dictates how deep to carve?
- Large scale variations in emission of moisture are what create cloud formations. We can express this by defining a cloud by its coverage of the sky at a given point. Also, because we are using our noise as a base probability for cloud density in a sample, we can apply the coverage value as an erosion to the product of our height gradient and noise. This allows the cloud appear to expand or contract over a gradient of coverage values in space. Here is an example.
- As the coverage value increases across the sky, the clouds in this image begin to inflate. This is useful when animating cloud coverage in transitions.
- So we have discussed the two probabilities that we use to model clouds but how to we build an entire sky full of clouds with varying type and coverage? We store coverage and cloud type probability values 2d texture that is mapped to the down on world position. In short, it’s a 2d Probability field that we simply call “The Cloud Map”. Red and green represent coverage and blue represents type. These textures can be created procedurally or manually for given weather situations and key events in the game, like cutscenes. Here are some examples.
- Mornings in the desert can be quite cold. If there is any vapor emission, it will probably create stratus clouds. So Cloud type was set to zero for this cloud map as you can see by the lack of Blue.
- In contrast, the probability of cumulus clouds can increase in the afternoons, So this cloud map uses a higher cloud type value.
- We have described how to model clouds using density but there is another modeling component that is equally as important: modeling with light. When you look at the surface of a cloud you aren’t actually looking at a surface. You are looking at a semi-transparent region that holds a collection of all of the photons of light from the sun which have ended up on an vector that points toward your eyes. Some photons are absorbed by dust in the cloud as indicated by the x’s This is called absorption. Some scatter away from your eye, this is called out-scattering. These two cases result in the attenuation or reduction of light. Others actually end up changing course join others heading to your eye. This is called in-scattering. It is responsible for the dark edges that you see here. There is also a higher probability that photons will scatter forward, in the original direction of the light ray. This produces the silver lining effect seen here.
- We define light energy at a given sample point in a cloud as a function of three components: Attenuation phase And in-scattering. Lets look at attenuation first.
- Back in 1852, a chemist and mathematician named Agustus Beer (great name)wanted to find a quick way to measure how much light was being absorbed by non scattering molecules in a solution. He came up with a way to express the reduction in light transmittance as a function of optical depth in a given medium. In other words if you are a photon the deeper you are in a translucent object, the lower the probability that you will escape and reach someones eyes. Since clouds are a translucent medium with absorption properties, thanks to the dust particles that all of those water molecules have bound themselves to, beer’s law applies.
- Here are some examples. On the left are some clouds with the beer-lambert function being used as the attenuation component. The right shows the same cloudscape without the attenuation component.
- Light also scatters in a forward direction, meaning that there is a higher probability that you will see scattered light when you look toward the sun, through a cloud. In 1941, Henyey and Greenstein developed a way to solve light directionality problems at galactic scales. Well, what works for astronomy also works for participating media at earth scales. For the sake of time, I’m going to treat it like a black box. For more information and a code implementation of this function, look at the SIGGRAPH 2017 slides. If you feed it a light direction, view direction and eccentricity, or directionality bias (either forward or backward) it will tell you how much more or less light is being scattered in those conditions.
- Here are some examples. We actually combined several of these functions with varying eccentricities in order to get a more art directed silver lining effect that we could control at sunset.
- Now, about those dark edges on clouds. Take a look at this photograph. What is actually happening here? First of all, in order for them to be edges there has to be a surface, which there is not. This effect is 3 dimensional. If you think in terms of attenuation or scattering direction alone, it’s more difficult to understand what is happening. Well, recall that photons enter the cloud and then scatter in many directions when they hit clumps of water molecules. The cause of this dark edge effect is revealed when we combine this scattering concept with a bit of geometry. Points that are deeper in the cloud will have more in-scattering contributors between them and the sun. How many more? Its better to ask what’s the probability of in-scattering occurring in these points? It can be described as the inverse of the beer-lambert-law. We call this component the In-Scattering Probability Function. If you found this slide confusing try looking at it this way: Look at the picture above and instead of seeing a white cloud with dark edges, first, imagine it as a black cloud with no light scattering happening on the edges, but deeper in the cloud, in-scattering is occurring more frequently, which shines through like a flashlight in a cotton sheet. Its like absorption, but in this case the cloud is the light source AND the medium that is absorbing the light. The rate at which it lights up seems to correspond to the inverse of how light attenuates in clouds. Now you have a proper understanding of what’s happening here. Lets look at an example.
- This is what a cloudscape looks like with only the Absorption and Phase components. And this is what it looks like when we apply our in-scattering probability function. There is a bit more involved in our current lighting model, but for times sake I will refer you to our previous publications.
- So, to sum up our lighting model, we say that the light energy at a point is the product of Attenuation, Phase and In-scattering probability. We have covered how to model clouds with density and light functions, so now its time to put it all together to make a cloud renderer.
- The industry standard method for rendering volumetrics involves something called a ray-march. You take a sample of density and lighting data along a vector that points away from the eye in order to determine Color and Alpha channels for that pixel. Every sample has the potential to be extremely costly for performance because in addition to sampling density for every step along the ray (represented by the red dots), you have to sample density several more times to calculate the lighting. So, knowing where you need to take expensive samples vs cheaper samples or no samples at all is key to building an efficient foundation for a cloud renderer.
- First, lets decide where we don’t need to take any samples at all. if objects like trees and mountains render front of the clouds, we do not march from those pixels. Also, when the alpha channel that we are building reaches 1, we can stop the ray march early.
- Next, lets define our ray-march by the parts of it that we need. We start our ray march taking large steps and only sampling the coverage and height gradient portions of the density model and skip noise and the associated instructions for applying noise to the cloud shape. The green sample points indicate these cheap samples. This reduces the number of samples by half. Recall that our noise erodes the base cloud shape created by the coverage and height gradient, so we are not going to miss any parts of the cloud. So, we are sort of testing for a non detailed outline of the cloud seen in green here. When we detect a non-zero density value we take a step back and switch to taking regular sized steps and full density samples. As indicated by the red sample points. If and only if the result of this expensive sample is non-zero, we take the 6 additional samples along the ray from the sample point to the sun that are required for lighting the clouds, but we reduce the resolution of the noise samples at each step until we are just taking a cheap sample on the last step. When we detect that we are no longer in the cloud, we switch back to large steps and cheap samples. So you can see, compared to the slide earlier, this is much less work and some of the work is cheaper. We take a maximum of 128 samples and 6 light samples for each of those. The original cost of our shader was 60 plus milliseconds, but after these steps its closer to 16 milliseconds. This is a huge improvement but it is still too slow. This is where the advantage of running tasks in parallel on a graphics card becomes apparent.
- We split our render up into 16 pieces that run in parallel and build an image out of these pieces. This brings performance to a safe 1.2ms for the standard use case.
- Here’s another way to break it down. For this frame, The shader without parallelization or any optimizations costs more than 60 milliseconds. After parallelization it drops to 22ms. Still pretty far from ideal. After the ray march optimizations I just described, it drops to 3.34ms. And after we don’t bother rendering below the horizon or behind large objects, the time drops to 1.2ms. To us, this is a reasonable cost to pay for something that takes up half of the frame.
- And finally, we have arrived at the slide you have been waiting for. I’m going to try to sum the whole thing up. First, we determine what pixels the shader needs to run on, then for each pixel in a block of 16 pixels, we march away from the camera using large cheap steps. Once we sample a non-zero density value, we switch to full samples and if those samples are non-zero, we do the light-ray march in 6 steps, reducing quality at every step. The light energy is then calculated using our lighting functions and attenuated by its depth in the cloud. The sampled density is also added to the alpha channel. If the Alpha channel goes reaches 1, we stop. Backing up a bit, If we detect that we have exited the cloud, we switch back to cheap samples and large steps. We map out lookup values for a set of probabilities that describe cloud density and lighting properties which, when combined in specific ways, as few times a possible and in parallel, create entire cloudscapes.
- That’s Nubis in a nutshell. There is plenty more to talk about on this subject, but I’m out of time. If this has piqued your interest, please check out the references here on the right. These slides should be available on the Guerrilla website by tomorrow. Follow @Guerrilla to get a notified when they are live. Thank you to my family, and colleagues at Guerrilla and in the field. Thank you for your time, thank you to Eurographcs for allowing us to share this work with you and have a great rest of the conference.