PRT(Precomputed Radiance Transfer) & SH(Spherical Harmonics)

1. PRT(Precomputed Radiance Transfer) 및
SH(Spherical Harmonics) 개괄
NEXON
CSO팀 클라이언트 파트
최지현

2. 발표자 소개
NEXON
카운터 스트라이크 온라인 팀
클라이언트 프로그래밍 파트장
최지현

3. Contents
01.Precomputed Radiance Transfer
02.Irradiance Environment Map
03.Spherical Harmonics Projection & Reconstruction
04.Spherical Harmonics “USEFUL” properties
05.Spherical Harmonics Example
06.QnA

4. 오늘 설명할 내용
1.모든 정점(맵)에
각 정점의 환경맵을 저장하자
Precomputed Radiance Transfer

5. 오늘 설명할 내용
2.환경맵을 압축해서 저장하자
Spherical Harmonics as a Compression Tool
2.1.압축된 환경맵을 렌더링에 이용하자

6. 오늘 설명할 내용
3.Then, in Detail...
그럼, SH는 어떻게 사용하나?
Spherical Harmonics USEFUL Property

7. STEP1
1.모든 정점(맵)에
각 정점의 환경맵을 저장하자
Precomputed Radiance Transfer

8. Precomputed Radiance Transfer

9. Precomputed Radiance Transfer
Direct Lighting
Self Shadow

10. Precomputed Radiance Transfer
Direct Lighting
Interreflection
Self Shadow

11. Precomputed Radiance Transfer
Direct Lighting
Interreflection
Subsurface Scattering
Self Shadow

12. Precomputed Radiance Transfer

13. Precomputed Radiance Transfer

14. Precomputed Radiance Transfer

15. Precomputed Radiance Transfer

16. STEP2
2.환경맵을 압축해서 저장하자
Spherical Harmonics as a Compression Tool
2.1.압축된 환경맵을 렌더링에 이용하자

17. Irradiance Environment Map
0 10 30 45 60 70 80 90 100 120 135 150 160 170 180
R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255
G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255
B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

18. Irradiance Environment Map
0 10 30 45 60 70 80 90 100 120 135 150 160 170 180
R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255
G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255
B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

19. Irradiance Environment Map
0 10 30 45 60 70 80 90 100 120 135 150 160 170 180
R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255
G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255
B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

20. Irradiance Environment Map
0 10 30 45 60 70 80 90 100 120 135 150 160 170 180
R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255
G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255
B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

21. Irradiance Environment Map
Environment Map 을
{θ  Env(Green)} 의 함수로 만들 수 있다!
Li (Circle)  f green ( )
+ r, b channel
Environment Map 을
{(θ, φ)  Env(R,G,B)} 의 함수로 만들 수 있다!
Li ( Sphere)  f rgb ( ,  )

22. Irradiance Environment Map
f
0 10 30 45 60 70 80 90 100 120 135 150 160 170 180
R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255
G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255
B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

23. Irradiance Environment Map
~
f
0 10 30 45 60 70 80 90 100 120 135 150 160 170 180
R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255
G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255
B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

24. Irradiance Environment Map
~
~
f
0 10 30 45 60 70 80 90 100 120 135 150 160 170 180
R 255 128 255 255 255 255 128 0 0 0 0 0 0 0 255
G 0 0 0 128 255 255 255 128 0 64 0 64 128 255 255
B 128 0 0 64 0 128 255 255 255 128 255 64 0 0 0

25. Irradiance Environment Map
+ r, b channel
Environment Map 을
{(θ, φ)  Env(R,G,B)} 의 함수로 만들 수 있다!
Li ( Sphere)  f rgb ( ,  )
Approximation
Environment Map Env(R,G,B) 함수를
여기에 근사하는 함수로 만들 수 있다!
~
~
f rgb ( ,  )  f rgb ( ,  )

26. Irradiance Environment Map
Low Pass Filtering
= Low Frequency Irradiance
= Compressing
64x64x8x3 9x3

27. STEP3
2.환경맵을 압축해서 저장하자

as a Compression Tool
2.1.압축된 환경맵을 렌더링에 이용하자

28. Spherical Harmonics Projection & Reconstruction
Projection To Basis
0
C 0
1
C 1
0
C 1
Li ( ,  )
1
C 1

29. Spherical Harmonics Projection & Reconstruction
Approximation
Environment Map Env(R,G,B) 함수를
여기에 근사하는 함수로 만들 수 있다!
~
~
f rgb ( ,  )  f rgb ( ,  )
Approximation
Environment Map을 근사하는 함수를
몇개의 상수로 표현할 수 있다!
 
~
~
f rgb ( ,  )  c0 ... cn1
0 n 1 T

30. Spherical Harmonics Projection & Reconstruction
Reconstruction By Basis
0
C 0
1
C 1
0
C 1
Li ( ,  )
1
C 1

31. 일반적으로, 앞서 언급된
Spherical Coordinate 상의 Basis 를
라고 함.
y ( ,  )
m
l

32. Spherical Harmonics Projection & Reconstruction
m = -4 -3 -2 -1 0 1 2 3 4
ylm ( ,  )  yi ( ,  )  i  l (l  1)  m l : band n  l  1 : order

33. Spherical Harmonics Projection & Reconstruction
Spherical Harmonics Math
 2 K lm cos(m ) Pl m (cos ), m  0

yl ( ,  )   2 K lm sin(m ) Pl m (cos ), m  0
m
 K l0 Pl 0 (cos ), m  0
 l  N ,l  m  l
Spherical Harmonics Definition
(1  m) Pl m ( x)  x(2l  1) Pl m ( x)  (1  m  1) Pl  2 ( x)
(2l  1) (1  m )!
m
P ( x)  (1) (2m  1)!!(1  x )
m m 2 m/ 2
K  m
4 (1  m )!
m l
Pm 1 ( x)  x(2m  1) Pm ( x)
m m
Associated Legendre Polynomials Recursive Formulae Spherical Harmonics Normalization Constants
이해하는 사람은 천재...

34. Spherical Harmonics Projection & Reconstruction
 c0 
0 n infinite
 1 
 c1 
 c10 
 
 ... 
Li ( ,  )  ...  Li ( ,  )
 n 1 
Projection To
Spherical Harmonics Basis
cn 1 
  Reconstruction By
Spherical Harmonics Basis
Function ylm ( ,  ) n
Function ylm ( ,  )

35. Spherical Harmonics Projection & Reconstruction
c 
0
0
n=3 order
 
1
3^3=9 coefficient
c 
1
c 
0
 
1
 ... 
Li ( ,  )
 ...  ~
 2 Li ( , )
Projection To  c2 
  Reconstruction By
Spherical Harmonics Basis n 3 Spherical Harmonics Basis
Function ylm ( ,  ) Function ylm ( ,  )

36. MATH TIME!
한방에 이해하는 사람은 천재!
나중에 집에서 천천히 생각해보세용~

37. Spherical Harmonics Projection & Reconstruction
m
Projection Math = I WANT c l
// 모든 방향의 Environment Map
c   f ( s ) y ( s )ds
m
l
m
l Pixel 값과 해당 방향의 SH Basis
s Function 를 곱해서 다~ 더하면 됨
ci   f ( s) yi ( s)ds  i  l (l  1)  m // band index
s
2 
ci    f ( ,  ) y ( ,  ) sin dd
0 0
i
// Spherical Coordinate
4 N
ci 
N
 f (x ) y (x )
j 1
j i j // Monte Carlo Integration

38. Spherical Harmonics Projection & Reconstruction
~
Reconstruction Math = I WANT Li ( , )
: Lii ( ,  )  C 0
0
0
0
 y0 ( ,  )
0
1
1
C  1
1
y11 ( ,  ) C  0
1
1
0
y10 ( ,  ) C  y ( ,)
1
1
1
1 1
1
2 1
1
C 
2
C 
2
2 y2 2 ( ,  )

C 
C 
2
2 y2 1 ( ,  )

C 
0
2
2
0
y2 ( ,  )
0
C 
1
1
2
2 y1 ( ,  )
2 C y ( ,)
2
2
2
2 2
2
..........
~ n 1 l
Li ( , )    c y ( ,  )
m m
l l
l 0 m   l

39. Spherical Harmonics Projection & Reconstruction
~
Reconstruction Math = I WANT Li ( , )
: Li ( ,  )  C 0
0  y0 ( ,  )
0
1
C  1
y11 ( ,  ) C  0
1
y10 ( ,  ) C  y ( ,)
1
1
1
1
1
2
C 
2 y2 2 ( ,  )

C 
2 y2 1 ( ,  )

C 
0
2 y2 ( ,  )
0
C 
1
2 y1 ( ,  )
2 C y ( ,)
2
2
2
2
..........
~ n 1 l
Li ( , )    c y ( ,  )
m m
l l
l 0 m   l

40. STEP4
[2부]

41. STEP3
시간 상 설명은
핵심적인 것만!

42. Spherical Harmonics
“USEFUL” properties
01.Quadratric Polynorminal Form
02.Double(Dot) Product
03.SH Rotation (ZYZ, Ivanic)
04.Zonal Harmonics Rotation
05.Analytic Light Source
06.Extract Dominant Direction Light

43. 01.Quadratric Polynorminal Form

y ( ,  ) 말고 y (n ) 을 사용하자!
m
l
m
l

y ( ,  )
m
l  x  sin  cos
m
y (n ( x, y, z ))
l

 y  sin  sin 
 z  cos


44. 01.Quadratric Polynorminal Form
Reconstruction 을 하더라도 ylm ( ,  ) 는 계산량이 많고
Shader 로 표현하기 어려우며,
게임에서 Spherical Coordinate 는 잘 사용하지도 않는다.
 2 K lm cos(m ) Pl m (cos ), m  0

ylm ( ,  )   2 K lm sin(m ) Pl  m (cos ), m  0
 K l0 Pl 0 (cos ), m  0

(1  m) Pl m ( x)  x(2l  1) Pl m ( x)  (1  m  1) Pl  2 ( x)
m
(2l  1) (1  m )!
Pm ( x)  (1) m (2m  1)!!(1  x 2 ) m / 2
m
Klm 
Pm 1 ( x)  x(2m  1) Pm ( x)
m m 4 (1  m )!
Spherical Coodinate (θ, φ) 를 Cartesian Coordinate (x, y, z)
로 변환해서 ylm ( ,  ) 를 직접 풀어 이용하자.
 x  sin  cos

 y  sin  sin 
 z  cos


45. 01.Quadratic Polynorminal Form

주어짂 normal y (n  ( x, y, z )) 에 대하여
m
l
주어짂 SH Coefficient Vector 의 SH Reconstruction 은?
0  1
y0 ( n )  ,
2 
 3y 0  3z 1  3x
y11 (n )   , y1 (n )  , y1 (n )   ,
2  2  2 
2 15 yx 1  15 yz 0  5 (3 z 2  1)
y (n )  , y2 (n )   , y2 (n )  ,
2  2  4 
2
 15xz 2  15( x 2  y 2 )
y (n )  
1
, y2 (n ) 
2  4 
2
 0 0 
Li (n  ( x, y, z ))  c0 y0 (n ) 
   0 0  1 1 
c1 1 y1 1 (n )  c1 y1 (n )  c1 y1 (n ) 
      0 0   2 2 
c2 2 y2 2 (n )  c2 1 y2 1 (n )  c2 y2 (n )  c1 y1 (n )  c2 y2 (n )
2 2

46. 02.Double(Dot) Product
두 함수의 곱을 Sphere 상에서
적분하는 Rendering Equation 을
빠르게 계산할 수 있다!
          
Lr ( x , r )   f r (x , i  r ) Li ( x , i )G ( x , x )V ( x , x )di
S
fL r i or  GL i or  LV i or ...

47. 02.Double(Dot) Product
( n 1) 2
 L(s)V (s)ds   c d
s i 0
i i
X =
L(s)  Ci V ( s)  d i  cd
c
0
0 ... c n 1
n 1  d 0
0 ... d n 1
n 1 T
 Lo
1x9 9x1 1x1

48. 02.Double(Dot) Product – PRT Revisted.
 c0 0 ... c0 2 
2
 v0 : EnvMap0To" SomeBasis  "
 0
0
2   0   v : EnvMap To" SomeBasis 
 vc0  ... c12  i0  1 "   Env 
 vc   ...
c10 1
     Light 
 1   ... ...  i11 

...
  
 ...    ... 
... ...    ...   ...    To 
    1   " Some
vcn  2   ... 0 ... ...   i2 

...
  
 vcn 1  cn  2 0
  
2
... cn  2 2   i2 
2
vn  2 : EnvMapn  2To" SomeBasis   Basis"
"  
 
 cn 12 ... cn 12 
2  v : EnvMap To" SomeBasis 
 2   n 1 n 1 "
each vertex color = dot(envmap@vertex.to.some.basis, envlight.to.some.basis)
 
V  BL
“Some Basis”  Spherical Harmonics
“Some Basis”  Ambient Cube Basis
“Some Basis”  Wavelet Basis

49. 03.SH Rotation
SH Rotation 은
마치 Vector3 의 Rotation 처럼
Linear Matrix 로
표현 가능하다!

50. 03.Rotation - ZYZ
SH – Rotation is Linear Operation!
 b00   c0 
0
 1   1 
 b1   c1 
 b10   c10 
 1  1
 b1   c1 
b  2 
 21  = x c  2 
 21 
 b2   c2 
 b0   c0 
 2  2
 b2 
1
 c1 
 2
2
 2
 b2   c2 
Spherical Coordinate 의 특성상 Z 축을 중심으로
φ 만큼 회전하는 matrix 는 만들기 쉽다
R( ,  ,  )  Z Y Z  Z X 90 Z  X 90 Z

51. 03.Rotation - Ivanic
ZYZ는 별로 nice 하지 않는데?
Spherical Harmonics 의 Rotation의 Recurrence Relation 한
성질을 이용해서 한방에(Fast) Matrix 를 구하자
= Rotation Matrices for Real Spherical Harmonics by Ivanic
 Rxx Rxy Rxz  Rotation Matrix
  By Local Frame
 R yx R yy R yz 
 Rzx Rzy Rzz 
 

52. 03.Rotation Invariant
Rotate Project
.BMP
Equal
Rotate

53. 03.Rotation – Ivanic - Shader

54. 04.Zonal Harmonics
Z축에 symmetric 한
는
Rotation 을 더 빠르게 할 수 있다!

55. 04.Zonal Harmonics
Circular Symmetry around Z-axis
Spherical Harmonic Subset
Simple & Cheap!

56. 04.Zonal Harmonics - Rotation
1 0 0 0 0 0 0 0 0  c0 
0
1 0 0 0 0 0 0 0 0 c0 
0
0  1 
x x x 0 0 0 0 0  c1 
0 x x x 0 0 0 0 0
 
    0
0 x x x 0 0 0 0 0  c10  0 x x x 0 0 0 0 0 c10 
   1    
0 x x x 0 0 0 0 0  c1  0 x x x 0 0 0 0 0 0
0 0 0 0 x x x x x c  2  0 0 0 0 x x x x x 0
   21     
0 0 0 0 x x x x x  c2  0 0 0 0 x x x x x 0
0 0 0 0 x x x x x  c0  0 0 0 0 x x x x x c 0 
   2    2
0 0 0 0 x x x x x  c1 
2 0 0 0 0 x x x x x 0
0 x  2 0  
 0 0 0 x x x x   c2   0 0 0 x x x x x
 0

57. 04.Zonal Harmonics
1개 : 9x3=27 float
모든 Vertex(Lumel) 에 대한 SH-Rotation 이 필요한 경우,
계산량이 많으므로 이경우 Zonal Harmonics 를 이용한다!
18개 : 9x3x18=486 float

58. 05.Analytic Light Source
우리는 Point(Direction)Light 에 익숙하니
이런 형태의 정형화된 Light 로부터
SH Coefficient 를 빠르게 계산 가능하다면
다양한 활용성이 기대되지 않을까?

59. 05.Analytic Light Source
우리는 임의의 Incident Light Source : Environment Map 을
SH Basis 에 Projection 해서 clm 를 얻어낼 수 있다.
그러면, 임의의 Environment Map 이 아니라 “자주 사용하는” 정형화된
형태의 Light Source 로부터 clm 를 얻어낼 수도 있지않을까?
Cone Light Analytic Models
2 
ci    f ( ,  ) y ( ,  ) sin dd
0 0
i
2 
ci    y ( ,  ) cos  sin dd
0 0
i

60. 05.Analytic Light Source
Cone Light Analytic Models
c0    (1  cos( ))
0
0-band
c11  0
1
c10  3  sin( ) 2 1-band
2
c1  0
1

c2 2  0

c2 1  0
Zonal Harmonics! c2 
0 1
5  cos( )(1  cos( ))(cos( )  1)

2
2-band
c1  0
2
c2  0
2

61. 06.Extract Dominant Directional Light
주어진 SH Coefficient 에서
를 찾아내서
다양한 곳에 사용해보자.

62. 06.Extract Dominant Directional Light
Shader 가 지원안되는 저사양 그래픽 카드에서 사용해보자
PRT Lighting 시 Specular BRDF Lighting 에 사용해보자
......등등
n 1  2
E   ( Li  cyi (d )) , E  0 // least square method
// minimizing E
i 0
n 1  n 1  2
c   ( Li yi (d )) /  yi (d ) // directional light color
i 0 i 0
// direction vector
1
dir.xyz  normalize c .xyz  c .xyz  c .xyz)
( 1
1 1
0
1
// using SH linear term
// using grayscale intens.

63. STEP4
EXAMPLE TIME!

64. SH – Applications
Halo3
Spherical Harmonics Lightmap
Diffuse : Quadradtic Polynormial Form
(SH Irradiance)
Specular :
SH Rotation(Local Frame),
Double Product(Light * BRDF),
Extract Dominant Directional Light
Crysis2
Injecting VPL to 3D Texture :
Cone Light Source (Analytic Model)
Zonal Harmonics Rotation

65. SH - Sample

67. SH - Sample

70. SH - Sample
Diffuse Shadowed (A)
+
Diffuse Interreflected (RGB)
Diffuse Shadow-Interreflected (NOT-USED)
SphereMap NormalMap DiffuseMap

71. END!
Q&A

72. Reference
1. An Efficient Representation for Irradiance Environment Maps - Ravi Ramamoorthi, Pat Hanrahan
2. Spherical Harmonics Lighting : The Gritty Details – Robin Green
3. Stupid Spherical Harmonics (SH) Tricks - Peter-Pike Sloan
4. Lighting and Material of Halo3 – Hao Chen, Xinguo Liu
5. Light Propagation Volumes in CryEngine 3 - Anton Kaplanyan
......and more!