2. Agenda
Game Forum Germany 2010 2
Wednesday, January 27, 2010
3. Agenda
• Goal:
Game Forum Germany 2010 2
Wednesday, January 27, 2010
4. Agenda
• Goal:
Learn about modern multiprocessor game engine update loops...
Game Forum Germany 2010 2
Wednesday, January 27, 2010
5. Agenda
• Goal:
Learn about modern multiprocessor game engine update loops...
... by investigating Naughty Dog’s train level.
Game Forum Germany 2010 2
Wednesday, January 27, 2010
7. Agenda
Game Forum Germany 2010 4
Wednesday, January 27, 2010
8. Agenda
• During our investigation, we’ll answer the following questions:
Game Forum Germany 2010 4
Wednesday, January 27, 2010
9. Agenda
• During our investigation, we’ll answer the following questions:
Why did we make the train itself dynamic (not static)?
Game Forum Germany 2010 4
Wednesday, January 27, 2010
10. Agenda
• During our investigation, we’ll answer the following questions:
Why did we make the train itself dynamic (not static)?
How does the train move?
Game Forum Germany 2010 4
Wednesday, January 27, 2010
11. Agenda
• During our investigation, we’ll answer the following questions:
Why did we make the train itself dynamic (not static)?
How does the train move?
How does it tie into the update of other engine systems?
Game Forum Germany 2010 4
Wednesday, January 27, 2010
12. Agenda
• During our investigation, we’ll answer the following questions:
Why did we make the train itself dynamic (not static)?
How does the train move?
How does it tie into the update of other engine systems?
How do player mechanics and animation work on the train?
Game Forum Germany 2010 4
Wednesday, January 27, 2010
13. Agenda
• During our investigation, we’ll answer the following questions:
Why did we make the train itself dynamic (not static)?
How does the train move?
How does it tie into the update of other engine systems?
How do player mechanics and animation work on the train?
How do game object attachment hierarchies work?
Game Forum Germany 2010 4
Wednesday, January 27, 2010
14. Agenda
• During our investigation, we’ll answer the following questions:
Why did we make the train itself dynamic (not static)?
How does the train move?
How does it tie into the update of other engine systems?
How do player mechanics and animation work on the train?
How do game object attachment hierarchies work?
How are ray and sphere casts used on the train?
Game Forum Germany 2010 4
Wednesday, January 27, 2010
15. Agenda
• During our investigation, we’ll answer the following questions:
Why did we make the train itself dynamic (not static)?
How does the train move?
How does it tie into the update of other engine systems?
How do player mechanics and animation work on the train?
How do game object attachment hierarchies work?
How are ray and sphere casts used on the train?
How do we utilize the PS3’s parallel computing resources?
Game Forum Germany 2010 4
Wednesday, January 27, 2010
17. Could the Train be Static?
• In the past, games with trains in them have used a static train
approach.
Train is actually stationary.
Background scrolls by to produce illusion of movement.
• This solves a lot of problems.
Player mechanics, NPC locomotion, weapon mechanics, etc.
are all the same as in a “regular” non-moving game level.
Game Forum Germany 2010 6
Wednesday, January 27, 2010
18. Was That Good Enough for Us?
Game Forum Germany 2010 7
Wednesday, January 27, 2010
19. Was That Good Enough for Us?
• No way!
Game Forum Germany 2010 7
Wednesday, January 27, 2010
20. Was That Good Enough for Us?
• No way!
• Stationary train design imposes undue limitations:
Game Forum Germany 2010 7
Wednesday, January 27, 2010
21. Was That Good Enough for Us?
• No way!
• Stationary train design imposes undue limitations:
Train must move in a perfectly straight line...
Game Forum Germany 2010 7
Wednesday, January 27, 2010
22. Was That Good Enough for Us?
• No way!
• Stationary train design imposes undue limitations:
Train must move in a perfectly straight line...
... or along a broad circular arc (constant curvature).
Game Forum Germany 2010 7
Wednesday, January 27, 2010
23. Was That Good Enough for Us?
• No way!
• Stationary train design imposes undue limitations:
Train must move in a perfectly straight line...
... or along a broad circular arc (constant curvature).
No ups and downs allowed in the track either.
Game Forum Germany 2010 7
Wednesday, January 27, 2010
24. Was That Good Enough for Us?
• No way!
• Stationary train design imposes undue limitations:
Train must move in a perfectly straight line...
... or along a broad circular arc (constant curvature).
No ups and downs allowed in the track either.
Not nearly as much fun to jump and shoot between trains.
Game Forum Germany 2010 7
Wednesday, January 27, 2010
25. Was That Good Enough for Us?
• No way!
• Stationary train design imposes undue limitations:
Train must move in a perfectly straight line...
... or along a broad circular arc (constant curvature).
No ups and downs allowed in the track either.
Not nearly as much fun to jump and shoot between trains.
∴ We decided to go for a fully dynamic train.
Game Forum Germany 2010 7
Wednesday, January 27, 2010
28. Spline Tracking
• The Uncharted 2 train follows a spline.
Catmull-Rom.
2 trackers for realistic movement.
Game Forum Germany 2010 10
Wednesday, January 27, 2010
29. The Master Car
• Each train car is an independent game object (GO).
• One car is designated as the master.
It moves without regard for the other cars.
Every other car is a slave: it simply maintains proper spacing with
the car(s) in front of and/or behind it.
Game Forum Germany 2010 11
Wednesday, January 27, 2010
30. The Master Car
• Each train car is an independent game object (GO).
• One car is designated as the master.
It moves without regard for the other cars.
Every other car is a slave: it simply maintains proper spacing with
the car(s) in front of and/or behind it.
Game Forum Germany 2010 11
Wednesday, January 27, 2010
31. Spacing and Speed
• To maintain proper spacing, we must work in terms of arc length.
Catmull-Roms are parameterized by a unitless quantity u.
Arc length (s) is not the same thing as u.
Careful—must use s for spacing and ds/dt for speed, not u, du/dt.
u = 0.5
u = 1
u = 0
Game Forum Germany 2010 12
Wednesday, January 27, 2010
32. Spacing and Speed
• To maintain proper spacing, we must work in terms of arc length.
Catmull-Roms are parameterized by a unitless quantity u.
Arc length (s) is not the same thing as u.
Careful—must use s for spacing and ds/dt for speed, not u, du/dt.
s = 14
u = 0.5
u = 1
s = 0
u = 0 s = 7
Game Forum Germany 2010 12
Wednesday, January 27, 2010
33. Updating the Cars
• Train car game objects need to update in a specific order:
Master first.
Then cars in front of master, from master to locomotive.
Then cars behind master, from master to caboose.
Game Forum Germany 2010 13
Wednesday, January 27, 2010
34. Updating the Cars
• Train car game objects need to update in a specific order:
Master first.
Then cars in front of master, from master to locomotive.
Then cars behind master, from master to caboose.
1
Game Forum Germany 2010 13
Wednesday, January 27, 2010
35. Updating the Cars
• Train car game objects need to update in a specific order:
Master first.
Then cars in front of master, from master to locomotive.
Then cars behind master, from master to caboose.
1 4 ... n
2 3
Game Forum Germany 2010 13
Wednesday, January 27, 2010
36. Updating the Cars
• Train car game objects need to update in a specific order:
Master first.
Then cars in front of master, from master to locomotive.
Then cars behind master, from master to caboose.
1 4 ... n
2 3
n + 1
n + 2 ... m
Game Forum Germany 2010 13
Wednesday, January 27, 2010
37. Updating the Cars
• In the Uncharted 2 engine, game objects are managed as a tree.
Children update after their parent.
• For the train...
M
M+1
M+2
M+3
M−1
M−2
M−3
Game Forum Germany 2010 14
Wednesday, January 27, 2010
38. Teleporting the Train
• The train sometimes teleports in the game.
e.g. To transition from a looped section to a straight-away.
Typically hidden by a camera cut.
Game Forum Germany 2010 15
Wednesday, January 27, 2010
39. Teleporting the Train
• The train sometimes teleports in the game.
e.g. To transition from a looped section to a straight-away.
Typically hidden by a camera cut.
Game Forum Germany 2010 15
Wednesday, January 27, 2010
40. Teleporting the Train
• Problem: Teleport the train such that whichever car the player is
on moves to a predefined location on the track.
To implement, change the master to be the player’s car...
... and teleport it to the desired location.
All other cars follow automatically.
Game Forum Germany 2010 16
Wednesday, January 27, 2010
41. Teleporting the Train
• Problem: Teleport the train such that whichever car the player is
on moves to a predefined location on the track.
To implement, change the master to be the player’s car...
... and teleport it to the desired location.
All other cars follow automatically.
Game Forum Germany 2010 16
Wednesday, January 27, 2010
42. Teleporting the Train
• Problem: Teleport the train such that whichever car the player is
on moves to a predefined location on the track.
To implement, change the master to be the player’s car...
... and teleport it to the desired location.
All other cars follow automatically.
Game Forum Germany 2010 16
Wednesday, January 27, 2010
44. Large-Scale Engine Systems
• Let’s define large-scale engine systems as:
Engine components that operate on lots of data...
... and require careful performance optimization.
• Examples include:
skeletal animation,
collision detection,
rigid body dynamics,
rendering, ...
Game Forum Germany 2010 18
Wednesday, January 27, 2010
45. A Simple Approach (That Doesn’t Work)
• Most game programmers first
learn about the game loop
from rendering tutorials.
Game Forum Germany 2010 19
Wednesday, January 27, 2010
46. A Simple Approach (That Doesn’t Work)
while (!quit)
{
• Most game programmers first ReadJoypad();
learn about the game loop UpdateScene();
from rendering tutorials. DrawScene();
FlipBuffers();
}
Game Forum Germany 2010 19
Wednesday, January 27, 2010
47. A Simple Approach (That Doesn’t Work)
• Since we need to update our game objects anyway...
UpdateScene() becomes UpdateGameObjects()
• In the spirit of good object-oriented design, we should let
the game objects drive the large-scale engine systems.
(Right???)
Game Forum Germany 2010 20
Wednesday, January 27, 2010
48. A Simple Approach (That Doesn’t Work)
while (!quit)
{
ReadJoypad();
UpdateGameObjects();
// DrawScene();
FlipBuffers();
}
Game Forum Germany 2010 21
Wednesday, January 27, 2010
49. A Simple Approach (That Doesn’t Work)
void Tank::Update ()
{
MoveTank();
while (!quit) AimTurret();
{ FireIfNecessary();
ReadJoypad();
UpdateGameObjects(); Animate();
// DrawScene(); DetectCollisions();
FlipBuffers(); SimulatePhysics();
} UpdateAudio();
Draw();
}
Game Forum Germany 2010 21
Wednesday, January 27, 2010
50. A Simple Approach (That Doesn’t Work)
• The only problem with this is...
it doesn’t work!
• For one thing, it’s just not feasible for some engine systems.
e.g. Collision detection cannot be done (properly) one object
at a time.
• Need to solve iteratively, account for time of impact (TOI),
• optimize collision detection via spatial subdivision (e.g.
broadphase AABB prune and sweep),
• optimize dynamics by grouping into islands, etc.
Game Forum Germany 2010 22
Wednesday, January 27, 2010
51. A Simple Approach (That Doesn’t Work)
Game Forum Germany 2010 23
Wednesday, January 27, 2010
52. A Simple Approach (That Doesn’t Work)
• It’s also terribly inefficient:
Game Forum Germany 2010 23
Wednesday, January 27, 2010
53. A Simple Approach (That Doesn’t Work)
• It’s also terribly inefficient:
potential for duplicated computations,
Game Forum Germany 2010 23
Wednesday, January 27, 2010
54. A Simple Approach (That Doesn’t Work)
• It’s also terribly inefficient:
potential for duplicated computations,
possible reallocation of resources,
Game Forum Germany 2010 23
Wednesday, January 27, 2010
55. A Simple Approach (That Doesn’t Work)
• It’s also terribly inefficient:
potential for duplicated computations,
possible reallocation of resources,
poor data and instruction cache coherence,
Game Forum Germany 2010 23
Wednesday, January 27, 2010
56. A Simple Approach (That Doesn’t Work)
• It’s also terribly inefficient:
potential for duplicated computations,
possible reallocation of resources,
poor data and instruction cache coherence,
not conducive to parallel computation.
Game Forum Germany 2010 23
Wednesday, January 27, 2010
57. Hardware in the Old Days
• On an Intel Pentium 286 (cira 1982):
CPU speed was the bottleneck (6 MHz).
• ∴ Use if() tests to avoid unnecessary computations.
Integer instructions faster than floating-point.
• ∴ Use fixed-point numbers, or FPU.
More compute power provided by adding transistors.
Game Forum Germany 2010 24
Wednesday, January 27, 2010
58. Modern Hardware
• On a Cell (PS3) or Xenon (Xbox 360):
Memory access time is the bottleneck.
• L1 cache miss = ~60 cycles.
• L2 cache miss = ~500 cycles.
• ∴ Organize data in compact, contiguous blocks.
• Use struct of arrays rather than array of structs.
Pipelined architecture = branching & data conversion are slow.
• ∴ Use duplicated computations to avoid if() tests.
Compute power is provided via parallelism.
Game Forum Germany 2010 25
Wednesday, January 27, 2010
59. Optimizing Large-Scale Updates
Inefficient
GameObject 1 GameObject 2 GameObject 3
Anim Coll. Draw Anim Coll. Draw Anim Coll. Draw ...
Data Data Data Data Data Data Data Data Data
Game Forum Germany 2010 26
Wednesday, January 27, 2010
60. Optimizing Large-Scale Updates
Much beEer!
Anima?on Engine Collision Engine Rendering Engine
GO 1 GO 2 GO 3 GO 1 GO 2 GO 3 GO 1 GO 2 GO 3
Anim Anim Anim ... Coll. Coll. Coll. ... Draw Draw Draw ...
Data Data Data Data Data Data Data Data Data
Game Forum Germany 2010 26
Wednesday, January 27, 2010
61. Optimizing Large-Scale Updates
• Game objects do not own their large-scale system data...
... they point to data that is owned by the large-scale systems.
• Game objects should not mutate their large-scale system data...
... they should only request changes.
• That way, each large-scale system can:
apply any requested changes as part of its batch update,
organize its data in whatever manner is most efficient.
Game Forum Germany 2010 27
Wednesday, January 27, 2010
62. Optimizing Large-Scale Updates
Anima?on
Engine
class Tank : public GameObject animControl15 skelInstance15
{
public: animControl42 skelInstance42
// interface...
animControl7 skelInstance7
private:
... ...
// components -- owned by engine subsystems
AnimControl* m_pAnimControl;
RigidBody* m_pRigidBody;
MeshInstance* m_pRenderable;
};
...
Game Forum Germany 2010 28
Wednesday, January 27, 2010
63. Optimizing Large-Scale Updates
class Tank : public GameObject
{
public: Collision/Physics World
// interface...
rigidBody9 collidable9
private:
// components -- owned by engine subsystems rigidBody3 collidable3
AnimControl* m_pAnimControl;
RigidBody* m_pRigidBody; rigidBody41 collidable41
MeshInstance* m_pRenderable;
}; ... ...
...
Game Forum Germany 2010 28
Wednesday, January 27, 2010
64. Optimizing Large-Scale Updates
class Tank : public GameObject
{ Scene Graph
public:
meshInstance8
// interface...
meshInstance5
private:
// components -- owned by engine subsystems meshInstance37
AnimControl* m_pAnimControl;
RigidBody* m_pRigidBody; ...
MeshInstance* m_pRenderable;
};
...
Game Forum Germany 2010 28
Wednesday, January 27, 2010
65. while (!quit)
{
ReadJoypad();
g_gameWorld.UpdateAllGameObjects();
//for (each GameObject* pGo)
// pGo->Update();
g_animationEngine.Update();
g_collisionWorld.DetectCollisions();
g_physicsWorld.Simulate();
g_audioEngine.Update();
g_renderingEngine.DrawAndFlipBuffers();
}
Game Forum Germany 2010 29
Wednesday, January 27, 2010
68. Player Mechanics on a Moving Object
• Player’s position and orientation represented as an attached frame
of reference:
reference to parent game object,
local transform (relative to parent),
cached world-space transform (with dirty flag).
• Not quite as simple as it may sound!
Just switching to attached frames got us ~60% of the way there.
Lots of bugs to fix.
Special-case handling of transitions between attached frames.
Game Forum Germany 2010 32
Wednesday, January 27, 2010
69. Animation Pipeline Review
1.Animation Update
Update clocks, trigger keyframed events, detect end-of-animation
conditions, take transitions to new animation states.
2.Pose Blending Phase
Extract poses from anim clips, blend individual joint poses.
• Generates local joint transforms (parent-relative).
Game Forum Germany 2010 33
Wednesday, January 27, 2010
70. Animation Pipeline Review
3.Global Pose Phase
Walk hierarchy, calculate global joint transforms (model-space or
world-space) and matrix palette (input to renderer).
4.Post-Processing
Apply IK, procedural animation, etc.
• Generates new local joint transforms.
• Recalculate global poses if necessary (re-run phase 3).
Game Forum Germany 2010 34
Wednesday, January 27, 2010
71. Game Object Hooks
• It’s convenient to provide game objects with hooks into the
various phases of the animation update.
• What we do at Naughty Dog:
GameObject::Update()
• Regular GO update; runs before animation.
GameObject::PostAnimUpdate()
• Runs after Animation Phase 1.
• Game objects may respond to animation events, force
transitions to new animation states, etc.
Game Forum Germany 2010 35
Wednesday, January 27, 2010
72. Game Object Hooks
GameObject::PostAnimBlending()
• Runs after Animation Phase 2.
• Game objects can apply procedural animation in local space.
GameObject::PostJointUpdate()
• Runs after Animation Phase 3.
• This is the first time during the frame that the global transform
of every joint is known.
• Game objects can apply IK, or use the global space joint
transforms in other ways.
Game Forum Germany 2010 36
Wednesday, January 27, 2010
73. while (!quit)
{
// ...
g_gameWorld.UpdateAllGameObjects();
g_animationEngine.RunPhase1();
g_gameWorld.CallPostAnimUpdateHooks();
g_animationEngine.RunPhase2();
g_gameWorld.CallPostAnimBlendingHooks();
g_animationEngine.RunPhase3();
g_gameWorld.CallPostJointUpdateHooks();
// ...
Game Forum Germany 2010 37
Wednesday, January 27, 2010
80. Bucketed Game Object Updates
• Can implement this by updating game objects in buckets.
Game object dependencies represented by a dependency tree.
Game Forum Germany 2010 40
Wednesday, January 27, 2010
81. Bucketed Game Object Updates
• Can implement this by updating game objects in buckets.
Game object dependencies represented by a dependency tree.
Each bucket corresponds to one level of the tree.
Bucket 0 Bucket 1 Bucket 2 Bucket 3
trainCar7
box12 weapon3
trainCar8 crate21 playerCharacter weapon17
npc65
trainCar9
... ... ... ...
Game Forum Germany 2010 40
Wednesday, January 27, 2010
82. Bucketed Game Object Updates
• Simpler to group objects into pre-determined buckets.
Vehicle Bucket Object Bucket Character Bucket Prop Bucket
trainCar7
box12 weapon3
trainCar8 crate21 playerCharacter weapon17
npc65
trainCar9
... ... ... ...
Game Forum Germany 2010 41
Wednesday, January 27, 2010
83. Bucketed Game Object Updates
while (!quit)
{
// ...
for (each bucket)
{
g_gameObjectMgr.UpdateObjects(bucket);
AnimateBucket(bucket);
}
g_collphysWorld.CollideAndSimulate(dt);
g_audioEngine.Update(dt);
g_renderingEngine.DrawAndFlipBuffers(dt);
}
Game Forum Germany 2010 42
Wednesday, January 27, 2010
84. void AnimateBucket(bucket)
{
g_animationEngine.UpdateClocks(bucket);
for (each GameObject* pGo in bucket)
pGo->PostAnimUpdate();
g_animationEngine.CalcLocalPoses(bucket);
for (each GameObject* pGo in bucket)
pGo->PostAnimBlending();
g_animationEngine.CalcGlobalPoses(bucket);
for (each GameObject* pGo in bucket)
pGo->PostJointUpdate();
}
Game Forum Germany 2010 43
Wednesday, January 27, 2010
86. Ray and Sphere Casts
• A collision cast is an instantaneous collision query.
Input:
• Snapshot of collision geometry in game world at time t.
• One or more rays / moving spheres (capsules) to cast.
Output:
• Would any of the rays/spheres strike anything?
• If so, what?
Game Forum Germany 2010 45
Wednesday, January 27, 2010
87. Ray and Sphere Casts
• A collision cast is an instantaneous collision query.
Input:
• Snapshot of collision geometry in game world at time t.
• One or more rays / moving spheres (capsules) to cast.
Output:
• Would any of the rays/spheres strike anything?
• If so, what?
Game Forum Germany 2010 45
Wednesday, January 27, 2010
88. Ray and Sphere Casts
• A collision cast is an instantaneous collision query.
Input:
• Snapshot of collision geometry in game world at time t.
• One or more rays / moving spheres (capsules) to cast.
Output:
• Would any of the rays/spheres strike anything?
• If so, what?
Game Forum Germany 2010 45
Wednesday, January 27, 2010
89. Ray and Sphere Casts
• A collision cast is an instantaneous collision query.
Input:
• Snapshot of collision geometry in game world at time t.
• One or more rays / moving spheres (capsules) to cast.
Output:
• Would any of the rays/spheres strike anything?
• If so, what?
Game Forum Germany 2010 45
Wednesday, January 27, 2010
90. Ray and Sphere Casts
• A collision cast is an instantaneous collision query.
Input:
• Snapshot of collision geometry in game world at time t.
• One or more rays / moving spheres (capsules) to cast.
Output:
• Would any of the rays/spheres strike anything?
• If so, what?
Game Forum Germany 2010 45
Wednesday, January 27, 2010
91. Uses for Collision Casting
• Player and NPCs use downward casts to determine what surface
they are standing on.
• NPCs use ray casts to answer line of sight questions.
• Weapons use ray casts to determine bullet impacts.
• and the list goes on...
Game Forum Germany 2010 46
Wednesday, January 27, 2010
92. Uses for Collision Casting
• Player and NPCs use downward casts to determine what surface
they are standing on.
• NPCs use ray casts to answer line of sight questions.
• Weapons use ray casts to determine bullet impacts.
• and the list goes on...
Game Forum Germany 2010 46
Wednesday, January 27, 2010
93. Uses for Collision Casting
• Player and NPCs use downward casts to determine what surface
they are standing on.
• NPCs use ray casts to answer line of sight questions.
• Weapons use ray casts to determine bullet impacts.
• and the list goes on...
Game Forum Germany 2010 46
Wednesday, January 27, 2010
94. Uses for Collision Casting
• Player and NPCs use downward casts to determine what surface
they are standing on.
• NPCs use ray casts to answer line of sight questions.
Game Forum Germany 2010 46
Wednesday, January 27, 2010
95. Uses for Collision Casting
• Player and NPCs use downward casts to determine what surface
they are standing on.
• NPCs use ray casts to answer line of sight questions.
• Weapons use ray casts to determine bullet impacts.
Game Forum Germany 2010 46
Wednesday, January 27, 2010
96. Uses for Collision Casting
• Player and NPCs use downward casts to determine what surface
they are standing on.
• NPCs use ray casts to answer line of sight questions.
• Weapons use ray casts to determine bullet impacts.
• and the list goes on...
Game Forum Germany 2010 46
Wednesday, January 27, 2010
97. Inter-Game-Object Queries and Synchronization
• Synchronization problems can arise whenever:
game object A...
... queries game object B.
• Not just limited to ray and sphere casts.
Any kind of inter-object query can be affected.
Game Forum Germany 2010 47
Wednesday, January 27, 2010
98. Game Object State Vectors
• Can think of a game object as a heterogeneous “state vector.”
• The state of the ith game object is a vector function of time t.
Si(t) = [ ri(t),
vi(t),
mi, ...,
healthi(t),
ammoi(t), ... ]
Game Forum Germany 2010 48
Wednesday, January 27, 2010
99. Bucketing and Game Object State Vectors
• Theoretically, the state vectors of all game objects are updated
instantaneously and in parallel.
t1
SA
SB
SC
Game Forum Germany 2010 49
Wednesday, January 27, 2010
100. Bucketing and Game Object State Vectors
• Theoretically, the state vectors of all game objects are updated
instantaneously and in parallel.
t1 t2 = t1 + ∆t
SA SA
SB SB
SC SC
Game Forum Germany 2010 49
Wednesday, January 27, 2010
101. Bucketing and Game Object State Vectors
• In practice, updates take time—object states can be inconsistent
during the update.
t1
SA
SB
SC
Game Forum Germany 2010 50
Wednesday, January 27, 2010
102. Bucketing and Game Object State Vectors
• In practice, updates take time—object states can be inconsistent
during the update.
t1 t2 = t1 + ∆t
SA SA
SB
SC
Game Forum Germany 2010 50
Wednesday, January 27, 2010
103. Bucketing and Game Object State Vectors
• In practice, updates take time—object states can be inconsistent
during the update.
t1 t2 = t1 + ∆t
SA SA
SB SB
SC
Game Forum Germany 2010 50
Wednesday, January 27, 2010
104. Bucketing and Game Object State Vectors
• In practice, updates take time—object states can be inconsistent
during the update.
t1 t2 = t1 + ∆t
SA SA
SB SB
SC
Game Forum Germany 2010 50
Wednesday, January 27, 2010
105. Bucketing and Game Object State Vectors
• In practice, updates take time—object states can be inconsistent
during the update.
t1 t2 = t1 + ∆t
SA SA
SB SB
SC SC
Game Forum Germany 2010 50
Wednesday, January 27, 2010
106. Bucketing and Game Object State Vectors
• Problems arise when we query the state of object A during the
update of object B or C.
t1 t2 = t1 + ∆t
SA SA
SB
SC
Game Forum Germany 2010 51
Wednesday, January 27, 2010
107. Bucketing and Game Object State Vectors
• Problems arise when we query the state of object A during the
update of object B or C.
t1 t2 = t1 + ∆t
SA SA
SB
query
SC
Game Forum Germany 2010 51
Wednesday, January 27, 2010
108. Bucketing and Game Object State Vectors
• Major contributor to the ubiquitous “one frame off bug.”
• Update ordering via bucketing helps, but only when the following
rule is adhered to:
An object in bucket b may only read the state
of objects in buckets (b - 1), (b - 2), ...
• You can’t reliably read the state of objects in your own bucket!
Game Forum Germany 2010 52
Wednesday, January 27, 2010
109. One Frame Off Bugs in Collision Queries
• Another complication arises because:
Collision and physics run after the game objects have updated.
So, the location of all collision geometry is one frame old when
we cast our rays/spheres.
Game Forum Germany 2010 53
Wednesday, January 27, 2010
110. One Frame Off Bugs in Collision Queries
• Another complication arises because:
Collision and physics run after the game objects have updated.
So, the location of all collision geometry is one frame old when
we cast our rays/spheres.
collision geo
is leR behind
Game Forum Germany 2010 53
Wednesday, January 27, 2010
111. One Frame Off Bugs in Collision Queries
• Another complication arises because:
Collision and physics run after the game objects have updated.
So, the location of all collision geometry is one frame old when
we cast our rays/spheres.
collision geo ray cast
is leR behind misses its target
Game Forum Germany 2010 53
Wednesday, January 27, 2010
112. One Frame Off Bugs in Collision Queries
• Problem:
What we really want is to update each game object’s collision
geometry with its bucket.
• Impossible, because coll/phys update is monolithic—happens
after all game objects have been updated.
Game Forum Germany 2010 54
Wednesday, January 27, 2010
113. One Frame Off Bugs in Collision Queries
• Problem:
What we really want is to update each game object’s collision
geometry with its bucket.
• Impossible, because coll/phys update is monolithic—happens
after all game objects have been updated.
... or is it?
Game Forum Germany 2010 54
Wednesday, January 27, 2010
114. One Frame Off Bugs in Collision Queries
• Observation:
Ray and sphere casts don’t care about the full collision/physics
update.
• Don’t need contact information, velocity, etc.
• Only need to know where the collision geo will be.
• Solution:
All we need to do is update the broadphase AABBs between
buckets.
Game Forum Germany 2010 55
Wednesday, January 27, 2010
115. Solving One Frame Off Bugs in Collision Queries
• Instead, we’ll update the AABBs early, in between game object
bucket updates.
This allows our collision queries to return correct results.
Game Forum Germany 2010 56
Wednesday, January 27, 2010
116. Solving One Frame Off Bugs in Collision Queries
• Instead, we’ll update the AABBs early, in between game object
bucket updates.
This allows our collision queries to return correct results.
collision geo
is leR behind
Game Forum Germany 2010 56
Wednesday, January 27, 2010
117. Solving One Frame Off Bugs in Collision Queries
• Instead, we’ll update the AABBs early, in between game object
bucket updates.
This allows our collision queries to return correct results.
broadphase
AABB update
Game Forum Germany 2010 56
Wednesday, January 27, 2010
118. Solving One Frame Off Bugs in Collision Queries
• Instead, we’ll update the AABBs early, in between game object
bucket updates.
This allows our collision queries to return correct results.
broadphase
AABB update
ray cast
hits
Game Forum Germany 2010 56
Wednesday, January 27, 2010
120. Game Console Architecture: Xbox 360
PowerPC PowerPC PowerPC
Core 0 Core 1 Core 2
L1 L1 L1 L1 L1 L1
Data Instr. Data Instr. Data Instr.
Shared L2 Cache (1 MB)
Unified RAM
GPU
(512 MB)
Game Forum Germany 2010 58
Wednesday, January 27, 2010
121. Game Console Architecture: PLAYSTATION 3
PPU SPU0 SPU1 SPU6
L1 L1
Data Instr. Local Store Local Store ... Local Store
L2 Cache (256 kB) (256 kB) (256 kB)
(512 kB)
EIB
DMAC
System RAM Video RAM
GPU
(256 MB) (256 MB)
Game Forum Germany 2010 59
Wednesday, January 27, 2010
122. Thinking About Modern Multiprocessor Hardware
• We’ll think in terms of multiple hardware threads.
• These could be provided by:
hyperthreaded CPU,
multiple cores,
SPUs on PS3/Cell.
Game Forum Germany 2010 60
Wednesday, January 27, 2010
123. Ways to Achieve Parallelism: Thread per Subsystem
game collision/
animaUon rendering audio
loop physics
Game Forum Germany 2010 61
Wednesday, January 27, 2010
124. Ways to Achieve Parallelism: Thread per Subsystem
game collision/
animaUon rendering audio
loop physics
Game Forum Germany 2010 61
Wednesday, January 27, 2010
125. Ways to Achieve Parallelism: Fork/Join
set up
Game Forum Germany 2010 62
Wednesday, January 27, 2010
126. Ways to Achieve Parallelism: Fork/Join
set up
fork
thread2
thread0 thread1 thread3
Game Forum Germany 2010 62
Wednesday, January 27, 2010
127. Ways to Achieve Parallelism: Fork/Join
set up
fork
thread2
thread0 thread1 thread3
process
join
results
Game Forum Germany 2010 62
Wednesday, January 27, 2010
128. Ways to Achieve Parallelism: Jobs
PPU SPU0 SPU1 SPU5
anim ray cast anim
pose pose
anim
ray cast pose anim
pose
game ...
broad broad
loop phase phase visibility
mesh
visibility visibility proc
Game Forum Germany 2010 63
Wednesday, January 27, 2010
129. Ways to Achieve Parallelism: Jobs
PPU SPU0 SPU1 SPU5
anim ray cast anim
pose pose
anim
kick ray cast pose anim
pose
game ...
broad broad
loop phase phase visibility
mesh
visibility visibility proc
Game Forum Germany 2010 63
Wednesday, January 27, 2010
130. Ways to Achieve Parallelism: Jobs
PPU SPU0 SPU1 SPU5
anim ray cast anim
pose pose
anim
kick ray cast pose anim
pose
game ...
broad broad
loop phase phase visibility
wait
mesh
visibility visibility proc
Game Forum Germany 2010 63
Wednesday, January 27, 2010
131. Ways to Achieve Parallelism: Jobs
• Job = [ (code + input data) → output data ]
Kick job = request job to be scheduled on a HW thread.
Must wait for job before processing its output data.
• If job is done, wait takes close to zero time.
• If job is not done, main thread (PPU) blocks until it is done.
• Job manager handles scheduling, allocates buffers in local store,
and coordinates DMAs.
Programmer specifies data sources & destinations via a DMA list.
Game Forum Germany 2010 64
Wednesday, January 27, 2010
132. Parallelism on the Train
while (!quit)
{
// ...
RayCastHandle hRayCast
= kickRayCast(...);
// do other useful work on PPU
// ...
waitRayCast(hRayCast);
processRayCastResults(hRayCast);
// ...
}
Game Forum Germany 2010 65
Wednesday, January 27, 2010
133. Parallelism on the Train
RayCastHandle hRayCast = INVALID;
while (!quit)
{
while (!quit)
// ...
{
// ...
RayCastHandle hRayCast
= kickRayCast(...); // wait and re-kick immediately
if (hRayCast.IsValid())
// do other useful work on PPU
{
// ...
waitRayCast(hRayCast);
processRayCastResults(hRayCast);
waitRayCast(hRayCast);
}
processRayCastResults(hRayCast);
hRayCast = kickRayCast(...);
// ...
// ...
}
}
Game Forum Germany 2010 65
Wednesday, January 27, 2010
134. Parallelism on the Train
• How does parallelism with jobs affect our train design?
Large-scale engine system updates and ray/sphere casts are
largely asynchronous in U2:AT.
• Main game loop (PPU) kicks jobs on SPU.
• Other work can proceed on the PPU while jobs are running.
• Results picked up later this frame... or next frame.
Game Forum Germany 2010 66
Wednesday, January 27, 2010
135. Parallelism on the Train
• Data must be compact and contiguous so it can be DMA’d to the
SPUs.
We’re doing this already, to maintain good cache coherency.
• The collision world must be locked in order to update broadphase
AABBs:
Wait until all outstanding ray/sphere jobs are done.
Lock.
Update AABBs.
Unlock.
Game Forum Germany 2010 67
Wednesday, January 27, 2010
136. Parallelism on the Train
• We can even do our broadphase AABB updates asynchronously.
PPU Update Bucket 0
Game Forum Germany 2010 68
Wednesday, January 27, 2010
137. Parallelism on the Train
• We can even do our broadphase AABB updates asynchronously.
Kick
PPU Update Bucket 0
BP Job
Game Forum Germany 2010 68
Wednesday, January 27, 2010
138. Parallelism on the Train
• We can even do our broadphase AABB updates asynchronously.
Kick
PPU Update Bucket 0
BP Job
Broadphase
SPU AABB Job
Game Forum Germany 2010 68
Wednesday, January 27, 2010
139. Parallelism on the Train
• We can even do our broadphase AABB updates asynchronously.
Kick
PPU Update Bucket 0
BP Job
Update Bucket 1
Broadphase
SPU AABB Job
Game Forum Germany 2010 68
Wednesday, January 27, 2010
140. Parallelism on the Train
• We can even do our broadphase AABB updates asynchronously.
query/cast
Kick
PPU Update Bucket 0
BP Job
Update Bucket 1
Broadphase
SPU AABB Job
Game Forum Germany 2010 68
Wednesday, January 27, 2010
142. Conclusions
• The combination of modern hardware restrictions and the
problems generated by the train level...
forced us to design our engine in a robust and efficient manner.
• Results:
U2:AT boasts near 100% hardware utilization on PS3
(PPU + all 6 SPUs).
Way-cool train level as pay-off for all the hard work.
Technology was the primary enabler for the convoy level as well.
Game Forum Germany 2010 70
Wednesday, January 27, 2010
143. Thanks For Listening!
• Free free to send questions to me at:
jason_gregory@naughtydog.com
Game Forum Germany 2010 71
Wednesday, January 27, 2010