1. GDC2011 | Jubok Kim, Choong-Hyo Kim
Implementation and Application of the
Real-Time Helper-Joint System
Ⓒ 2011 NEXON Corporation & devCAT Studio. All Rights Reserved
M2 team, Game Development Team for Project M2 in longCAT (The 3rd New Development Division in NEXON Corp.). M2 team Director is Kim, Dong-Gun | Project M2 is produced by Kim, Dong-Gun
GT-R team, Engine Development Team for Project M2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

2. Jubok Kim
Technical Director of “Mabinogi 2”
Worked as a game programmer for 10 years in Nexon
eias@nexon.co.kr
http://twitter.com/eiaserinnys
Choong-Hyo Kim
Technical Art Director of “Mabinogi 2”
Worked as a 3D artist for 10 years in Nexon
uc2612@nexon.co.kr
http://twitter.com/siyoskii

3. Nexon?
Nexon is the first Korean online game company to direct its attention to the
overseas market
Cartoon Rendered-MMORPG,
Mabinogi
A Casual MMORPG
MapleStory MMO-Action
Vindictus
(Mabinogi:Heroes in Korea)

4. Mabinogi 2
“Mabinogi 2” is the third one of the series and the official sequel to “Mabinogi.” We
are aiming very high quality bar of graphics and animation unlike its casual-looking
prequel

5. What we’re gonna talk.
1. Real-time helper-joint system
Problem definition
Study about helper-joints
Replication of helper-joints
2. Layering the helper-joints
Concept / Authoring conventions
Implementation
3. Optimization
4. Conclusion
Pros/Cons/Considerations
**Annotation slides

6. Problem
Definition

7. Candy-wrap problem
Basic bone structure + Skinning = Twisted wrist
You will see problems like this even at shoulder or other complex joint

8. Twist bone is a solution
Measure and calculate the average angle between forearm and wrist to get the
rotation of the twist bone

9. Helper-Joint
A joint which measures the movement of other joints and settle the movement of
itself
Many DCC tools provide features which are
actually helper-joints
Popular controllers in 3DS MAX :
Position Constraint
Look At Constraint
Orientation Constraint…

10. So, what’s the problem left?
All we’ve got helper-joints in our DCC tools already.

11. Hard to modify behavior
All animation asset should re-baked if you change behavior of a helper-joint
It becomes more and more
painful to modify behavior of the
skeletal structure in the later
stage of development because
animation asset size is growing
bigger and bigger

12. Procedural Motion
Helper-joint does not react properly to procedural animation like ragdoll or full-
body IK
http://forum.unity3d.com/threads/51805-Additive-animations-mess-up-skinning

13. Customization
Hard to author and modify costumes if you need to bake all the animation for them.
(Most of typical MMOGs have rich character customization feature)
Costumes from Vindictus, Nexon

14. We needed Helper-Joint which
does not require pre-baked animation
Let’s call it “Real-Time Helper-Joint”

15. Expected effects
Expected effects with Real-Time Helper-Joint system are…
Flexibility of the content creation pipeline
Smaller package size to distribute
Proper reaction to procedural animation
Low cost for authoring various and complex
costumes

16. Problems predicted
We predicted some problems before beginning to build the system
Inexperience
Artists were not experienced to the concept of Helper-Joint. We did not use
Helper-Joint for the previous game, and materials are very rare in Korea.
Hardness of replication
We did not know even if it was possible to replicate arbitrary Helper-Joint in
DCC tool. There was a high chance of failing if we try to replicate too
complex and unpredictable Helper-Joint.
Performance Issue
Even if we succeeded to replicate Helper-Joint in the game code, can we
get enough performance to process many characters in the crowd scene of
an MMOG?

17. Solving Strategy
We prepared clear strategies for each predicted problems before starting to build
the system
for Inexperience
Technical artist decided to study about human skin changes rather than
muscle movement, because Helper-Joint is about skinning
for Hardness of replication
Technical artist and programmer reached an agreement to maintain the list
of Helper-Joints of 3DS MAX compact by choosing easy and intuitive ones
for Performance Issue
Programmer decided to adopt simple and rigid architecture to support
multi-threading

18. Study about
Helper-Joints

19. Where do we need it?
We collected cases which is hard to set up skinning without Helper-Joint first
Usually hard part to set up

20. Parts requiring twist bone

21. Parts with caution for
rubbery joint problem

22. Parts affect to attractive character silhouette

23. Focused on female shoulder
Female shoulder was the best study subject with all the reviewed cases
Why shoulder?
Shoulder has 3 degrees of freedom
Most skinning issues get resolved if we
solve the issues around shoulder

24. Focused on female shoulder
Female shoulder was the best study subject with all the reviewed cases
Why female?
Expression of subtle silhouette of a female character is much more
harder than tough expression of muscles of a male character

25. More early decisions
We made some more decisions before start to set up female shoulder
3DS MAX native features only
Maintenance cost of the system might grow bigger,
if we develop our own Helper-Joint system
(what if there is 3DS MAX upgrade?)

26. More early decisions
We made some more decisions before start to set up female shoulder
Focus on movement of skin only
The system would be an over-engineered one if we have studied and
replicated the movement of muscles

27. Rotation around the longitudinal axis

28. Rotation with the arm forward

29. Rotation with the arm forward

30. Rotation with the arm upward
Noticeable crease of the skin surface

31. Basic Components
Basic components to settle movement of a Helper-Joint
Animation of the base framework
Nodes for Measurement
Controllers for intermediate calculation

32. No detail for the rig
Will not talk about it right now.
There’s No Fancy technique.
Keep it simple.

33. Final rig
Preview with 3ds max

34. Demonstration
Video 1
This demonstration shows the our final rig with
3DS MAX native Helper-Joints

36. Final list to replicate
We determined the minimal list of Helper-Joints to replicate by the study
PositionConstraint
LookAtConstraint
OrientationConstraint
ListController (S/R/T)
ReactionController (S/R/T/Float)
ExpressionController (S/R/T/Float)
WireParameter
we decided not to use WireParameter later, because it is too slow and hard to modify
ExposeTM

37. Replication of
Helper-Joints

38. PositionConstraint
Let’s start with the simplest Helper-Joint in the list mentioned before
Simply adds and calculates
the average of position targets
multiplied by weight

39. Preparation
Technical artist part
Sample Scene
Technical artist created a scene 2~3 bones using position constraints and
export its setting as an XML file by MAXScript

40. Preparation
Programmer part
Interface Definition
Programmer defined the simple interface for Helper-Joint and integrate it
into the existing animation system
struct HelperJointResult
{
enum Type { Scale, Rotation, Position, };
Type type;
Vector4 result;
};
class IHelperJoint
{
virtual void Process(CurrentPose& pose) = 0;
virtual const HelperJointResult& GetResult() const = 0;
};

41. Pseudo-code
Really easy one, huh?
class PositionConstraint : public IHelperJoint
{
virtual void Process(const CurrentPose& pose)
{
Calculate the matrix T which transforms
a ‘world space coordinate’ into a ‘parent space coordinate’
Calculate the average position P of the positions of
target bones multiplied by weight in the world space
Transform P by T and store it
}
virtual const HelperJointResult& GetResult() const
{ return P; }
};

42. Pseudo-code
Really easy one, huh?
class PositionConstraint : public IHelperJoint
{
virtual void Process(const CurrentPose& pose)
{
Calculate the matrix T which transforms
a ‘world space coordinate’ into a ‘parent space coordinate’
Calculate the average position P of the positions of
target bones multiplied by weight in the world space
Transform P by T and store it
}
virtual const HelperJointResult& GetResult() const
{ return P; }
};

43. Life isn’t that simple…
3DS MAX calculates and stores the differences between bind pose and the weighted
average of target positions in the bind pose, and add them to later calculation
results if ‘Keep Initial Offset’ option is checked
Bone position from
PositionAtConstraint
Actual bind position
Position adjusted by
‘Keep initial offset’ option

44. Introducing preprocess step
Preprocess() function has been added to IHelperJoint interface to handle the ‘Keep
Initial Offset’ option
class PositionConstraint : public IHelperJoint
{
virtual void Preprocess(const BindPose& bindPose)
{
if ‘keep initial offset’ option is cheked
{
Run Process() with bind pose and store the result in P_base
Calculate the differences of position between P_base and
actual bind pose and store the results in Offset
}
}
virtual void Process(const CurrentPose& pose)
{
Calculate the matrix T which transform a ‘world space coordinate’ into a ‘parent space coordinate’
Calculate the averaged position P of the positions of target bones
multiplied by weight in the world space
Transform P by T and store it
if ‘keep initial offset’ option is cheked
Add the preprocessed Offset to P
}
}

45. Replicate the remainder
It is straightforward to replicate except non-intuitive options like ‘Keep Initial Offset’
Detailed implementation note of other
Helper-Joints will be added in appendix

46. PositionConstraint
LookAtConstraint
OrientationConstraint
ListController (S/R/T)
ReactionController (S/R/T/Float)
ExpressionController (S/R/T/Float)
WireParameter
ExposeTM
Cool, it’s all xxxxing done
All replications are done, but this is too easy,
and bad premonition always proved right…

47. Unexpected problem
The order of evaluation was not a straightforward one
0 In general animation
calculation process,
if you calculate the
1 8 world transform in order,
accurate result will be
guaranteed
2 5 9 10
3 4 6 7 11 12 13 14

48. 0
Unfortunately,
this simple and easy
1 8 solution does not work
LookAt
Constraint with Helper-Joint
LookAt
2 5 9 Target 10 A Helper-Joint can
refer to the bone which
is not its own ancestor
3 4 6 7 11 12 13 14

49. A naïve solution?
The order of evaluation can be calculated by traversing Helper-Joints in postorder
To traverse Helper-Joints, we should figure out their dependency first
/* Let’s add a function to the IHelperJoint interface to return the
list of bone indices which the Helper-Joint referred to */
class LookAtConstraint
{
virtual void GetDependency(
vector<BoneIndex>& dependency)
{
Add indices of look-at targets to dependency
Add index of upnode to dependency
}
/* Other functions */
}

50. is not a rigid solution…
We have a lot of Helper-Joints to implement, so more solid and human error proof
solution is required
/* What if there is more complicated Helper-Joint like an expression controller? */
/* Furthermore, how do I validate and convince the dependency list, and evaluate
that order is correct? */
class ReactorExpressionBinary {
void GetDependancy(std::vector<int>& dependancies);
};
class ReactorExpressionID {
void GetDependancy(std::vector<int>& dependancies);
};
class ReactorExpressionFunction {
void GetDependancy(std::vector<int>& dependancies);
};
class ReactorExpressionConstant {
void GetDependancy(std::vector<int>& dependancies);
};

51. Concrete solution
Use evaluation step itself as dependency traversal step
Each implementation of Helper-Joint is modified not to access the
animation pose array directly. Instead, Helper-Joint uses
IHelperJointPoseSource whenever it requires pose of other bones
PositionConstraint
IHelperJointPoseSource
LookAtConstraint GetWorldTranslation(int boneIndex)
OrientationConstraint GetWorldRotation(int boneIndex)
GetWorldScale(int boneIndex)
PositionReactionController GetLocalTranslation(int boneIndex)
GetLocalRotation(int boneIndex)
RotationReactionController
GetLocalScale(int boneIndex)
ScaleReactionConstraint GetWorldTransform(int boneIndex)
…
FloatWireConstraint

52. A special implementation of IHelperJointPoseSource -
EvaluationOrderBuilder is used when order of evaluation is
calculated.
PositionConstraint
9, 10
OrientationConstraint Assumes that bones with index less
12 than 11 are already evaluated
ScaleXYZ
bone11 EvaluationOrderBuilder :
public IHelperJointPoseSource
PositionXYZ
… 10
LookAtConstraint
24
ScaleXYZ
bone12

53. Bone 9, 10 are already
evaulated, so they can
PositionConstraint be referenced without
9, 10 problem.
OrientationConstraint
12
ScaleXYZ
bone11 EvaluationOrderBuilder :
public IHelperJointPoseSource
PositionXYZ
… 10
LookAtConstraint
24
ScaleXYZ
bone12

54. PositionConstraint Bone 12 is not traversed yet.
9, 10 So evaluation of bone 11 is holded,
and evaluation of bone 12 begins
OrientationConstraint instead.
12
ScaleXYZ
bone11 EvaluationOrderBuilder :
public IHelperJointPoseSource
PositionXYZ
… 10
LookAtConstraint
24
ScaleXYZ
bone12

55. If a bone is calculated by pre-baked
animation, then it can be referenced
PositionConstraint anytime.
9, 10 Let’s assume bone 24 is settled by
pre-baked animation. Bone 24 is
OrientationConstraint assumed to be traversed now, and
12 added to the traversal order list.
ScaleXYZ
bone11 EvaluationOrderBuilder :
public IHelperJointPoseSource
PositionXYZ
… 10 24
LookAtConstraint
24
ScaleXYZ
bone12

56. PositionConstraint
9, 10
Bone 12 is evaluated, so add it to the
OrientationConstraint traversal list.
12
ScaleXYZ
bone11 EvaluationOrderBuilder :
public IHelperJointPoseSource
PositionXYZ
… 10 24 12
LookAtConstraint
24
ScaleXYZ
bone12

57. PositionConstraint
9, 10
OrientationConstraint Now bone 11 can reference bone 12
12 safely.
ScaleXYZ
bone11 EvaluationOrderBuilder :
public IHelperJointPoseSource
PositionXYZ
… 10 24 12
LookAtConstraint
24
ScaleXYZ
bone12

58. PositionConstraint
9, 10
OrientationConstraint Bone 11 is evaluated now, so add it
12 to the list too
ScaleXYZ
bone11 EvaluationOrderBuilder :
public IHelperJointPoseSource
PositionXYZ
… 10 24 12 11
LookAtConstraint Bone 12 is already evaluated, so
24 evaluate bone 13 next.
ScaleXYZ
bone12

59. Demonstration
Video 2
This demonstration shows the basic implementation result.
(Shoulder, Knee, reaction to IK)

61. Layering
Helper-Joints

62. Customization
Let’s extend the realtime Helper-Joint system to character customization
MMOGs have many kinds of costumes
Many of them require its own animation.
And costumes can be changed at any time.
Too Big to bake them out
We can’t export all the costume animations.
Single base animation
…for all the costumes.

63. Layering skeletal structure
We introduced the concept of layer of the skeletal structure
Most pre-baked animation Contains Helper-Joint settings Contains Helper-Joint settings
Face and separate pose clips
contain poses for this layer of shoulder, knee and others of costumes
Body Layer Face & Hands Helper-Joint for body Helper-Joint for costume

64. Layering example
Basic concept for framework layering
Sets of various frameworks Nude body
Hand Long Skirt
Face
Short Skirt
Muscle
Mantle
Base

65. Layering example
Basic concept for framework layering
Sets of various frameworks Face Mesh
Hand Long Skirt
Face
Short Skirt
Muscle
Mantle
Base

66. Layering example
Basic concept for framework layering
Sets of various frameworks Robe Mesh
Hand Long Skirt
Face
Short Skirt
Muscle
Mantle
Base

67. Layering example
Basic concept for framework layering
Sets of various frameworks Short Skirt
Hand Long Skirt
Face
Short Skirt
Muscle
Mantle
Base

68. Authoring convention
We used a naming convention to determine the layer where a bone is contained
and what the bone is for
_0Base{B}#Spine1
Skin Weight Layer Pre-baked/Runtime Name
_ 0Base {B} # Spine1
_ 0Base
1Face {B} : Baked
+ 1Hand
1Muscle {D} : Deformable
~ 2Costume
2Tool

69. Authoring convention
We used a naming convention to determine the layer where a bone is contained
and what the bone is for
_0Base{B}#Spine1
Skin Weight Layer Pre-baked/Runtime Name
_ 0Base {B} # Spine1
_ 0Base
1Face {B} : Baked
+ 1Hand
1Muscle {D} : Deformable
~ 2Costume
2Tool

70. Authoring convention
We used a naming convention to determine the layer where a bone is contained
and what the bone is for
_0Base{B}#Spine1
Skin Weight Layer Pre-baked/Runtime Name
_ 0Base {B} # Spine1
_ 0Base
1Face {B} : Baked
+ 1Hand
1Muscle {D} : Deformable
~ 2Costume
2Tool

71. Authoring convention
We used a naming convention to determine the layer where a bone is contained
and what the bone is for
_0Base{B}#Spine1
Skin Weight Layer Pre-baked/Runtime Name
_ 0Base {B} # Spine1
_ 0Base
1Face {B} : Baked
+ 1Hand
1Muscle {D} : Deformable
~ 2Costume
2Tool

72. Authoring convention
A layer should not be cyclic, or bi-directional
(Frm Layer 0)
_Layer0{B}#AAA
(Layer 0 + Layer 1) (Animation Data)
_Layer0{B}#BBB
_Layer0{B}#AAA _Layer0{B}#AAA
_Layer0{B}#BBB _Layer0{B}#BBB
_Layer1{B}#CCC _Layer1{B}#CCC
(Frm Layer 1)
_Layer0{B}#BBB
_Layer1{B}#CCC

73. Export and import
Exporting from and importing back into the DCC tool is done layer by layer
AAA.max
AAA.Mesh AAA.Layer0.Framework AAA.Layer0.Animation
Pre-baked Layer
“I’ll use these layers.” AAA.Layer1.Framework
AAA.Layer1.HelperJointSetting
AAA.Layer2.Framework
AAA.Layer2.HelperJointSetting

74. Framework template
We maintained a list of typical framework settings as templates
Predefined framework templates
Popular fantasy costumes like skirts, and robes
Delivered to the outsourcing company
Low cost for modification
Behavior of the procedural layers can be changed at any time with low cost
Just re-export the helper-joint setting for the layer
Great benefit to quality control

75. Implementation
Animation pose player and Helper-Joint processor should be modified to handle
framework layers
Framework class is introduced
A Framework instance maintains current stack of layers and bind poses
AnimationPose class is introduced
A AnimationPose instance contains current bone poses and flows through each
animation module.

76. One animation player for one layer
There can be multiple instances of animation players but there is one AnimationPose instance only. Each
animation player matches the bone name to the framework to get the indices to fill the final pose array in
AnimationPose instance
One Helper-Joint processor for one layer
There can be multiple instances of Helper-Joint processors and works similar to an animation player.
Actually, the animation player and the Helper-Joint processor are implemented as a node in the general
module based on the animation system
Pose Pose
SRT (Layer0) SRT (Layer0)
SRT SRT
AnimationPlayer SRT SRT
HelperJointProc
Bind pose Bind Pose
AdvanceProcessor HelperJointLayerProc
0Logic Bind pose Bind Pose
ILayerPlayer HelperJointLayerProc
SRT (Layer1) SRT (Layer1)
1Base ILayerPlayer HelperJointLayerProc
SRT SRT
ILayerPlayer
2Hand SRT SRT
Bind pose SRT (Layer3)
Bind pose SRT

77. Demonstration
Video 3
This demonstration shows several costumes
which have their own Helper-Joint setting

79. Optimization

80. Unexpected problem again
Not like that we expected, multi-threading was an easy part
Multithreading was an easy part
Data parallel approach and Intel TBB did all the work
Thanks, Intel!
World transform calculation
Performance loss on scale calculation

81. World transform calculation
Usually, multiplying local transform with parent’s world transform occurs once for a
bone in a frame
When the
The Helper-Joint evaluation
calculation result of a process updates local poses
Helper-Joint is applied… of bones almost randomly
To make things worse,
implementation of Helper-
Joints require world
transform of other bones in
many cases
then all world
transform of entire
sub-tree are
invalidated

82. Typical character framework contains
100~200 bones
So performance hit will be drastic if
you re-calculate world transform of all
bones whenever world transform of a
bone is referenced

83. If a world transform for a bone is calculated, we would set the dirty
mask for the bone
If local bone pose of a bone is updated, we would reset the dirty
mask of the bone and its descendants
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Local pose of bone 8 is
0 0
1 8 updated, then reset the dirty
234567 9 10 11 12 13 14 mask of the bone and its
descendants
01 08
34 2 5 9 12 13 14
012 0 8 12
3 4 6 7 10 11 13 14

84. Dirty mask
This performance problem can be simply solved if each entry in the world transform
array has dirty mask
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 All entries in the world
transform array have
0
Ancestor of
bone 8 is 0 0
dirty mask
234567 1 only 8 9 10 11 12 13 14
Decendants of All bones have list of
bone 8 is 9~14
their own ancestors
01
34 2 5 9 12
08
13 14
and descendants. These
lists are maintained by
Framework instance
012 0 8 12
3 4 6 7 10 11 13 14

85. If world transform of a bone is referred when its dirty mask is reset,
we would check and calculate world transforms of its ancestors and
the bone itself
It can be done recursively without list of ancestors and descendants, but there is performance
hit by function call in that case
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
0 0
234567 1 8 9 10 11 12 13 14
When we need world
01 08 transform of bone 12, then
34 2 5 9 12 13 14 update ancestors of bone 12
and bone 12 itself only
012 0 8 12
3 4 6 7 10 11 13 14

86. Improved, but not enough
The result of performance measurement showed that much more time consumed
than expected time from the complexity of Helper-Joint calculation code
L2 cache miss was the cause
Fine measurement revealed that L2 cache miss is the cause
Helper-Joint is evaluated in random order, and references to world transform
of other bones occur in random and sporadic manner

87. Access should be gathered
Accesses to memory should be predicted and gathered
Calculating the order of reference
The order of referencing to world transform is calculated in similar way to
the way order of evaluation of Helper-Joint is calculated
Ready world transforms in advance
Calculate world transform in the order before evaluating a Helper-Joint so
the world transform can be loaded in L2 cache
Prefetch worked well
many L2 cache hot spots have been removed by this approach by
combination with some prefetch instructions

88. Performance loss on scale
Scale transform is just one multiplication step and does not impact the performance
in usual animation system, but again, it is a different story from the real-time
Helper-Joint system
Getting SRT from a matrix is
expensive too
Many Helper-Joint
requires inverse matrix
of world transform of
arbitrary bone
Calculate inverse transform
of a matrix is expensive.

89. But non-trivial scale is rare
If scale is trivial, inverse transform calculation becomes getting transpose
Getting SRT from a matrix becomes much simpler too
R-1 = RT
SRT(M) =
{
(1, 1, 1),
QuaternionFromMatrix(M),
(M[3][1], M[3][2], M[3][3]
}

90. Scale mask
Scale mask which is similar to world transform dirty mask introduced into
AnimationPose and array of world transform
We would set scale mask for the
bone if local pose of a bone
contains non-trivial scale
Getting SRT from a world
transform is replaced by matrix
into quaternion conversion if
scale mask is not set.
Matrix inversion is replaced by
matrix transpose if scale mask
is not set.

91. Result is good
Performance gain comes from the fact that animation containing non-trivial scale is
very rare
Further optimization is possible for the cases
with non-trivial scale.
For example, code calculating x component
transformed by inverse of parent’s world
transform can be reduced into one line.

92. Performance chart
Here goes actual performance chart for current implementation
Avg count in Time per
HelperJoint Total Clocks No.of Calls Time per Call (ms) one char one char (ms)

93. Conclusion

94. pros.
Animation asset maintenance became easier
Polishing the behaviors of Helper-Joint became easier
Authoring high quality costumes became easier
Variance in monsters can be introduced easily with
additional Helper-Joint layer
Package size to distribute got reduced

95. cons.
Riggers should get more familiar to the concept of real-
time Helper-Joint system
Relation between costume and Helper-Joint layer
introduces another complexity in asset definition
management
Performance hit is not ignorable

96. Considerations
Do enough research before you begin to implement. Avoid
over design
Make the best use of native features of DCC tools
Maintenance will be hard if there are too many in-house tools
Runtime Helper-Joint system has a very different runtime
execution path
Integrating it into the existing system needs careful consideration

97. Considerations
Set a standard of expression and performance which artist
and programmer both can reach an agreement

98. Jubok Kim
eias@nexon.co.kr | twitter.com/eiaserinnys (Korean)
Choong-Hyo Kim
uc2612@nexon.co.kr | twitter.com/siyoskii (Korean)
Q&A

99. Thank you

100. ADDITIONAL NOTE
Replicating 3DS MAX Native Controllers
LookAtConstraint | OrientationConstraint | ReactionController | ExpressionController&FloatWire | XYZ&ListControllers
Ⓒ 2011 NEXON Corporation & devCAT Studio. All Rights Reserved
M2 team, Game Development Team for Project M2 in longCAT (The 3rd New Development Division in NEXON Corp.). M2 team Director is Kim, Dong-Gun | Project M2 is produced by Kim, Dong-Gun
GT-R team, Engine Development Team for Project M2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

101. Make it look at targets
LookAtConstraint first calculates the weighted average of look-at targets, then
makes the constrainted bone to point the position

102. Relatively easy to replicate
LookAtConstraint works in a very clear and intuitive manner, and the first version of
replication is also simple
void LookAtConstraint::Process(/* Arguments omitted */)
{
Calculate the transform T doing [World space]→[Parent space]
Tranform the positon of constrainted bone into its parental space with T
Calculated the weighted average position of targets P
Transform the P with T, and calculate the forward vector F
Get the upward vector U by transform the upward target with T
Calculate the result matrix from F and U, and convert it into quaternion
}

103. Prepare the basis to look at
LookAtConstraint acually works in the world space, but results in local space
It is favorable to do the calcuation with the inverse of parent’s world transform
void LookAtConstraint::Process(/* Arguments omitted */)
{
Calculate the transform T doing [World space]→[Parent space]
Tranform the positon of constrainted bone into its parental space with T
Calculated the weighted average position of targets P
Transform the P with T, and calculate the forward vector F
Get the upward vector U by transform the upward target with T
Calculate the result matrix from F and U, and convert it into quaternion
}

104. Finding the forward vector
Calculate the weighted average position of targets, and transform it with the basis
transform T to find the forward vector
void LookAtConstraint::Process(/* Arguments omitted */)
{
Calculate the transform T doing [World space]→[Parent space]
Tranform the positon of constrainted bone into its parental space with T
Calculated the weighted average position of targets P
Transform the P with T, and calculate the forward vector F
Get the upward vector U by transform the upward target with T
Calculate the result matrix from F and U, and convert it into quaternion
}

105. Finding the upward vector
Simple and straight?
void LookAtConstraint::Process(/* Arguments omitted */)
{
Calculate the transform T doing [World space]→[Parent space]
Tranform the positon of constrainted bone into its parental space with T
Calculated the weighted average position of targets P
Transform the P with T, and calculate the forward vector F
Get the upward vector U by transform the upward target with T
Calculate the result matrix from F and U, and convert it into quaternion
}

106. …takes a lot of care
There are so many options for upnode control

107. Finding the result rotation
We’ve get the forward and upward vectors, so cross them to get the result rotation
Does this concludes the replication of LookAtConstraint?
void LookAtConstraint::Process(/* Arguments omitted */)
{
Calculate the transform T doing [World space]→[Parent space]
Tranform the positon of constrainted bone into its parental space with T
Calculated the weighted average position of targets P
Transform the P with T, and calculate the forward vector F
Get the upward vector U by transform the upward target with T
Calculate the result matrix from F and U, and convert it into quaternion
}

108. Some options left in rollout
Wow, the “Keep Initial Offset” checkbox AGAIN!

109. Preserving the difference
The “Keep Initial Offset” option of LookAtConstraint works similar to the same
option of PositionConstaint
The bone which is
constrainted by
LookAtConstraint

110. …concludes the replication
The final replication which calculates and preserves the difference by “Keep Initial
Offset”
void LookAtConstraint::Preprocess()
{
Preserve the difference of rotation between the bind pose and
the result with Process() only when “Keep Initial Offset” is checked
}
void LookAtConstraint::Process(/* Arguments Omitted */)
{
/* Ommitted */
Add the preserved rotation O to F when the “Keep Initial Offset” is checked
Calculate the result matrix from F and U, and convert it into quaternion
}

111. ADDITIONAL NOTE
Replicating 3DS MAX Native Controllers
LookAtConstraint | OrientationConstraint | ReactionController | ExpressionController&FloatWire | XYZ&ListControllers
Ⓒ 2011 NEXON Corporation & devCAT Studio. All Rights Reserved
M2 team, Game Development Team for Project M2 in longCAT (The 3rd New Development Division in NEXON Corp.). M2 team Director is Kim, Dong-Gun | Project M2 is produced by Kim, Dong-Gun
GT-R team, Engine Development Team for Project M2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

112. Takes the average rotation
OrientationConstraint calculates the weighted average of rotations of targets

113. The simplest one ever?
It even looks like that it is simpler than PositionConstraint
void OrientationConstraint::Process(/* Omitted */)
{
Get local rotations of targets
Calculate the weighted average of the rotations
}

114. No, it is not a simple one
It has the “Keep initial offset” checkbox too, and a very complex option which
controls blend method
void OrientationConstraint::Preprocess()
{
Preserve the difference of rotation D between the bind pose and
the result with Process() only when “Keep Initial Offset” is checked
}
void OrientationConstraint::Process(/* Omitted */)
{
if (“Transform rule” is “Local to Local”)
Preserve the local rotations of targets
else // “World to World”
Transform the rotation of targets to the parental space and preserve them
Calculate the weighted average of preserved rotations
Add preserved difference D to the average when “Keep initial offset” is checked
Preserved the result
}

115. Usual “Keep Initial Offset”
This option works in the same manner to LookAtConstraint
void OrientationConstraint::Preprocess()
{
Preserve the difference of rotation D between the bind pose and
the result with Process() only when “Keep Initial Offset” is checked
}
void OrientationConstraint::Process(/* Omitted */)
{
if (“Transform rule” is “Local to Local”)
Preserve the local rotations of targets
else // “World to World”
Transform the rotation of targets to the parental space and preserve them
Calculate the weighted average of preserved rotations
Add preserved difference D to the average when “Keep initial offset” is checked
Preserved the result
}

116. Strange “Transform Rules”
This option betrays the usual concept of animation blending
void OrientationConstraint::Preprocess()
{
Preserve the difference of rotation D between the bind pose and
the result with Process() only when “Keep Initial Offset” is checked
}
void OrientationConstraint::Process(/* Omitted */)
{
if (“Transform rule” is “Local to Local”)
Preserve the local rotations of targets
else // “World to World”
Transform the rotation of targets to the parental space and preserve them
Calculate the weighted average of preserved rotations
Add preserved difference D to the average when “Keep initial offset” is checked
Preserved the result
}

117. Blending what you “see”
This option is needed because target bones generally have different parents, so
blended result can be very different to the result which an artist expects
15 degree rotated
(30 degree in world)
Child
“Local→Local” case
A bone which is (15 + 15) / 2 = 15
constrainted by World→World case
Parent OrientationConstraint (15 + 30) / 2 = 22.5 degree
with “World→World”
transform rule
15 degree rotated
(15 degree in world)

118. ADDITIONAL NOTE
Replicating 3DS MAX Native Controllers
LookAtConstraint | OrientationConstraint | ReactionController | ExpressionController&FloatWire | XYZ&ListControllers
Ⓒ 2011 NEXON Corporation & devCAT Studio. All Rights Reserved
M2 team, Game Development Team for Project M2 in longCAT (The 3rd New Development Division in NEXON Corp.). M2 team Director is Kim, Dong-Gun | Project M2 is produced by Kim, Dong-Gun
GT-R team, Engine Development Team for Project M2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

119. Input-Graph-Output
All *ReactionManagers calculate the output value through the plotted graph
This UI is VEEEEEEERY unintuitive
DO SOMETHING, AUTODESK!

120. Too many input types!
You need to standarize the way to retrieve all the possible inputs
Local Rotation X/Y/Z
World Rotation X/Y/Z
Local Position X/Y/Z
World Position X/Y/Z
Distance

121. Abstract the input at first
The main function of the *ReactionManager is the “float →Graph →result”
transformation
DistanceEvaluator
PositionEvaluator RotationEvaluator

122. PositionEvaluator
This class handles input types “Local Position X/Y/Z” and “World Position X/Y/Z”
const float PositionEvaluator::Process(/* Omitted */)
{
if (demands the position in local space)
if (relative to the parent bone)
return (local translation)[channel]
else if (relative to a reference bone)
transform into the space of the reference bone
return (transformed translation)[channel]
else // reference is undefined
return (world translation)[channel]
else
return (world translation)[channel]
}

123. Relative to a reference?
Relative to the parent bone, and world space is very intuitive
But the correct replication requires translation relative to the reference bone
const float PositionEvaluator::Process(/* Omitted */)
{
if (demands the position in local space)
if (relative to the parent bone)
return (local translation)[channel]
else if (relative to a reference bone)
transform into the space of the reference bone
return (transformed translation)[channel]
else // reference is undefined
return (world translation)[channel]
else
return (world translation)[channel]
}

124. …comes from helper-bone
Helper-bone rollout have many options which are hard to implement
You should agree upon the options which would be supported by the replication

125. RotationEvaluator
This class handles input types “Local Rotation X/Y/Z” and “World Rotation X/Y/Z”
const float RotationEvaluator::Process(/* Omitted */)
{
if (demands the rotation in local space)
if (relative to the parent bone)
return (local rotation)[channel]
else if (relative to a reference bone)
transform into the space of the reference bone
return (transformed rotation)[channel]
else // reference is undefined, then returns in world space
return (world rotation)[channel]
else
return (world rotation)[channel]
}

126. Quaternion→EulerXYZ
It can be derived from a conversion of a quaternion to a matrix and decomposition
of a matrix to euler angles

127. DistanceEvaluator
This handles input type “Distance”
const float DistanceEvaluator::Process(/* Omitted */)
{
if (the reference bone is valid)
returns the distance between
the base bone and the reference bone
else
return the distance between
the base bone and the world origin
}

128. Graph lookup is easy
It can be implemented in the same way as a key frame animation lookup routine
const ReactorResult ReactionControllerBase::Process(/* Omitted */)
{
Find the largest master/slave index
whose master value is not larger than
the given master value in the sorted master/slave table
Interpolate the reation and the next to it
returns the result
}

129. ADDITIONAL NOTE
Replicating 3DS MAX Native Controllers
LookAtConstraint | OrientationConstraint | ReactionController | ExpressionController&FloatWire | XYZ&ListControllers
Ⓒ 2011 NEXON Corporation & devCAT Studio. All Rights Reserved
M2 team, Game Development Team for Project M2 in longCAT (The 3rd New Development Division in NEXON Corp.). M2 team Director is Kim, Dong-Gun | Project M2 is produced by Kim, Dong-Gun
GT-R team, Engine Development Team for Project M2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

130. Uses a raw expression
Both FloatWire and *ExpressionControl need parsing an expression

131. Replication is simple
…except you need to implement an expression parser
Interpreting the given expression every frame
would hurt runtime performance,
so you should build a parser for it
But explaining how to build a parser is beyond the scope,
so I will skip this part

132. Reuses the implementations
The implementations of ID of the expression are almost same as handlers of input
types of *ReactionManager
RotationEvaluator,
PositionEvaluator can be used
in these controllers again

133. ADDITIONAL NOTE
Replicating 3DS MAX Native Controllers
LookAtConstraint | OrientationConstraint | ReactionController | ExpressionController&FloatWire | XYZ&ListControllers
Ⓒ 2011 NEXON Corporation & devCAT Studio. All Rights Reserved
M2 team, Game Development Team for Project M2 in longCAT (The 3rd New Development Division in NEXON Corp.). M2 team Director is Kim, Dong-Gun | Project M2 is produced by Kim, Dong-Gun
GT-R team, Engine Development Team for Project M2 and more. GT-R Team Technical Director is Jeon, Hyeong-Kyu

134. Group of float controllers
We replicated 3 types of XYZ controllers - PositionXYZ, EulerXYZ, and ScaleXYZ

135. Need an explaination?
In special case, it performs radian to degree conversion
void EulerXYZ::Process(/* Omitted */)
{
if (is it dynamically changing?)
preserve the default value
foreach (controllers associated with X/Y/Z channel)
result[channel] = calculate the float controller
if (the controller is FloatReactionControl)
convert result[channel] to radian
convert the result euler angles into a quaternion
else
preserve the default value
}

136. Group of controllers
We replicated 3 types of lists – PositionList, RotationList, and ScaleList

137. Need an explaination? (2)
In special case, the calculation process changes!
void RotationList::Process(/* Omitted */)
{
foreach (controllers in the list)
[current] = result of the controller
if (controller is a *Constraint)
[result] = Slerp([result], [current], [weight])
else
[result] = [result] * [current]
}

138. Jubok Kim
eias@nexon.co.kr | twitter.com/eiaserinnys (Korean)
Choong-Hyo Kim
uc2612@nexon.co.kr | twitter.com/siyoskii (Korean)
EoD