This presentation provides first an abstract definition of Data Centric Architecture. Then it shows the mechanism and techniques that can be used to architect Data Centric Architectures with DDS.
4. Data-Centric Architecture (DCA)
Abstract Definition
A Data Centric Architecture consists of:
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ Abstract Data Types: A, B, C, ...
☐ Arrows: f, g, h, ...
where arrows satisfy the following properties:
OpenSplice DDS
Given f: A → B, g: B → C, and h: C → D
then the following holds true:
∃ g ∘ f: A → C [Composition]
∃ 1A : A → A [Identity Arrow]
h ∘ (g ∘ f) = (h ∘ g) ∘ f [Associativity]
f ∘ 1A = f = 1B ∘ f [Unit]
A Data Centric Architecture is a Category
5. Data-Centric Architecture (DCA)
The abstract definition of a Data-Centric Architecture
formally captures its key properties:
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ The Abstraction Barrier is defined in terms of Abstract
Data Types
OpenSplice DDS
☐ The System Behaviour is specified in terms of
transformations (functions or arrows) over these
Abstract Data Types
From this emerges a natural relationship between DCA and
Functional Programming Languages
6. Example: Space Traveller [1/3]
☐ Suppose we want to create a
distributed game in which we have
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
(some) UFOs that are trying to
destroy our spaceship
☐ UFO can be our allies/enemies and
can be played by other players or
OpenSplice DDS
the computer
☐ Some UFOs, might just be space
travellers and should not be
destroyed
☐ The goal of the game is to destroy
the UFOs that have been classified
as threats
7. Example: Space Traveller [2/3]
☐ The DCA for this game could be described as follows
Copyright
2011,
PrismTech
–
All
Rights
Reserved.
(keeping it simple...)
Data Types Arrows
OpenSplice DDS
☐ FlyingObject ☐ animate: Dynamics → Dynamics
☐ Dynamics ☐ classify: FlyingObject x Dynamics → Classification
☐ Classification ☐ collide: [FlyingObject x Dynamics] → [Collision]
☐ Collision ☐ pilot: IO → Dynamics x FlyingObject
☐ fire: IO → FlyingObject x Dynamics
☐ display: FlyingObject x Dynamics x Classification x Collision
→ IO
8. Example: Space Traveller [3/3]
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
FlyingObject FlyingObject
IO pilot Dynamics animate Dynamics classify Classification
Dynamics Dynamics
OpenSplice DDS
FlyingObject
FlyingObject [FlyingObject] Dynamics
IO fire [Dynamics]
collide [Collision] display IO
Dynamics Classification
Collision
9. Data-Centric vs. Service-Centric
Data-Centric and Service-Centric architectures, such as SOA, Distributed
Objects, etc., differ in several dimensions:
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ Abstraction Barrier:
☐ Data-Centric architectures rely on data abstractions
☐ Service-Centric architectures rely on service abstractions
OpenSplice DDS
☐ Level of Coupling
☐ Data-Centric architectures rely solely on the shared knowledge of abstract data
types (agree on the “what”)
☐ Service-Centric architectures rely on the shared knowledge of operational
interfaces along with the knowledge of the service provider and of its existence
(agree on “what”, “who”, and “when”)
10. DCA Benefits
☐ Loose Coupling
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ Anonymicity (e.g. only things to be known are the data types
and not who produces/consumes them)
Composability (especially when combined with functional
OpenSplice DDS
☐
languages)
☐ Extensibility
☐ Ease of Integration
☐ Performance (easier to parallelize)
12. DCAs with DDS
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ DDS provides first class support for DCAs
☐ DCAs Abstract Data Types are captured in DDS via
Topics
OpenSplice DDS
☐ DDS does not provide primitive support for specifying
DCAs Arrows. These are defined by the architect using
her/his favourite formalism
13. DDS Topics
“net.gva.VehiclePosition”
☐ A Topic defines a typed data
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
stream Name
☐ A Topic has associated a data
type and QoS
Topic
Typ
S
OpenSplice DDS
Qo
☐ The Topic name, type and QoS DURABILITY,
e
DEADLINE,
defines the key functional and PRIORITY,
…
non-functional invariants
struct Position2D {
☐ Topics can be discovered or long
long
vid; //@Key
x;
locally defined };
long y;
15. Back to the Space Traveller
☐ In the next few slides,
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
we’ll explore the steps
required to design
extensible and efficient
OpenSplice DDS
data models with DDS
☐ We’ll use the Space
Traveller as our running
example
16. Space Traveller Topic Types
//@Nested //@Nested enum Classifier {
struct Bounds { struct Vector { UNKNOWN,
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
long width; long x; FRIEND,
long height; long y; THREAT};
}; };
struct Classification {
struct FlyingObject { struct Dynamics { long oid; // @key
long oid; //@Key long oid; //@Key Classifier kind;
OpenSplice DDS
long kind; Vector pos; };
Bounds bounds; Vector speed;
}; }; struct Collision {
long oid;
long coid;
};
17. From 2D to 3D
☐ Our original data model was designed for a 2D world
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ Let’s assume that we want to be able to run our game
in both 3D as well as 2D. In addition we don’t want to
OpenSplice DDS
break interoperability between the 3D and the (older)
2D version
☐ Afterall the 2D visualization is simply a projection of the
3D visualization
☐ How can we do this?
18. 2D to 3D w/o X-Types
☐ If we don’t want to break interoperability with existing games, as we are not going
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
to be able to upgrade them all, we can’t change the existing topic types
☐ As a result we need to define another topic type to capture the 3rd dimension. This
way the older version of the game will not notice the difference, while the new
version will be made aware of the additional topic-type providing the 3rd
OpenSplice DDS
dimension
//@Nested
struct Vector {
long x;
long y;
}; struct DynamicsEx {
long oid; //@Key
struct Dynamics { long z;
long oid; //@Key long dz;
Vector pos; };
Vector speed;
};
19. Extensibility w/o X-Types
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ Extending and evolving a DDS data model w/o the
use of X-Types introduces (1) fragmentation, (2) can
OpenSplice DDS
potentially introduce update anomalies (e.g.
inconsistency), and (3) adds complexity for both the
architect and the developer
20. X-Types Background Info
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ The DDS X-Types specification extends DDS with:
☐ A polymorphic structural type system
OpenSplice DDS
☐ A set of concrete syntaxes for expressing DDS Topic Types
☐ Extensible Type Encoding
☐ A Dynamic API for runtime creation and manipulation of
Topic Types, DataReaders and Writers
21. Types Assignability
☐ The main mechanism provided by the X-Types specification
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
to support interoperability among different types is the
concept of “assignability”
☐ The rules of assignability depends on the kind of extensions/
OpenSplice DDS
modifications allowed for the type:
☐ FINAL => Types are identical
☐ EXTENSIBLE => Types are one the monotonic extension of the other
☐ MUTABLE => Types can be projected w/o violating attribute
invariants
22. 2D to 3D with X-Types
☐ The first thing to observe from our current data model is that it does not
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
provide any extensibility annotations
☐ This means that by default it is considered “Extensible”, or to be more
precise, monotonically extensible
OpenSplice DDS
//@Nested
struct Vector {
long x;
long y;
};
struct Dynamics {
long oid; //@Key
Vector pos;
Vector speed;
};
23. Extensible X-Types (EXT)
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ Monotonically Extensible Types (EXT), as suggested by
their name, imply extensions only at the “tail” of the
type.
OpenSplice DDS
☐ Can we go from 2D to 3D using Monotonic Extensible
Types?
☐ Let’s give it a try ...
24. 2D to 3D with EXT? [1/3]
☐ Let’s add explicit extensibility annotations first, so that
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
we don’t get confused later on
//@Nested @Extensibility(EXTENSIBLE_EXTENSIBILITY)
struct Vector {
OpenSplice DDS
long x;
long y;
};
//@Extensibility(EXTENSIBLE_EXTENSIBILITY)
struct Dynamics {
long oid; //@Key
Vector pos;
Vector speed;
};
25. 2D to 3D with EXT? [2/3]
☐ Let’s now extend the Vector Type:
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
//@Nested @Extensibility(EXTENSIBLE_EXTENSIBILITY)
struct Vector {
long x;
long y;
OpenSplice DDS
long z;
};
☐ Everything is fine as this extension is “adding to the tail”
of our type
26. 2D to 3D with EXT? [3/3]
☐ The resulting types are now:
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
//@Nested @Extensibility(EXTENSIBLE_EXTENSIBILITY)
struct Vector {
long x;
long y;
long z;
};
//@Extensibility(EXTENSIBLE_EXTENSIBILITY)
OpenSplice DDS
struct Dynamics {
long oid; //@Key
Vector pos;
Vector speed;
};
☐ Wait a second... The monotonic-extension on the type Vector is
leading to a non-monotonic-extension of the type Dynamics!
☐ We need a “more extensible” Dynamics Type!
27. Mutable X-Types (MXT)
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ Mutable X-Types (MXT) provide the most generic
OpenSplice DDS
form of extensibility. Attributes can be added in any
order, and they can equally be removed
28. 2D to 3D with MXT? [1/3]
☐ As our extension to the Vector type is monotonic,
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
we can keep it as a EXT
OpenSplice DDS
//@Nested @Extensibility(EXTENSIBLE_EXTENSIBILITY)
struct Vector {
long x;
long y;
};
29. 2D to 3D with MXT? [2/3]
☐ The Dynamics type needs to be MXT so to allow non-
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
monotonic extensions.
//@Nested @Extensibility(EXTENSIBLE_EXTENSIBILITY)
struct Vector {
long x;
OpenSplice DDS
long y;
long z;
};
//@Extensibility(MUTABLE_EXTENSIBILITY)
struct Dynamics {
long oid; //@Key
Vector pos;
Vector speed;
};
30. 2D to 3D with MXT? [3/3]
☐ The resulting type ensures that a DataReader of the old
version can safely project the new types on the old types
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
//@Nested @Extensibility(EXTENSIBLE_EXTENSIBILITY)
struct Vector {
long x;
OpenSplice DDS
long y;
long z;
};
//@Extensibility(MUTABLE_EXTENSIBILITY)
struct Dynamics {
long oid; //@Key
Vector pos;
Vector speed;
};
☐ Yet, there are still a few issues. Can you see them?
31. Open Issues
☐ The “legacy” DataWriter will not match DataReaders for the new type as
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
the Vector type would have to be “widened” as opposed to “projected”.
This means that the 3D version of the game won’t see the flying objects on
the 2D game. While the 2D will see a projection of the 3D objects
OpenSplice DDS
☐ If we change the extensibility of the Vector type to MUTABLE we still have
an issue. How can we figure out, from the 3D application, when a “z” is not
available? We need this information to be able to assign a arbitrary flight
level to 2D UFOs as well as being able to distinguish when the “z” value is
really “0”
32. 2D to 3D with MXT Revised [1/2]
☐ To address the issues just noted, we can simply change the Vector extensibility type, and
also tag the z attribute as Optional
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ Optional attributes can have a special “nil” value that allow us to detect when the field
was not set
//@Nested @Extensibility(MUTABLE_EXTENSIBILITY)
struct Vector {
long x;
OpenSplice DDS
long y;
long z; //@Optional
};
//@Extensibility(MUTABLE_EXTENSIBILITY)
struct Dynamics {
long oid; //@Key
Vector pos;
Vector speed;
};
☐ As defined the types allow the level of interoperability required by our application. However,
our type definition is a bit loose as we are not defining the strong invariants...
33. 2D to 3D with MXT Revised [2/2]
☐ For mutable types it is important to identify which attributes represent the
strong invariants of the type. These are the attributes that should always be
present to allow assignability. These invariants are described via the
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
MustUnderstand annotation
//@Nested @Extensibility(MUTABLE_EXTENSIBILITY)
struct Vector {
long x; //@MustUndertand @Id(1)
long y; //@MustUndertand @Id(2)
OpenSplice DDS
long z; //@Optional @Id(3)
};
//@Extensibility(MUTABLE_EXTENSIBILITY)
struct Dynamics {
long oid; //@Key @Id(1)
Vector pos; //@MustUndertand @Id(2)
Vector speed; //@MustUndertand @Id(3)
};
☐ To allow attribute re-ordering/removal it is a good practice to explicitly assign
attributes Ids, otherwise those will be deduced by the order of declaration
thus impacting the extensibility of a type
34. Resulting Types
//@Nested @Extensibility(MUTABLE_EXTENSIBILITY)
//@Nested @Extensibility(MUTABLE_EXTENSIBILITY)
struct Bounds {
struct Vector {
long width; //@MustUndertand @Id(1)
long x; //@MustUndertand @Id(1)
long height; //@MustUndertand @Id(2)
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
long y; //@MustUndertand @Id(2)
long depth; //@Optional @Id(3)
long z; //@Optional @Id(3)
};
};
//@Extensibility(MUTABLE_EXTENSIBILITY)
//@Extensibility(MUTABLE_EXTENSIBILITY)
struct Dynamics {
struct FlyingObject {
long oid; //@Key @Id(1)
long oid; //@Key @Id(1)
Vector pos; //@MustUndertand @Id(2)
long kind; //@MustUndertand @Id(2)
OpenSplice DDS
Vector speed; //@MustUndertand @Id(3)
Bounds bounds; //@MustUndertand @Id(3)
};
};
// @Extensibility(FINAL_EXTENSIBILITY) // @Extensibility(FINAL_EXTENSIBILITY)
enum Classifier { enum Classifier {
UNKNOWN,FRIEND,THREAT}; UNKNOWN,FRIEND,THREAT};
// @Extensibility(FINAL_EXTENSIBILITY) // @Extensibility(FINAL_EXTENSIBILITY)
struct Classification { struct Classification {
long oid; // @key long oid; // @key
Classifier kind;}; Classifier kind;};
NOTE: Some Final types to demonstrate their use
35. Are we Done?
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ Our game has become very popular and friends play
with it across the internet
However, some people are complaining that in some
OpenSplice DDS
☐
cases we are consuming a bit too much bandwith,
especially when playing on narrow-band links
☐ How can we fix this problem?
36. Copyright
2012,
PrismTech
–
All
Rights
Reserved.
N’ouvre la bouche que lorque tu es sur que ce que tu
vas dire est plus beau que le silence.
OpenSplice DDS
[Proverbe Arabe]
37. Is our Data Model too Chatty?
☐ If we look at the Dynamics
type, it includes the position --
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
which will change regularly
and relatively often -- as well as //@Nested @Extensibility(MUTABLE_EXTENSIBILITY)
the speed vector -- which is struct Vector {
long x; //@MustUndertand @Id(1)
subject to change less often
OpenSplice DDS
long y; //@MustUndertand @Id(2)
long z; //@Optional @Id(3)
(especially for UFOs) };
//@Extensibility(MUTABLE_EXTENSIBILITY)
☐ As a consequence of mixing struct Dynamics {
long oid; //@Key @Id(1)
the same topic-type attributes Vector pos; //@MustUndertand @Id(2)
Vector speed; //@MustUndertand @Id(3)
with different update rates, we };
are sending more data than
we really should
38. Frequency Normalization
☐ To address this issue we can split the topic type into as many
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
topic types as necessary to ensure that attributes are updated
at similar rates
//@Extensibility(MUTABLE_EXTENSIBILITY) //@Extensibility(MUTABLE_EXTENSIBILITY)
OpenSplice DDS
struct FOPosition { struct FOSpeed {
long oid; //@Key @Id(1) long oid; //@Key @Id(1)
Vector pos; //@MustUndertand @Id(2) Vector speed; //@MustUndertand @Id(2)
}; };
☐ Notice that splitting the topic can lead to temporal inconsistencies,
thus either (1) these are acceptable to your application or (2) we
use coherent updates, or (3) we add sequence numbers to
correlate the corresponding positions/speeds
39. Delta Updates
☐ This splitting does not feel
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
completely right, does it? //@Nested @Extensibility(MUTABLE_EXTENSIBILITY)
struct Vector {
☐ It would be nice if the middleware long x; //@MustUndertand @Id(1)
long y; //@MustUndertand @Id(2)
could figure out what had long z; //@Optional @Id(3)
};
changed and only send deltas
OpenSplice DDS
//@Extensibility(MUTABLE_EXTENSIBILITY)
struct Dynamics {
☐ Delta-updates are not currently long oid;
Vector pos;
//@Key @Id(1)
//@MustUndertand @Id(2)
available in DDS, these will be Vector speed; //@MustUndertand @Id(3)
};
included in future versions of the
standard
☐ Is there a way we can emulate
delta-updates?
40. Emulating Delta Updates
☐ The @Optional annotation
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
can be used to emulate
//@Nested @Extensibility(MUTABLE_EXTENSIBILITY)
delta updates. struct Vector {
long x; //@MustUndertand @Id(1)
☐ This works since optional long y; //@MustUndertand @Id(2)
long z; //@Optional @Id(3)
attributes set to nil are never };
OpenSplice DDS
sent on the wire //@Extensibility(MUTABLE_EXTENSIBILITY)
struct Dynamics {
long oid; //@Key @Id(1)
☐ This comes at a cost, as it Vector pos; //@MustUndertand @Id(2)
requires some work -- ideally Vector speed; //@MustUndertand @Optional @Id(3)
};
embedded into a layer
sitting between the
middleware and the
application
42. DCA Benefits
☐ Loose Coupling
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
☐ Anonymicity (e.g. only things to be known are data types
and not who produces/consumes them)
Composability (especially when combined with functional
OpenSplice DDS
☐
languages)
☐ Extensibility
☐ Ease of Integration
☐ Performance (easier to parallelize)
43. My Advice on X-Types
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
Start using the X-Types annotation system immediately!
☐ DDS vendors are still working on the implementions of this
specification, yet comment-style annotations //@ will be
OpenSplice DDS
ignored by non-compliant IDL compilers
44. My Advice on X-Types
When in doubt, make outer types Mutable X-Types. Inner types
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
are usually better candidates for Extensible X-Types (yet keep in
mind our example)
☐ The main trade-off between Mutable and Extensible X-Types
OpenSplice DDS
lies in the space/time efficiency. The latter is more efficient
than the former.
45. My Advice on X-Types
Always identify @MustUnderstand attributes of Mutable X-Types.
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
Then declare as @Optional whatever is not @MustUnderstand
☐ This will ensure that you’ll always be able to distinguish
OpenSplice DDS
between default constructor values and nil
46. My Advice on X-Types
Copyright
2012,
PrismTech
–
All
Rights
Reserved.
Always explicitly define @Id for the attributes of Mutable X-Types
☐ This will ensure you’ll be to handle attribute re-ordering and
removal
OpenSplice DDS