This presentation includes : - Introduction - Methodology - Data Fusion Techniques - ATC Applications - Current works

1. Multi-sensor Data Fusion
Techno Briefing
Mr. Paveen Juntama
Air Traffic Service engineering
Research & Development Department
(RD.AS.)
Presented by

2. Contents
 Overview
 Methodology
 Fusion Techniques
 ATC Applications
 Current works in RD.AS. (วว.สว.)
2

3. Overview
 Overview
 Methodology
 Fusion Techniques
 ATC Applications
 Current works in RD.AS. (วว.สว.)
3

4.  Problem-solving techniques based on the idea of
integrating many answers into a single; the best
answer
 Process of combining data or information from
various sensors to provide a robust and complete
description of an process of interest
 Multilevel process dealing with automatic detection,
association, correlation, estimation and combination
of data or information from single or multiple sources
Definitions :
Overview
Multisensor Data Fusion (MDF)
4

5.  Location and characterization of
enemy units & weapons
 Air to air / surface to air defense
 Battlefield intelligence
 Strategic warning etc.
Military applications :
Overview
MDF Applications
5
 Central Monitoring systems (CMS)
 System Faults Detection
 Location & Identification
 Robotics & UAVs
 Medical etc.
Non military applications :

6.  Improves accuracy
 Improves precision
 Improves availability
 Reduces uncertainty
 Supports effective decision making
MDF provides advantages over a single sensor :
Overview
Why MDF ?
6

7. Methodology
 Overview
 Methodology
 Fusion Techniques
 ATC Applications
 Current works in RD.AS. (วว.สว.)
7

8. Methodology
Fusion Architectures
8
 Measurement Fusion (Sensor data Fusion)
 Feature-level Fusion
 Decision-level Fusion (High-level data Fusion)
Data Fusion requires combining expertise in 2 areas :
 Sensors
 Information integration
Data fusion is essentially an information integration problem.
Data fusion can be categorized into 3 main classes based on the
level of data abstraction used for fusion :

9.  Direct fusion of data sensor
 The sensors measuring the same physical phenomena
are required.
Measurement Fusion (Sensor Data Fusion) :
Methodology
Fusion Architectures
9
S1
Data Level
Fusion
Association
S2
Sn
Feature
Extraction
Identity
Declaration

10.  Involves the extraction of representative features from
sensor data
 Features is combined into a single concatenated feature
vector that is an input to a fusion node
Feature-level Fusion :
Methodology
Fusion Architectures
10
S1
Association
S2
Sn
Feature
Extraction
Feature Level
Fusion
+
Identity
Declaration

11.  Each sensor has made a preliminary determination of an
entity’s location, attributes and identity before combining
 Decision-level fusion algorithms are used such as weighted
decision, Bayesian inference and Dempster-Shafer’s method
Decision-level Fusion :
Methodology
Fusion Architectures
11
S1
S2
Sn
Identity
Declaration
Feature
Extraction
Identity
Declaration
Identity
Declaration
Association
Declaration
Level
Fusion

12. Fusion Techniques
 Overview
 Methodology
 Fusion Techniques
 ATC Applications
 Current works in RD.AS. (วว.สว.)
12

13. Fusion Techniques
13
The available data fusion techniques can be classified into
3 categories
Data
Fusion
Data Association
Decision Fusion
State Estimation

14. The process of assign and compute the weight that relates the
observations or tracks from one set to the observation of tracks of
another set.
Fusion Techniques
Data Fusion Techniques
14
Data Association Techniques
Algorithms commonly used
Nearest Neighbors(NN), Probabilistic Data Association(PDA),
Joint PDA(JPDA), Multiple Hypothesis Test (MHT) etc.

15. State estimation techniques aim to determine the state of the target
under movement (typically the position) given the observation or
measurement.
Fusion Techniques
Data Fusion Techniques
15
State Estimation (Tracking)
Algorithms commonly used
Maximum Likelihood (ML) & Maximum Posterior, Kalman Filter,
Particle Filter, Covariance Consistency Methods etc.

16. Decision Fusion techniques aim to make a high-level inference about
the events and activities produced from the detected targets.
Fusion Techniques
Data Fusion Techniques
16
Decision Fusion
Algorithms commonly used
Bayesian Methods & Dempster-Shafer Inference, Abductive
Reasoning, Semantic Methods etc.
𝑥1(𝑛)
𝑥2(𝑛)
𝑥 𝑛(𝑛)
𝑥(𝑛)
|
|
|

17. Fusion Techniques
Data Fusion Techniques
17
Nearest Neighbors
Probabilistic Data
Association
Joint PDA
Multiple Hypothesis
Test
Maximum Likelihood
Kalman Filter*
Particle Filter
Covariance Consistency
Methods
Bayesian Methods*
Dempster-Shafer
Inference
Abductive Reasoning
Semantic Methods
*Bayesian approaches
Data Association State Estimation Decision Fusion

18. Fusion Techniques
Bayesian Approaches
18
Bayes’ theorem
where the posterior probability, P(Y|X), represents the belief in the
hypothesis Y given the information X. This probability is obtained by
multiplying the a priori probability of the hypothesis P(Y) by the
probability of having X given that Y is true, P(X|Y)

19. Fusion Techniques
Bayesian Approaches
19
A Recursive Bayesian Estimator : Kalman Filter
 Address the general problem of trying to estimate the state
of a discrete time process
 Estimate a process using a recursive algorithm :
– Prediction : estimate the process state at a certain time
– Correction : obtain feedback from noisy measurement

20. Fusion Techniques
Bayesian Approaches
20
The need of Kalman Filter ?
System
Measuring
Device
System
Error Sources
Control
Unknown
System State
Measurement
Error Sources
 System state cannot be measures directly
 Estimation “optimally” from measurements is required
Correction
PredictionPrediction
++
Measurement
Model
Process
Model
Updated+
-
Error
Kalman Filter
𝑥(𝑛)
𝑥(𝑛)

21. Fusion Techniques
Bayesian Approaches
21
Data Fusion with Kalman filter
Measurement
Fusion
Track-to-track
Fusion

22. Fusion Techniques
Bayesian Approaches
22
Example results of Kalman filtering

23. ATC Applications
 Overview
 Methodology
 Fusion Techniques
 ATC Applications
 Current works in RD.AS. (วว.สว.)
23

24. ATC Applications
Surveillance Data Processing
24
VHF GS
SAT GS
ATC CENTRE
ADS GS
MLAT/WAMMODE SSSRPSR
SAT NAVINMARSATSAT COM
Surveillance sensor environment

25. 25
ATC Applications
Surveillance Data Processing

26. 26
ATC Applications
Surveillance Data Processing
Selection techniques
Radar 1
Radar 2
|
|
|
Radar N
Multiple
plots
switching
Selected Plots
plots
plots
plots
Radar 1
Radar 2
|
|
|
Radar N
Mono radar tracking
Mono radar tracking
Mono radar tracking
Multiple
tracks
switching
plots
plots
tracks
tracks
Multiple plots
switching method
Multiple tracks
switching method
Selected Tracks
plots tracks

27. 27
ATC Applications
Surveillance Data Processing
Average techniques
Multiple track
average method
Multiple plot
average method
Radar 1
Radar 2
|
|
|
Radar N
Mono radar tracking
Mono radar tracking
Mono radar tracking
Track-to-
track
correlation
Track-to-
track
fusion
Fused
Tracks
plots
plots
plots
tracks
tracks
tracks
Radar 1
Radar 2
|
|
|
Radar N
Plot-to-
plot
correlation
Plot-to-
plot fusion
Fused
Plots
plots
plots
plots

28.  The technique consists in using all plots coming from any radar to update a unique
synthetic common track
 The track update is performed in the fly as soon as sensor report are received so that the
reduction of the meantime update in multi-radar configuration improves the accuracy of
the track parameter estimation.
 These techniques contain more complex algorithms (Association + State Estimation +
Decision Fusion)
Variable update techniques :
28
ATC Applications
Surveillance Data Processing
N
N-1
N-2
Correlation
Track
Management
Track Update
Track
Initiation
N-k
Output
Tracks created
Tracks initiated
Non
associated
plots
Association
Pairing
Non
associated
tracks
Tracks to update

29. 29
ATC Applications
Surveillance Data Processing
Comparison between techniques :
Selection & Average Techniques Variable Update Techniques
Low CPU load Medium to high CPU load
Low track accuracy Good track accuracy
Low track discrimination Good track discrimination
Manoeuvre detection in long time Manoeuvre detection in short time

30. Current works in RD.AS.
 Overview
 Methodology
 Fusion Techniques
 ATC Applications
 Current works in RD.AS. (วว.สว.)
30

31. Current works in RD.AS.
31
System Architecture :
Multi Radar Tracking
System (MRTS)
Fusion
System
SSR
SSR
SSR
ADS-B
Ground Station
Local Tracks
ADS-B Reports
System Tracks

32. 32
Study of System tracks & ADS-B reports
Characteristics System tracks ADS-B reports
Update rates 500 ms 0.3-3 ms
Update rates / target 5 s 1 s
Data source MRTS GNSS
Identification Mode 3/A Callsign, Mode S
Performance
High Availability
Low Accuracy
High Accuracy
Low Availability
Current works in RD.AS.

33. Current works in RD.AS.
33
Study of System tracks & ADS-B reports
Horizontal Zoom
 ADS-B reports lost in some periods (Low Availability)
 System tracks are less accurate in positioning compared to ADS-B
reports

34. Current works in RD.AS.
34
Fusion system
Sensor data Feature vector Identity
Declaration
System tracks
(CAT62),
ADS-B reports
(CAT21)
Metadata Fused Tracks
 Track Management, Track Initiation and
Filtering are responsible for the
association, correlation and state
estimation techniques.
 While Track-to-track Fusion corresponds
to a Decision-level Fusion scheme.

35. Current works in RD.AS.
35
Results :
Remark :
There is only 1 ADS-B
ground station in
operation

36. Current works in RD.AS.
36
Problems & Difficulties
 The difficult synchronization due
to different time sources between
ADS-B GS & MRTS
 Difficulty of track correlation due
to target identification problems

37. Current works in RD.AS.
37
Future works
 Improve the synchronization mechanisms
 Improve Fusion algorithms
 Evaluate performance of the fusion system
 Study possibilities for integrating data from new sensor types such as
MLAT, WAM etc.
 Study and characterize the system closing to the realistic environment
as possible including a process model, a measurement model, Radar
biases and ADS-B receiver biases.

38. Conclusion
38
 Data fusion can be performed at 3 levels :
– Sensor data
– Feature vectors
– High level inferences
 Several techniques has been developed to process data fusion at each
level.
 Fusion techniques can be used with one or more techniques; data
association, state estimation or decision fusion, each technique
contains various algorithms.
 The use of fusion techniques and methodology depends on the
environment of the system which include sensor characteristics,
integrated information etc.