Compressive Oversampling for Robust Data Transmission in Sensor Networks - Presented at INFOCOM 2010

Recovering Lost Sensor
Data through Compressed
Sensing
Zainul Charbiwala

Collaborators:Younghun Kim, Sadaf Zahedi, Supriyo Chakraborty,
Ting He (IBM), Chatschik Bisdikian (IBM), Mani Srivastava

The Big Picture

Lossy Communication Link

zainul@ee.ucla.edu - CSEC - Infocom - March 2010 2

The Big Picture


How do we recover from this loss?


The Big Picture


• Retransmit the lost packets


The Big Picture

Generate Error
Correction Bits


• Proactively encode the data with some protection bits


The Big Picture

Generate Error
Correction Bits


• Proactively encode the data with some protection bits
• Can we do something better ?


The Big Picture - Using Compressed Sensing


CSEC


Generate Compressed
Measurements


CSEC


Generate Compressed
Measurements


CSEC
Recover from
Received Compressed
Measurements

How does this work ?



Generate Compressed
Measurements


CSEC
Recover from
Received Compressed
Measurements

• Use knowledge of signal model and channel



Generate Compressed
Measurements


CSEC
Recover from
Received Compressed
Measurements

• CS uses randomized sampling/projections



Generate Compressed
Measurements


CSEC
Recover from
Received Compressed
Measurements

• Random losses look like additional randomness !



Generate Compressed
Measurements


CSEC
Recover from
Received Compressed
Measurements

• Random losses look like additional randomness !
Rest of this talk focuses on describing
“How” and “How Well” this works

Talk Outline

‣ A Quick Intro to Compressed Sensing
‣ CS Erasure Coding for Recovering Lost Sensor Data
‣ Evaluating CSEC’s cost and performance
‣ Concluding Remarks


Why Compressed Sensing ?
Physical Sampling Compression Communication Application
Signal
Computationally
expensive


Why Compressed Sensing ?
Physical Sampling Compression Communication Application
Signal
Computationally
expensive

Physical Compressive
Communication Decoding Application
Signal Sampling

Shifts computation to
a capable server


Compressed Sensing - Some Intuition

Bandwidth Frequency



Bandwidth Frequency

How do you acquire this signal?



Bandwidth Frequency

• Nyquist rate - twice the bandwidth



Bandwidth Frequency

• But what if you knew more about the signal?



Bandwidth Frequency

• But what if you knew more about the signal?
• CS enables signal acquisition based on information content


Transform Domain Analysis


‣ We usually acquire signals in the time or spatial domain


‣ By looking at the signal in another domain, the signal may be
represented more compactly



‣ Eg: a sine wave can be expressed by
3 parameters: frequency, amplitude
and phase.



and phase.
‣ Or, in this case, by the index of the
FFT coefﬁcient and its complex
value



and phase.
‣ Or, in this case, by the index of the
FFT coefﬁcient and its complex
value
‣ Sine wave is sparse in frequency
domain

Acquiring a Sine Wave



‣ Assume we’re interesting in acquiring a single sine wave x(t) in a
noiseless environment



‣ An inﬁnite duration sine wave can be expressed using three
parameters: frequency f, amplitude a and phase φ.



‣ An inﬁnite duration sine wave can be expressed using three
parameters: frequency f, amplitude a and phase φ.
‣ Question: What’s the best way to ﬁnd the parameters ?



‣ Technically, to estimate three parameters one needs three good
measurements



measurements
‣ Questions:



measurements
‣ Questions:
‣ What are “good” measurements ?



measurements
‣ Questions:
‣ What are “good” measurements ?

‣ How do you estimate f, a, φ from three measurements ?


Compressed Sensing


Compressed Sensing

‣ With three samples: z1, z2, z3 of the sine wave at times t1, t2, t3


Compressed Sensing

‣ We know that any solution of f, a and φ must meet the three
constraints and spans a 3D space:


Compressed Sensing

z i = x(t i ) = a sin(2π ft i + φ )
∀i ∈{1, 2, 3}


Compressed Sensing

∀i ∈{1, 2, 3}
φ
‣ Feasible solution space is much smaller

a


Compressed Sensing

∀i ∈{1, 2, 3}
φ
‣ Feasible solution space is much smaller

‣ As the number of constraints grows from
more measurements, the feasible solution
a
space shrinks
‣ Exhaustive search over this space reveals the
right answer knowing presence of one sine

Formulating the Problem
‣ We could also represent f, a and φ as a very long, but mostly
empty FFT coefﬁcient vector.


empty FFT coefﬁcient vector. Sine wave.
Amplitude
represented by
x color


Amplitude
represented by
Ψ (Fourier Transform) x color


Amplitude
represented by
y = Ψ (Fourier Transform) x color


Amplitude
represented by
y = Ψ (Fourier Transform) x color

− j2 π ft + φ
ae


Sampling Matrix

‣ We could also write out the sampling process in matrix form


Sampling Matrix


x


Sampling Matrix


Φ x


Sampling Matrix


z = Φ x


Sampling Matrix


z = Φ x

Three non-zero entries at some
“good” locations


Sampling Matrix


Three measurements
z = Φ x



Sampling Matrix


Three measurements
z = Φ x
k

n



Exhaustive Search
‣ Objective of exhaustive search:
‣ Find an estimate of the vector y that meets the constraints and is the most
compact representation of x (also called the sparsest representation)

‣ Our search is now guided by the fact that y is a sparse vector
‣ Rewriting constraints:
z = Φx
y = Ψx
−1
z = ΦΨ y
Constraints from
measurements


Exhaustive Search

ˆ %
y = arg min y l 0
%
y
z = Φx
y = Ψx %
s.t. z = ΦΨ y −1

z = ΦΨ y −1 y l0
@ {i : yi ≠ 0}

Constraints from
measurements


Exhaustive Search

ˆ %
y = arg min y l 0
%
y
z = Φx
y = Ψx %
s.t. z = ΦΨ y −1

z = ΦΨ y −1 y l0
@ {i : yi ≠ 0}

Constraints from This optimization problem
measurements is NP-Hard !

l1 Minimization

‣ Approximate the l0 norm to an l1 norm

ˆ %
y = arg min y l 1
%
y y = ∑ yi
l1

% −1 i
s.t. z = ΦΨ y

‣ This problem can now be solved efﬁciently using linear
programming techniques
‣ This approximation was not new
‣ The big leap in Compressed Sensing was a theorem that showed
that under the right conditions, this approximation is exact!


Some CS Results

‣ Theorem: If k samples of a length n signal are acquired uniformly
randomly (if each sample is equiprobable) and reconstruction is
performed in the Fourier basis:
k
[Rudelson06] s≤C · 4 ′
w.h.p.
log (n)

‣ Where s is the sparsity of the signal


Handling Missing Data - Traditional Approach
Physical Compressed
Sampling Compression
Signal domain samples
n
x ∈° z = In x y = Ψz
nxn


Physical Compressed
n
nxn

Missing
Communication
samples

When communication channel is lossy:


Physical Compressed
n
nxn

Missing
Communication
samples

• Use retransmissions to recover lost data


Physical Compressed
n
nxn

Missing
Communication
samples

• Use retransmissions to recover lost data
• Or, use error (erasure) correcting codes


Physical Compressed
n
nxn

Missing
Communication
samples


Physical Compressed
n
nxn

Recovered
Channel Channel
Missing
Communication compressed
Coding Decoding
samples domain samples


Physical Compressed
n
nxn

Recovered
Channel Channel
Missing
Coding Decoding
+
w = Ωy wl = Cw y = ( CΩ ) wl
ˆ
mxn
m>n


Physical Compressed
n
x ∈° z = In x y = Ψz Done at
nxn application layer

Recovered
Channel Channel
Missing
Coding Decoding
+
ˆ
mxn
m>n


Physical Compressed
n
x ∈° z = In x y = Ψz Done at
nxn application layer

Recovered
Channel Channel
Missing
Coding Decoding
+
ˆ
mxn
m>n Done at physical layer
Can’t exploit signal
characteristics


CS Erasure Coding Approach
Physical Compressive Compressed
Communication Decoding
Signal Sampling domain samples
n
x ∈° z = Φx zl = Cz %
y = arg min y l 1
kxn %
y
k<n
%
s.t. zl = CΦΨ y −1


n
x ∈° z = Φx zl = Cz %
y = arg min y l 1
kxn %
y
k<n
%

n
x ∈° z = Φx zl = Cz %
y = arg min y l 1
mxn %
y
k<m<n
%


n
x ∈° z = Φx zl = Cz %
y = arg min y l 1
kxn %
y
k<n
%

Over-sampling in CS is
Erasure Coding !

n
x ∈° z = Φx zl = Cz %
y = arg min y l 1
mxn %
y
k<m<n
%


Effects of Missing Samples on CS

z = Φ x



z = Φ x

Missing
samples at the
receiver



z = Φ x

Missing
samples at the
Same as missing
receiver
rows in the
sampling matrix



z = Φ x

What happens if we over-sample?



z = Φ x

• Can we recover the lost data?



z = Φ x

• Can we recover the lost data?
• How much over-sampling is needed?


Extending CS Results

‣ Claim: When m>k samples are acquired uniformly randomly and
communicated through a memoryless binary erasure channel that
drops m-k samples, the received k samples are still equiprobable.

‣ Implies that bound on sparsity condition should hold.

‣ If bound is tight, over-sampling rate (m-k) is same as loss rate

[This paper]


Features of CS Erasure Coding
‣ No need of additional channel coding block
‣ Redundancy achieved by oversampling
‣ Recovery is resilient to incorrect channel estimates
‣ Traditional channel coding fails if redundancy is inadequate

‣ Decoding is free if CS was used for compression anyway


Features of CS Erasure Coding
‣ No need of additional channel coding block
‣ Redundancy achieved by oversampling
‣ Recovery is resilient to incorrect channel estimates
‣ Traditional channel coding fails if redundancy is inadequate

‣ Decoding is free if CS was used for compression anyway
‣ Intuition:
‣ Channel Coding spreads information out over measurements

‣ Compression (Source Coding) - compact information in few measurements

‣ CSEC - spreads information while compacting !


Signal Recovery Performance Evaluation

Create CS Lossy CS Reconstruction
Signal Sampling Channel Recovery Error?


In Memoryless Channels
Baseline performance - No Loss



20 % Loss - Drop in
recovery probability



20 % Loss - Drop in

20 % Oversampling -
complete recovery



20 % Loss - Drop in

20 % Oversampling -
complete recovery

Less than 20 %
Oversampling -
recovery does not fail
completely


In Bursty Channels


In Bursty Channels

20 % Loss - Drop in


In Bursty Channels

20 % Loss - Drop in

20 % Oversampling -
doesn’t recover
completely


In Bursty Channels

20 % Loss - Drop in
Oversampling + Interleaving
- Still incomplete recovery
20 % Oversampling -
doesn’t recover
completely


In Bursty Channels
Worse than baseline Baseline performance - No Loss

20 % Loss - Drop in
Oversampling + Interleaving
- Still incomplete recovery
20 % Oversampling -
doesn’t recover
completely

Better than baseline

‣ Recovery incomplete because of low interleaving depth
‣ Recovery better at high sparsity because bursty channels deliver
bigger packets on average, but with higher variance


In Real 802.15.4 Channel



15 % Loss - Drop in



15 % Loss - Drop in

15 % Oversampling -
complete recovery



15 % Loss - Drop in

15 % Oversampling -
complete recovery

Less than 15 %
Oversampling -
recovery does not fail
completely


Cost of CSEC
5
Rnd ADC FFT Radio TX RS

4
Energy/block (mJ)

3

2

1

0
m=256 S-n-S m=10 C-n-S m=64 CS k=320 S-n-S+RS k=16 C-n-S+RS k=80 CSEC

Sense Sense, CS Sense Sense, CSEC
and Compress and and Compress and
Send (FFT) Send Send and Send
and (1/4th with Send
Send rate) Reed with
Solomon RS

No robustness
guarantees

Cost of CSEC
5

4
Energy/block (mJ)

3

2

1

0

Solomon RS

No robustness All options equally
guarantees robust (w.h.p.)

Cost of CSEC
5

4

2.5x
Energy/block (mJ)

lower
3

2 energy
1

0

Solomon RS

No robustness All options equally
guarantees robust (w.h.p.)

Summary

‣ Oversampling is a valid erasure coding strategy for compressive
reconstruction
‣ For binary erasure channels, an oversampling rate equal to loss
rate is sufﬁcient
‣ CS erasure coding can be rate-less like fountain codes
‣ Allows adaptation to varying channel conditions

‣ Can be computationally more efﬁcient on transmit side than
traditional erasure codes


Closing Remarks

‣ CSEC spreads information out while compacting
‣ No free lunch syndrome: Data rate requirement is higher than if using good
source and channel coding independently

‣ But, then, computation cost is higher too

‣ CSEC requires knowledge of signal model
‣ If signal is non-stationary, model needs to be updated during recovery

‣ This can be done using over-sampling too

‣ CSEC requires knowledge of channel conditions
‣ Can use CS streaming with feedback


Compressive Oversampling for Robust Data Transmission in Sensor Networks - Presented at INFOCOM 2010

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (6)

Recently uploaded

Recently uploaded (20)

Compressive Oversampling for Robust Data Transmission in Sensor Networks - Presented at INFOCOM 2010

Editor's Notes