The document discusses exact repair problems with multiple sources in distributed storage systems. It presents the problem definition for (n,k,d) exact repair with multiple sources. It then provides an example of a 2-source (3,2,2) exact repair problem and outlines techniques for computing achievable rate regions using polyhedral bounds and a projection algorithm. The techniques are implemented in software that can be used to obtain rate regions and optimal codes for coded distributed storage networks.

1. CISS 2014, Princeton NJ 1
Exact Repair Problems with Multiple
Sources
Jayant Apte*, Congduan Li,
John MacLaren Walsh, Steven Weber
ECE Dept. Drexel University

2. CISS 2014, Princeton NJ 2
Outline
● Problem Definition
● Computer assisted proofs: General Structure
● Polyhedral bounds on
● Polyhedral computation interpretation of rate
region computation
● A projection technique for computing
achievable rate region

3. CISS 2014, Princeton NJ 3
Outline
● Problem Definition
● Computer assisted proofs: General structure
● Polyhedral bounds on
● Polyhedral computation interpretation of rate
region computation
● A projection technique for computing
achievable rate region

4. CISS 2014, Princeton NJ 4
(n,k,d) Exact Repair with multiple
sources

5. CISS 2014, Princeton NJ 5
(n,k,d) Exact Repair with multiple
sources

6. CISS 2014, Princeton NJ 6
2-source (3,2,2) exact
repair problem

7. CISS 2014, Princeton NJ 7
2-source (3,2,2) exact
repair problem
2 sources

8. CISS 2014, Princeton NJ 8
2-source (3,2,2) exact
repair problem
3 encoding
functions

9. CISS 2014, Princeton NJ 9
2-source (3,2,2) exact
repair problem
3 storage
random
variables

10. CISS 2014, Princeton NJ 10
2-source (3,2,2) exact
repair problem
3 decoders
with different
demands

11. CISS 2014, Princeton NJ 11
2-source (3,2,2) exact
repair problem
6 repair
encoding
functions

12. CISS 2014, Princeton NJ 12
2-source (3,2,2) exact
repair problem
3 repair
decoding
functions

13. CISS 2014, Princeton NJ 13
2-source (3,2,2) exact
repair problem
Total 11 random variables

14. CISS 2014, Princeton NJ 14
Implicit characterization
of rate region(Yan et al.)

15. CISS 2014, Princeton NJ 15
Outline
● Problem Definition
● Computer assisted proofs: General structure
● Polyhedral bounds on
● Polyhedral computation interpretation of rate
region computation
● A projection technique for computing
achievable rate region

16. CISS 2014, Princeton NJ 16
Motivation
Sources
Decoder
Demands
SoftwareNetwork

17. CISS 2014, Princeton NJ 17
Motivation
Software
Sources
Decoder
Demands
Network
Rate Region
and
optimal codes

18. CISS 2014, Princeton NJ 18
Software for computer
assisted proofs

19. CISS 2014, Princeton NJ 19
Computer assisted converse

20. CISS 2014, Princeton NJ 20
Computer assisted converse
Inequalities obtained as an
implication of linear Shannon-type,
non-Shannon-type, non-linear
non-Shannon type inequalities and
network constraints

21. CISS 2014, Princeton NJ 21
Computer assisted converse

22. CISS 2014, Princeton NJ 22
Computer assisted achievability

23. CISS 2014, Princeton NJ 23
Software for computer
assisted proofs

24. CISS 2014, Princeton NJ 24
Outline
● Problem Definition
● Computer assisted proofs: General structure
● Polyhedral bounds on
● Polyhedral computation interpretation of rate
region computation
● A projection technique for computing
achievable rate region

25. CISS 2014, Princeton NJ 25
● Closure of set of all 'entropic' vectors
arising from N-variable probability
distributions
3-D rendition of

26. CISS 2014, Princeton NJ 26
● Closure of set of all 'entropic' vectors
arising from N-variable probability
distributions
● Each entropic vector is formed by
stacking entropies of subsets of N
random variables
3-D rendition of

27. CISS 2014, Princeton NJ 27
● Closure of set of all 'entropic' vectors
arising from N-variable probability
distributions
● Each entropic vector is formed by
stacking entropies of subsets of N
random variables
● Cone:
3-D rendition of

28. CISS 2014, Princeton NJ 28
● Cannot be expressed as intersection
of finite number of linear inequalities
for N>3
● For N=4, existence of single nonlinear
● non-Shannon inequality(necessary and
sufficient) is known [Liu & Walsh 2014]
● Additionally, several hundred linear
non-Shannon inequalities are known
[DFZ 2011, Csirmaz 2013]
3-D rendition of

29. CISS 2014, Princeton NJ 29

30. CISS 2014, Princeton NJ 30

31. CISS 2014, Princeton NJ 31
Outline
● Problem Definition
● Computer assisted proofs: General structure
● Polyhedral bounds on
– Shannon (Outer) bound
● Polyhedral computation interpretation of rate
region computation
● A projection technique for computing
achievable rate region

32. CISS 2014, Princeton NJ 32
Shannon Outer Bound

33. CISS 2014, Princeton NJ 33
Shannon Outer Bound

34. CISS 2014, Princeton NJ 34
Shannon Outer Bound

35. CISS 2014, Princeton NJ 35
Outline
● Problem Definition
● Computer assisted proofs: General structure
● Polyhedral bounds on
– Shannon (Outer) bound
– (Representable) Matroid (Inner) bound(s)
● Polyhedral computation interpretation of rate region
computation
● A projection technique for computing achievable rate
region

36. CISS 2014, Princeton NJ 36
(Representable) Matroid Inner bound(s)

37. CISS 2014, Princeton NJ 37
(Representable) Matroid Inner bound(s)

38. CISS 2014, Princeton NJ 38
(Representable) Matroid Inner bound(s)

39. CISS 2014, Princeton NJ 39
(Representable) Matroid Inner bound(s)

40. CISS 2014, Princeton NJ 40
(Representable) Matroid Inner bound(s)

41. CISS 2014, Princeton NJ 41
(Representable) Matroid Inner bound(s)

42. CISS 2014, Princeton NJ 42
(Representable) Matroid Inner Bound(s)

43. CISS 2014, Princeton NJ 43
Outline
● Problem Definition
● Computer assisted proofs: General structure
● Polyhedral bounds on
– Shannon (Outer) bound
– Matroid (Inner) bound(s)
– Subspace (Inner) bounds
● Polyhedral computation interpretation of rate region
computation
● A projection technique for computing achievable rate region

44. CISS 2014, Princeton NJ 44
Subspace Inner Bound(s)

45. CISS 2014, Princeton NJ 45
Subspace Inner Bound(s)

46. CISS 2014, Princeton NJ 46
Subspace Inner Bound(s)

47. CISS 2014, Princeton NJ 47
Subspace Inner Bound(s)

48. CISS 2014, Princeton NJ 48
Subspace Inner Bound(s)

49. CISS 2014, Princeton NJ 49
Software for computer
assisted proofs

50. CISS 2014, Princeton NJ 50
Polyhedral bounds on rate region
● Using polyhedral inner/outer bound on yields
polyhedral inner/outer bounds on rate region
● Lemma 1: Inner bounds on rate region
computed using or are achievable
using linear codes

51. CISS 2014, Princeton NJ 51
Outline
● Problem Definition
● Computer assisted proofs: General structure
● Polyhedral bounds on
– Shannon (Outer) bound
– Matroid (Inner) bound(s)
– Subspace (Inner) bounds
● Polyhedral computation interpretation of rate region
computation
● A projection technique for computing achievable rate region

52. CISS 2014, Princeton NJ 52
Network Coding constraints

53. CISS 2014, Princeton NJ 53
Network Coding constraints
● Consider a type 1 or type 2 constraint H
● In general, computing extreme rays of given H and
extreme rays of is equivalent to an iteration of Double
Description Method of polyhedral representation conversion
● Lemma 2 [Li et al. 2013]: An extreme ray of is an extreme
ray of if it is contained in the hyperplane corresponding
to H
● Hence, simple membership check suffices to find extreme rays of

54. CISS 2014, Princeton NJ 54
Software for computer
assisted proofs

55. CISS 2014, Princeton NJ 55
Rate constraints
Storage Bandwidth

56. CISS 2014, Princeton NJ 56
Rate constraints
Repair Bandwidth
Storage Bandwidth

57. CISS 2014, Princeton NJ 57
A projection technique for computing
achievable rate region

58. CISS 2014, Princeton NJ 58
A projection technique for computing
achievable rate region

59. CISS 2014, Princeton NJ 59
A projection technique for computing
achievable rate region

60. CISS 2014, Princeton NJ 60
Polyhedral projection via chm
● chm is an implementation of polyhedral projection
algorithm called Convex Hull Method by Jayant Apte*
● chmlib v0.x is available at:
http://www.ece.drexel.edu/walsh/aspitrg/software.html
● Rational arithmetic using FLINT: Fast Library for
Number Theory
● Rational LP solver based on qsopt

61. CISS 2014, Princeton NJ 61
Polyhedral projection via chm
● Has been used for
– The current work
– Computer assisted converse proofs of rate regions of Multilevel
Diversity Coding Systems(a special case of multi-source network
coding)
– Finding non-Shannon Information Inequalities via Generalized
Copy Lemma of Csirmaz
● Can be used for
– Finding necessary conditions for non-contexuality of small
marginal scenarios(Quantum Information)

62. CISS 2014, Princeton NJ 62
Results
SoftwareNetwork

63. CISS 2014, Princeton NJ 63
Rate region for H(S1)=1 and
H(S2)=1

64. CISS 2014, Princeton NJ 64
Rate region for H(S1)=1 and
H(S2)=2

65. CISS 2014, Princeton NJ 65
References
● X. Yan, R.W. Yeung, and Zhen Zhang. An implicit characterization of the achievable rate region for acyclic
multisource multisink network coding. Information Theory, IEEE Transactions on, 58(9):5625–5639, 2012.
● Dougherty, Randall, Chris Freiling, and Kenneth Zeger. "Non-Shannon information inequalities in four
random variables." arXiv preprint arXiv:1104.3602 (2011).
● Csirmaz, László. "Information inequalities for four variables." CEU (2013).
● Yunshu Liu and John M. Walsh, "Only One Nonlinear Non-Shannon Inequality is Necessary for Four
Variables", submitted to IEEE Int. Symp. Information Theory (ISIT2014)
● Congduan Li, J. Apte, J.M. Walsh, and S. Weber. A new computational approach for determining rate regions
and optimal codes for coded networks. In Network Coding (NetCod), 2013 International Symposium on,
pages 1–6, 2013.
● Congduan Li, John MacLaren Walsh, Steven Weber. Matroid bounds on the region of entropic vectors. In
51th Annual Allerton Conference on Communication, Control and Computing, October 2013.