introduction to random walk and its applications on the machine learning

1. Random Walks
Shuai Zhang, UNSW
cheungshuai@outlook.com

2. What is
Random Walk?
A drunk man will find his
way home, but a drunk
bird may get lost forever.

3. What is Random Walk?
• Random walk is a process, a model or a rule to generate path
sequence of random motion.
• A random walk is a mathematical object, known as a stochastic or
random process, that describes a path that consists of a succession of
random steps on some mathematical space such as the integers.

4. What is Random Walk?
• Many natural phenomena can be modelled as random walk
• the path traced by a molecule as it travels in a liquid or a gas,
• the search path of a foraging animal,
• the price of a fluctuating stock and
• the financial status of a gambler
• PageRank
• Recommender Systems
• Investment theory of stock market
• Generate fractal images
• Even though they may not be truly random in reality

5. Simulation of Normally Distributed Random
Walk
• We start with initial location 100 and generate the random walk
based on normal probability distribution.

6. Simulation on Higher Dimensions
• In higher dimensions, the set of randomly walked points has
interesting geometric properties. In fact, one gets a discrete fractal.

7. Simulation on
Higher
Dimensions

8. Simple random walks on graphs
Let G=(V, E) be a connected graph, |V|=n and |E|=m
• Given an initial vertex 𝑣0, select “at random” an adjacent vertex 𝑣1,
and move to this neighbour
• Then select “at random” a neighbour 𝑣2 of 𝑣1, and move to it.
• etc.

14. Simple random walks on graphs
The sequence of vertices 𝑣0, 𝑣1, 𝑣2, … , 𝑣 𝑘, … selected in this way is a
simple random walk on G.
At each step 𝑘, we have a random variable 𝑋 𝑘 taking values on 𝑉.
Hence, the random sequence
𝑋0, 𝑋1, 𝑋2, … , 𝑋 𝑘, …
Is a discrete time stochastic process defined on the state space 𝑉.

15. Simple random walks on graphs
What does “at random” mean?
If at time 𝑘 we are at vertex 𝑖, choose uniformly an adjacent vertex 𝑗 to
move to.
Let d(i) denote the degree of vertex i.
These transition
probabilities do
not depend on
“time” 𝑘

16. Simple random walks on graphs

17. Random Walk and Markov chain
Correspondence between terminology of random walks and Markov chains

18. Random Walk and Markov chain
• The Markov property holds: conditional on the present, the future is
independent of the past
• The random sequence of vertices visited by the walk,
𝑋0, 𝑋1, 𝑋2, … , 𝑋 𝑘, …
• Is a Markov chain with state space V and matrix of transition
probabilities
𝑃 = 𝑝𝑖𝑗 𝑖, 𝑗 ∈ 𝑉

19. Random Walk and Markov chain
• A Markov chain describes a stochastic process over a set of states
according to a transition probability matrix
• Markov chains are memoryless
• Random walks correspond to Markov chains:
• The set of states is the set of nodes in the graph.
• The elements of the transition probability matrix are the probabilities to
follow and edge from one node to another.

20. Random Walk and Markov chain
Stochastic matrix

21. Random Walk and Markov chain
𝑗∈𝑉
𝑝𝑖𝑗 = 1
Let D be the diagonal matrix with 𝐷 𝑖𝑗 = 1/𝑑(𝑖) and A be the
adjacency matrix of G. Then
P = DA
In particular , if G is d-regular,
𝑃 =
1
𝑑
𝐴

22. Random Walk and Markov chain
Let 𝜌 𝑘 be the row vector giving the probability distribution of 𝑋 𝑘,
𝜌 𝑘 𝑖 = 𝑃 𝑋 𝑘 = 𝑖 , 𝑖 ∈ 𝑉
The rule of the walk is expressed by the simple equation
𝜌 𝑘+1 = 𝜌 𝑘 𝑃
That is, if 𝜌0 is the initial distribution from which the starting vertex 𝑣0
is drawn,
𝜌 𝑘 = 𝜌0 𝑃 𝑘, 𝑘 ≥ 0

23. Random Walk and Markov chain
• Since G is connected, the random walk on G corresponds to an
irreducible Markov chain.
Irreducible: There is a path from every node to every other node.
• The Perron-Frobenius theorem for nonnegative matrices implies the
existence of a unique probability distribution π, which is a positive
left eigenvector of P associated to its dominant eigenvalue λ = 1

24. Random Walk and Markov chain
If the initial vertex of the walk is drawn from 𝜋 , then the probability
distribution at time k is
𝜌 𝑘 = 𝜋𝑃 𝑘
= π
Hence, for all time k >=0 ,
𝑃 𝑋 𝑘 = 𝑖 = 𝜋 𝑖 , 𝑖 ∈ 𝑉
The random walk is a stationary stochastic process
𝜋 is the
stationary
distribution

25. Random Walk and Markov chain
Does a stationary distribution always exist? Is it unique?
Yes, if the graph is “well-behaved”.
Irreducible and Aperiodic, a directed graph is said to be aperiodic if
there is no integer k > 1 that divides the length of every cycle of the
graph

26. Several concepts
How fast will the random surfer approach this stationary distribution?
Mixing Time!
The access time or hitting time H(i,j) is the expected number of step before
node j is visited, starting from node i. In general
𝐻 𝑖, 𝑗 ≠ 𝐻 𝑗, 𝑖
The commute time is
𝐻 𝑖, 𝑗 + 𝐻 𝑗, 𝑖
The cover time (starting from a given distribution) is the expected number of
steps to reach every node.

27. Random Walk with Recommender Systems
A
B
C
D
a
b
c
d
e

28. Random Walk with Recommender Systems
A
B
C
D
a
b
c
d
e
A to c

29. Random Walk with Recommender Systems
A
B
C
D
a
b
c
d
e
A to e
A
B
C
b
c
d
e
a
D

30. Random Walk for Recommendation
TrustWalker: A Random Walk Model for Combining Trust-based and
Item-based Recommendation

31. Random Walk algorithms

32. Random Walk algorithms
A subset of Vc of a set genes V have “a prori” known property C
Can we rank the other genes in the set VVc w.r.t their likelihood to
belong to Vc?

33. Random Walk in DeepWalk
DeepWalk, learning latent representations of vertices in a network

34. Random Walk in DeepWalk
DeepWalk, learning latent representations of vertices in a network

35. Reference
• http://people.revoledu.com/kardi/tutorial/StochasticProcess/Rando
mWalk/index.html
• http://www.mit.edu/~kardar/teaching/projects/chemotaxis(AndreaS
chmidt)/random.htm
• https://www.math.uchicago.edu/~lawler/srwbook.pdf
• https://www.cs.cmu.edu/~avrim/598/chap5only.pdf
• http://www.lirmm.fr/~sau/JCALM/Josep.pdf
• http://homes.di.unimi.it/valentini/MB201213/slide/RandomWalksGr
aphs.pdf
• https://arxiv.org/pdf/1403.6652.pdf

36. Weekly Report
• Released the DeepRec
• Revised the sequential recommendation paper
• I implemented a similar idea for rating prediction-> does not work
• Prepare for the learning group
Next Week:
• Some review work
• Revision