This document discusses matrix factorization techniques. Matrix factorization involves decomposing a large, sparse matrix into more compact, ordered matrices. Common factorization methods include singular value decomposition (SVD), principal component analysis (PCA), and non-negative matrix factorization (NMF). NMF decomposes a matrix into two lower rank, non-negative matrices. It can reduce space needed to store data compared to SVD. The document provides an example of using NMF to factorize a user-item rating matrix to make recommendations.
2. What is Matrix Factorisation?
• For some domain, we have matrix that are:
very ‘big’
sparse
and don’t have any order
• Factoring it yield a set of:
More manageable
compact
and ordered matrices
4. SVD (Singular Value
Decomposition)
• The singular value decomposition of an m × n matrix M is a
factorisation of the form :
M = UΣV∗,
where U is an m × m unitary matrix,
Σ is an m × n rectangular diagonal matrix with non-negative real
numbers on the diagonal (known as the singular values of M),
and V∗ (the transpose of V) is an n × n real or complex unitary
matrix.
The m columns of U and the n columns of V are called the left-
singular vectors and right-singular vectors of M, respectively.
5. Limitation SVD
• One of the most general algorithm for
decomposition. SVD is an amelioration from
eigenvalue decomposition (was only used with n x
n matrix)
• Didn’t really reduce the space use to store our
data.
• From a m x n (M) matrix, we end up with m x m
(U), m x n (Σ) and n x n (V) matrices…
6. Matrix Factorisation
• Using Non-Negative Matrix Factorisation
V ≈ WH
• V, W and H are all non-negative
• V is a n x m matrix
• W is a n x r matrix, H is a r x m matrix
• and r ≪ min(m, n)
7. How to decompose?
• Minimize with respect to W and H,
subject to the constraints W, H ≥ 0.
• Multiplication Update Algorithm
• Alternating Least Square Algorithm
10. Back to the math
• We have a matrix of ratings we want to approximate:
• We need to construct P and Q by minimising:
• We have an optimisation problem:
• Update are then give by:
http://www.quuxlabs.com/blog/2010/09/matrix-factorization-a-simple-tutorial-and-implementation-in-python/
11. Easy to implement
(for toy example)
http://www.quuxlabs.com/blog/2010/09/matrix-factorization-a-simple-tutorial-and-implementation-in-python/
