Mixture-Rank Matrix Approximation for Collaborative Filtering

MIXTURE-RANK
MATRIX APPROXIMATION
FOR COLLABORATIVE FILTERING
2019. 06. 25.
JoonyoungYi
joonyoung.yi@kaist.ac.kr

CONTENTS
1. Preliminary
2. Mixture Rank Matrix Approximation
3. Experiments
4. Discussion

4
• Let are matrices.
• Low-rank Matrix Factorization Problem:  
We know only 1% of entries in R, how to complete low-rank matrix R?
• Optimization form:
LOW-RANK MATRIX FACTORIZATION (LRMF)
R U VT≒ xm m
n
n
r
r
R 2 Rm⇥n
, U 2 Rm⇥r
, V 2 Rn⇥r
<latexit sha1_base64="WI/r+BC8V9iiHIKaNDQmKJSxYmg=">AAACQ3icdZDNSsNAFIUn9a/Wv6hLN4NFcCElEUHdFdy4klpMW0himUyn7dDJJMxMhBLycG58AHc+gRsXKm4FJ21AbfXCwOF893LvnCBmVCrLejJKC4tLyyvl1cra+sbmlrm905JRIjBxcMQi0QmQJIxy4iiqGOnEgqAwYKQdjC5y3r4jQtKI36hxTPwQDTjtU4yUtrqm24Qe5dALkRoGQdrMbtMQeoqGREKeHUHnfyw0bs1h/o27ZtWqWZOC88IuRBUU1eiaj14vwklIuMIMSenaVqz8FAlFMSNZxUskiREeoQFxteRI7/HTSQgZPNBOD/YjoR9XcOL+nEhRKOU4DHRnfq6cZbn5F3MT1T/zU8rjRBGOp4v6CYMqgnmisEcFwYqNtUBYUH0rxEMkEFY694oOwZ798rxwjmvnNfv6pFq/KtIogz2wDw6BDU5BHVyCBnAABvfgGbyCN+PBeDHejY9pa8koZnbBrzI+vwDdMbEZ</latexit><latexit sha1_base64="WI/r+BC8V9iiHIKaNDQmKJSxYmg=">AAACQ3icdZDNSsNAFIUn9a/Wv6hLN4NFcCElEUHdFdy4klpMW0himUyn7dDJJMxMhBLycG58AHc+gRsXKm4FJ21AbfXCwOF893LvnCBmVCrLejJKC4tLyyvl1cra+sbmlrm905JRIjBxcMQi0QmQJIxy4iiqGOnEgqAwYKQdjC5y3r4jQtKI36hxTPwQDTjtU4yUtrqm24Qe5dALkRoGQdrMbtMQeoqGREKeHUHnfyw0bs1h/o27ZtWqWZOC88IuRBUU1eiaj14vwklIuMIMSenaVqz8FAlFMSNZxUskiREeoQFxteRI7/HTSQgZPNBOD/YjoR9XcOL+nEhRKOU4DHRnfq6cZbn5F3MT1T/zU8rjRBGOp4v6CYMqgnmisEcFwYqNtUBYUH0rxEMkEFY694oOwZ798rxwjmvnNfv6pFq/KtIogz2wDw6BDU5BHVyCBnAABvfgGbyCN+PBeDHejY9pa8koZnbBrzI+vwDdMbEZ</latexit><latexit sha1_base64="WI/r+BC8V9iiHIKaNDQmKJSxYmg=">AAACQ3icdZDNSsNAFIUn9a/Wv6hLN4NFcCElEUHdFdy4klpMW0himUyn7dDJJMxMhBLycG58AHc+gRsXKm4FJ21AbfXCwOF893LvnCBmVCrLejJKC4tLyyvl1cra+sbmlrm905JRIjBxcMQi0QmQJIxy4iiqGOnEgqAwYKQdjC5y3r4jQtKI36hxTPwQDTjtU4yUtrqm24Qe5dALkRoGQdrMbtMQeoqGREKeHUHnfyw0bs1h/o27ZtWqWZOC88IuRBUU1eiaj14vwklIuMIMSenaVqz8FAlFMSNZxUskiREeoQFxteRI7/HTSQgZPNBOD/YjoR9XcOL+nEhRKOU4DHRnfq6cZbn5F3MT1T/zU8rjRBGOp4v6CYMqgnmisEcFwYqNtUBYUH0rxEMkEFY694oOwZ798rxwjmvnNfv6pFq/KtIogz2wDw6BDU5BHVyCBnAABvfgGbyCN+PBeDHejY9pa8koZnbBrzI+vwDdMbEZ</latexit><latexit sha1_base64="WI/r+BC8V9iiHIKaNDQmKJSxYmg=">AAACQ3icdZDNSsNAFIUn9a/Wv6hLN4NFcCElEUHdFdy4klpMW0himUyn7dDJJMxMhBLycG58AHc+gRsXKm4FJ21AbfXCwOF893LvnCBmVCrLejJKC4tLyyvl1cra+sbmlrm905JRIjBxcMQi0QmQJIxy4iiqGOnEgqAwYKQdjC5y3r4jQtKI36hxTPwQDTjtU4yUtrqm24Qe5dALkRoGQdrMbtMQeoqGREKeHUHnfyw0bs1h/o27ZtWqWZOC88IuRBUU1eiaj14vwklIuMIMSenaVqz8FAlFMSNZxUskiREeoQFxteRI7/HTSQgZPNBOD/YjoR9XcOL+nEhRKOU4DHRnfq6cZbn5F3MT1T/zU8rjRBGOp4v6CYMqgnmisEcFwYqNtUBYUH0rxEMkEFY694oOwZ798rxwjmvnNfv6pFq/KtIogz2wDw6BDU5BHVyCBnAABvfgGbyCN+PBeDHejY9pa8koZnbBrzI+vwDdMbEZ</latexit>

5
• Each set(training set, test set) is a set of (i, j, Rij)s.
• i: row index, j: column index
• Rij: true rating of row i to column j.
• : the set of (i, j) corresponding to the training set (=The set of known entries).
• m is the number of rows, n is the number of columns.
• RMSE (Root Mean Square Error) was used for measuring accuracy.
• is the cardinality of the .
• Indicator Operator:
•
NOTATION
⌦<latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit>
ˆR, R 2 Rm⇥n
<latexit sha1_base64="EGo0+wpGbNsctCEWF04o7ynaSbg=">AAACDnicbVBNS8NAEN3Ur1q/oh69LBbFg5REBPVW9OKxFmMLTSyb7bZdutmE3YlQQv6BF/+KFw8qXj1789+4/Tho64OBx3szzMwLE8E1OM63VVhYXFpeKa6W1tY3Nrfs7Z07HaeKMo/GIlbNkGgmuGQecBCsmShGolCwRji4GvmNB6Y0j+UtDBMWRKQneZdTAkZq24d+n0BWz49xHftcYj8i0A9Do9xnEfaBR0xjmbftslNxxsDzxJ2SMpqi1ra//E5M04hJoIJo3XKdBIKMKOBUsLzkp5olhA5Ij7UMlcTsCbLxPzk+MEoHd2NlSgIeq78nMhJpPYxC0zk6V896I/E/r5VC9zzIuExSYJJOFnVTgSHGo3BwhytGQQwNIVRxcyumfaIIBRNhyYTgzr48T7yTykXFvTktVy+naRTRHtpHR8hFZ6iKrlENeYiiR/SMXtGb9WS9WO/Wx6S1YE1ndtEfWJ8/jaeb8g==</latexit><latexit sha1_base64="EGo0+wpGbNsctCEWF04o7ynaSbg=">AAACDnicbVBNS8NAEN3Ur1q/oh69LBbFg5REBPVW9OKxFmMLTSyb7bZdutmE3YlQQv6BF/+KFw8qXj1789+4/Tho64OBx3szzMwLE8E1OM63VVhYXFpeKa6W1tY3Nrfs7Z07HaeKMo/GIlbNkGgmuGQecBCsmShGolCwRji4GvmNB6Y0j+UtDBMWRKQneZdTAkZq24d+n0BWz49xHftcYj8i0A9Do9xnEfaBR0xjmbftslNxxsDzxJ2SMpqi1ra//E5M04hJoIJo3XKdBIKMKOBUsLzkp5olhA5Ij7UMlcTsCbLxPzk+MEoHd2NlSgIeq78nMhJpPYxC0zk6V896I/E/r5VC9zzIuExSYJJOFnVTgSHGo3BwhytGQQwNIVRxcyumfaIIBRNhyYTgzr48T7yTykXFvTktVy+naRTRHtpHR8hFZ6iKrlENeYiiR/SMXtGb9WS9WO/Wx6S1YE1ndtEfWJ8/jaeb8g==</latexit><latexit sha1_base64="EGo0+wpGbNsctCEWF04o7ynaSbg=">AAACDnicbVBNS8NAEN3Ur1q/oh69LBbFg5REBPVW9OKxFmMLTSyb7bZdutmE3YlQQv6BF/+KFw8qXj1789+4/Tho64OBx3szzMwLE8E1OM63VVhYXFpeKa6W1tY3Nrfs7Z07HaeKMo/GIlbNkGgmuGQecBCsmShGolCwRji4GvmNB6Y0j+UtDBMWRKQneZdTAkZq24d+n0BWz49xHftcYj8i0A9Do9xnEfaBR0xjmbftslNxxsDzxJ2SMpqi1ra//E5M04hJoIJo3XKdBIKMKOBUsLzkp5olhA5Ij7UMlcTsCbLxPzk+MEoHd2NlSgIeq78nMhJpPYxC0zk6V896I/E/r5VC9zzIuExSYJJOFnVTgSHGo3BwhytGQQwNIVRxcyumfaIIBRNhyYTgzr48T7yTykXFvTktVy+naRTRHtpHR8hFZ6iKrlENeYiiR/SMXtGb9WS9WO/Wx6S1YE1ndtEfWJ8/jaeb8g==</latexit><latexit sha1_base64="EGo0+wpGbNsctCEWF04o7ynaSbg=">AAACDnicbVBNS8NAEN3Ur1q/oh69LBbFg5REBPVW9OKxFmMLTSyb7bZdutmE3YlQQv6BF/+KFw8qXj1789+4/Tho64OBx3szzMwLE8E1OM63VVhYXFpeKa6W1tY3Nrfs7Z07HaeKMo/GIlbNkGgmuGQecBCsmShGolCwRji4GvmNB6Y0j+UtDBMWRKQneZdTAkZq24d+n0BWz49xHftcYj8i0A9Do9xnEfaBR0xjmbftslNxxsDzxJ2SMpqi1ra//E5M04hJoIJo3XKdBIKMKOBUsLzkp5olhA5Ij7UMlcTsCbLxPzk+MEoHd2NlSgIeq78nMhJpPYxC0zk6V896I/E/r5VC9zzIuExSYJJOFnVTgSHGo3BwhytGQQwNIVRxcyumfaIIBRNhyYTgzr48T7yTykXFvTktVy+naRTRHtpHR8hFZ6iKrlENeYiiR/SMXtGb9WS9WO/Wx6S1YE1ndtEfWJ8/jaeb8g==</latexit>
⌦0
<latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit>
|⌦0
|<latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit>
1i,j =
⇢
1 if (i, j) 2 ⌦
0 otherwise<latexit sha1_base64="CFycjGWklxDDCf6zjqtrsLtGCfg=">AAACZ3icbVFdaxQxFM2MX3Xrx/qJ4MvVRWmhLDMiqA9C0RffrODawmZZMtk7s2kzmSG5oy4h+B998wf44q8wu50H23ogcDj33HuTk6LVylGW/UrSS5evXL22dX2wfePmrdvDO3e/uKazEiey0Y09KoRDrQxOSJHGo9aiqAuNh8XJ+3X98CtapxrzmVYtzmpRGVUqKShK8+EPXgtaFqXPw9yrvePwlmssiXteYKWMF9aKVfBah+iA5+A54XcCD6qEsKP24HgXuDLAP9ZYiQCcg8/OGBtaov2mHEIIHM2iH8mtqpY0ng9H2TjbAC6SvCcj1uNgPvzJF43sajQktXBummctzeJQUlJjGPDOYSvkiahwGqkRNbqZ3wQV4FlUFlA2Nh5DsFH/7fCidm5VF9G5jsWdr63F/9WmHZWvZ16ZtiM08nRR2WmgBtapw0JZlKRXkQhpVbwryKWwQlL8m0EMIT//5Itk8mL8Zpx/ejnaf9enscUes6dsh+XsFdtnH9gBmzDJfifbyf3kQfInHaYP00en1jTpe+6xM0if/AXzmLlG</latexit><latexit sha1_base64="CFycjGWklxDDCf6zjqtrsLtGCfg=">AAACZ3icbVFdaxQxFM2MX3Xrx/qJ4MvVRWmhLDMiqA9C0RffrODawmZZMtk7s2kzmSG5oy4h+B998wf44q8wu50H23ogcDj33HuTk6LVylGW/UrSS5evXL22dX2wfePmrdvDO3e/uKazEiey0Y09KoRDrQxOSJHGo9aiqAuNh8XJ+3X98CtapxrzmVYtzmpRGVUqKShK8+EPXgtaFqXPw9yrvePwlmssiXteYKWMF9aKVfBah+iA5+A54XcCD6qEsKP24HgXuDLAP9ZYiQCcg8/OGBtaov2mHEIIHM2iH8mtqpY0ng9H2TjbAC6SvCcj1uNgPvzJF43sajQktXBummctzeJQUlJjGPDOYSvkiahwGqkRNbqZ3wQV4FlUFlA2Nh5DsFH/7fCidm5VF9G5jsWdr63F/9WmHZWvZ16ZtiM08nRR2WmgBtapw0JZlKRXkQhpVbwryKWwQlL8m0EMIT//5Itk8mL8Zpx/ejnaf9enscUes6dsh+XsFdtnH9gBmzDJfifbyf3kQfInHaYP00en1jTpe+6xM0if/AXzmLlG</latexit><latexit sha1_base64="CFycjGWklxDDCf6zjqtrsLtGCfg=">AAACZ3icbVFdaxQxFM2MX3Xrx/qJ4MvVRWmhLDMiqA9C0RffrODawmZZMtk7s2kzmSG5oy4h+B998wf44q8wu50H23ogcDj33HuTk6LVylGW/UrSS5evXL22dX2wfePmrdvDO3e/uKazEiey0Y09KoRDrQxOSJHGo9aiqAuNh8XJ+3X98CtapxrzmVYtzmpRGVUqKShK8+EPXgtaFqXPw9yrvePwlmssiXteYKWMF9aKVfBah+iA5+A54XcCD6qEsKP24HgXuDLAP9ZYiQCcg8/OGBtaov2mHEIIHM2iH8mtqpY0ng9H2TjbAC6SvCcj1uNgPvzJF43sajQktXBummctzeJQUlJjGPDOYSvkiahwGqkRNbqZ3wQV4FlUFlA2Nh5DsFH/7fCidm5VF9G5jsWdr63F/9WmHZWvZ16ZtiM08nRR2WmgBtapw0JZlKRXkQhpVbwryKWwQlL8m0EMIT//5Itk8mL8Zpx/ejnaf9enscUes6dsh+XsFdtnH9gBmzDJfifbyf3kQfInHaYP00en1jTpe+6xM0if/AXzmLlG</latexit><latexit sha1_base64="CFycjGWklxDDCf6zjqtrsLtGCfg=">AAACZ3icbVFdaxQxFM2MX3Xrx/qJ4MvVRWmhLDMiqA9C0RffrODawmZZMtk7s2kzmSG5oy4h+B998wf44q8wu50H23ogcDj33HuTk6LVylGW/UrSS5evXL22dX2wfePmrdvDO3e/uKazEiey0Y09KoRDrQxOSJHGo9aiqAuNh8XJ+3X98CtapxrzmVYtzmpRGVUqKShK8+EPXgtaFqXPw9yrvePwlmssiXteYKWMF9aKVfBah+iA5+A54XcCD6qEsKP24HgXuDLAP9ZYiQCcg8/OGBtaov2mHEIIHM2iH8mtqpY0ng9H2TjbAC6SvCcj1uNgPvzJF43sajQktXBummctzeJQUlJjGPDOYSvkiahwGqkRNbqZ3wQV4FlUFlA2Nh5DsFH/7fCidm5VF9G5jsWdr63F/9WmHZWvZ16ZtiM08nRR2WmgBtapw0JZlKRXkQhpVbwryKWwQlL8m0EMIT//5Itk8mL8Zpx/ejnaf9enscUes6dsh+XsFdtnH9gBmzDJfifbyf3kQfInHaYP00en1jTpe+6xM0if/AXzmLlG</latexit>
RMSE =
v
u
u
t
1
|⌦|
X
(i,j)2⌦
⇣
ˆRi,j Ri,j
⌘2
=
v
u
u
t 1
|⌦|
mX
i=1
nX
j=1
1i,j
⇣
ˆRi,j Ri,j
⌘2
<latexit sha1_base64="jvinTaGm+A+OeUSYy58E379ffRQ=">AAACyniclVFLbxMxEPYurxJeAY5cLCKkVCrRboVEe4hUgZB6qSiB0EpxuvI63sSp7V3sWSByfeMXcuPKL8Gb7AFaODCSpW++eXm+ySspLCTJjyi+dv3GzVtbtzt37t67/6D78NFHW9aG8TErZWlOc2q5FJqPQYDkp5XhVOWSn+Tnr5v4yWdurCj1B1hVfKroXItCMAqByro/SVlxQ6E0miruRkfv3/ghsZ8MOEwKQ5lLPXYX5K3ic3rhMbG1ylxf7Cy3MREabwKBl7yAPllQcCOfObGDl/75qAXEiPkCts/crvd4iDub/v9uL4apP1Ots2ycMEhRWORFyG+aLv9jYtbtJYNkbfgqSFvQQ60dZ93vZFayWnENTFJrJ2lSwdRRA4JJ7juktryi7JzO+STARjg7detbePwsMDNclCY8DXjN/l7hqLJ2pfKQ2WxkL8ca8m+xSQ3F3tQJXdXANdsMKmqJocTNYfFMGM5ArgKgzIjwV8wWNEgM4fydIEJ6eeWrYLw72B+k7170Dl61amyhJ+gp6qMUvUQH6BAdozFi0WGkoy/R1/gotvEqdpvUOGprHqM/LP72Cw/c38M=</latexit><latexit sha1_base64="jvinTaGm+A+OeUSYy58E379ffRQ=">AAACyniclVFLbxMxEPYurxJeAY5cLCKkVCrRboVEe4hUgZB6qSiB0EpxuvI63sSp7V3sWSByfeMXcuPKL8Gb7AFaODCSpW++eXm+ySspLCTJjyi+dv3GzVtbtzt37t67/6D78NFHW9aG8TErZWlOc2q5FJqPQYDkp5XhVOWSn+Tnr5v4yWdurCj1B1hVfKroXItCMAqByro/SVlxQ6E0miruRkfv3/ghsZ8MOEwKQ5lLPXYX5K3ic3rhMbG1ylxf7Cy3MREabwKBl7yAPllQcCOfObGDl/75qAXEiPkCts/crvd4iDub/v9uL4apP1Ots2ycMEhRWORFyG+aLv9jYtbtJYNkbfgqSFvQQ60dZ93vZFayWnENTFJrJ2lSwdRRA4JJ7juktryi7JzO+STARjg7detbePwsMDNclCY8DXjN/l7hqLJ2pfKQ2WxkL8ca8m+xSQ3F3tQJXdXANdsMKmqJocTNYfFMGM5ArgKgzIjwV8wWNEgM4fydIEJ6eeWrYLw72B+k7170Dl61amyhJ+gp6qMUvUQH6BAdozFi0WGkoy/R1/gotvEqdpvUOGprHqM/LP72Cw/c38M=</latexit><latexit sha1_base64="jvinTaGm+A+OeUSYy58E379ffRQ=">AAACyniclVFLbxMxEPYurxJeAY5cLCKkVCrRboVEe4hUgZB6qSiB0EpxuvI63sSp7V3sWSByfeMXcuPKL8Gb7AFaODCSpW++eXm+ySspLCTJjyi+dv3GzVtbtzt37t67/6D78NFHW9aG8TErZWlOc2q5FJqPQYDkp5XhVOWSn+Tnr5v4yWdurCj1B1hVfKroXItCMAqByro/SVlxQ6E0miruRkfv3/ghsZ8MOEwKQ5lLPXYX5K3ic3rhMbG1ylxf7Cy3MREabwKBl7yAPllQcCOfObGDl/75qAXEiPkCts/crvd4iDub/v9uL4apP1Ots2ycMEhRWORFyG+aLv9jYtbtJYNkbfgqSFvQQ60dZ93vZFayWnENTFJrJ2lSwdRRA4JJ7juktryi7JzO+STARjg7detbePwsMDNclCY8DXjN/l7hqLJ2pfKQ2WxkL8ca8m+xSQ3F3tQJXdXANdsMKmqJocTNYfFMGM5ArgKgzIjwV8wWNEgM4fydIEJ6eeWrYLw72B+k7170Dl61amyhJ+gp6qMUvUQH6BAdozFi0WGkoy/R1/gotvEqdpvUOGprHqM/LP72Cw/c38M=</latexit><latexit sha1_base64="jvinTaGm+A+OeUSYy58E379ffRQ=">AAACyniclVFLbxMxEPYurxJeAY5cLCKkVCrRboVEe4hUgZB6qSiB0EpxuvI63sSp7V3sWSByfeMXcuPKL8Gb7AFaODCSpW++eXm+ySspLCTJjyi+dv3GzVtbtzt37t67/6D78NFHW9aG8TErZWlOc2q5FJqPQYDkp5XhVOWSn+Tnr5v4yWdurCj1B1hVfKroXItCMAqByro/SVlxQ6E0miruRkfv3/ghsZ8MOEwKQ5lLPXYX5K3ic3rhMbG1ylxf7Cy3MREabwKBl7yAPllQcCOfObGDl/75qAXEiPkCts/crvd4iDub/v9uL4apP1Ots2ycMEhRWORFyG+aLv9jYtbtJYNkbfgqSFvQQ60dZ93vZFayWnENTFJrJ2lSwdRRA4JJ7juktryi7JzO+STARjg7detbePwsMDNclCY8DXjN/l7hqLJ2pfKQ2WxkL8ca8m+xSQ3F3tQJXdXANdsMKmqJocTNYfFMGM5ArgKgzIjwV8wWNEgM4fydIEJ6eeWrYLw72B+k7170Dl61amyhJ+gp6qMUvUQH6BAdozFi0WGkoy/R1/gotvEqdpvUOGprHqM/LP72Cw/c38M=</latexit>

6
• Optimization form of PMF:
• To find U and V via MAP (Maximum A Posteriori).
• A low-rank assumption ( ) and Gaussian noise assumptions.
• Also, place zero-mean spherical Gaussian priors on U and V,
PROBABILISTIC MATRIX FACTORIZATION (PMF)
R ' UV T
<latexit sha1_base64="jCmwqZd86gOUnUt5v9urx4yIgss=">AAAB83icbVBNTwIxEJ3FL8Qv1KOXRmLiiewaE/VG9OIRDQsksJJu6UJD213aLgkh/A4vHtR49c94899YYA8KvmSSl/dmMjMvTDjTxnW/ndza+sbmVn67sLO7t39QPDyq6zhVhPok5rFqhlhTziT1DTOcNhNFsQg5bYSDu5nfGFGlWSxrZpzQQOCeZBEj2FgpeERtzQQdIr/+VOsUS27ZnQOtEi8jJchQ7RS/2t2YpIJKQzjWuuW5iQkmWBlGOJ0W2qmmCSYD3KMtSyUWVAeT+dFTdGaVLopiZUsaNFd/T0yw0HosQtspsOnrZW8m/ue1UhNdBxMmk9RQSRaLopQjE6NZAqjLFCWGjy3BRDF7KyJ9rDAxNqeCDcFbfnmV+Bflm7L3cFmq3GZp5OEETuEcPLiCCtxDFXwgMIRneIU3Z+S8OO/Ox6I152Qzx/AHzucPIU+ROg==</latexit><latexit sha1_base64="jCmwqZd86gOUnUt5v9urx4yIgss=">AAAB83icbVBNTwIxEJ3FL8Qv1KOXRmLiiewaE/VG9OIRDQsksJJu6UJD213aLgkh/A4vHtR49c94899YYA8KvmSSl/dmMjMvTDjTxnW/ndza+sbmVn67sLO7t39QPDyq6zhVhPok5rFqhlhTziT1DTOcNhNFsQg5bYSDu5nfGFGlWSxrZpzQQOCeZBEj2FgpeERtzQQdIr/+VOsUS27ZnQOtEi8jJchQ7RS/2t2YpIJKQzjWuuW5iQkmWBlGOJ0W2qmmCSYD3KMtSyUWVAeT+dFTdGaVLopiZUsaNFd/T0yw0HosQtspsOnrZW8m/ue1UhNdBxMmk9RQSRaLopQjE6NZAqjLFCWGjy3BRDF7KyJ9rDAxNqeCDcFbfnmV+Bflm7L3cFmq3GZp5OEETuEcPLiCCtxDFXwgMIRneIU3Z+S8OO/Ox6I152Qzx/AHzucPIU+ROg==</latexit><latexit sha1_base64="jCmwqZd86gOUnUt5v9urx4yIgss=">AAAB83icbVBNTwIxEJ3FL8Qv1KOXRmLiiewaE/VG9OIRDQsksJJu6UJD213aLgkh/A4vHtR49c94899YYA8KvmSSl/dmMjMvTDjTxnW/ndza+sbmVn67sLO7t39QPDyq6zhVhPok5rFqhlhTziT1DTOcNhNFsQg5bYSDu5nfGFGlWSxrZpzQQOCeZBEj2FgpeERtzQQdIr/+VOsUS27ZnQOtEi8jJchQ7RS/2t2YpIJKQzjWuuW5iQkmWBlGOJ0W2qmmCSYD3KMtSyUWVAeT+dFTdGaVLopiZUsaNFd/T0yw0HosQtspsOnrZW8m/ue1UhNdBxMmk9RQSRaLopQjE6NZAqjLFCWGjy3BRDF7KyJ9rDAxNqeCDcFbfnmV+Bflm7L3cFmq3GZp5OEETuEcPLiCCtxDFXwgMIRneIU3Z+S8OO/Ox6I152Qzx/AHzucPIU+ROg==</latexit><latexit sha1_base64="jCmwqZd86gOUnUt5v9urx4yIgss=">AAAB83icbVBNTwIxEJ3FL8Qv1KOXRmLiiewaE/VG9OIRDQsksJJu6UJD213aLgkh/A4vHtR49c94899YYA8KvmSSl/dmMjMvTDjTxnW/ndza+sbmVn67sLO7t39QPDyq6zhVhPok5rFqhlhTziT1DTOcNhNFsQg5bYSDu5nfGFGlWSxrZpzQQOCeZBEj2FgpeERtzQQdIr/+VOsUS27ZnQOtEi8jJchQ7RS/2t2YpIJKQzjWuuW5iQkmWBlGOJ0W2qmmCSYD3KMtSyUWVAeT+dFTdGaVLopiZUsaNFd/T0yw0HosQtspsOnrZW8m/ue1UhNdBxMmk9RQSRaLopQjE6NZAqjLFCWGjy3BRDF7KyJ9rDAxNqeCDcFbfnmV+Bflm7L3cFmq3GZp5OEETuEcPLiCCtxDFXwgMIRneIU3Z+S8OO/Ox6I152Qzx/AHzucPIU+ROg==</latexit>
Pr R|U, V, 2
=
mY
i=1
nY
j=1
⇥
N Ri,j|UiV T
j , 2
⇤1i,j
<latexit sha1_base64="rzUcx0Crp/tm/RsFTd0lzBU239I=">AAACknicbVFdb9MwFHUyYKN8BRBPvFhUSEOqqmRCA4QmCnvhAaGClm5Sk1aO67Te7DiybyZVxn+In8Mb/wanDRJsu5Llcz/Ovb7HRS24gTj+HYQ7t27f2d2727t3/8HDR9HjJxOjGk1ZSpVQ+qwghglesRQ4CHZWa0ZkIdhpcXHc5k8vmTZcVSewrlkuybLiJacEfGge/cxUzTQBpSsimR1rlwlWwv53/AOnAzwZ4MzwpSQze+AyzZcreHWU1Vot5pYfJW5mpcOdf77xK++3HaY4kwRWlAj79W9Pzxngc9e29tDhiSd5yom7Ycr2ymc26VhuHvXjYbwxfB0kHeijzsbz6Fe2ULSRrAIqiDHTJK4ht0QDp4K5XtYYVhN6QZZs6mErgMntRlOHX/rIApdK+1MB3kT/ZVgijVnLwle2e5qruTZ4U27aQPk2t7yqG2AV3Q4qG4FB4faD8IJrRkGsPSBUc/9WTFdEEwr+G3tehOTqytdBejB8N0y+ve6PPnVq7KHn6AXaRwl6g0boMxqjFNEgCg6DD8EofBa+Dz+Gx9vSMOg4T9F/Fn75A/PmyUk=</latexit><latexit sha1_base64="rzUcx0Crp/tm/RsFTd0lzBU239I=">AAACknicbVFdb9MwFHUyYKN8BRBPvFhUSEOqqmRCA4QmCnvhAaGClm5Sk1aO67Te7DiybyZVxn+In8Mb/wanDRJsu5Llcz/Ovb7HRS24gTj+HYQ7t27f2d2727t3/8HDR9HjJxOjGk1ZSpVQ+qwghglesRQ4CHZWa0ZkIdhpcXHc5k8vmTZcVSewrlkuybLiJacEfGge/cxUzTQBpSsimR1rlwlWwv53/AOnAzwZ4MzwpSQze+AyzZcreHWU1Vot5pYfJW5mpcOdf77xK++3HaY4kwRWlAj79W9Pzxngc9e29tDhiSd5yom7Ycr2ymc26VhuHvXjYbwxfB0kHeijzsbz6Fe2ULSRrAIqiDHTJK4ht0QDp4K5XtYYVhN6QZZs6mErgMntRlOHX/rIApdK+1MB3kT/ZVgijVnLwle2e5qruTZ4U27aQPk2t7yqG2AV3Q4qG4FB4faD8IJrRkGsPSBUc/9WTFdEEwr+G3tehOTqytdBejB8N0y+ve6PPnVq7KHn6AXaRwl6g0boMxqjFNEgCg6DD8EofBa+Dz+Gx9vSMOg4T9F/Fn75A/PmyUk=</latexit><latexit sha1_base64="rzUcx0Crp/tm/RsFTd0lzBU239I=">AAACknicbVFdb9MwFHUyYKN8BRBPvFhUSEOqqmRCA4QmCnvhAaGClm5Sk1aO67Te7DiybyZVxn+In8Mb/wanDRJsu5Llcz/Ovb7HRS24gTj+HYQ7t27f2d2727t3/8HDR9HjJxOjGk1ZSpVQ+qwghglesRQ4CHZWa0ZkIdhpcXHc5k8vmTZcVSewrlkuybLiJacEfGge/cxUzTQBpSsimR1rlwlWwv53/AOnAzwZ4MzwpSQze+AyzZcreHWU1Vot5pYfJW5mpcOdf77xK++3HaY4kwRWlAj79W9Pzxngc9e29tDhiSd5yom7Ycr2ymc26VhuHvXjYbwxfB0kHeijzsbz6Fe2ULSRrAIqiDHTJK4ht0QDp4K5XtYYVhN6QZZs6mErgMntRlOHX/rIApdK+1MB3kT/ZVgijVnLwle2e5qruTZ4U27aQPk2t7yqG2AV3Q4qG4FB4faD8IJrRkGsPSBUc/9WTFdEEwr+G3tehOTqytdBejB8N0y+ve6PPnVq7KHn6AXaRwl6g0boMxqjFNEgCg6DD8EofBa+Dz+Gx9vSMOg4T9F/Fn75A/PmyUk=</latexit><latexit sha1_base64="rzUcx0Crp/tm/RsFTd0lzBU239I=">AAACknicbVFdb9MwFHUyYKN8BRBPvFhUSEOqqmRCA4QmCnvhAaGClm5Sk1aO67Te7DiybyZVxn+In8Mb/wanDRJsu5Llcz/Ovb7HRS24gTj+HYQ7t27f2d2727t3/8HDR9HjJxOjGk1ZSpVQ+qwghglesRQ4CHZWa0ZkIdhpcXHc5k8vmTZcVSewrlkuybLiJacEfGge/cxUzTQBpSsimR1rlwlWwv53/AOnAzwZ4MzwpSQze+AyzZcreHWU1Vot5pYfJW5mpcOdf77xK++3HaY4kwRWlAj79W9Pzxngc9e29tDhiSd5yom7Ycr2ymc26VhuHvXjYbwxfB0kHeijzsbz6Fe2ULSRrAIqiDHTJK4ht0QDp4K5XtYYVhN6QZZs6mErgMntRlOHX/rIApdK+1MB3kT/ZVgijVnLwle2e5qruTZ4U27aQPk2t7yqG2AV3Q4qG4FB4faD8IJrRkGsPSBUc/9WTFdEEwr+G3tehOTqytdBejB8N0y+ve6PPnVq7KHn6AXaRwl6g0boMxqjFNEgCg6DD8EofBa+Dz+Gx9vSMOg4T9F/Fn75A/PmyUk=</latexit>
Pr U| 2
U =
mY
i=1
N Ui|0, 2
U I , Pr V | 2
V =
nY
j=1
N Vj|0, 2
V I
<latexit sha1_base64="ldTOt0DBLUYCU7PVN1Jq2Yx3kiQ=">AAACznicdVLRatswFJXddeuybs22x72IhUELI9il0O6hUNaXDUpJYXYLcWquZTlRK8meJG8E1ex137e3/sC+Y0riji7NLggO59x7z9WVsoozbYLg1vPXHq0/frLxtPNs8/mLre7LV7Eua0VoREpeqosMNOVM0sgww+lFpSiIjNPz7Pp4pp9/o0qzUn4x04qOBIwlKxgB46i0+zsZqITTwmxH+AYnmo0FpDZqLu1ukyg2npidw6RSZZ5adhg6WjQ4EWAmBLg9bdpSJzauPHi/1GGRmhX2810zl/G1hhz/tY3v2carbK/mtnKFbezEJdv4P7Zptxf0g3nghyBsQQ+1MUi7v5K8JLWg0hAOWg/DoDIjC8owwmnTSWpNKyDXMKZDByUIqkd2/h4NfueYHBelckcaPGfvV1gQWk9F5jJng+plbUau0oa1KQ5GlsmqNlSShVFRc2xKPHtcnDNFieFTB4Ao5mbFZAIKiHFfoOOWEC5f+SGIdvsf+uHZXu/oY7uNDfQGvUXbKET76Ah9QgMUIeKdeMqz3o1/5n/3G//HItX32prX6J/wf/4B4Mbh1g==</latexit><latexit sha1_base64="ldTOt0DBLUYCU7PVN1Jq2Yx3kiQ=">AAACznicdVLRatswFJXddeuybs22x72IhUELI9il0O6hUNaXDUpJYXYLcWquZTlRK8meJG8E1ex137e3/sC+Y0riji7NLggO59x7z9WVsoozbYLg1vPXHq0/frLxtPNs8/mLre7LV7Eua0VoREpeqosMNOVM0sgww+lFpSiIjNPz7Pp4pp9/o0qzUn4x04qOBIwlKxgB46i0+zsZqITTwmxH+AYnmo0FpDZqLu1ukyg2npidw6RSZZ5adhg6WjQ4EWAmBLg9bdpSJzauPHi/1GGRmhX2810zl/G1hhz/tY3v2carbK/mtnKFbezEJdv4P7Zptxf0g3nghyBsQQ+1MUi7v5K8JLWg0hAOWg/DoDIjC8owwmnTSWpNKyDXMKZDByUIqkd2/h4NfueYHBelckcaPGfvV1gQWk9F5jJng+plbUau0oa1KQ5GlsmqNlSShVFRc2xKPHtcnDNFieFTB4Ao5mbFZAIKiHFfoOOWEC5f+SGIdvsf+uHZXu/oY7uNDfQGvUXbKET76Ah9QgMUIeKdeMqz3o1/5n/3G//HItX32prX6J/wf/4B4Mbh1g==</latexit><latexit sha1_base64="ldTOt0DBLUYCU7PVN1Jq2Yx3kiQ=">AAACznicdVLRatswFJXddeuybs22x72IhUELI9il0O6hUNaXDUpJYXYLcWquZTlRK8meJG8E1ex137e3/sC+Y0riji7NLggO59x7z9WVsoozbYLg1vPXHq0/frLxtPNs8/mLre7LV7Eua0VoREpeqosMNOVM0sgww+lFpSiIjNPz7Pp4pp9/o0qzUn4x04qOBIwlKxgB46i0+zsZqITTwmxH+AYnmo0FpDZqLu1ukyg2npidw6RSZZ5adhg6WjQ4EWAmBLg9bdpSJzauPHi/1GGRmhX2810zl/G1hhz/tY3v2carbK/mtnKFbezEJdv4P7Zptxf0g3nghyBsQQ+1MUi7v5K8JLWg0hAOWg/DoDIjC8owwmnTSWpNKyDXMKZDByUIqkd2/h4NfueYHBelckcaPGfvV1gQWk9F5jJng+plbUau0oa1KQ5GlsmqNlSShVFRc2xKPHtcnDNFieFTB4Ao5mbFZAIKiHFfoOOWEC5f+SGIdvsf+uHZXu/oY7uNDfQGvUXbKET76Ah9QgMUIeKdeMqz3o1/5n/3G//HItX32prX6J/wf/4B4Mbh1g==</latexit><latexit sha1_base64="ldTOt0DBLUYCU7PVN1Jq2Yx3kiQ=">AAACznicdVLRatswFJXddeuybs22x72IhUELI9il0O6hUNaXDUpJYXYLcWquZTlRK8meJG8E1ex137e3/sC+Y0riji7NLggO59x7z9WVsoozbYLg1vPXHq0/frLxtPNs8/mLre7LV7Eua0VoREpeqosMNOVM0sgww+lFpSiIjNPz7Pp4pp9/o0qzUn4x04qOBIwlKxgB46i0+zsZqITTwmxH+AYnmo0FpDZqLu1ukyg2npidw6RSZZ5adhg6WjQ4EWAmBLg9bdpSJzauPHi/1GGRmhX2810zl/G1hhz/tY3v2carbK/mtnKFbezEJdv4P7Zptxf0g3nghyBsQQ+1MUi7v5K8JLWg0hAOWg/DoDIjC8owwmnTSWpNKyDXMKZDByUIqkd2/h4NfueYHBelckcaPGfvV1gQWk9F5jJng+plbUau0oa1KQ5GlsmqNlSShVFRc2xKPHtcnDNFieFTB4Ao5mbFZAIKiHFfoOOWEC5f+SGIdvsf+uHZXu/oY7uNDfQGvUXbKET76Ah9QgMUIeKdeMqz3o1/5n/3G//HItX32prX6J/wf/4B4Mbh1g==</latexit>
arg max
U,V
Pr R|U, V, 2
s.t. rank(U) = rank(V ) = k
<latexit sha1_base64="bY6ARNoDq/sEODPYgRv/Km1OuwA=">AAACWHicbZFNa9tAEIZXavPlfjnJsZehpuBAEFIItD0UQnvp0S2VE/C6ZrReyYtXH+yOSoyiP9lDofSv5NCVo0ObdGDh5Xln2Nl3k0orS2H4y/MfPd7Z3ds/GDx5+uz5i+Hh0dSWtREyFqUuzVWCVmpVyJgUaXlVGYl5ouVlsv7Y+ZffpbGqLL7SppLzHLNCpUogObQYVhxNBjzHa1g08SlMW+BlJQ1SaQrMZTMxLdcypfEXuIGu4RS4VVmO35qzlhuVregEOMlrasAGFIDBYt3COD6B9z3vybQj68VwFAbhtuChiHoxYn1NFsMffFmKOpcFCY3WzqKwonmDhpTQsh3w2soKxRozOXOyW9rOm20yLbx2ZAlpadwpCLb074kGc2s3eeI6c6SVve918H/erKb07bxRRVWTLMTdRWmtgUroYoalMlKQ3jiBwii3K4gVGhTkPmPgQojuP/mhiM+Cd0H0+Xx08aFPY5+9ZK/YmEXsDbtgn9iExUywn+zW2/F2vd++5+/5B3etvtfPHLN/yj/6A/TzsSU=</latexit><latexit sha1_base64="bY6ARNoDq/sEODPYgRv/Km1OuwA=">AAACWHicbZFNa9tAEIZXavPlfjnJsZehpuBAEFIItD0UQnvp0S2VE/C6ZrReyYtXH+yOSoyiP9lDofSv5NCVo0ObdGDh5Xln2Nl3k0orS2H4y/MfPd7Z3ds/GDx5+uz5i+Hh0dSWtREyFqUuzVWCVmpVyJgUaXlVGYl5ouVlsv7Y+ZffpbGqLL7SppLzHLNCpUogObQYVhxNBjzHa1g08SlMW+BlJQ1SaQrMZTMxLdcypfEXuIGu4RS4VVmO35qzlhuVregEOMlrasAGFIDBYt3COD6B9z3vybQj68VwFAbhtuChiHoxYn1NFsMffFmKOpcFCY3WzqKwonmDhpTQsh3w2soKxRozOXOyW9rOm20yLbx2ZAlpadwpCLb074kGc2s3eeI6c6SVve918H/erKb07bxRRVWTLMTdRWmtgUroYoalMlKQ3jiBwii3K4gVGhTkPmPgQojuP/mhiM+Cd0H0+Xx08aFPY5+9ZK/YmEXsDbtgn9iExUywn+zW2/F2vd++5+/5B3etvtfPHLN/yj/6A/TzsSU=</latexit><latexit sha1_base64="bY6ARNoDq/sEODPYgRv/Km1OuwA=">AAACWHicbZFNa9tAEIZXavPlfjnJsZehpuBAEFIItD0UQnvp0S2VE/C6ZrReyYtXH+yOSoyiP9lDofSv5NCVo0ObdGDh5Xln2Nl3k0orS2H4y/MfPd7Z3ds/GDx5+uz5i+Hh0dSWtREyFqUuzVWCVmpVyJgUaXlVGYl5ouVlsv7Y+ZffpbGqLL7SppLzHLNCpUogObQYVhxNBjzHa1g08SlMW+BlJQ1SaQrMZTMxLdcypfEXuIGu4RS4VVmO35qzlhuVregEOMlrasAGFIDBYt3COD6B9z3vybQj68VwFAbhtuChiHoxYn1NFsMffFmKOpcFCY3WzqKwonmDhpTQsh3w2soKxRozOXOyW9rOm20yLbx2ZAlpadwpCLb074kGc2s3eeI6c6SVve918H/erKb07bxRRVWTLMTdRWmtgUroYoalMlKQ3jiBwii3K4gVGhTkPmPgQojuP/mhiM+Cd0H0+Xx08aFPY5+9ZK/YmEXsDbtgn9iExUywn+zW2/F2vd++5+/5B3etvtfPHLN/yj/6A/TzsSU=</latexit><latexit sha1_base64="bY6ARNoDq/sEODPYgRv/Km1OuwA=">AAACWHicbZFNa9tAEIZXavPlfjnJsZehpuBAEFIItD0UQnvp0S2VE/C6ZrReyYtXH+yOSoyiP9lDofSv5NCVo0ObdGDh5Xln2Nl3k0orS2H4y/MfPd7Z3ds/GDx5+uz5i+Hh0dSWtREyFqUuzVWCVmpVyJgUaXlVGYl5ouVlsv7Y+ZffpbGqLL7SppLzHLNCpUogObQYVhxNBjzHa1g08SlMW+BlJQ1SaQrMZTMxLdcypfEXuIGu4RS4VVmO35qzlhuVregEOMlrasAGFIDBYt3COD6B9z3vybQj68VwFAbhtuChiHoxYn1NFsMffFmKOpcFCY3WzqKwonmDhpTQsh3w2soKxRozOXOyW9rOm20yLbx2ZAlpadwpCLb074kGc2s3eeI6c6SVve918H/erKb07bxRRVWTLMTdRWmtgUroYoalMlKQ3jiBwii3K4gVGhTkPmPgQojuP/mhiM+Cd0H0+Xx08aFPY5+9ZK/YmEXsDbtgn9iExUywn+zW2/F2vd++5+/5B3etvtfPHLN/yj/6A/TzsSU=</latexit>

7
• The log of the posterior distribution over U and V,
• We can see sigmas as hyper-parameters.
• Graphical Model for PMF:
PROBABILISTIC MATRIX FACTORIZATION (PMF)
l =
1
2 2
mX
i=1
nX
j=1
1i,j Ri,j UiV T
j
2 1
2 2
U
kUkF
1
2 2
V
kV kF
<latexit sha1_base64="4XOOk2l9NvGz4IhzPsuhSl4W/wA=">AAACpXicbVFdb9MwFHXC1ygfK/DIi0WFVCRWJRUS8DBpAgnBCxqjySY1XeS4TuvOdiLbQaqMfxp/gjf+DTdZELDuSrbOvfccX/u4qAU3Nop+BeGNm7du39m7O7h3/8HD/eGjx6mpGk1ZQitR6bOCGCa4YonlVrCzWjMiC8FOi4v3bf/0G9OGV2pmtzVbSLJSvOSUWCjlwx/i8CArNaEu9m6KM8NXkpy7qfeAG5k7fhj7cyf/pJsuVZBKYtdFCTLgvNz4TLDSjk/aBG/8QQLA4xQEQJ8BXfPV2r5oT96dl7vE9zO/4wS2/MN1pPQvKe1IOB+OoknUBd4FcQ9GqI/jfPgzW1a0kUxZKogx8ziq7cIRbTkVzA+yxrCa0AuyYnOAikhmFq6z2ePnUFnistKwlMVd9V+FI9KYrSyA2Zpjrvba4nW9eWPLNwvHVd1YpujloLIR2Fa4/TO85JpRK7YACNUc7orpmoA/Fn52ACbEV5+8C5Lp5O0k/vJqdPSud2MPPUXP0BjF6DU6Qh/RMUoQDUbBp+Ak+BqOw8/hLEwvqWHQa56g/yLMfwP2ndAg</latexit><latexit sha1_base64="4XOOk2l9NvGz4IhzPsuhSl4W/wA=">AAACpXicbVFdb9MwFHXC1ygfK/DIi0WFVCRWJRUS8DBpAgnBCxqjySY1XeS4TuvOdiLbQaqMfxp/gjf+DTdZELDuSrbOvfccX/u4qAU3Nop+BeGNm7du39m7O7h3/8HD/eGjx6mpGk1ZQitR6bOCGCa4YonlVrCzWjMiC8FOi4v3bf/0G9OGV2pmtzVbSLJSvOSUWCjlwx/i8CArNaEu9m6KM8NXkpy7qfeAG5k7fhj7cyf/pJsuVZBKYtdFCTLgvNz4TLDSjk/aBG/8QQLA4xQEQJ8BXfPV2r5oT96dl7vE9zO/4wS2/MN1pPQvKe1IOB+OoknUBd4FcQ9GqI/jfPgzW1a0kUxZKogx8ziq7cIRbTkVzA+yxrCa0AuyYnOAikhmFq6z2ePnUFnistKwlMVd9V+FI9KYrSyA2Zpjrvba4nW9eWPLNwvHVd1YpujloLIR2Fa4/TO85JpRK7YACNUc7orpmoA/Fn52ACbEV5+8C5Lp5O0k/vJqdPSud2MPPUXP0BjF6DU6Qh/RMUoQDUbBp+Ak+BqOw8/hLEwvqWHQa56g/yLMfwP2ndAg</latexit><latexit sha1_base64="4XOOk2l9NvGz4IhzPsuhSl4W/wA=">AAACpXicbVFdb9MwFHXC1ygfK/DIi0WFVCRWJRUS8DBpAgnBCxqjySY1XeS4TuvOdiLbQaqMfxp/gjf+DTdZELDuSrbOvfccX/u4qAU3Nop+BeGNm7du39m7O7h3/8HD/eGjx6mpGk1ZQitR6bOCGCa4YonlVrCzWjMiC8FOi4v3bf/0G9OGV2pmtzVbSLJSvOSUWCjlwx/i8CArNaEu9m6KM8NXkpy7qfeAG5k7fhj7cyf/pJsuVZBKYtdFCTLgvNz4TLDSjk/aBG/8QQLA4xQEQJ8BXfPV2r5oT96dl7vE9zO/4wS2/MN1pPQvKe1IOB+OoknUBd4FcQ9GqI/jfPgzW1a0kUxZKogx8ziq7cIRbTkVzA+yxrCa0AuyYnOAikhmFq6z2ePnUFnistKwlMVd9V+FI9KYrSyA2Zpjrvba4nW9eWPLNwvHVd1YpujloLIR2Fa4/TO85JpRK7YACNUc7orpmoA/Fn52ACbEV5+8C5Lp5O0k/vJqdPSud2MPPUXP0BjF6DU6Qh/RMUoQDUbBp+Ak+BqOw8/hLEwvqWHQa56g/yLMfwP2ndAg</latexit><latexit sha1_base64="4XOOk2l9NvGz4IhzPsuhSl4W/wA=">AAACpXicbVFdb9MwFHXC1ygfK/DIi0WFVCRWJRUS8DBpAgnBCxqjySY1XeS4TuvOdiLbQaqMfxp/gjf+DTdZELDuSrbOvfccX/u4qAU3Nop+BeGNm7du39m7O7h3/8HD/eGjx6mpGk1ZQitR6bOCGCa4YonlVrCzWjMiC8FOi4v3bf/0G9OGV2pmtzVbSLJSvOSUWCjlwx/i8CArNaEu9m6KM8NXkpy7qfeAG5k7fhj7cyf/pJsuVZBKYtdFCTLgvNz4TLDSjk/aBG/8QQLA4xQEQJ8BXfPV2r5oT96dl7vE9zO/4wS2/MN1pPQvKe1IOB+OoknUBd4FcQ9GqI/jfPgzW1a0kUxZKogx8ziq7cIRbTkVzA+yxrCa0AuyYnOAikhmFq6z2ePnUFnistKwlMVd9V+FI9KYrSyA2Zpjrvba4nW9eWPLNwvHVd1YpujloLIR2Fa4/TO85JpRK7YACNUc7orpmoA/Fn52ACbEV5+8C5Lp5O0k/vJqdPSud2MPPUXP0BjF6DU6Qh/RMUoQDUbBp+Ak+BqOw8/hLEwvqWHQa56g/yLMfwP2ndAg</latexit>
Ui Vj
Ri,j
i=1,…,m
j=1,…,n
σ
σU σV
ln Pr U, V |R, 2
, 2
V , 2
U =
1
2 2
mX
i=1
nX
j=1
1i,j Ri,j UiV T
j
2 1
2 2
U
kUkF
1
2 2
V
kV kF
1
2
0
@
0
@
mX
i=1
nX
j=1
1i,j
1
A ln 2
+ mk ln 2
U + nk ln 2
V
1
A + C
<latexit sha1_base64="TSI5jjMzDLa+cypiTq7ef+J6qfQ=">AAADiHiclVJdb9MwFHUTGKMD1sEjL1dUTEP9UFOxjT1MmjYJ8VimJZvUlMhxndZt7ESxg1Rl+S38J974Nzhpytp1L1wp1jnX5/p+5PpxyKTq9f7UDPPZ850Xuy/re69ev9lvHLx1ZJQmhNokCqPkzseShkxQWzEV0rs4oZj7Ib3151fF/e1PmkgWiRu1iOmI44lgASNYaZd3UPvlhgLcQQJuSAN1ZLfBgXu4boMr2YTjH1k/X2Evc/JNbpfcTdhkqj6dH3bcIMEks/Ksvxaea5xyL2PnlpbzFZ2VVGjKsZr6gQ7TmvYsXxZyXRCY5R1bgxwcHaDlN1q+zFa8vJ1vVZKW3YOtD+/rUyLnQeSUInDd+iF04EFblVGd/9fAskIoJ/tvDC3g8zXXqtIWiE23sz7T1pXXaPa6vdJgG1gVaKLKBl7jtzuOSMqpUCTEUg6tXqxGGU4UIyHN624qaYzJHE/oUEOBOZWjrNykHD5qzxiCKNGfUFB61yMyzKVccF8ri6bl47vC+dTdMFXBl1HGRJwqKsgyUZCGoCIo1hLGLKFEhQsNMEmYrhXIFOufofTy1vUQrMctbwO73z3rWt8/Ny8uq2nsovfoAzpCFjpFF+gbGiAbEWPHaBvHxom5Z1rmqXm2lBq1KuYd2jDz8i9U6x++</latexit><latexit sha1_base64="TSI5jjMzDLa+cypiTq7ef+J6qfQ=">AAADiHiclVJdb9MwFHUTGKMD1sEjL1dUTEP9UFOxjT1MmjYJ8VimJZvUlMhxndZt7ESxg1Rl+S38J974Nzhpytp1L1wp1jnX5/p+5PpxyKTq9f7UDPPZ850Xuy/re69ev9lvHLx1ZJQmhNokCqPkzseShkxQWzEV0rs4oZj7Ib3151fF/e1PmkgWiRu1iOmI44lgASNYaZd3UPvlhgLcQQJuSAN1ZLfBgXu4boMr2YTjH1k/X2Evc/JNbpfcTdhkqj6dH3bcIMEks/Ksvxaea5xyL2PnlpbzFZ2VVGjKsZr6gQ7TmvYsXxZyXRCY5R1bgxwcHaDlN1q+zFa8vJ1vVZKW3YOtD+/rUyLnQeSUInDd+iF04EFblVGd/9fAskIoJ/tvDC3g8zXXqtIWiE23sz7T1pXXaPa6vdJgG1gVaKLKBl7jtzuOSMqpUCTEUg6tXqxGGU4UIyHN624qaYzJHE/oUEOBOZWjrNykHD5qzxiCKNGfUFB61yMyzKVccF8ri6bl47vC+dTdMFXBl1HGRJwqKsgyUZCGoCIo1hLGLKFEhQsNMEmYrhXIFOufofTy1vUQrMctbwO73z3rWt8/Ny8uq2nsovfoAzpCFjpFF+gbGiAbEWPHaBvHxom5Z1rmqXm2lBq1KuYd2jDz8i9U6x++</latexit><latexit sha1_base64="TSI5jjMzDLa+cypiTq7ef+J6qfQ=">AAADiHiclVJdb9MwFHUTGKMD1sEjL1dUTEP9UFOxjT1MmjYJ8VimJZvUlMhxndZt7ESxg1Rl+S38J974Nzhpytp1L1wp1jnX5/p+5PpxyKTq9f7UDPPZ850Xuy/re69ev9lvHLx1ZJQmhNokCqPkzseShkxQWzEV0rs4oZj7Ib3151fF/e1PmkgWiRu1iOmI44lgASNYaZd3UPvlhgLcQQJuSAN1ZLfBgXu4boMr2YTjH1k/X2Evc/JNbpfcTdhkqj6dH3bcIMEks/Ksvxaea5xyL2PnlpbzFZ2VVGjKsZr6gQ7TmvYsXxZyXRCY5R1bgxwcHaDlN1q+zFa8vJ1vVZKW3YOtD+/rUyLnQeSUInDd+iF04EFblVGd/9fAskIoJ/tvDC3g8zXXqtIWiE23sz7T1pXXaPa6vdJgG1gVaKLKBl7jtzuOSMqpUCTEUg6tXqxGGU4UIyHN624qaYzJHE/oUEOBOZWjrNykHD5qzxiCKNGfUFB61yMyzKVccF8ri6bl47vC+dTdMFXBl1HGRJwqKsgyUZCGoCIo1hLGLKFEhQsNMEmYrhXIFOufofTy1vUQrMctbwO73z3rWt8/Ny8uq2nsovfoAzpCFjpFF+gbGiAbEWPHaBvHxom5Z1rmqXm2lBq1KuYd2jDz8i9U6x++</latexit><latexit sha1_base64="TSI5jjMzDLa+cypiTq7ef+J6qfQ=">AAADiHiclVJdb9MwFHUTGKMD1sEjL1dUTEP9UFOxjT1MmjYJ8VimJZvUlMhxndZt7ESxg1Rl+S38J974Nzhpytp1L1wp1jnX5/p+5PpxyKTq9f7UDPPZ850Xuy/re69ev9lvHLx1ZJQmhNokCqPkzseShkxQWzEV0rs4oZj7Ib3151fF/e1PmkgWiRu1iOmI44lgASNYaZd3UPvlhgLcQQJuSAN1ZLfBgXu4boMr2YTjH1k/X2Evc/JNbpfcTdhkqj6dH3bcIMEks/Ksvxaea5xyL2PnlpbzFZ2VVGjKsZr6gQ7TmvYsXxZyXRCY5R1bgxwcHaDlN1q+zFa8vJ1vVZKW3YOtD+/rUyLnQeSUInDd+iF04EFblVGd/9fAskIoJ/tvDC3g8zXXqtIWiE23sz7T1pXXaPa6vdJgG1gVaKLKBl7jtzuOSMqpUCTEUg6tXqxGGU4UIyHN624qaYzJHE/oUEOBOZWjrNykHD5qzxiCKNGfUFB61yMyzKVccF8ri6bl47vC+dTdMFXBl1HGRJwqKsgyUZCGoCIo1hLGLKFEhQsNMEmYrhXIFOufofTy1vUQrMctbwO73z3rWt8/Ny8uq2nsovfoAzpCFjpFF+gbGiAbEWPHaBvHxom5Z1rmqXm2lBq1KuYd2jDz8i9U6x++</latexit>

8
• In vanilla Low-rank Matrix Factorization Problem,  
a solution find one pair of U and V (single model).
• To enhance performance of single model,  
we can use ensemble methods.
• We can say some algorithm is low-rank matrix approximation 
when it finds multiple low-rank matrices to complete final matrix.
• For example,
• [ICML’13] LLoRMA: Local Low-rank Matrix Approximation
• [ICML’16] SMA: Low-rank Matrix Approximation with Stability
• [AAAI’17] GLOMA: Global Information in Local Matrix Approximation
• [NIPS’17] MRMA: Mixture Rank Matrix Approximation
• The current states-of-the-arts model in Matrix Completion  
on Movielens 10M and Netflix dataset.
LOW-RANK MATRIX APPROXIMATION

10
• Matrix Approximation methods proceed with ensembles in their own way.
• MRMA ensembles matrices with different ranks.
• Because we do not know which rank is appropriate. 
• Observation:
• For rows and columns with fewer samples, a lower rank is more appropriate.
• Otherwise, a higher rank is more appropriate.
OBSERVATION OF MRMA

11
• Probabilistic Matrix Factorization (PMF):
• Mixture-rank Matrix Approximation (MRMA):  
• : the weight for rank k of row i.
• : the weight for rank k of column j.
• are the factorized matrices of the rank-k matrix approximation model.
CONDITIONAL PROBABILITY
Pr R|U, V, ↵, , 2
=
mY
i=1
nY
j=1
" KX
k=1
↵k
i
k
j N
⇣
Ri,j|Uk
i V kT
j , 2
⌘
#1i,j
<latexit sha1_base64="Bov3h2AWc9NxaycTX6VmxFdJPI4=">AAAC3HicbVJbi9QwFE7rbR1voz76EhyEFYahXRbUh4VFXwRBRtnOLk47Jc2kM9lJmpCkwhADvvig4qs/zDf/hz/A9KLo7h4I5zuX7+vJSQvJqDZR9DMIL12+cvXazvXBjZu3bt8Z3r0306JWmCRYMKFOCqQJoxVJDDWMnEhFEC8YOS42L5r68XuiNBXVkdlKknG0qmhJMTI+lQ9/pUIShYxQFeLETpVLGSnN7lv4ASZjOBvDFDG5Rt4XxDRO0xVHC7vnUkVXa/P4IJVKLHNLD2K3sNzBPj5t46rTm6e65rndtLlXrhf1JB9uXKftKX3EkVljxOzrP8P4xjE8dc1Mfzmzvn9hj5y7YK7OZQvbyhWljV0v4/LhKJpErcHzIO7BCPQ2zYc/0qXANSeVwQxpPY8jaTKLlKGYETdIa00kwhu0InMPm1XqzLav4+Ajn1nCUih/KgPb7L8Mi7jWW174zmZSfbbWJC+qzWtTPs0srWRtSIW7D5U1g0bA5qnhkiqCDdt6gLCiflaI10ghbPwPMfBLiM9e+TxI9ibPJvGb/dHh834bO+ABeAh2QQyegEPwEkxBAnDwLvgYfA6+hHn4Kfwafutaw6Dn3Af/Wfj9N7/p6WY=</latexit><latexit sha1_base64="Bov3h2AWc9NxaycTX6VmxFdJPI4=">AAAC3HicbVJbi9QwFE7rbR1voz76EhyEFYahXRbUh4VFXwRBRtnOLk47Jc2kM9lJmpCkwhADvvig4qs/zDf/hz/A9KLo7h4I5zuX7+vJSQvJqDZR9DMIL12+cvXazvXBjZu3bt8Z3r0306JWmCRYMKFOCqQJoxVJDDWMnEhFEC8YOS42L5r68XuiNBXVkdlKknG0qmhJMTI+lQ9/pUIShYxQFeLETpVLGSnN7lv4ASZjOBvDFDG5Rt4XxDRO0xVHC7vnUkVXa/P4IJVKLHNLD2K3sNzBPj5t46rTm6e65rndtLlXrhf1JB9uXKftKX3EkVljxOzrP8P4xjE8dc1Mfzmzvn9hj5y7YK7OZQvbyhWljV0v4/LhKJpErcHzIO7BCPQ2zYc/0qXANSeVwQxpPY8jaTKLlKGYETdIa00kwhu0InMPm1XqzLav4+Ajn1nCUih/KgPb7L8Mi7jWW174zmZSfbbWJC+qzWtTPs0srWRtSIW7D5U1g0bA5qnhkiqCDdt6gLCiflaI10ghbPwPMfBLiM9e+TxI9ibPJvGb/dHh834bO+ABeAh2QQyegEPwEkxBAnDwLvgYfA6+hHn4Kfwafutaw6Dn3Af/Wfj9N7/p6WY=</latexit><latexit sha1_base64="Bov3h2AWc9NxaycTX6VmxFdJPI4=">AAAC3HicbVJbi9QwFE7rbR1voz76EhyEFYahXRbUh4VFXwRBRtnOLk47Jc2kM9lJmpCkwhADvvig4qs/zDf/hz/A9KLo7h4I5zuX7+vJSQvJqDZR9DMIL12+cvXazvXBjZu3bt8Z3r0306JWmCRYMKFOCqQJoxVJDDWMnEhFEC8YOS42L5r68XuiNBXVkdlKknG0qmhJMTI+lQ9/pUIShYxQFeLETpVLGSnN7lv4ASZjOBvDFDG5Rt4XxDRO0xVHC7vnUkVXa/P4IJVKLHNLD2K3sNzBPj5t46rTm6e65rndtLlXrhf1JB9uXKftKX3EkVljxOzrP8P4xjE8dc1Mfzmzvn9hj5y7YK7OZQvbyhWljV0v4/LhKJpErcHzIO7BCPQ2zYc/0qXANSeVwQxpPY8jaTKLlKGYETdIa00kwhu0InMPm1XqzLav4+Ajn1nCUih/KgPb7L8Mi7jWW174zmZSfbbWJC+qzWtTPs0srWRtSIW7D5U1g0bA5qnhkiqCDdt6gLCiflaI10ghbPwPMfBLiM9e+TxI9ibPJvGb/dHh834bO+ABeAh2QQyegEPwEkxBAnDwLvgYfA6+hHn4Kfwafutaw6Dn3Af/Wfj9N7/p6WY=</latexit><latexit sha1_base64="Bov3h2AWc9NxaycTX6VmxFdJPI4=">AAAC3HicbVJbi9QwFE7rbR1voz76EhyEFYahXRbUh4VFXwRBRtnOLk47Jc2kM9lJmpCkwhADvvig4qs/zDf/hz/A9KLo7h4I5zuX7+vJSQvJqDZR9DMIL12+cvXazvXBjZu3bt8Z3r0306JWmCRYMKFOCqQJoxVJDDWMnEhFEC8YOS42L5r68XuiNBXVkdlKknG0qmhJMTI+lQ9/pUIShYxQFeLETpVLGSnN7lv4ASZjOBvDFDG5Rt4XxDRO0xVHC7vnUkVXa/P4IJVKLHNLD2K3sNzBPj5t46rTm6e65rndtLlXrhf1JB9uXKftKX3EkVljxOzrP8P4xjE8dc1Mfzmzvn9hj5y7YK7OZQvbyhWljV0v4/LhKJpErcHzIO7BCPQ2zYc/0qXANSeVwQxpPY8jaTKLlKGYETdIa00kwhu0InMPm1XqzLav4+Ajn1nCUih/KgPb7L8Mi7jWW174zmZSfbbWJC+qzWtTPs0srWRtSIW7D5U1g0bA5qnhkiqCDdt6gLCiflaI10ghbPwPMfBLiM9e+TxI9ibPJvGb/dHh834bO+ABeAh2QQyegEPwEkxBAnDwLvgYfA6+hHn4Kfwafutaw6Dn3Af/Wfj9N7/p6WY=</latexit>
↵k
i<latexit sha1_base64="gbISuVdzwYW/m14D6P+ZAhTCrlk=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3gRTxVMLbYxjLZbtqlm03Y3Qgl9F948aDi1Z/jzX/jts1BWx8MPN6bYWZemAqujet+O0vLK6tr66WN8ubW9s5uZW//XieZosyniUhUK0TNBJfMN9wI1koVwzgUrBkOryZ+84kpzRN5Z0YpC2LsSx5xisZKDx0U6QC7/HHYrVTdmjsFWSReQapQoNGtfHV6Cc1iJg0VqHXbc1MT5KgMp4KNy51MsxTpEPusbanEmOkgn148JsdW6ZEoUbakIVP190SOsdajOLSdMZqBnvcm4n9eOzPRRZBzmWaGSTpbFGWCmIRM3ic9rhg1YmQJUsXtrYQOUCE1NqSyDcGbf3mR+Ke1y5p3e1at3xRplOAQjuAEPDiHOlxDA3ygIOEZXuHN0c6L8+58zFqXnGLmAP7A+fwB+EaQrQ==</latexit><latexit sha1_base64="gbISuVdzwYW/m14D6P+ZAhTCrlk=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3gRTxVMLbYxjLZbtqlm03Y3Qgl9F948aDi1Z/jzX/jts1BWx8MPN6bYWZemAqujet+O0vLK6tr66WN8ubW9s5uZW//XieZosyniUhUK0TNBJfMN9wI1koVwzgUrBkOryZ+84kpzRN5Z0YpC2LsSx5xisZKDx0U6QC7/HHYrVTdmjsFWSReQapQoNGtfHV6Cc1iJg0VqHXbc1MT5KgMp4KNy51MsxTpEPusbanEmOkgn148JsdW6ZEoUbakIVP190SOsdajOLSdMZqBnvcm4n9eOzPRRZBzmWaGSTpbFGWCmIRM3ic9rhg1YmQJUsXtrYQOUCE1NqSyDcGbf3mR+Ke1y5p3e1at3xRplOAQjuAEPDiHOlxDA3ygIOEZXuHN0c6L8+58zFqXnGLmAP7A+fwB+EaQrQ==</latexit><latexit sha1_base64="gbISuVdzwYW/m14D6P+ZAhTCrlk=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3gRTxVMLbYxjLZbtqlm03Y3Qgl9F948aDi1Z/jzX/jts1BWx8MPN6bYWZemAqujet+O0vLK6tr66WN8ubW9s5uZW//XieZosyniUhUK0TNBJfMN9wI1koVwzgUrBkOryZ+84kpzRN5Z0YpC2LsSx5xisZKDx0U6QC7/HHYrVTdmjsFWSReQapQoNGtfHV6Cc1iJg0VqHXbc1MT5KgMp4KNy51MsxTpEPusbanEmOkgn148JsdW6ZEoUbakIVP190SOsdajOLSdMZqBnvcm4n9eOzPRRZBzmWaGSTpbFGWCmIRM3ic9rhg1YmQJUsXtrYQOUCE1NqSyDcGbf3mR+Ke1y5p3e1at3xRplOAQjuAEPDiHOlxDA3ygIOEZXuHN0c6L8+58zFqXnGLmAP7A+fwB+EaQrQ==</latexit><latexit sha1_base64="gbISuVdzwYW/m14D6P+ZAhTCrlk=">AAAB8HicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3gRTxVMLbYxjLZbtqlm03Y3Qgl9F948aDi1Z/jzX/jts1BWx8MPN6bYWZemAqujet+O0vLK6tr66WN8ubW9s5uZW//XieZosyniUhUK0TNBJfMN9wI1koVwzgUrBkOryZ+84kpzRN5Z0YpC2LsSx5xisZKDx0U6QC7/HHYrVTdmjsFWSReQapQoNGtfHV6Cc1iJg0VqHXbc1MT5KgMp4KNy51MsxTpEPusbanEmOkgn148JsdW6ZEoUbakIVP190SOsdajOLSdMZqBnvcm4n9eOzPRRZBzmWaGSTpbFGWCmIRM3ic9rhg1YmQJUsXtrYQOUCE1NqSyDcGbf3mR+Ke1y5p3e1at3xRplOAQjuAEPDiHOlxDA3ygIOEZXuHN0c6L8+58zFqXnGLmAP7A+fwB+EaQrQ==</latexit>
k
i<latexit sha1_base64="stQpZfKHtvzvglg1miBXfyqafYM=">AAAB73icbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3gRTxVMLbSxrLZbtqlu0nYnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/naXlldW19dJGeXNre2e3srd/b5JMM+6zRCa6FVLDpYi5jwIlb6WaUxVK3gyHVxO/+cS1EUl8h6OUB4r2YxEJRtFKD52QI+2Kx2G3UnVr7hRkkXgFqUKBRrfy1eklLFM8RiapMW3PTTHIqUbBJB+XO5nhKWVD2udtS2OquAny6cFjcmyVHokSbStGMlV/T+RUGTNSoe1UFAdm3puI/3ntDKOLIBdxmiGP2WxRlEmCCZl8T3pCc4ZyZAllWthbCRtQTRnajMo2BG/+5UXin9Yua97tWbV+U6RRgkM4ghPw4BzqcA0N8IGBgmd4hTdHOy/Ou/Mxa11yipkD+APn8wcvZZA5</latexit><latexit sha1_base64="stQpZfKHtvzvglg1miBXfyqafYM=">AAAB73icbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3gRTxVMLbSxrLZbtqlu0nYnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/naXlldW19dJGeXNre2e3srd/b5JMM+6zRCa6FVLDpYi5jwIlb6WaUxVK3gyHVxO/+cS1EUl8h6OUB4r2YxEJRtFKD52QI+2Kx2G3UnVr7hRkkXgFqUKBRrfy1eklLFM8RiapMW3PTTHIqUbBJB+XO5nhKWVD2udtS2OquAny6cFjcmyVHokSbStGMlV/T+RUGTNSoe1UFAdm3puI/3ntDKOLIBdxmiGP2WxRlEmCCZl8T3pCc4ZyZAllWthbCRtQTRnajMo2BG/+5UXin9Yua97tWbV+U6RRgkM4ghPw4BzqcA0N8IGBgmd4hTdHOy/Ou/Mxa11yipkD+APn8wcvZZA5</latexit><latexit sha1_base64="stQpZfKHtvzvglg1miBXfyqafYM=">AAAB73icbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3gRTxVMLbSxrLZbtqlu0nYnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/naXlldW19dJGeXNre2e3srd/b5JMM+6zRCa6FVLDpYi5jwIlb6WaUxVK3gyHVxO/+cS1EUl8h6OUB4r2YxEJRtFKD52QI+2Kx2G3UnVr7hRkkXgFqUKBRrfy1eklLFM8RiapMW3PTTHIqUbBJB+XO5nhKWVD2udtS2OquAny6cFjcmyVHokSbStGMlV/T+RUGTNSoe1UFAdm3puI/3ntDKOLIBdxmiGP2WxRlEmCCZl8T3pCc4ZyZAllWthbCRtQTRnajMo2BG/+5UXin9Yua97tWbV+U6RRgkM4ghPw4BzqcA0N8IGBgmd4hTdHOy/Ou/Mxa11yipkD+APn8wcvZZA5</latexit><latexit sha1_base64="stQpZfKHtvzvglg1miBXfyqafYM=">AAAB73icbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3gRTxVMLbSxrLZbtqlu0nYnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/naXlldW19dJGeXNre2e3srd/b5JMM+6zRCa6FVLDpYi5jwIlb6WaUxVK3gyHVxO/+cS1EUl8h6OUB4r2YxEJRtFKD52QI+2Kx2G3UnVr7hRkkXgFqUKBRrfy1eklLFM8RiapMW3PTTHIqUbBJB+XO5nhKWVD2udtS2OquAny6cFjcmyVHokSbStGMlV/T+RUGTNSoe1UFAdm3puI/3ntDKOLIBdxmiGP2WxRlEmCCZl8T3pCc4ZyZAllWthbCRtQTRnajMo2BG/+5UXin9Yua97tWbV+U6RRgkM4ghPw4BzqcA0N8IGBgmd4hTdHOy/Ou/Mxa11yipkD+APn8wcvZZA5</latexit>
Uk
, V k
<latexit sha1_base64="M9YARhy7znpFfvb+2o28pT0ng6U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBg5REBPVW8CKeKpi20KZls920SzabuLsRSuif8OJBxau/x5v/xm2ag7Y+GHi8N8PMPD/hTGnb/rZKK6tr6xvlzcrW9s7uXnX/oKXiVBLqkpjHsuNjRTkT1NVMc9pJJMWRz2nbD29mfvuJSsVi8aAnCfUiPBIsYARrI3XcfniGWv1wUK3ZdTsHWiZOQWpQoDmofvWGMUkjKjThWKmuYyfay7DUjHA6rfRSRRNMQjyiXUMFjqjysvzeKToxyhAFsTQlNMrV3xMZjpSaRL7pjLAeq0VvJv7ndVMdXHkZE0mqqSDzRUHKkY7R7Hk0ZJISzSeGYCKZuRWRMZaYaBNRxYTgLL68TNzz+nXdub+oNe6KNMpwBMdwCg5cQgNuoQkuEODwDK/wZj1aL9a79TFvLVnFzCH8gfX5A4INjy8=</latexit><latexit sha1_base64="M9YARhy7znpFfvb+2o28pT0ng6U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBg5REBPVW8CKeKpi20KZls920SzabuLsRSuif8OJBxau/x5v/xm2ag7Y+GHi8N8PMPD/hTGnb/rZKK6tr6xvlzcrW9s7uXnX/oKXiVBLqkpjHsuNjRTkT1NVMc9pJJMWRz2nbD29mfvuJSsVi8aAnCfUiPBIsYARrI3XcfniGWv1wUK3ZdTsHWiZOQWpQoDmofvWGMUkjKjThWKmuYyfay7DUjHA6rfRSRRNMQjyiXUMFjqjysvzeKToxyhAFsTQlNMrV3xMZjpSaRL7pjLAeq0VvJv7ndVMdXHkZE0mqqSDzRUHKkY7R7Hk0ZJISzSeGYCKZuRWRMZaYaBNRxYTgLL68TNzz+nXdub+oNe6KNMpwBMdwCg5cQgNuoQkuEODwDK/wZj1aL9a79TFvLVnFzCH8gfX5A4INjy8=</latexit><latexit sha1_base64="M9YARhy7znpFfvb+2o28pT0ng6U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBg5REBPVW8CKeKpi20KZls920SzabuLsRSuif8OJBxau/x5v/xm2ag7Y+GHi8N8PMPD/hTGnb/rZKK6tr6xvlzcrW9s7uXnX/oKXiVBLqkpjHsuNjRTkT1NVMc9pJJMWRz2nbD29mfvuJSsVi8aAnCfUiPBIsYARrI3XcfniGWv1wUK3ZdTsHWiZOQWpQoDmofvWGMUkjKjThWKmuYyfay7DUjHA6rfRSRRNMQjyiXUMFjqjysvzeKToxyhAFsTQlNMrV3xMZjpSaRL7pjLAeq0VvJv7ndVMdXHkZE0mqqSDzRUHKkY7R7Hk0ZJISzSeGYCKZuRWRMZaYaBNRxYTgLL68TNzz+nXdub+oNe6KNMpwBMdwCg5cQgNuoQkuEODwDK/wZj1aL9a79TFvLVnFzCH8gfX5A4INjy8=</latexit><latexit sha1_base64="M9YARhy7znpFfvb+2o28pT0ng6U=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBg5REBPVW8CKeKpi20KZls920SzabuLsRSuif8OJBxau/x5v/xm2ag7Y+GHi8N8PMPD/hTGnb/rZKK6tr6xvlzcrW9s7uXnX/oKXiVBLqkpjHsuNjRTkT1NVMc9pJJMWRz2nbD29mfvuJSsVi8aAnCfUiPBIsYARrI3XcfniGWv1wUK3ZdTsHWiZOQWpQoDmofvWGMUkjKjThWKmuYyfay7DUjHA6rfRSRRNMQjyiXUMFjqjysvzeKToxyhAFsTQlNMrV3xMZjpSaRL7pjLAeq0VvJv7ndVMdXHkZE0mqqSDzRUHKkY7R7Hk0ZJISzSeGYCKZuRWRMZaYaBNRxYTgLL68TNzz+nXdub+oNe6KNMpwBMdwCg5cQgNuoQkuEODwDK/wZj1aL9a79TFvLVnFzCH8gfX5A4INjy8=</latexit>
Pr R|U, V, 2
=
mY
i=1
nY
j=1
⇥
N Ri,j|UiV T
j , 2
⇤1i,j
<latexit sha1_base64="rzUcx0Crp/tm/RsFTd0lzBU239I=">AAACknicbVFdb9MwFHUyYKN8BRBPvFhUSEOqqmRCA4QmCnvhAaGClm5Sk1aO67Te7DiybyZVxn+In8Mb/wanDRJsu5Llcz/Ovb7HRS24gTj+HYQ7t27f2d2727t3/8HDR9HjJxOjGk1ZSpVQ+qwghglesRQ4CHZWa0ZkIdhpcXHc5k8vmTZcVSewrlkuybLiJacEfGge/cxUzTQBpSsimR1rlwlWwv53/AOnAzwZ4MzwpSQze+AyzZcreHWU1Vot5pYfJW5mpcOdf77xK++3HaY4kwRWlAj79W9Pzxngc9e29tDhiSd5yom7Ycr2ymc26VhuHvXjYbwxfB0kHeijzsbz6Fe2ULSRrAIqiDHTJK4ht0QDp4K5XtYYVhN6QZZs6mErgMntRlOHX/rIApdK+1MB3kT/ZVgijVnLwle2e5qruTZ4U27aQPk2t7yqG2AV3Q4qG4FB4faD8IJrRkGsPSBUc/9WTFdEEwr+G3tehOTqytdBejB8N0y+ve6PPnVq7KHn6AXaRwl6g0boMxqjFNEgCg6DD8EofBa+Dz+Gx9vSMOg4T9F/Fn75A/PmyUk=</latexit><latexit sha1_base64="rzUcx0Crp/tm/RsFTd0lzBU239I=">AAACknicbVFdb9MwFHUyYKN8BRBPvFhUSEOqqmRCA4QmCnvhAaGClm5Sk1aO67Te7DiybyZVxn+In8Mb/wanDRJsu5Llcz/Ovb7HRS24gTj+HYQ7t27f2d2727t3/8HDR9HjJxOjGk1ZSpVQ+qwghglesRQ4CHZWa0ZkIdhpcXHc5k8vmTZcVSewrlkuybLiJacEfGge/cxUzTQBpSsimR1rlwlWwv53/AOnAzwZ4MzwpSQze+AyzZcreHWU1Vot5pYfJW5mpcOdf77xK++3HaY4kwRWlAj79W9Pzxngc9e29tDhiSd5yom7Ycr2ymc26VhuHvXjYbwxfB0kHeijzsbz6Fe2ULSRrAIqiDHTJK4ht0QDp4K5XtYYVhN6QZZs6mErgMntRlOHX/rIApdK+1MB3kT/ZVgijVnLwle2e5qruTZ4U27aQPk2t7yqG2AV3Q4qG4FB4faD8IJrRkGsPSBUc/9WTFdEEwr+G3tehOTqytdBejB8N0y+ve6PPnVq7KHn6AXaRwl6g0boMxqjFNEgCg6DD8EofBa+Dz+Gx9vSMOg4T9F/Fn75A/PmyUk=</latexit><latexit sha1_base64="rzUcx0Crp/tm/RsFTd0lzBU239I=">AAACknicbVFdb9MwFHUyYKN8BRBPvFhUSEOqqmRCA4QmCnvhAaGClm5Sk1aO67Te7DiybyZVxn+In8Mb/wanDRJsu5Llcz/Ovb7HRS24gTj+HYQ7t27f2d2727t3/8HDR9HjJxOjGk1ZSpVQ+qwghglesRQ4CHZWa0ZkIdhpcXHc5k8vmTZcVSewrlkuybLiJacEfGge/cxUzTQBpSsimR1rlwlWwv53/AOnAzwZ4MzwpSQze+AyzZcreHWU1Vot5pYfJW5mpcOdf77xK++3HaY4kwRWlAj79W9Pzxngc9e29tDhiSd5yom7Ycr2ymc26VhuHvXjYbwxfB0kHeijzsbz6Fe2ULSRrAIqiDHTJK4ht0QDp4K5XtYYVhN6QZZs6mErgMntRlOHX/rIApdK+1MB3kT/ZVgijVnLwle2e5qruTZ4U27aQPk2t7yqG2AV3Q4qG4FB4faD8IJrRkGsPSBUc/9WTFdEEwr+G3tehOTqytdBejB8N0y+ve6PPnVq7KHn6AXaRwl6g0boMxqjFNEgCg6DD8EofBa+Dz+Gx9vSMOg4T9F/Fn75A/PmyUk=</latexit><latexit sha1_base64="rzUcx0Crp/tm/RsFTd0lzBU239I=">AAACknicbVFdb9MwFHUyYKN8BRBPvFhUSEOqqmRCA4QmCnvhAaGClm5Sk1aO67Te7DiybyZVxn+In8Mb/wanDRJsu5Llcz/Ovb7HRS24gTj+HYQ7t27f2d2727t3/8HDR9HjJxOjGk1ZSpVQ+qwghglesRQ4CHZWa0ZkIdhpcXHc5k8vmTZcVSewrlkuybLiJacEfGge/cxUzTQBpSsimR1rlwlWwv53/AOnAzwZ4MzwpSQze+AyzZcreHWU1Vot5pYfJW5mpcOdf77xK++3HaY4kwRWlAj79W9Pzxngc9e29tDhiSd5yom7Ycr2ymc26VhuHvXjYbwxfB0kHeijzsbz6Fe2ULSRrAIqiDHTJK4ht0QDp4K5XtYYVhN6QZZs6mErgMntRlOHX/rIApdK+1MB3kT/ZVgijVnLwle2e5qruTZ4U27aQPk2t7yqG2AV3Q4qG4FB4faD8IJrRkGsPSBUc/9WTFdEEwr+G3tehOTqytdBejB8N0y+ve6PPnVq7KHn6AXaRwl6g0boMxqjFNEgCg6DD8EofBa+Dz+Gx9vSMOg4T9F/Fn75A/PmyUk=</latexit>

12
• By placing a zero mean isotropic Gaussian prior similar to PMF,
• For , the paper choose a Laplacian prior,
• Because the models with most suitable ranks for each row/column should be with
large weights, i.e., should be sparse.
• : location parameters of .
• : scale parameters of .
CONDITIONAL PROBABILITY
Pr Uk
| 2
U =
mY
i=1
N Uk
i |0, 2
U I , Pr V k
| 2
V =
nY
j=1
N V k
j |0, 2
V I
<latexit sha1_base64="ibGcVTj/nlukwg02DHvII4ki5Cs=">AAAC6XicdVJNb9QwEHXCR8vy0S0cuViskIqEVkmF1HKoVIkLcECLRNJKm200cZysWccOtoO0MvkJXDi0iCv/iBv/BmcbJLpdRrL09J5n/GbGWc2ZNkHw2/Nv3Lx1e2v7zuDuvfsPdoa7D2MtG0VoRCSX6jQDTTkTNDLMcHpaKwpVxulJtnjV6SefqdJMig9mWdNZBaVgBSNgHJXueiiRNVVgpBJQUTtRbcJpYfaiM7to8RecaFZWkNqoPbP7baJYOTfPjpJayTy17Ch0dNXipAIzJ8Dtu7/pTmz7EsHztSr4TV/HCZ8ayPF/PMRrHuJNHj6uPIgNHmInbvAQX/WQDkfBOFgFvg7CHoxQH5N0+CvJJWkqKgzhoPU0DGozs6AMI5y2g6TRtAaygJJOHexa0jO7WlWLnzomx4VU7giDV+y/GRYqrZdV5m527eh1rSM3adPGFIczy0TdGCrI5UNFw7GRuNs7zpmixPClA0AUc14xmYMCYtzvGLghhOstXwfR/vjlOHz/YnT8tp/GNnqMnqA9FKIDdIxeowmKEPFK76t37l343P/mf/d/XF71vT7nEboS/s8/faXtbw==</latexit><latexit sha1_base64="ibGcVTj/nlukwg02DHvII4ki5Cs=">AAAC6XicdVJNb9QwEHXCR8vy0S0cuViskIqEVkmF1HKoVIkLcECLRNJKm200cZysWccOtoO0MvkJXDi0iCv/iBv/BmcbJLpdRrL09J5n/GbGWc2ZNkHw2/Nv3Lx1e2v7zuDuvfsPdoa7D2MtG0VoRCSX6jQDTTkTNDLMcHpaKwpVxulJtnjV6SefqdJMig9mWdNZBaVgBSNgHJXueiiRNVVgpBJQUTtRbcJpYfaiM7to8RecaFZWkNqoPbP7baJYOTfPjpJayTy17Ch0dNXipAIzJ8Dtu7/pTmz7EsHztSr4TV/HCZ8ayPF/PMRrHuJNHj6uPIgNHmInbvAQX/WQDkfBOFgFvg7CHoxQH5N0+CvJJWkqKgzhoPU0DGozs6AMI5y2g6TRtAaygJJOHexa0jO7WlWLnzomx4VU7giDV+y/GRYqrZdV5m527eh1rSM3adPGFIczy0TdGCrI5UNFw7GRuNs7zpmixPClA0AUc14xmYMCYtzvGLghhOstXwfR/vjlOHz/YnT8tp/GNnqMnqA9FKIDdIxeowmKEPFK76t37l343P/mf/d/XF71vT7nEboS/s8/faXtbw==</latexit><latexit sha1_base64="ibGcVTj/nlukwg02DHvII4ki5Cs=">AAAC6XicdVJNb9QwEHXCR8vy0S0cuViskIqEVkmF1HKoVIkLcECLRNJKm200cZysWccOtoO0MvkJXDi0iCv/iBv/BmcbJLpdRrL09J5n/GbGWc2ZNkHw2/Nv3Lx1e2v7zuDuvfsPdoa7D2MtG0VoRCSX6jQDTTkTNDLMcHpaKwpVxulJtnjV6SefqdJMig9mWdNZBaVgBSNgHJXueiiRNVVgpBJQUTtRbcJpYfaiM7to8RecaFZWkNqoPbP7baJYOTfPjpJayTy17Ch0dNXipAIzJ8Dtu7/pTmz7EsHztSr4TV/HCZ8ayPF/PMRrHuJNHj6uPIgNHmInbvAQX/WQDkfBOFgFvg7CHoxQH5N0+CvJJWkqKgzhoPU0DGozs6AMI5y2g6TRtAaygJJOHexa0jO7WlWLnzomx4VU7giDV+y/GRYqrZdV5m527eh1rSM3adPGFIczy0TdGCrI5UNFw7GRuNs7zpmixPClA0AUc14xmYMCYtzvGLghhOstXwfR/vjlOHz/YnT8tp/GNnqMnqA9FKIDdIxeowmKEPFK76t37l343P/mf/d/XF71vT7nEboS/s8/faXtbw==</latexit><latexit sha1_base64="ibGcVTj/nlukwg02DHvII4ki5Cs=">AAAC6XicdVJNb9QwEHXCR8vy0S0cuViskIqEVkmF1HKoVIkLcECLRNJKm200cZysWccOtoO0MvkJXDi0iCv/iBv/BmcbJLpdRrL09J5n/GbGWc2ZNkHw2/Nv3Lx1e2v7zuDuvfsPdoa7D2MtG0VoRCSX6jQDTTkTNDLMcHpaKwpVxulJtnjV6SefqdJMig9mWdNZBaVgBSNgHJXueiiRNVVgpBJQUTtRbcJpYfaiM7to8RecaFZWkNqoPbP7baJYOTfPjpJayTy17Ch0dNXipAIzJ8Dtu7/pTmz7EsHztSr4TV/HCZ8ayPF/PMRrHuJNHj6uPIgNHmInbvAQX/WQDkfBOFgFvg7CHoxQH5N0+CvJJWkqKgzhoPU0DGozs6AMI5y2g6TRtAaygJJOHexa0jO7WlWLnzomx4VU7giDV+y/GRYqrZdV5m527eh1rSM3adPGFIczy0TdGCrI5UNFw7GRuNs7zpmixPClA0AUc14xmYMCYtzvGLghhOstXwfR/vjlOHz/YnT8tp/GNnqMnqA9FKIDdIxeowmKEPFK76t37l343P/mf/d/XF71vT7nEboS/s8/faXtbw==</latexit>
↵k
and k
<latexit sha1_base64="vv5EClcgQJall3ZFxRQQPhiWDNM=">AAACCnicbVA9SwNBEN2L3/ErammzGASrcCeC2gVsxErBGCGJYW4zSZbs7R27c2I40tv4V2wsVGz9BXb+GzcfhSY+GHj73gw788JESUu+/+3l5uYXFpeWV/Kra+sbm4Wt7Rsbp0ZgRcQqNrchWFRSY4UkKbxNDEIUKqyGvbOhX71HY2Wsr6mfYCOCjpZtKYCc1Czs1UElXbjLegNeJ3wgnnHQLe5eIdJIbxaKfskfgc+SYEKKbILLZuGr3opFGqEmocDaWuAn1MjAkBQKB/l6ajEB0YMO1hzVEKFtZKNbBnzfKS3ejo0rTXyk/p7IILK2H4WuMwLq2mlvKP7n1VJqnzQyqZOUUIvxR+1UcYr5MBjekgYFqb4jIIx0u3LRBQOCXHx5F0IwffIsqRyWTkvB1VGxfDFJY5ntsj12wAJ2zMrsnF2yChPskT2zV/bmPXkv3rv3MW7NeZOZHfYH3ucPB/Caow==</latexit><latexit sha1_base64="vv5EClcgQJall3ZFxRQQPhiWDNM=">AAACCnicbVA9SwNBEN2L3/ErammzGASrcCeC2gVsxErBGCGJYW4zSZbs7R27c2I40tv4V2wsVGz9BXb+GzcfhSY+GHj73gw788JESUu+/+3l5uYXFpeWV/Kra+sbm4Wt7Rsbp0ZgRcQqNrchWFRSY4UkKbxNDEIUKqyGvbOhX71HY2Wsr6mfYCOCjpZtKYCc1Czs1UElXbjLegNeJ3wgnnHQLe5eIdJIbxaKfskfgc+SYEKKbILLZuGr3opFGqEmocDaWuAn1MjAkBQKB/l6ajEB0YMO1hzVEKFtZKNbBnzfKS3ejo0rTXyk/p7IILK2H4WuMwLq2mlvKP7n1VJqnzQyqZOUUIvxR+1UcYr5MBjekgYFqb4jIIx0u3LRBQOCXHx5F0IwffIsqRyWTkvB1VGxfDFJY5ntsj12wAJ2zMrsnF2yChPskT2zV/bmPXkv3rv3MW7NeZOZHfYH3ucPB/Caow==</latexit><latexit sha1_base64="vv5EClcgQJall3ZFxRQQPhiWDNM=">AAACCnicbVA9SwNBEN2L3/ErammzGASrcCeC2gVsxErBGCGJYW4zSZbs7R27c2I40tv4V2wsVGz9BXb+GzcfhSY+GHj73gw788JESUu+/+3l5uYXFpeWV/Kra+sbm4Wt7Rsbp0ZgRcQqNrchWFRSY4UkKbxNDEIUKqyGvbOhX71HY2Wsr6mfYCOCjpZtKYCc1Czs1UElXbjLegNeJ3wgnnHQLe5eIdJIbxaKfskfgc+SYEKKbILLZuGr3opFGqEmocDaWuAn1MjAkBQKB/l6ajEB0YMO1hzVEKFtZKNbBnzfKS3ejo0rTXyk/p7IILK2H4WuMwLq2mlvKP7n1VJqnzQyqZOUUIvxR+1UcYr5MBjekgYFqb4jIIx0u3LRBQOCXHx5F0IwffIsqRyWTkvB1VGxfDFJY5ntsj12wAJ2zMrsnF2yChPskT2zV/bmPXkv3rv3MW7NeZOZHfYH3ucPB/Caow==</latexit><latexit sha1_base64="vv5EClcgQJall3ZFxRQQPhiWDNM=">AAACCnicbVA9SwNBEN2L3/ErammzGASrcCeC2gVsxErBGCGJYW4zSZbs7R27c2I40tv4V2wsVGz9BXb+GzcfhSY+GHj73gw788JESUu+/+3l5uYXFpeWV/Kra+sbm4Wt7Rsbp0ZgRcQqNrchWFRSY4UkKbxNDEIUKqyGvbOhX71HY2Wsr6mfYCOCjpZtKYCc1Czs1UElXbjLegNeJ3wgnnHQLe5eIdJIbxaKfskfgc+SYEKKbILLZuGr3opFGqEmocDaWuAn1MjAkBQKB/l6ajEB0YMO1hzVEKFtZKNbBnzfKS3ejo0rTXyk/p7IILK2H4WuMwLq2mlvKP7n1VJqnzQyqZOUUIvxR+1UcYr5MBjekgYFqb4jIIx0u3LRBQOCXHx5F0IwffIsqRyWTkvB1VGxfDFJY5ntsj12wAJ2zMrsnF2yChPskT2zV/bmPXkv3rv3MW7NeZOZHfYH3ucPB/Caow==</latexit>
↵k
and k
Pr ↵k
|µ↵, b↵ =
mY
i=1
L ↵k
i |µ↵, b↵ , Pr k
|µ , b =
nY
j=1
L k
j |µ , b
<latexit sha1_base64="jSJigXbmFhgDb+ubvgD1rs7wJBg=">AAADFXicjVJNi9RAEO3Ej13Hr1k9emkchBWGIRHB9bCw4EXEwwiOuzCZDZVOZ6adTjp2V4Shza/w4l/x4kHFq+DNf2MnE8SZXWELGh6vXle/qq6klMJgEPz2/EuXr1zd2b3Wu37j5q3b/b07b4yqNOMTpqTSJwkYLkXBJyhQ8pNSc8gTyY+T5bMmf/yeayNU8RpXJZ/lMC9EJhigo+I9bxipkmtApQvIuR3rOpI8w/0IZLmAU7us6Qca5VVs10w9pMlfHGkxX+DDw6jUKo2tOAzrU5vXTg+4YCDty41qTlFfrOKQRu8qSOn/zCUcN701RFeohVvO3rbOivOcNXInqC9SLu4PglHQBj0Lwg4MSBfjuP8rShWrcl4gk2DMNAxKnFnQKJjkdS+qDC+BLWHOpw42XZqZbb+1pg8ck9JMaXcKpC377w0LuTGrPHHKpiuznWvI83LTCrODmRVFWSEv2PqhrJIUFW12hKZCc4Zy5QAwLZxXyhaggaHbpJ4bQrjd8lkweTR6OgpfPR4cveimsUvukftkn4TkCTkiz8mYTAjzPnqfva/eN/+T/8X/7v9YS32vu3OXbIT/8w+rCAIQ</latexit><latexit sha1_base64="jSJigXbmFhgDb+ubvgD1rs7wJBg=">AAADFXicjVJNi9RAEO3Ej13Hr1k9emkchBWGIRHB9bCw4EXEwwiOuzCZDZVOZ6adTjp2V4Shza/w4l/x4kHFq+DNf2MnE8SZXWELGh6vXle/qq6klMJgEPz2/EuXr1zd2b3Wu37j5q3b/b07b4yqNOMTpqTSJwkYLkXBJyhQ8pNSc8gTyY+T5bMmf/yeayNU8RpXJZ/lMC9EJhigo+I9bxipkmtApQvIuR3rOpI8w/0IZLmAU7us6Qca5VVs10w9pMlfHGkxX+DDw6jUKo2tOAzrU5vXTg+4YCDty41qTlFfrOKQRu8qSOn/zCUcN701RFeohVvO3rbOivOcNXInqC9SLu4PglHQBj0Lwg4MSBfjuP8rShWrcl4gk2DMNAxKnFnQKJjkdS+qDC+BLWHOpw42XZqZbb+1pg8ck9JMaXcKpC377w0LuTGrPHHKpiuznWvI83LTCrODmRVFWSEv2PqhrJIUFW12hKZCc4Zy5QAwLZxXyhaggaHbpJ4bQrjd8lkweTR6OgpfPR4cveimsUvukftkn4TkCTkiz8mYTAjzPnqfva/eN/+T/8X/7v9YS32vu3OXbIT/8w+rCAIQ</latexit><latexit sha1_base64="jSJigXbmFhgDb+ubvgD1rs7wJBg=">AAADFXicjVJNi9RAEO3Ej13Hr1k9emkchBWGIRHB9bCw4EXEwwiOuzCZDZVOZ6adTjp2V4Shza/w4l/x4kHFq+DNf2MnE8SZXWELGh6vXle/qq6klMJgEPz2/EuXr1zd2b3Wu37j5q3b/b07b4yqNOMTpqTSJwkYLkXBJyhQ8pNSc8gTyY+T5bMmf/yeayNU8RpXJZ/lMC9EJhigo+I9bxipkmtApQvIuR3rOpI8w/0IZLmAU7us6Qca5VVs10w9pMlfHGkxX+DDw6jUKo2tOAzrU5vXTg+4YCDty41qTlFfrOKQRu8qSOn/zCUcN701RFeohVvO3rbOivOcNXInqC9SLu4PglHQBj0Lwg4MSBfjuP8rShWrcl4gk2DMNAxKnFnQKJjkdS+qDC+BLWHOpw42XZqZbb+1pg8ck9JMaXcKpC377w0LuTGrPHHKpiuznWvI83LTCrODmRVFWSEv2PqhrJIUFW12hKZCc4Zy5QAwLZxXyhaggaHbpJ4bQrjd8lkweTR6OgpfPR4cveimsUvukftkn4TkCTkiz8mYTAjzPnqfva/eN/+T/8X/7v9YS32vu3OXbIT/8w+rCAIQ</latexit><latexit sha1_base64="jSJigXbmFhgDb+ubvgD1rs7wJBg=">AAADFXicjVJNi9RAEO3Ej13Hr1k9emkchBWGIRHB9bCw4EXEwwiOuzCZDZVOZ6adTjp2V4Shza/w4l/x4kHFq+DNf2MnE8SZXWELGh6vXle/qq6klMJgEPz2/EuXr1zd2b3Wu37j5q3b/b07b4yqNOMTpqTSJwkYLkXBJyhQ8pNSc8gTyY+T5bMmf/yeayNU8RpXJZ/lMC9EJhigo+I9bxipkmtApQvIuR3rOpI8w/0IZLmAU7us6Qca5VVs10w9pMlfHGkxX+DDw6jUKo2tOAzrU5vXTg+4YCDty41qTlFfrOKQRu8qSOn/zCUcN701RFeohVvO3rbOivOcNXInqC9SLu4PglHQBj0Lwg4MSBfjuP8rShWrcl4gk2DMNAxKnFnQKJjkdS+qDC+BLWHOpw42XZqZbb+1pg8ck9JMaXcKpC377w0LuTGrPHHKpiuznWvI83LTCrODmRVFWSEv2PqhrJIUFW12hKZCc4Zy5QAwLZxXyhaggaHbpJ4bQrjd8lkweTR6OgpfPR4cveimsUvukftkn4TkCTkiz8mYTAjzPnqfva/eN/+T/8X/7v9YS32vu3OXbIT/8w+rCAIQ</latexit>
µ↵ and µ<latexit sha1_base64="HVwf6SKT/zrTDH+W1lNdUUPuwng=">AAACDnicbVA9SwNBEN3z2/gVtbRZDIpVuBNB7QQbsYpgTCAXwtxmYhb39o7dOTEc+Qc2/hUbCxVbazv/jZuPQhMfDLx9b4adeVGqpCXf//ZmZufmFxaXlgsrq2vrG8XNrRubZEZgVSQqMfUILCqpsUqSFNZTgxBHCmvR3fnAr92jsTLR19RLsRnDrZYdKYCc1Cruh3HWykNQaRf6PCR8IJ5z0G3uXkMrQoJ+q1jyy/4QfJoEY1JiY1Raxa+wnYgsRk1CgbWNwE+pmYMhKRT2C2FmMQVxB7fYcFRDjLaZD+/p8z2ntHknMa408aH6eyKH2NpeHLnOGKhrJ72B+J/XyKhz0sylTjNCLUYfdTLFKeGDcHhbGhSkeo6AMNLtykUXDAhyERZcCMHkydOkelg+LQdXR6Wzy3EaS2yH7bIDFrBjdsYuWIVVmWCP7Jm9sjfvyXvx3r2PUeuMN57ZZn/gff4AVwuccw==</latexit><latexit sha1_base64="HVwf6SKT/zrTDH+W1lNdUUPuwng=">AAACDnicbVA9SwNBEN3z2/gVtbRZDIpVuBNB7QQbsYpgTCAXwtxmYhb39o7dOTEc+Qc2/hUbCxVbazv/jZuPQhMfDLx9b4adeVGqpCXf//ZmZufmFxaXlgsrq2vrG8XNrRubZEZgVSQqMfUILCqpsUqSFNZTgxBHCmvR3fnAr92jsTLR19RLsRnDrZYdKYCc1Cruh3HWykNQaRf6PCR8IJ5z0G3uXkMrQoJ+q1jyy/4QfJoEY1JiY1Raxa+wnYgsRk1CgbWNwE+pmYMhKRT2C2FmMQVxB7fYcFRDjLaZD+/p8z2ntHknMa408aH6eyKH2NpeHLnOGKhrJ72B+J/XyKhz0sylTjNCLUYfdTLFKeGDcHhbGhSkeo6AMNLtykUXDAhyERZcCMHkydOkelg+LQdXR6Wzy3EaS2yH7bIDFrBjdsYuWIVVmWCP7Jm9sjfvyXvx3r2PUeuMN57ZZn/gff4AVwuccw==</latexit><latexit sha1_base64="HVwf6SKT/zrTDH+W1lNdUUPuwng=">AAACDnicbVA9SwNBEN3z2/gVtbRZDIpVuBNB7QQbsYpgTCAXwtxmYhb39o7dOTEc+Qc2/hUbCxVbazv/jZuPQhMfDLx9b4adeVGqpCXf//ZmZufmFxaXlgsrq2vrG8XNrRubZEZgVSQqMfUILCqpsUqSFNZTgxBHCmvR3fnAr92jsTLR19RLsRnDrZYdKYCc1Cruh3HWykNQaRf6PCR8IJ5z0G3uXkMrQoJ+q1jyy/4QfJoEY1JiY1Raxa+wnYgsRk1CgbWNwE+pmYMhKRT2C2FmMQVxB7fYcFRDjLaZD+/p8z2ntHknMa408aH6eyKH2NpeHLnOGKhrJ72B+J/XyKhz0sylTjNCLUYfdTLFKeGDcHhbGhSkeo6AMNLtykUXDAhyERZcCMHkydOkelg+LQdXR6Wzy3EaS2yH7bIDFrBjdsYuWIVVmWCP7Jm9sjfvyXvx3r2PUeuMN57ZZn/gff4AVwuccw==</latexit><latexit sha1_base64="HVwf6SKT/zrTDH+W1lNdUUPuwng=">AAACDnicbVA9SwNBEN3z2/gVtbRZDIpVuBNB7QQbsYpgTCAXwtxmYhb39o7dOTEc+Qc2/hUbCxVbazv/jZuPQhMfDLx9b4adeVGqpCXf//ZmZufmFxaXlgsrq2vrG8XNrRubZEZgVSQqMfUILCqpsUqSFNZTgxBHCmvR3fnAr92jsTLR19RLsRnDrZYdKYCc1Cruh3HWykNQaRf6PCR8IJ5z0G3uXkMrQoJ+q1jyy/4QfJoEY1JiY1Raxa+wnYgsRk1CgbWNwE+pmYMhKRT2C2FmMQVxB7fYcFRDjLaZD+/p8z2ntHknMa408aH6eyKH2NpeHLnOGKhrJ72B+J/XyKhz0sylTjNCLUYfdTLFKeGDcHhbGhSkeo6AMNLtykUXDAhyERZcCMHkydOkelg+LQdXR6Wzy3EaS2yH7bIDFrBjdsYuWIVVmWCP7Jm9sjfvyXvx3r2PUeuMN57ZZn/gff4AVwuccw==</latexit>
b↵ and b<latexit sha1_base64="Xn4WAGv913gDz0QXwVabXM2HrLc=">AAACCnicbVA9SwNBEN3z2/gVtbRZEgSrcCeC2gVsxErBmEASwtxmYhb39o7dOTEc19v4V2wsVGz9BXb+GzcfhSY+GHi8N8PMvDBR0pLvf3tz8wuLS8srq4W19Y3NreL2zo2NUyOwJmIVm0YIFpXUWCNJChuJQYhChfXw7mzo1+/RWBnraxok2I7gVsueFEBO6hRLYSdrgUr6kPMW4QPxjIPu8pwPjRAJ8k6x7Ff8EfgsCSakzCa47BS/Wt1YpBFqEgqsbQZ+Qu0MDEmhMC+0UosJiDu4xaajGiK07Wz0S873ndLlvdi40sRH6u+JDCJrB1HoOiOgvp32huJ/XjOl3kk7kzpJCbUYL+qlilPMh8HwrjQoSA0cAWGku5WLPhgQ5OIruBCC6ZdnSe2wcloJro7K1YtJGitsj5XYAQvYMauyc3bJakywR/bMXtmb9+S9eO/ex7h1zpvM7LI/8D5/AO8/mpM=</latexit><latexit sha1_base64="Xn4WAGv913gDz0QXwVabXM2HrLc=">AAACCnicbVA9SwNBEN3z2/gVtbRZEgSrcCeC2gVsxErBmEASwtxmYhb39o7dOTEc19v4V2wsVGz9BXb+GzcfhSY+GHi8N8PMvDBR0pLvf3tz8wuLS8srq4W19Y3NreL2zo2NUyOwJmIVm0YIFpXUWCNJChuJQYhChfXw7mzo1+/RWBnraxok2I7gVsueFEBO6hRLYSdrgUr6kPMW4QPxjIPu8pwPjRAJ8k6x7Ff8EfgsCSakzCa47BS/Wt1YpBFqEgqsbQZ+Qu0MDEmhMC+0UosJiDu4xaajGiK07Wz0S873ndLlvdi40sRH6u+JDCJrB1HoOiOgvp32huJ/XjOl3kk7kzpJCbUYL+qlilPMh8HwrjQoSA0cAWGku5WLPhgQ5OIruBCC6ZdnSe2wcloJro7K1YtJGitsj5XYAQvYMauyc3bJakywR/bMXtmb9+S9eO/ex7h1zpvM7LI/8D5/AO8/mpM=</latexit><latexit sha1_base64="Xn4WAGv913gDz0QXwVabXM2HrLc=">AAACCnicbVA9SwNBEN3z2/gVtbRZEgSrcCeC2gVsxErBmEASwtxmYhb39o7dOTEc19v4V2wsVGz9BXb+GzcfhSY+GHi8N8PMvDBR0pLvf3tz8wuLS8srq4W19Y3NreL2zo2NUyOwJmIVm0YIFpXUWCNJChuJQYhChfXw7mzo1+/RWBnraxok2I7gVsueFEBO6hRLYSdrgUr6kPMW4QPxjIPu8pwPjRAJ8k6x7Ff8EfgsCSakzCa47BS/Wt1YpBFqEgqsbQZ+Qu0MDEmhMC+0UosJiDu4xaajGiK07Wz0S873ndLlvdi40sRH6u+JDCJrB1HoOiOgvp32huJ/XjOl3kk7kzpJCbUYL+qlilPMh8HwrjQoSA0cAWGku5WLPhgQ5OIruBCC6ZdnSe2wcloJro7K1YtJGitsj5XYAQvYMauyc3bJakywR/bMXtmb9+S9eO/ex7h1zpvM7LI/8D5/AO8/mpM=</latexit><latexit sha1_base64="Xn4WAGv913gDz0QXwVabXM2HrLc=">AAACCnicbVA9SwNBEN3z2/gVtbRZEgSrcCeC2gVsxErBmEASwtxmYhb39o7dOTEc19v4V2wsVGz9BXb+GzcfhSY+GHi8N8PMvDBR0pLvf3tz8wuLS8srq4W19Y3NreL2zo2NUyOwJmIVm0YIFpXUWCNJChuJQYhChfXw7mzo1+/RWBnraxok2I7gVsueFEBO6hRLYSdrgUr6kPMW4QPxjIPu8pwPjRAJ8k6x7Ff8EfgsCSakzCa47BS/Wt1YpBFqEgqsbQZ+Qu0MDEmhMC+0UosJiDu4xaajGiK07Wz0S873ndLlvdi40sRH6u+JDCJrB1HoOiOgvp32huJ/XjOl3kk7kzpJCbUYL+qlilPMh8HwrjQoSA0cAWGku5WLPhgQ5OIruBCC6ZdnSe2wcloJro7K1YtJGitsj5XYAQvYMauyc3bJakywR/bMXtmb9+S9eO/ex7h1zpvM7LI/8D5/AO8/mpM=</latexit>
↵ and<latexit sha1_base64="yWpuuevthvDA3PuWyS+zg8ynjn8=">AAACAnicbVDLSgNBEJz1bXxFvellMAiewq4I6k3wIp4iGBPIhtA76SRDZmeXmV4xLAEv/ooXDype/Qpv/o2Tx8FXQUNR1U13V5Qqacn3P72Z2bn5hcWl5cLK6tr6RnFz68YmmRFYFYlKTD0Ci0pqrJIkhfXUIMSRwlrUPx/5tVs0Vib6mgYpNmPoatmRAshJreJOCCrtAQ8J74jnHHSbD3kYIUGrWPLL/hj8LwmmpMSmqLSKH2E7EVmMmoQCaxuBn1IzB0NSKBwWwsxiCqIPXWw4qiFG28zHPwz5vlPavJMYV5r4WP0+kUNs7SCOXGcM1LO/vZH4n9fIqHPSzKVOM0ItJos6meKU8FEgvC0NClIDR0AY6W7logcGBLnYCi6E4PfLf0n1sHxaDq6OSmeX0zSW2C7bYwcsYMfsjF2wCqsywe7ZI3tmL96D9+S9em+T1hlvOrPNfsB7/wIye5bR</latexit><latexit sha1_base64="yWpuuevthvDA3PuWyS+zg8ynjn8=">AAACAnicbVDLSgNBEJz1bXxFvellMAiewq4I6k3wIp4iGBPIhtA76SRDZmeXmV4xLAEv/ooXDype/Qpv/o2Tx8FXQUNR1U13V5Qqacn3P72Z2bn5hcWl5cLK6tr6RnFz68YmmRFYFYlKTD0Ci0pqrJIkhfXUIMSRwlrUPx/5tVs0Vib6mgYpNmPoatmRAshJreJOCCrtAQ8J74jnHHSbD3kYIUGrWPLL/hj8LwmmpMSmqLSKH2E7EVmMmoQCaxuBn1IzB0NSKBwWwsxiCqIPXWw4qiFG28zHPwz5vlPavJMYV5r4WP0+kUNs7SCOXGcM1LO/vZH4n9fIqHPSzKVOM0ItJos6meKU8FEgvC0NClIDR0AY6W7logcGBLnYCi6E4PfLf0n1sHxaDq6OSmeX0zSW2C7bYwcsYMfsjF2wCqsywe7ZI3tmL96D9+S9em+T1hlvOrPNfsB7/wIye5bR</latexit><latexit sha1_base64="yWpuuevthvDA3PuWyS+zg8ynjn8=">AAACAnicbVDLSgNBEJz1bXxFvellMAiewq4I6k3wIp4iGBPIhtA76SRDZmeXmV4xLAEv/ooXDype/Qpv/o2Tx8FXQUNR1U13V5Qqacn3P72Z2bn5hcWl5cLK6tr6RnFz68YmmRFYFYlKTD0Ci0pqrJIkhfXUIMSRwlrUPx/5tVs0Vib6mgYpNmPoatmRAshJreJOCCrtAQ8J74jnHHSbD3kYIUGrWPLL/hj8LwmmpMSmqLSKH2E7EVmMmoQCaxuBn1IzB0NSKBwWwsxiCqIPXWw4qiFG28zHPwz5vlPavJMYV5r4WP0+kUNs7SCOXGcM1LO/vZH4n9fIqHPSzKVOM0ItJos6meKU8FEgvC0NClIDR0AY6W7logcGBLnYCi6E4PfLf0n1sHxaDq6OSmeX0zSW2C7bYwcsYMfsjF2wCqsywe7ZI3tmL96D9+S9em+T1hlvOrPNfsB7/wIye5bR</latexit><latexit sha1_base64="yWpuuevthvDA3PuWyS+zg8ynjn8=">AAACAnicbVDLSgNBEJz1bXxFvellMAiewq4I6k3wIp4iGBPIhtA76SRDZmeXmV4xLAEv/ooXDype/Qpv/o2Tx8FXQUNR1U13V5Qqacn3P72Z2bn5hcWl5cLK6tr6RnFz68YmmRFYFYlKTD0Ci0pqrJIkhfXUIMSRwlrUPx/5tVs0Vib6mgYpNmPoatmRAshJreJOCCrtAQ8J74jnHHSbD3kYIUGrWPLL/hj8LwmmpMSmqLSKH2E7EVmMmoQCaxuBn1IzB0NSKBwWwsxiCqIPXWw4qiFG28zHPwz5vlPavJMYV5r4WP0+kUNs7SCOXGcM1LO/vZH4n9fIqHPSzKVOM0ItJos6meKU8FEgvC0NClIDR0AY6W7logcGBLnYCi6E4PfLf0n1sHxaDq6OSmeX0zSW2C7bYwcsYMfsjF2wCqsywe7ZI3tmL96D9+S9em+T1hlvOrPNfsB7/wIye5bR</latexit>
↵ and<latexit sha1_base64="yWpuuevthvDA3PuWyS+zg8ynjn8=">AAACAnicbVDLSgNBEJz1bXxFvellMAiewq4I6k3wIp4iGBPIhtA76SRDZmeXmV4xLAEv/ooXDype/Qpv/o2Tx8FXQUNR1U13V5Qqacn3P72Z2bn5hcWl5cLK6tr6RnFz68YmmRFYFYlKTD0Ci0pqrJIkhfXUIMSRwlrUPx/5tVs0Vib6mgYpNmPoatmRAshJreJOCCrtAQ8J74jnHHSbD3kYIUGrWPLL/hj8LwmmpMSmqLSKH2E7EVmMmoQCaxuBn1IzB0NSKBwWwsxiCqIPXWw4qiFG28zHPwz5vlPavJMYV5r4WP0+kUNs7SCOXGcM1LO/vZH4n9fIqHPSzKVOM0ItJos6meKU8FEgvC0NClIDR0AY6W7logcGBLnYCi6E4PfLf0n1sHxaDq6OSmeX0zSW2C7bYwcsYMfsjF2wCqsywe7ZI3tmL96D9+S9em+T1hlvOrPNfsB7/wIye5bR</latexit><latexit sha1_base64="yWpuuevthvDA3PuWyS+zg8ynjn8=">AAACAnicbVDLSgNBEJz1bXxFvellMAiewq4I6k3wIp4iGBPIhtA76SRDZmeXmV4xLAEv/ooXDype/Qpv/o2Tx8FXQUNR1U13V5Qqacn3P72Z2bn5hcWl5cLK6tr6RnFz68YmmRFYFYlKTD0Ci0pqrJIkhfXUIMSRwlrUPx/5tVs0Vib6mgYpNmPoatmRAshJreJOCCrtAQ8J74jnHHSbD3kYIUGrWPLL/hj8LwmmpMSmqLSKH2E7EVmMmoQCaxuBn1IzB0NSKBwWwsxiCqIPXWw4qiFG28zHPwz5vlPavJMYV5r4WP0+kUNs7SCOXGcM1LO/vZH4n9fIqHPSzKVOM0ItJos6meKU8FEgvC0NClIDR0AY6W7logcGBLnYCi6E4PfLf0n1sHxaDq6OSmeX0zSW2C7bYwcsYMfsjF2wCqsywe7ZI3tmL96D9+S9em+T1hlvOrPNfsB7/wIye5bR</latexit><latexit sha1_base64="yWpuuevthvDA3PuWyS+zg8ynjn8=">AAACAnicbVDLSgNBEJz1bXxFvellMAiewq4I6k3wIp4iGBPIhtA76SRDZmeXmV4xLAEv/ooXDype/Qpv/o2Tx8FXQUNR1U13V5Qqacn3P72Z2bn5hcWl5cLK6tr6RnFz68YmmRFYFYlKTD0Ci0pqrJIkhfXUIMSRwlrUPx/5tVs0Vib6mgYpNmPoatmRAshJreJOCCrtAQ8J74jnHHSbD3kYIUGrWPLL/hj8LwmmpMSmqLSKH2E7EVmMmoQCaxuBn1IzB0NSKBwWwsxiCqIPXWw4qiFG28zHPwz5vlPavJMYV5r4WP0+kUNs7SCOXGcM1LO/vZH4n9fIqHPSzKVOM0ItJos6meKU8FEgvC0NClIDR0AY6W7logcGBLnYCi6E4PfLf0n1sHxaDq6OSmeX0zSW2C7bYwcsYMfsjF2wCqsywe7ZI3tmL96D9+S9em+T1hlvOrPNfsB7/wIye5bR</latexit><latexit sha1_base64="yWpuuevthvDA3PuWyS+zg8ynjn8=">AAACAnicbVDLSgNBEJz1bXxFvellMAiewq4I6k3wIp4iGBPIhtA76SRDZmeXmV4xLAEv/ooXDype/Qpv/o2Tx8FXQUNR1U13V5Qqacn3P72Z2bn5hcWl5cLK6tr6RnFz68YmmRFYFYlKTD0Ci0pqrJIkhfXUIMSRwlrUPx/5tVs0Vib6mgYpNmPoatmRAshJreJOCCrtAQ8J74jnHHSbD3kYIUGrWPLL/hj8LwmmpMSmqLSKH2E7EVmMmoQCaxuBn1IzB0NSKBwWwsxiCqIPXWw4qiFG28zHPwz5vlPavJMYV5r4WP0+kUNs7SCOXGcM1LO/vZH4n9fIqHPSzKVOM0ItJos6meKU8FEgvC0NClIDR0AY6W7logcGBLnYCi6E4PfLf0n1sHxaDq6OSmeX0zSW2C7bYwcsYMfsjF2wCqsywe7ZI3tmL96D9+S9em+T1hlvOrPNfsB7/wIye5bR</latexit>

13
• MRMA can be viewed as a generalized form of PMF.
• Setting K = 1 in MRMA is the same problem as PMF.
GRAPHICAL MODELS
Ui Vj
Ri,j
i=1,…,m
j=1,…,n
σ
σU σV
[ MRMA ][ PMF ]

14
• The log of the posterior distribution:
POSTERIOR DISTRIBUTION
l = ln Pr U, V, ↵, |R, 2
, 2
U , 2
V , µ↵, b↵, µ , b
/ ln
⇥
Pr R|U, V, ↵, , 2
Pr U| 2
U Pr V | 2
V Pr (↵|µ↵, b↵) Pr ( |µ , b )
⇤
=
mX
i=1
nX
j=1
1i,j
"
ln
KX
k=1
↵k
i
k
j N
⇣
Ri,j|Uk
i V k
j
T
, 2
I
⌘
#
1
2 2
U
KX
k=1
mX
i=1
Uk
i
2 1
2 2
V
KX
k=1
nX
j=1
V k
i
2 1
2
Km ln 2
U
1
2
Kn ln 2
V
1
b↵
KX
k=1
mX
i=1
↵k
i µ↵
1
b
KX
k=1
nX
j=1
k
j µ
1
2
KX
k=1
m ln b2
↵
1
2
KX
k=1
n ln b2
+ C
<latexit sha1_base64="cG+osX+yrHKPMSA8qVPE0o9O51o=">AAAGlnicjVRba9swFHa7JuuyW7vtZexFrKx0LC1xGOwCZYVu7FIYWWnSQpwGWZUTNfIFSR4URT9pf2Zv+zeTbTm1nfQiMD5HOufTd74jyY0o4aLV+re0fGelVr+7eq9x/8HDR4/X1p/0eBgzhLsopCE7cSHHlAS4K4ig+CRiGPouxcfuZD9ZP/6NGSdhcCQuIjzw4SggHkFQ6Knh+sofurnr0AA4YYQZFCELoI9lhymHYk9sdZug1wQOpNEY6r+LBQRTcKhNTkY+PJVtldtD2VVlv5f7fjyUGYb23IKdrSSoZiE1HUZGY/EaOE5j04lYGIkQpBwTSv0rmB5qXgvYFpnOcK+oVSOUS7khoVdI6N0mIWOWZF2hyE35Rv9rZct+g1S9XYfH/lCSXVvT8xXI3PPUDbTrQzF2PWkrHdME58oonGidRk7SyANlNNVR2p2oTFoNZLwEBkEqf+a9MHBJS/IcI1meZMieyiNVbBH4vqCKbcdjEGmWsl1pkJrjWarX9HVGId+zrRZB9q6DzDWblTEHCQqYChwAPzuzJb6VkKAUku1fqfjybNym1mm5T9vFc5ZRnZah04Nzm5KnpZZvF06gwa00ag4zk+OynDk5KvHBLD7dJQl/sz9c22jttNIB5g3bGBuWGZ3h2l/nLESxjwOBKOS8b7ciMZCQCYIoVg0n5jiCaAJHuK/N5LrxgUzfVQVe6Zkz4IVMf4EA6WwxQ0Kf8wvf1ZHJDeDVtWRy0Vo/Ft77gSRBFAscoGwjL6ZAv3PJIw3OCMNI0AttQMSI5grQGGqphH7KG1oEu1ryvNFt73zYsX+93dj7YdRYtV5YL60ty7beWXvWN6tjdS1Ue1b7WNuvfa4/r3+qf6l/zUKXl0zOU6s06p3/ZwdK7Q==</latexit><latexit sha1_base64="cG+osX+yrHKPMSA8qVPE0o9O51o=">AAAGlnicjVRba9swFHa7JuuyW7vtZexFrKx0LC1xGOwCZYVu7FIYWWnSQpwGWZUTNfIFSR4URT9pf2Zv+zeTbTm1nfQiMD5HOufTd74jyY0o4aLV+re0fGelVr+7eq9x/8HDR4/X1p/0eBgzhLsopCE7cSHHlAS4K4ig+CRiGPouxcfuZD9ZP/6NGSdhcCQuIjzw4SggHkFQ6Knh+sofurnr0AA4YYQZFCELoI9lhymHYk9sdZug1wQOpNEY6r+LBQRTcKhNTkY+PJVtldtD2VVlv5f7fjyUGYb23IKdrSSoZiE1HUZGY/EaOE5j04lYGIkQpBwTSv0rmB5qXgvYFpnOcK+oVSOUS7khoVdI6N0mIWOWZF2hyE35Rv9rZct+g1S9XYfH/lCSXVvT8xXI3PPUDbTrQzF2PWkrHdME58oonGidRk7SyANlNNVR2p2oTFoNZLwEBkEqf+a9MHBJS/IcI1meZMieyiNVbBH4vqCKbcdjEGmWsl1pkJrjWarX9HVGId+zrRZB9q6DzDWblTEHCQqYChwAPzuzJb6VkKAUku1fqfjybNym1mm5T9vFc5ZRnZah04Nzm5KnpZZvF06gwa00ag4zk+OynDk5KvHBLD7dJQl/sz9c22jttNIB5g3bGBuWGZ3h2l/nLESxjwOBKOS8b7ciMZCQCYIoVg0n5jiCaAJHuK/N5LrxgUzfVQVe6Zkz4IVMf4EA6WwxQ0Kf8wvf1ZHJDeDVtWRy0Vo/Ft77gSRBFAscoGwjL6ZAv3PJIw3OCMNI0AttQMSI5grQGGqphH7KG1oEu1ryvNFt73zYsX+93dj7YdRYtV5YL60ty7beWXvWN6tjdS1Ue1b7WNuvfa4/r3+qf6l/zUKXl0zOU6s06p3/ZwdK7Q==</latexit><latexit sha1_base64="cG+osX+yrHKPMSA8qVPE0o9O51o=">AAAGlnicjVRba9swFHa7JuuyW7vtZexFrKx0LC1xGOwCZYVu7FIYWWnSQpwGWZUTNfIFSR4URT9pf2Zv+zeTbTm1nfQiMD5HOufTd74jyY0o4aLV+re0fGelVr+7eq9x/8HDR4/X1p/0eBgzhLsopCE7cSHHlAS4K4ig+CRiGPouxcfuZD9ZP/6NGSdhcCQuIjzw4SggHkFQ6Knh+sofurnr0AA4YYQZFCELoI9lhymHYk9sdZug1wQOpNEY6r+LBQRTcKhNTkY+PJVtldtD2VVlv5f7fjyUGYb23IKdrSSoZiE1HUZGY/EaOE5j04lYGIkQpBwTSv0rmB5qXgvYFpnOcK+oVSOUS7khoVdI6N0mIWOWZF2hyE35Rv9rZct+g1S9XYfH/lCSXVvT8xXI3PPUDbTrQzF2PWkrHdME58oonGidRk7SyANlNNVR2p2oTFoNZLwEBkEqf+a9MHBJS/IcI1meZMieyiNVbBH4vqCKbcdjEGmWsl1pkJrjWarX9HVGId+zrRZB9q6DzDWblTEHCQqYChwAPzuzJb6VkKAUku1fqfjybNym1mm5T9vFc5ZRnZah04Nzm5KnpZZvF06gwa00ag4zk+OynDk5KvHBLD7dJQl/sz9c22jttNIB5g3bGBuWGZ3h2l/nLESxjwOBKOS8b7ciMZCQCYIoVg0n5jiCaAJHuK/N5LrxgUzfVQVe6Zkz4IVMf4EA6WwxQ0Kf8wvf1ZHJDeDVtWRy0Vo/Ft77gSRBFAscoGwjL6ZAv3PJIw3OCMNI0AttQMSI5grQGGqphH7KG1oEu1ryvNFt73zYsX+93dj7YdRYtV5YL60ty7beWXvWN6tjdS1Ue1b7WNuvfa4/r3+qf6l/zUKXl0zOU6s06p3/ZwdK7Q==</latexit><latexit sha1_base64="cG+osX+yrHKPMSA8qVPE0o9O51o=">AAAGlnicjVRba9swFHa7JuuyW7vtZexFrKx0LC1xGOwCZYVu7FIYWWnSQpwGWZUTNfIFSR4URT9pf2Zv+zeTbTm1nfQiMD5HOufTd74jyY0o4aLV+re0fGelVr+7eq9x/8HDR4/X1p/0eBgzhLsopCE7cSHHlAS4K4ig+CRiGPouxcfuZD9ZP/6NGSdhcCQuIjzw4SggHkFQ6Knh+sofurnr0AA4YYQZFCELoI9lhymHYk9sdZug1wQOpNEY6r+LBQRTcKhNTkY+PJVtldtD2VVlv5f7fjyUGYb23IKdrSSoZiE1HUZGY/EaOE5j04lYGIkQpBwTSv0rmB5qXgvYFpnOcK+oVSOUS7khoVdI6N0mIWOWZF2hyE35Rv9rZct+g1S9XYfH/lCSXVvT8xXI3PPUDbTrQzF2PWkrHdME58oonGidRk7SyANlNNVR2p2oTFoNZLwEBkEqf+a9MHBJS/IcI1meZMieyiNVbBH4vqCKbcdjEGmWsl1pkJrjWarX9HVGId+zrRZB9q6DzDWblTEHCQqYChwAPzuzJb6VkKAUku1fqfjybNym1mm5T9vFc5ZRnZah04Nzm5KnpZZvF06gwa00ag4zk+OynDk5KvHBLD7dJQl/sz9c22jttNIB5g3bGBuWGZ3h2l/nLESxjwOBKOS8b7ciMZCQCYIoVg0n5jiCaAJHuK/N5LrxgUzfVQVe6Zkz4IVMf4EA6WwxQ0Kf8wvf1ZHJDeDVtWRy0Vo/Ft77gSRBFAscoGwjL6ZAv3PJIw3OCMNI0AttQMSI5grQGGqphH7KG1oEu1ryvNFt73zYsX+93dj7YdRYtV5YL60ty7beWXvWN6tjdS1Ue1b7WNuvfa4/r3+qf6l/zUKXl0zOU6s06p3/ZwdK7Q==</latexit>
Difficult to solve
directly!

15
• Given i and j,
• It is mixture of Gaussian distribution. 
• If K > 1, it is difficult to calculate
• We need another way to compute the gradient of l.
LOWER BOUND OF LOG POSTERIOR
f(U, V ) = ln
" KX
k=1
↵k
i
k
j N
⇣
Ri,j|Uk
i V k
j
T
, 2
I
⌘
#
<latexit sha1_base64="tF3HAQdNxQuE99Q2y7NEXf7f7oE=">AAAChnicbVFdb9MwFHXCgK18FXjk5YoKqUOlS8bQ4GFSBS8gJDTQ0k1q0shxndar40T2DVJl8lf4Ubzxb3DbTIKNK1k+95x7dH2vs0oKg0Hw2/Nv7dy+c3d3r3Pv/oOHj7qPn4xNWWvGI1bKUl9k1HApFI9QoOQXlea0yCQ/z5Yf1vr5d66NKNUZriqeFHSuRC4YRUel3Z95PxrAeB9O4FUsFcSS5ziB2NRFapcnYTO1nxuIqawWNLXCpUuXZhxddtlmBcUFo9J+aTbu/jdXOIDLBn5AdOXZKuMrU6zFfIH7U3vWDFwzMS/o1B428KkVYHsnabcXDINNwE0QtqBH2jhNu7/iWcnqgitkkhozCYMKE0s1CiZ504lrwyvKlnTOJw4qWnCT2M0eG3jhmBnkpXZHIWzYvx2WFsasisxVrmc217U1+T9tUmP+NrFCVTVyxbaN8loClrD+FJgJzRnKlQOUaeHeCmxBNWXovq7jlhBeH/kmiA6H74bh16Pe6H27jV3yjDwnfRKSYzIiH8kpiQjzdryX3mvvyN/zD/w3/vG21Pdaz1PyT/ijPw2gw8A=</latexit><latexit sha1_base64="tF3HAQdNxQuE99Q2y7NEXf7f7oE=">AAAChnicbVFdb9MwFHXCgK18FXjk5YoKqUOlS8bQ4GFSBS8gJDTQ0k1q0shxndar40T2DVJl8lf4Ubzxb3DbTIKNK1k+95x7dH2vs0oKg0Hw2/Nv7dy+c3d3r3Pv/oOHj7qPn4xNWWvGI1bKUl9k1HApFI9QoOQXlea0yCQ/z5Yf1vr5d66NKNUZriqeFHSuRC4YRUel3Z95PxrAeB9O4FUsFcSS5ziB2NRFapcnYTO1nxuIqawWNLXCpUuXZhxddtlmBcUFo9J+aTbu/jdXOIDLBn5AdOXZKuMrU6zFfIH7U3vWDFwzMS/o1B428KkVYHsnabcXDINNwE0QtqBH2jhNu7/iWcnqgitkkhozCYMKE0s1CiZ504lrwyvKlnTOJw4qWnCT2M0eG3jhmBnkpXZHIWzYvx2WFsasisxVrmc217U1+T9tUmP+NrFCVTVyxbaN8loClrD+FJgJzRnKlQOUaeHeCmxBNWXovq7jlhBeH/kmiA6H74bh16Pe6H27jV3yjDwnfRKSYzIiH8kpiQjzdryX3mvvyN/zD/w3/vG21Pdaz1PyT/ijPw2gw8A=</latexit><latexit sha1_base64="tF3HAQdNxQuE99Q2y7NEXf7f7oE=">AAAChnicbVFdb9MwFHXCgK18FXjk5YoKqUOlS8bQ4GFSBS8gJDTQ0k1q0shxndar40T2DVJl8lf4Ubzxb3DbTIKNK1k+95x7dH2vs0oKg0Hw2/Nv7dy+c3d3r3Pv/oOHj7qPn4xNWWvGI1bKUl9k1HApFI9QoOQXlea0yCQ/z5Yf1vr5d66NKNUZriqeFHSuRC4YRUel3Z95PxrAeB9O4FUsFcSS5ziB2NRFapcnYTO1nxuIqawWNLXCpUuXZhxddtlmBcUFo9J+aTbu/jdXOIDLBn5AdOXZKuMrU6zFfIH7U3vWDFwzMS/o1B428KkVYHsnabcXDINNwE0QtqBH2jhNu7/iWcnqgitkkhozCYMKE0s1CiZ504lrwyvKlnTOJw4qWnCT2M0eG3jhmBnkpXZHIWzYvx2WFsasisxVrmc217U1+T9tUmP+NrFCVTVyxbaN8loClrD+FJgJzRnKlQOUaeHeCmxBNWXovq7jlhBeH/kmiA6H74bh16Pe6H27jV3yjDwnfRKSYzIiH8kpiQjzdryX3mvvyN/zD/w3/vG21Pdaz1PyT/ijPw2gw8A=</latexit><latexit sha1_base64="tF3HAQdNxQuE99Q2y7NEXf7f7oE=">AAAChnicbVFdb9MwFHXCgK18FXjk5YoKqUOlS8bQ4GFSBS8gJDTQ0k1q0shxndar40T2DVJl8lf4Ubzxb3DbTIKNK1k+95x7dH2vs0oKg0Hw2/Nv7dy+c3d3r3Pv/oOHj7qPn4xNWWvGI1bKUl9k1HApFI9QoOQXlea0yCQ/z5Yf1vr5d66NKNUZriqeFHSuRC4YRUel3Z95PxrAeB9O4FUsFcSS5ziB2NRFapcnYTO1nxuIqawWNLXCpUuXZhxddtlmBcUFo9J+aTbu/jdXOIDLBn5AdOXZKuMrU6zFfIH7U3vWDFwzMS/o1B428KkVYHsnabcXDINNwE0QtqBH2jhNu7/iWcnqgitkkhozCYMKE0s1CiZ504lrwyvKlnTOJw4qWnCT2M0eG3jhmBnkpXZHIWzYvx2WFsasisxVrmc217U1+T9tUmP+NrFCVTVyxbaN8loClrD+FJgJzRnKlQOUaeHeCmxBNWXovq7jlhBeH/kmiA6H74bh16Pe6H27jV3yjDwnfRKSYzIiH8kpiQjzdryX3mvvyN/zD/w3/vG21Pdaz1PyT/ijPw2gw8A=</latexit>
d
dU
f(U, V ) and
d
dV
f(U, V )
<latexit sha1_base64="Oi9isIP3rIBSbS8PrVoZlYHBdGA=">AAACIHicbVDLSgMxFM3UV62vqks3wSJUkDIjgnVXdOOygtMW2lIymUwbmskMyR2xDPMrbvwVNy5UdKdfY/pAtPVA4HDOudzc48WCa7DtTyu3tLyyupZfL2xsbm3vFHf3GjpKFGUujUSkWh7RTHDJXOAgWCtWjISeYE1veDX2m3dMaR7JWxjFrBuSvuQBpwSM1CtWO4EiNPUznPpuhoOye4Ibx7gD7B5STKSPM/wr0viJ9Iolu2JPgBeJMyMlNEO9V/zo+BFNQiaBCqJ127Fj6KZEAaeCZYVOollM6JD0WdtQSUKmu+nkwgwfGcXHQaTMk4An6u+JlIRaj0LPJEMCAz3vjcX/vHYCQbWbchknwCSdLgoSgSHC47qwzxWjIEaGEKq4+SumA2L6AFNqwZTgzJ+8SNzTykXFuTkr1S5nbeTRATpEZeSgc1RD16iOXETRA3pCL+jVerSerTfrfRrNWbOZffQH1tc3cEehYg==</latexit><latexit sha1_base64="Oi9isIP3rIBSbS8PrVoZlYHBdGA=">AAACIHicbVDLSgMxFM3UV62vqks3wSJUkDIjgnVXdOOygtMW2lIymUwbmskMyR2xDPMrbvwVNy5UdKdfY/pAtPVA4HDOudzc48WCa7DtTyu3tLyyupZfL2xsbm3vFHf3GjpKFGUujUSkWh7RTHDJXOAgWCtWjISeYE1veDX2m3dMaR7JWxjFrBuSvuQBpwSM1CtWO4EiNPUznPpuhoOye4Ibx7gD7B5STKSPM/wr0viJ9Iolu2JPgBeJMyMlNEO9V/zo+BFNQiaBCqJ127Fj6KZEAaeCZYVOollM6JD0WdtQSUKmu+nkwgwfGcXHQaTMk4An6u+JlIRaj0LPJEMCAz3vjcX/vHYCQbWbchknwCSdLgoSgSHC47qwzxWjIEaGEKq4+SumA2L6AFNqwZTgzJ+8SNzTykXFuTkr1S5nbeTRATpEZeSgc1RD16iOXETRA3pCL+jVerSerTfrfRrNWbOZffQH1tc3cEehYg==</latexit><latexit sha1_base64="Oi9isIP3rIBSbS8PrVoZlYHBdGA=">AAACIHicbVDLSgMxFM3UV62vqks3wSJUkDIjgnVXdOOygtMW2lIymUwbmskMyR2xDPMrbvwVNy5UdKdfY/pAtPVA4HDOudzc48WCa7DtTyu3tLyyupZfL2xsbm3vFHf3GjpKFGUujUSkWh7RTHDJXOAgWCtWjISeYE1veDX2m3dMaR7JWxjFrBuSvuQBpwSM1CtWO4EiNPUznPpuhoOye4Ibx7gD7B5STKSPM/wr0viJ9Iolu2JPgBeJMyMlNEO9V/zo+BFNQiaBCqJ127Fj6KZEAaeCZYVOollM6JD0WdtQSUKmu+nkwgwfGcXHQaTMk4An6u+JlIRaj0LPJEMCAz3vjcX/vHYCQbWbchknwCSdLgoSgSHC47qwzxWjIEaGEKq4+SumA2L6AFNqwZTgzJ+8SNzTykXFuTkr1S5nbeTRATpEZeSgc1RD16iOXETRA3pCL+jVerSerTfrfRrNWbOZffQH1tc3cEehYg==</latexit><latexit sha1_base64="Oi9isIP3rIBSbS8PrVoZlYHBdGA=">AAACIHicbVDLSgMxFM3UV62vqks3wSJUkDIjgnVXdOOygtMW2lIymUwbmskMyR2xDPMrbvwVNy5UdKdfY/pAtPVA4HDOudzc48WCa7DtTyu3tLyyupZfL2xsbm3vFHf3GjpKFGUujUSkWh7RTHDJXOAgWCtWjISeYE1veDX2m3dMaR7JWxjFrBuSvuQBpwSM1CtWO4EiNPUznPpuhoOye4Ibx7gD7B5STKSPM/wr0viJ9Iolu2JPgBeJMyMlNEO9V/zo+BFNQiaBCqJ127Fj6KZEAaeCZYVOollM6JD0WdtQSUKmu+nkwgwfGcXHQaTMk4An6u+JlIRaj0LPJEMCAz3vjcX/vHYCQbWbchknwCSdLgoSgSHC47qwzxWjIEaGEKq4+SumA2L6AFNqwZTgzJ+8SNzTykXFuTkr1S5nbeTRATpEZeSgc1RD16iOXETRA3pCL+jVerSerTfrfRrNWbOZffQH1tc3cEehYg==</latexit>

16
• Solution: Jensen’s Inequality
• For a convex function f,
• On the other hand, f(U,V) is convex. Why?
• Gaussian function and -log(x) function is convex.
• Sum of multiple convex function is convex.
• The composite function of two convex function is convex.
80   1, f ( x1 + (1 )x2)  f (x1) + (1 )f (x2)<latexit sha1_base64="ek3/fThD/KmpXpQa7LNtF4N0GHo=">AAACdXicbVHLTgIxFO2ML8QX6sqYmEbUQFScISbqjujGJSaiJAwhndKBhs7D9o6RTPgEf86d3+HGpQXGB+BN2pyec89t760bCa7Ast4Nc25+YXEps5xdWV1b38htbj2oMJaU1WgoQll3iWKCB6wGHASrR5IR3xXs0e3dDPXHZyYVD4N76Ees6ZNOwD1OCWiqlXt1vFASIbCFHcGe9Ka9bTI+2CfY08CDwjf90krswXHBPk2J4pApDxzJO10oTpZIrSNLmjDh/NV/CrRyeatkjQLPAjsFeZRGtZV7c9ohjX0WABVEqYZtRdBMiAROBRtknVixiNAe6bCGhgHxmWomo7EN8KFm2lj3r1cAeMT+dSTEV6rvuzrTJ9BV09qQ/E9rxOBdNhMeRDGwgI4v8mKBIcTDP8BtLhkF0deAUMn1WzHtEkko6J/K6iHY0y3Pglq5dFWy787zlet0Ghm0i/ZRAdnoAlXQLaqiGqLow9gx9o288WnumQfm0TjVNFLPNpoI8+wLu4e72g==</latexit><latexit sha1_base64="ek3/fThD/KmpXpQa7LNtF4N0GHo=">AAACdXicbVHLTgIxFO2ML8QX6sqYmEbUQFScISbqjujGJSaiJAwhndKBhs7D9o6RTPgEf86d3+HGpQXGB+BN2pyec89t760bCa7Ast4Nc25+YXEps5xdWV1b38htbj2oMJaU1WgoQll3iWKCB6wGHASrR5IR3xXs0e3dDPXHZyYVD4N76Ees6ZNOwD1OCWiqlXt1vFASIbCFHcGe9Ka9bTI+2CfY08CDwjf90krswXHBPk2J4pApDxzJO10oTpZIrSNLmjDh/NV/CrRyeatkjQLPAjsFeZRGtZV7c9ohjX0WABVEqYZtRdBMiAROBRtknVixiNAe6bCGhgHxmWomo7EN8KFm2lj3r1cAeMT+dSTEV6rvuzrTJ9BV09qQ/E9rxOBdNhMeRDGwgI4v8mKBIcTDP8BtLhkF0deAUMn1WzHtEkko6J/K6iHY0y3Pglq5dFWy787zlet0Ghm0i/ZRAdnoAlXQLaqiGqLow9gx9o288WnumQfm0TjVNFLPNpoI8+wLu4e72g==</latexit><latexit sha1_base64="ek3/fThD/KmpXpQa7LNtF4N0GHo=">AAACdXicbVHLTgIxFO2ML8QX6sqYmEbUQFScISbqjujGJSaiJAwhndKBhs7D9o6RTPgEf86d3+HGpQXGB+BN2pyec89t760bCa7Ast4Nc25+YXEps5xdWV1b38htbj2oMJaU1WgoQll3iWKCB6wGHASrR5IR3xXs0e3dDPXHZyYVD4N76Ees6ZNOwD1OCWiqlXt1vFASIbCFHcGe9Ka9bTI+2CfY08CDwjf90krswXHBPk2J4pApDxzJO10oTpZIrSNLmjDh/NV/CrRyeatkjQLPAjsFeZRGtZV7c9ohjX0WABVEqYZtRdBMiAROBRtknVixiNAe6bCGhgHxmWomo7EN8KFm2lj3r1cAeMT+dSTEV6rvuzrTJ9BV09qQ/E9rxOBdNhMeRDGwgI4v8mKBIcTDP8BtLhkF0deAUMn1WzHtEkko6J/K6iHY0y3Pglq5dFWy787zlet0Ghm0i/ZRAdnoAlXQLaqiGqLow9gx9o288WnumQfm0TjVNFLPNpoI8+wLu4e72g==</latexit><latexit sha1_base64="ek3/fThD/KmpXpQa7LNtF4N0GHo=">AAACdXicbVHLTgIxFO2ML8QX6sqYmEbUQFScISbqjujGJSaiJAwhndKBhs7D9o6RTPgEf86d3+HGpQXGB+BN2pyec89t760bCa7Ast4Nc25+YXEps5xdWV1b38htbj2oMJaU1WgoQll3iWKCB6wGHASrR5IR3xXs0e3dDPXHZyYVD4N76Ees6ZNOwD1OCWiqlXt1vFASIbCFHcGe9Ka9bTI+2CfY08CDwjf90krswXHBPk2J4pApDxzJO10oTpZIrSNLmjDh/NV/CrRyeatkjQLPAjsFeZRGtZV7c9ohjX0WABVEqYZtRdBMiAROBRtknVixiNAe6bCGhgHxmWomo7EN8KFm2lj3r1cAeMT+dSTEV6rvuzrTJ9BV09qQ/E9rxOBdNhMeRDGwgI4v8mKBIcTDP8BtLhkF0deAUMn1WzHtEkko6J/K6iHY0y3Pglq5dFWy787zlet0Ghm0i/ZRAdnoAlXQLaqiGqLow9gx9o288WnumQfm0TjVNFLPNpoI8+wLu4e72g==</latexit>

17
• Applying Jensen’s inequality to f(U,V),
• The lower bound of the log posterior distribution:
f(U, V ) = ln
" KX
k=1
↵k
i
k
j N
⇣
Ri,j|Uk
i V k
j
T
, 2
I
⌘
#
KX
k=1
↵k
i
k
j ln N
⇣
Ri,j|Uk
i V k
j
T
, 2
I
⌘
=
1
2 2
KX
k=1
↵k
i
k
j
⇣
Ri,j Uk
i V k
j
T
⌘2 1
2
ln 2
+ C0
<latexit sha1_base64="RXMQ8PXD6sDLkW5Has/7aAoIQe4=">AAADuHictVJbixMxFE5nvKz11tVHXw4W1y62pVPEC1JY3BdFkFV2ugtNWzJpZppt5kKSEUqc3yj45r8x05nVtSuoDx4I+c453yHfOTlBJrjSg8G3huNeuXrt+s6N5s1bt+/cbe3eG6s0l5T5NBWpPA2IYoInzNdcC3aaSUbiQLCTYHVY5k8+Mal4mhzrdcamMYkSHnJKtA3Ndxtfwo7fhfH+aK+HRQJYsFBPsMrjuVmNvGJm3hWAiciWZG64dVfWDZi23lntxUQvKRHmfbEp7ny0xC6cFfAZ/POaKjM+L8KSR0u9PzPHRRew4lFMZmZYwNs6UV1TAIybOGJ70IMtSX/QVHbyH3TBqFQSSkKNV5hhxYCZZfyjvIt6en+h5ge0T/V+Cqg7rYQO4QkcPp632oP+YGNwGXg1aKPajuatr3iR0jxmiaaCKDXxBpmeGiI1p4IVTZwrlhG6IhGbWJiQmKmp2WxeAY9sZAFhKu1JNGyiFysMiZVax4Fllt+htnNl8He5Sa7DF1PDkyzXLKHVQ2EuQKdQrjEsuGRUi7UFhEputQJdEjsXbZe9aYfgbbd8GfjD/su+9+Fp++B1PY0d9AA9RB3koefoAL1BR8hH1HnmYIc5ofvKJW7k8orqNOqa++gXc+V30PM1lQ==</latexit><latexit sha1_base64="RXMQ8PXD6sDLkW5Has/7aAoIQe4=">AAADuHictVJbixMxFE5nvKz11tVHXw4W1y62pVPEC1JY3BdFkFV2ugtNWzJpZppt5kKSEUqc3yj45r8x05nVtSuoDx4I+c453yHfOTlBJrjSg8G3huNeuXrt+s6N5s1bt+/cbe3eG6s0l5T5NBWpPA2IYoInzNdcC3aaSUbiQLCTYHVY5k8+Mal4mhzrdcamMYkSHnJKtA3Ndxtfwo7fhfH+aK+HRQJYsFBPsMrjuVmNvGJm3hWAiciWZG64dVfWDZi23lntxUQvKRHmfbEp7ny0xC6cFfAZ/POaKjM+L8KSR0u9PzPHRRew4lFMZmZYwNs6UV1TAIybOGJ70IMtSX/QVHbyH3TBqFQSSkKNV5hhxYCZZfyjvIt6en+h5ge0T/V+Cqg7rYQO4QkcPp632oP+YGNwGXg1aKPajuatr3iR0jxmiaaCKDXxBpmeGiI1p4IVTZwrlhG6IhGbWJiQmKmp2WxeAY9sZAFhKu1JNGyiFysMiZVax4Fllt+htnNl8He5Sa7DF1PDkyzXLKHVQ2EuQKdQrjEsuGRUi7UFhEputQJdEjsXbZe9aYfgbbd8GfjD/su+9+Fp++B1PY0d9AA9RB3koefoAL1BR8hH1HnmYIc5ofvKJW7k8orqNOqa++gXc+V30PM1lQ==</latexit><latexit sha1_base64="RXMQ8PXD6sDLkW5Has/7aAoIQe4=">AAADuHictVJbixMxFE5nvKz11tVHXw4W1y62pVPEC1JY3BdFkFV2ugtNWzJpZppt5kKSEUqc3yj45r8x05nVtSuoDx4I+c453yHfOTlBJrjSg8G3huNeuXrt+s6N5s1bt+/cbe3eG6s0l5T5NBWpPA2IYoInzNdcC3aaSUbiQLCTYHVY5k8+Mal4mhzrdcamMYkSHnJKtA3Ndxtfwo7fhfH+aK+HRQJYsFBPsMrjuVmNvGJm3hWAiciWZG64dVfWDZi23lntxUQvKRHmfbEp7ny0xC6cFfAZ/POaKjM+L8KSR0u9PzPHRRew4lFMZmZYwNs6UV1TAIybOGJ70IMtSX/QVHbyH3TBqFQSSkKNV5hhxYCZZfyjvIt6en+h5ge0T/V+Cqg7rYQO4QkcPp632oP+YGNwGXg1aKPajuatr3iR0jxmiaaCKDXxBpmeGiI1p4IVTZwrlhG6IhGbWJiQmKmp2WxeAY9sZAFhKu1JNGyiFysMiZVax4Fllt+htnNl8He5Sa7DF1PDkyzXLKHVQ2EuQKdQrjEsuGRUi7UFhEputQJdEjsXbZe9aYfgbbd8GfjD/su+9+Fp++B1PY0d9AA9RB3koefoAL1BR8hH1HnmYIc5ofvKJW7k8orqNOqa++gXc+V30PM1lQ==</latexit><latexit sha1_base64="RXMQ8PXD6sDLkW5Has/7aAoIQe4=">AAADuHictVJbixMxFE5nvKz11tVHXw4W1y62pVPEC1JY3BdFkFV2ugtNWzJpZppt5kKSEUqc3yj45r8x05nVtSuoDx4I+c453yHfOTlBJrjSg8G3huNeuXrt+s6N5s1bt+/cbe3eG6s0l5T5NBWpPA2IYoInzNdcC3aaSUbiQLCTYHVY5k8+Mal4mhzrdcamMYkSHnJKtA3Ndxtfwo7fhfH+aK+HRQJYsFBPsMrjuVmNvGJm3hWAiciWZG64dVfWDZi23lntxUQvKRHmfbEp7ny0xC6cFfAZ/POaKjM+L8KSR0u9PzPHRRew4lFMZmZYwNs6UV1TAIybOGJ70IMtSX/QVHbyH3TBqFQSSkKNV5hhxYCZZfyjvIt6en+h5ge0T/V+Cqg7rYQO4QkcPp632oP+YGNwGXg1aKPajuatr3iR0jxmiaaCKDXxBpmeGiI1p4IVTZwrlhG6IhGbWJiQmKmp2WxeAY9sZAFhKu1JNGyiFysMiZVax4Fllt+htnNl8He5Sa7DF1PDkyzXLKHVQ2EuQKdQrjEsuGRUi7UFhEputQJdEjsXbZe9aYfgbbd8GfjD/su+9+Fp++B1PY0d9AA9RB3koefoAL1BR8hH1HnmYIc5ofvKJW7k8orqNOqa++gXc+V30PM1lQ==</latexit>

18
• On the other hands, the loss function is difficult to compute.
• Too many parameters and hyper-parameters.
• Iterated Conditional Modes (ICM) Steps [J. Besag, 1986]:
• Conditioned on the rest variables, 
then iterate until convergence. 
(Alternating minimization)
• To estimate optimal value of each variables  
by conditioning other variables.
• Similar to works of Kittler and Foglein [1984],  
Kiiveri and Campbell [1986].
• In MRMA, the hyper-parameters are also trained by ICM.
ITERATED CONDITIONAL MODES (ICM)

19
• Each parameters can be updated by solving the following minimization:
• These all convex optimization!
ICM OF MRMA (PARAMETERS)

20
• Each hyper-parameters can be updated by solving the following minimization:
• The hyper-parameters can be learned as their maximum likelihood
estimates by setting their partial derivatives on l’ to 0. Why?
• These all convex optimization!
• Cons: the closed form solution exists.
• Without ICM, we can not get good performance.
ICM OF MRMA (HYPER-PARAMETERS)

21
• U,V updating procedures can be seen as some form of ridge regression.
• Therefore, the closed form solution exists!
• Then, why not MRMA use closed form solution for U and V?
• Netflix Prize Winner said that  
“SGD shows better performance compared to closed form solution”.
• Many empirical studies have supported this claim. Still open question!
• (CF) Two main algorithms for matrix factorization:
• 1. Stochastic Gradient Descent (SGD)
• 2.Alternating Minimization (closed form solution)
• Sub-optimality has been proven [Jain et al 2013].
• But performance is worse than SGD and slow on GPU (SVD calculation)
WHY NOT UPDATING U AND V BY CLOSED FORM?

22
• Besag [1986] said that the performance of ICM is highly sensitive  
to initial value of the parameters and hyper-parameters.
• Therefore, MRMA present the initial values.
• U,V : the solution of classical PMF algorithm.
• : (Bagging) 
• I reproduced MRMA using tensorflow. Difficult to reproduce!
• Even, the authors of RaFM (ICML 2019) said that they can not reproduce the
performance of MRMA paper.
• I can get better performance than MRMA’s report on Movielen 1M (RMSE 0.833).
• Really sensitive to initial value.
• My implementation: https://github.com/JoonyoungYi/MRMA-tensorflow
TRAINING OF MRMA
↵k
and k

23
• I guessed how to estimate R because it did not appear in the MRMA paper.
• Even if we have all the parameters, it is difficult to estimate R using l.
• They have to estimate R using its lower bound l’.  
• In l’, .  
• This can be seen as weighted average of each sub-models.
• Hence,
HOW TO COMPUTE THE PREDICTION OF R?
ˆRi,j =
KX
k=1
↵k
i
k
j Uk
i V k
j
T
/
KX
k=1
↵k
i
k
j
<latexit sha1_base64="8QceK8Ex9TlMga2634ozLlSbEsg=">AAACc3icfVFLSxxBEO6ZvHTz2iSHHPTQcRMwkGxmREhyEEQvghcTHBV21qGmt2an3Z4H3TWBpZlfkH+Xmz8jl5ztfQhGQwoavkdVdXdVWitpKAguPf/e/QcPH62sdh4/efrseffFyxNTNVpgJCpV6bMUDCpZYkSSFJ7VGqFIFZ6mk/2Zf/oDtZFVeUzTGocFjEuZSQHkpKT7M86B7Pc2sfLDRct3eGyaIrGTnbA9t4ctj0HVOTjX0YmjKZJjFwsWXeuxwow2T66NWMtxTu/P7XHLP93sePjffkm3F/SDefC7IFyCHlvGUdL9FY8q0RRYklBgzCAMahpa0CSFwrYTNwZrEBMY48DBEgo0QzsfWsvfOWXEs0q7UxKfqzcrLBTGTIvUZRZAubntzcR/eYOGsi9DK8u6ISzF4qKsUZwqPtsAH0mNgtTUARBaurdykYMGQW5PHTeE8PaX74Joq/+1H37b7u3uLaexwtbYBttkIfvMdtkBO2IRE+y399rj3hvvj7/ub/hvF6m+t6x5xf4K/+MV/ojAqQ==</latexit><latexit sha1_base64="8QceK8Ex9TlMga2634ozLlSbEsg=">AAACc3icfVFLSxxBEO6ZvHTz2iSHHPTQcRMwkGxmREhyEEQvghcTHBV21qGmt2an3Z4H3TWBpZlfkH+Xmz8jl5ztfQhGQwoavkdVdXdVWitpKAguPf/e/QcPH62sdh4/efrseffFyxNTNVpgJCpV6bMUDCpZYkSSFJ7VGqFIFZ6mk/2Zf/oDtZFVeUzTGocFjEuZSQHkpKT7M86B7Pc2sfLDRct3eGyaIrGTnbA9t4ctj0HVOTjX0YmjKZJjFwsWXeuxwow2T66NWMtxTu/P7XHLP93sePjffkm3F/SDefC7IFyCHlvGUdL9FY8q0RRYklBgzCAMahpa0CSFwrYTNwZrEBMY48DBEgo0QzsfWsvfOWXEs0q7UxKfqzcrLBTGTIvUZRZAubntzcR/eYOGsi9DK8u6ISzF4qKsUZwqPtsAH0mNgtTUARBaurdykYMGQW5PHTeE8PaX74Joq/+1H37b7u3uLaexwtbYBttkIfvMdtkBO2IRE+y399rj3hvvj7/ub/hvF6m+t6x5xf4K/+MV/ojAqQ==</latexit><latexit sha1_base64="8QceK8Ex9TlMga2634ozLlSbEsg=">AAACc3icfVFLSxxBEO6ZvHTz2iSHHPTQcRMwkGxmREhyEEQvghcTHBV21qGmt2an3Z4H3TWBpZlfkH+Xmz8jl5ztfQhGQwoavkdVdXdVWitpKAguPf/e/QcPH62sdh4/efrseffFyxNTNVpgJCpV6bMUDCpZYkSSFJ7VGqFIFZ6mk/2Zf/oDtZFVeUzTGocFjEuZSQHkpKT7M86B7Pc2sfLDRct3eGyaIrGTnbA9t4ctj0HVOTjX0YmjKZJjFwsWXeuxwow2T66NWMtxTu/P7XHLP93sePjffkm3F/SDefC7IFyCHlvGUdL9FY8q0RRYklBgzCAMahpa0CSFwrYTNwZrEBMY48DBEgo0QzsfWsvfOWXEs0q7UxKfqzcrLBTGTIvUZRZAubntzcR/eYOGsi9DK8u6ISzF4qKsUZwqPtsAH0mNgtTUARBaurdykYMGQW5PHTeE8PaX74Joq/+1H37b7u3uLaexwtbYBttkIfvMdtkBO2IRE+y399rj3hvvj7/ub/hvF6m+t6x5xf4K/+MV/ojAqQ==</latexit><latexit sha1_base64="8QceK8Ex9TlMga2634ozLlSbEsg=">AAACc3icfVFLSxxBEO6ZvHTz2iSHHPTQcRMwkGxmREhyEEQvghcTHBV21qGmt2an3Z4H3TWBpZlfkH+Xmz8jl5ztfQhGQwoavkdVdXdVWitpKAguPf/e/QcPH62sdh4/efrseffFyxNTNVpgJCpV6bMUDCpZYkSSFJ7VGqFIFZ6mk/2Zf/oDtZFVeUzTGocFjEuZSQHkpKT7M86B7Pc2sfLDRct3eGyaIrGTnbA9t4ctj0HVOTjX0YmjKZJjFwsWXeuxwow2T66NWMtxTu/P7XHLP93sePjffkm3F/SDefC7IFyCHlvGUdL9FY8q0RRYklBgzCAMahpa0CSFwrYTNwZrEBMY48DBEgo0QzsfWsvfOWXEs0q7UxKfqzcrLBTGTIvUZRZAubntzcR/eYOGsi9DK8u6ISzF4qKsUZwqPtsAH0mNgtTUARBaurdykYMGQW5PHTeE8PaX74Joq/+1H37b7u3uLaexwtbYBttkIfvMdtkBO2IRE+y399rj3hvvj7/ub/hvF6m+t6x5xf4K/+MV/ojAqQ==</latexit>
weight each sub-model

25
• Is MRMA effective compared to PMF? (Experiments on Movielens 1M) 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
• I think experimental results of PMF are not natural.  
(The relationship between RMSE and k is inconsistent.)
VS. FIXED-RANK MATRIX APPROXIMATION
approximately 0.840

26
• The set of ranks decide the performance of the final model.
• However, it is not necessary to choose all the ranks in [1, 2, …, K].
• The parameter may be too large and the calculation may not be efficient. 
• Assumption: Overlapping structure
• A rank-k approx. will be very similar to rank-(k − 1) and rank-(k + 1) approx.. 
• Experimental Settings:
• set 1 = {10, 20, 30, …, 300}
• set 2 = {20, 40, 60, …, 300}
• set 3 = {30, 60, 90, …, 300}
• set 4 = {50, 100, 150, …, 300}
• set 5 = {100, 200, 300} 
SENSITIVITY OF RANK

27
• The set of ranks: {10, 20, 50, 100, 150, 200, 250, 300}
• This set showed good performance emperically. 
• The state-of-the-arts performance on Movielens 10M and Netflix datasets.
• Table summarized RMSE with 95% confidence levels 
 
• Better performance compared to other matrix approximation medthods.
• Better than the studies after MRMA (ex. GLOMA,ABCF).
ACCURACY COMPARISON (RATING PREDICTION)
single model

28
• NDCG@N:Another metric that measures recommendation performance.
• Unlike other algorithms, this algorithm showed good performance  
in both NDCG and RMSE.
ACCURACY COMPARISON (ITEM RANKING)

29
• Does MRMA tackle the issue related to their observation?
• Top 10 movies with largest values with rank k=20 and k=200: 
• The movies appropriated to a smaller rank are less evaluated.
• The movies appropriated to a higher rank are more evaluated.
INTERPRETATION OF MRMA
<latexit sha1_base64="iMKl04By15BQHnx60/B/fYUHUAc=">AAAB63icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF/FUwdhCG8pmO2mXbjZhdyKU0t/gxYOKV/+QN/+N2zYHrT4YeLw3w8y8KJPCkOd9OaWV1bX1jfJmZWt7Z3evun/wYNJccwx4KlPdjphBKRQGJEhiO9PIkkhiKxpdz/zWI2ojUnVP4wzDhA2UiAVnZKWgGyGxXrXm1b053L/EL0gNCjR71c9uP+V5goq4ZMZ0fC+jcMI0CS5xWunmBjPGR2yAHUsVS9CEk/mxU/fEKn03TrUtRe5c/TkxYYkx4ySynQmjoVn2ZuJ/Xien+DKcCJXlhIovFsW5dCl1Z5+7faGRkxxbwrgW9laXD5lmnGw+FRuCv/zyXxKc1a/q/t15rXFbpFGGIziGU/DhAhpwA00IgIOAJ3iBV0c5z86b875oLTnFzCH8gvPxDTMgjoA=</latexit><latexit sha1_base64="iMKl04By15BQHnx60/B/fYUHUAc=">AAAB63icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF/FUwdhCG8pmO2mXbjZhdyKU0t/gxYOKV/+QN/+N2zYHrT4YeLw3w8y8KJPCkOd9OaWV1bX1jfJmZWt7Z3evun/wYNJccwx4KlPdjphBKRQGJEhiO9PIkkhiKxpdz/zWI2ojUnVP4wzDhA2UiAVnZKWgGyGxXrXm1b053L/EL0gNCjR71c9uP+V5goq4ZMZ0fC+jcMI0CS5xWunmBjPGR2yAHUsVS9CEk/mxU/fEKn03TrUtRe5c/TkxYYkx4ySynQmjoVn2ZuJ/Xien+DKcCJXlhIovFsW5dCl1Z5+7faGRkxxbwrgW9laXD5lmnGw+FRuCv/zyXxKc1a/q/t15rXFbpFGGIziGU/DhAhpwA00IgIOAJ3iBV0c5z86b875oLTnFzCH8gvPxDTMgjoA=</latexit><latexit sha1_base64="iMKl04By15BQHnx60/B/fYUHUAc=">AAAB63icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF/FUwdhCG8pmO2mXbjZhdyKU0t/gxYOKV/+QN/+N2zYHrT4YeLw3w8y8KJPCkOd9OaWV1bX1jfJmZWt7Z3evun/wYNJccwx4KlPdjphBKRQGJEhiO9PIkkhiKxpdz/zWI2ojUnVP4wzDhA2UiAVnZKWgGyGxXrXm1b053L/EL0gNCjR71c9uP+V5goq4ZMZ0fC+jcMI0CS5xWunmBjPGR2yAHUsVS9CEk/mxU/fEKn03TrUtRe5c/TkxYYkx4ySynQmjoVn2ZuJ/Xien+DKcCJXlhIovFsW5dCl1Z5+7faGRkxxbwrgW9laXD5lmnGw+FRuCv/zyXxKc1a/q/t15rXFbpFGGIziGU/DhAhpwA00IgIOAJ3iBV0c5z86b875oLTnFzCH8gvPxDTMgjoA=</latexit><latexit sha1_base64="iMKl04By15BQHnx60/B/fYUHUAc=">AAAB63icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF/FUwdhCG8pmO2mXbjZhdyKU0t/gxYOKV/+QN/+N2zYHrT4YeLw3w8y8KJPCkOd9OaWV1bX1jfJmZWt7Z3evun/wYNJccwx4KlPdjphBKRQGJEhiO9PIkkhiKxpdz/zWI2ojUnVP4wzDhA2UiAVnZKWgGyGxXrXm1b053L/EL0gNCjR71c9uP+V5goq4ZMZ0fC+jcMI0CS5xWunmBjPGR2yAHUsVS9CEk/mxU/fEKn03TrUtRe5c/TkxYYkx4ySynQmjoVn2ZuJ/Xien+DKcCJXlhIovFsW5dCl1Z5+7faGRkxxbwrgW9laXD5lmnGw+FRuCv/zyXxKc1a/q/t15rXFbpFGGIziGU/DhAhpwA00IgIOAJ3iBV0c5z86b875oLTnFzCH8gvPxDTMgjoA=</latexit>

31
• Is there any idea for replacing ICM?
• The ICM is too dependent on the initial value.
• It takes too long to run the classical PMF algorithms for every rank k. 
• Can not we estimate in the original model rather than estimate in the
lower bound?
• The estimation of MRMA comes from its lower bound.
• Is there any way to use the tight lower bound or expectation?
• The current rank setting is too heuristic. Is there any better way?
DISCUSSION POINTS
ˆR<latexit sha1_base64="SH5F9i/jR7x8G2Fq2m0En9PunPg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL+KpirGFNpTNdtMu3WzC7kQooT/CiwcVr/4fb/4bt20O2vpg4PHeDDPzwlQKg6777ZRWVtfWN8qbla3tnd296v7Bo0kyzbjPEpnodkgNl0JxHwVK3k41p3EoeSscXU/91hPXRiTqAccpD2I6UCISjKKVWt0hxfx+0qvW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx27oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpr+TvtCcoRxbQpkW9lbChlRThjahig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCCZ3iFNyd1Xpx352PeWnKKmUP4A+fzB+TMj38=</latexit><latexit sha1_base64="SH5F9i/jR7x8G2Fq2m0En9PunPg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL+KpirGFNpTNdtMu3WzC7kQooT/CiwcVr/4fb/4bt20O2vpg4PHeDDPzwlQKg6777ZRWVtfWN8qbla3tnd296v7Bo0kyzbjPEpnodkgNl0JxHwVK3k41p3EoeSscXU/91hPXRiTqAccpD2I6UCISjKKVWt0hxfx+0qvW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx27oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpr+TvtCcoRxbQpkW9lbChlRThjahig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCCZ3iFNyd1Xpx352PeWnKKmUP4A+fzB+TMj38=</latexit><latexit sha1_base64="SH5F9i/jR7x8G2Fq2m0En9PunPg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL+KpirGFNpTNdtMu3WzC7kQooT/CiwcVr/4fb/4bt20O2vpg4PHeDDPzwlQKg6777ZRWVtfWN8qbla3tnd296v7Bo0kyzbjPEpnodkgNl0JxHwVK3k41p3EoeSscXU/91hPXRiTqAccpD2I6UCISjKKVWt0hxfx+0qvW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx27oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpr+TvtCcoRxbQpkW9lbChlRThjahig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCCZ3iFNyd1Xpx352PeWnKKmUP4A+fzB+TMj38=</latexit><latexit sha1_base64="SH5F9i/jR7x8G2Fq2m0En9PunPg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL+KpirGFNpTNdtMu3WzC7kQooT/CiwcVr/4fb/4bt20O2vpg4PHeDDPzwlQKg6777ZRWVtfWN8qbla3tnd296v7Bo0kyzbjPEpnodkgNl0JxHwVK3k41p3EoeSscXU/91hPXRiTqAccpD2I6UCISjKKVWt0hxfx+0qvW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx27oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpr+TvtCcoRxbQpkW9lbChlRThjahig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCCZ3iFNyd1Xpx352PeWnKKmUP4A+fzB+TMj38=</latexit>
ˆR<latexit sha1_base64="SH5F9i/jR7x8G2Fq2m0En9PunPg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL+KpirGFNpTNdtMu3WzC7kQooT/CiwcVr/4fb/4bt20O2vpg4PHeDDPzwlQKg6777ZRWVtfWN8qbla3tnd296v7Bo0kyzbjPEpnodkgNl0JxHwVK3k41p3EoeSscXU/91hPXRiTqAccpD2I6UCISjKKVWt0hxfx+0qvW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx27oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpr+TvtCcoRxbQpkW9lbChlRThjahig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCCZ3iFNyd1Xpx352PeWnKKmUP4A+fzB+TMj38=</latexit><latexit sha1_base64="SH5F9i/jR7x8G2Fq2m0En9PunPg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL+KpirGFNpTNdtMu3WzC7kQooT/CiwcVr/4fb/4bt20O2vpg4PHeDDPzwlQKg6777ZRWVtfWN8qbla3tnd296v7Bo0kyzbjPEpnodkgNl0JxHwVK3k41p3EoeSscXU/91hPXRiTqAccpD2I6UCISjKKVWt0hxfx+0qvW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx27oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpr+TvtCcoRxbQpkW9lbChlRThjahig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCCZ3iFNyd1Xpx352PeWnKKmUP4A+fzB+TMj38=</latexit><latexit sha1_base64="SH5F9i/jR7x8G2Fq2m0En9PunPg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL+KpirGFNpTNdtMu3WzC7kQooT/CiwcVr/4fb/4bt20O2vpg4PHeDDPzwlQKg6777ZRWVtfWN8qbla3tnd296v7Bo0kyzbjPEpnodkgNl0JxHwVK3k41p3EoeSscXU/91hPXRiTqAccpD2I6UCISjKKVWt0hxfx+0qvW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx27oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpr+TvtCcoRxbQpkW9lbChlRThjahig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCCZ3iFNyd1Xpx352PeWnKKmUP4A+fzB+TMj38=</latexit><latexit sha1_base64="SH5F9i/jR7x8G2Fq2m0En9PunPg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL+KpirGFNpTNdtMu3WzC7kQooT/CiwcVr/4fb/4bt20O2vpg4PHeDDPzwlQKg6777ZRWVtfWN8qbla3tnd296v7Bo0kyzbjPEpnodkgNl0JxHwVK3k41p3EoeSscXU/91hPXRiTqAccpD2I6UCISjKKVWt0hxfx+0qvW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx27oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpr+TvtCcoRxbQpkW9lbChlRThjahig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCCZ3iFNyd1Xpx352PeWnKKmUP4A+fzB+TMj38=</latexit>

32
• The current state-of-the-arts table [4] on Movielens 100k, 1M, 10M: 
• RMSE of MRMA on Movielens 100k: 0.893 (reproduction)
• RMSE of MRMA on Movielens 1M: 0.833 (reproduction)
• Why we can’t get the best performance with relatively small dataset  
such as 100k and 1m?
THE PERFORMANCE ON SMALL SIZE DATASET

33
• Inspired by Slimmable Network [2].
• The first column of U1 and U2 may be similar.
• Sharing the first column of U1 and U2 would reduce the number of
parameters drastically and enable efficient learning.
• If using weight sharing,
• would not be necessary to worry about how to set a rank set.
• would not be necessary to initialize using PMF?
WEIGHT SHARING

35
• [1] Li, Dongsheng, et al. "Mixture-rank matrix approximation for
collaborative filtering." Advances in Neural Information Processing Systems.
2017.
• [2]Yu, Jiahui, and Thomas Huang. "Universally Slimmable Networks and
Improved Training Techniques." arXiv preprint arXiv:1903.05134 (2019).
• [3] Chen, Xiaoshuang, et al. "RaFM: Rank-Aware Factorization
Machines." arXiv preprint arXiv:1905.07570 (2019).
• [4]Yi, Joonyoung, et al. "Sparsity Normalization: Stabilizing the Expected
Outputs of Deep Networks." arXiv preprint arXiv:1906.00150 (2019).
REFERENCE

Mixture-Rank Matrix Approximation for Collaborative Filtering

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

Mixture-Rank Matrix Approximation for Collaborative Filtering