Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Neural Autoregressive Approach to Collaborative Filtering (CF-NADE) Slide

PPT for A Neural Autoregressive Approach to Collaborative Filtering (CF-NADE).
I made ppt for explaining the paper.

Abstract of the paper:
This paper proposes CF-NADE, a neural autoregressive architecture for collaborative filtering (CF) tasks, which is inspired by the Restricted Boltzmann Machine (RBM) based CF model and the Neural Autoregressive Distribution Estimator (NADE). We first describe the basic CF-NADE model for CF tasks. Then we propose to improve the model by sharing parameters between different ratings. A factored version of CF-NADE is also proposed for better scalability. Furthermore, we take the ordinal nature of the preferences into consideration and propose an ordinal cost to optimize CF-NADE, which shows superior performance. Finally, CF-NADE can be extended to a deep model, with only moderately increased computational complexity. Experimental results show that CF-NADE with a single hidden layer beats all previous state-of-the-art methods on MovieLens 1M, MovieLens 10M, and Netflix datasets, and adding more hidden layers can further improve the performance.

  • Be the first to comment

  • Be the first to like this

A Neural Autoregressive Approach to Collaborative Filtering (CF-NADE) Slide

  1. 1. CF-NADE A NEURAL AUTOREGRESSIVE APPROACH TO COLLABORATIVE FILTERING 2018. 10. 31. JoonyoungYi joonyoung.yi@kaist.ac.kr *This slide is adopted from the author’s slide.
  2. 2. CONTENTS 1. Motivation 2. NADE 3. Basic Model of CF-NADE 4. Parameters Sharing 5. Ordinal Cost 6. Deep Models and Augmentation 7. Experiments 8. Future Works and Discussion
  3. 3. 3 • NADE (Neural Autoregressive Density Estimator) • 2011 AISTAT. • Alternative to Restricted Boltzmann Machine (RBM). • CF-NADE (NADE to Collaborative Filtering) • 2016 ICML. • Apply NADE to Collaborative Filtering (CF) Problem. • Matrix Completion is one of the popular algorithms to solve CF Problem. TODAY’S PAPERS
  4. 4. 4 • The Netflix recommendation engine use hybrid models combined with matrix factorization based models (Biased-MF, …) and deep networks [3]. • Not Netflix Prize. • The main model used for deep networks is RBM-CF [9]. • The RBM-CF apply RBM to CF problem. • An algorithm that works well with practical data. • Authors of CF-NADE were in Hulu, which is a competitor of the Netflix. • To enhance their recommendation engine. • By replacing RBM-CF to CF-NADE. • Personally, the CF-NADE paper is able to get several insights on engineering. MOTIVATION
  5. 5. 5 MOTIVATION RBM NADE RBM-CF CF-NADE
  6. 6. CONTENTS 1. Motivation 2. NADE 3. Basic Model of CF-NADE 4. Parameters Sharing 5. Ordinal Cost 6. Deep Models and Augmentation 7. Experiments 8. Future Works and Discussion
  7. 7. 7 • RBM is not tractable by the partition function. • The goal of NADE: • Make RBM be tractable. • To do this, the author of NADE convert RBM to a Bayesian network. RBM TO NADE Z is the partition function. We should calculate Z by considering all possible v, h The thing we need to train in NADE
  8. 8. 8 • The model of NADE: • It is a tractable.There is no partition function or similar things. • For fixed length binary vectors. • Practical for high dimensional data. • Practical version of NADE: NADE p(vi = 1|v<i) = sigm(bi + Vi,:hi) hi = sigm(c + X j<i W:,j) <latexit sha1_base64="4YOzk+sFkqupXgiISHGBsvBHvLw=">AAACdHicbVFNaxsxENVuvxL3I05zCZTCNCZg02B2S6BtSCDQS6GXFOo44DWLVpZtJdLuIs2aGkX/IL+ut/6NXHKN7CwkdTogePPmPY00k5VSGIyiv0H45Omz5y/W1hsvX71+s9HcfHtqikoz3mOFLPRZRg2XIuc9FCj5Wak5VZnk/ezi26Len3FtRJH/wnnJh4pOcjEWjKKn0uZVCW2YpQKOIIZLSLJCjuzMpfZQuI4nE+S/0RoxUQ7amdd9rDWnXiP24MDV+dSlogNJ0rhPV+21krnFJaZSqT33XaCf2oM9OHfQSZutqBstAx6DuAYtUsdJ2vyTjApWKZ4jk9SYQRyVOLRUo2CSu0ZSGV5SdkEnfOBhThU3Q7ucmoNdz4xgXGh/coQl+9BhqTJmrjKvVBSnZrW2IP9XG1Q4/jK0Ii8r5Dm7azSuJGABixXASGjOUM49oEwL/1ZgU6opQ7+ohh9CvPrlx6D3qfu1G//cbx3/qKexRt6RHdImMflMjsl3ckJ6hJHrYDv4EOwEN+H7sBXu3knDoPZskX8i7N4CrD66zw==</latexit><latexit sha1_base64="4YOzk+sFkqupXgiISHGBsvBHvLw=">AAACdHicbVFNaxsxENVuvxL3I05zCZTCNCZg02B2S6BtSCDQS6GXFOo44DWLVpZtJdLuIs2aGkX/IL+ut/6NXHKN7CwkdTogePPmPY00k5VSGIyiv0H45Omz5y/W1hsvX71+s9HcfHtqikoz3mOFLPRZRg2XIuc9FCj5Wak5VZnk/ezi26Len3FtRJH/wnnJh4pOcjEWjKKn0uZVCW2YpQKOIIZLSLJCjuzMpfZQuI4nE+S/0RoxUQ7amdd9rDWnXiP24MDV+dSlogNJ0rhPV+21krnFJaZSqT33XaCf2oM9OHfQSZutqBstAx6DuAYtUsdJ2vyTjApWKZ4jk9SYQRyVOLRUo2CSu0ZSGV5SdkEnfOBhThU3Q7ucmoNdz4xgXGh/coQl+9BhqTJmrjKvVBSnZrW2IP9XG1Q4/jK0Ii8r5Dm7azSuJGABixXASGjOUM49oEwL/1ZgU6opQ7+ohh9CvPrlx6D3qfu1G//cbx3/qKexRt6RHdImMflMjsl3ckJ6hJHrYDv4EOwEN+H7sBXu3knDoPZskX8i7N4CrD66zw==</latexit><latexit sha1_base64="4YOzk+sFkqupXgiISHGBsvBHvLw=">AAACdHicbVFNaxsxENVuvxL3I05zCZTCNCZg02B2S6BtSCDQS6GXFOo44DWLVpZtJdLuIs2aGkX/IL+ut/6NXHKN7CwkdTogePPmPY00k5VSGIyiv0H45Omz5y/W1hsvX71+s9HcfHtqikoz3mOFLPRZRg2XIuc9FCj5Wak5VZnk/ezi26Len3FtRJH/wnnJh4pOcjEWjKKn0uZVCW2YpQKOIIZLSLJCjuzMpfZQuI4nE+S/0RoxUQ7amdd9rDWnXiP24MDV+dSlogNJ0rhPV+21krnFJaZSqT33XaCf2oM9OHfQSZutqBstAx6DuAYtUsdJ2vyTjApWKZ4jk9SYQRyVOLRUo2CSu0ZSGV5SdkEnfOBhThU3Q7ucmoNdz4xgXGh/coQl+9BhqTJmrjKvVBSnZrW2IP9XG1Q4/jK0Ii8r5Dm7azSuJGABixXASGjOUM49oEwL/1ZgU6opQ7+ohh9CvPrlx6D3qfu1G//cbx3/qKexRt6RHdImMflMjsl3ckJ6hJHrYDv4EOwEN+H7sBXu3knDoPZskX8i7N4CrD66zw==</latexit><latexit sha1_base64="4YOzk+sFkqupXgiISHGBsvBHvLw=">AAACdHicbVFNaxsxENVuvxL3I05zCZTCNCZg02B2S6BtSCDQS6GXFOo44DWLVpZtJdLuIs2aGkX/IL+ut/6NXHKN7CwkdTogePPmPY00k5VSGIyiv0H45Omz5y/W1hsvX71+s9HcfHtqikoz3mOFLPRZRg2XIuc9FCj5Wak5VZnk/ezi26Len3FtRJH/wnnJh4pOcjEWjKKn0uZVCW2YpQKOIIZLSLJCjuzMpfZQuI4nE+S/0RoxUQ7amdd9rDWnXiP24MDV+dSlogNJ0rhPV+21krnFJaZSqT33XaCf2oM9OHfQSZutqBstAx6DuAYtUsdJ2vyTjApWKZ4jk9SYQRyVOLRUo2CSu0ZSGV5SdkEnfOBhThU3Q7ucmoNdz4xgXGh/coQl+9BhqTJmrjKvVBSnZrW2IP9XG1Q4/jK0Ii8r5Dm7azSuJGABixXASGjOUM49oEwL/1ZgU6opQ7+ohh9CvPrlx6D3qfu1G//cbx3/qKexRt6RHdImMflMjsl3ckJ6hJHrYDv4EOwEN+H7sBXu3knDoPZskX8i7N4CrD66zw==</latexit>
  9. 9. CONTENTS 1. Motivation 2. NADE 3. Basic Model of CF-NADE 4. Parameters Sharing 5. Ordinal Cost 6. Deep Models and Augmentation 7. Experiments 8. Future Works and Discussion
  10. 10. 10 • The training case for user u: • o is a D-tuple in the set of permutations of (1, 2, …, D). • The items . • . • For simplicity, the index u of ru can be omitted. • We want to model the below equation by NADE. • : the first i-1 elements of r indexed by o. • D is the number of ratings by user u. • o: ordering. NOTATION AND PROBABILISTIC MODEL
  11. 11. • The objective function: • where • g( ) : activation function like tanh. • is a latent matrix. is a weight matrix. • and are the bias terms. 11 THE BASIC MODEL no activation function
  12. 12. 12 FIGURES OF THE MODEL W1 W2 Wk…
  13. 13. 13 FIGURES OF THE MODEL W1 W2 Wk… + W rmo1 :,mo1<latexit sha1_base64="iJwv7B/8dh/Y1fnz6gNS8tSGnig=">AAACBXicbVDLSgMxFM34rPU16lKEYBFcSJkRwceq4EZwU8GxhXYcMmnahiaTIckIJczKjb/ixoWKW//BnX9j2s5CWw9cODnnXnLviVNGlfa8b2dufmFxabm0Ul5dW9/YdLe275TIJCYBFkzIZowUYTQhgaaakWYqCeIxI414cDnyGw9EKiqSWz1MSchRL6FdipG2UuTuNSJzcQR5ZETk5zC/NzIyxSvPI7fiVb0x4CzxC1IBBeqR+9XuCJxxkmjMkFIt30t1aJDUFDOSl9uZIinCA9QjLUsTxIkKzfiMHB5YpQO7QtpKNByrvycM4koNeWw7OdJ9Ne2NxP+8Vqa7Z6GhSZppkuDJR92MQS3gKBPYoZJgzYaWICyp3RXiPpIIa5tc2YbgT588S4Lj6nnVvzmp1K6LNEpgF+yDQ+CDU1ADV6AOAoDBI3gGr+DNeXJenHfnY9I65xQzO+APnM8fTteYow==</latexit><latexit sha1_base64="iJwv7B/8dh/Y1fnz6gNS8tSGnig=">AAACBXicbVDLSgMxFM34rPU16lKEYBFcSJkRwceq4EZwU8GxhXYcMmnahiaTIckIJczKjb/ixoWKW//BnX9j2s5CWw9cODnnXnLviVNGlfa8b2dufmFxabm0Ul5dW9/YdLe275TIJCYBFkzIZowUYTQhgaaakWYqCeIxI414cDnyGw9EKiqSWz1MSchRL6FdipG2UuTuNSJzcQR5ZETk5zC/NzIyxSvPI7fiVb0x4CzxC1IBBeqR+9XuCJxxkmjMkFIt30t1aJDUFDOSl9uZIinCA9QjLUsTxIkKzfiMHB5YpQO7QtpKNByrvycM4koNeWw7OdJ9Ne2NxP+8Vqa7Z6GhSZppkuDJR92MQS3gKBPYoZJgzYaWICyp3RXiPpIIa5tc2YbgT588S4Lj6nnVvzmp1K6LNEpgF+yDQ+CDU1ADV6AOAoDBI3gGr+DNeXJenHfnY9I65xQzO+APnM8fTteYow==</latexit><latexit sha1_base64="iJwv7B/8dh/Y1fnz6gNS8tSGnig=">AAACBXicbVDLSgMxFM34rPU16lKEYBFcSJkRwceq4EZwU8GxhXYcMmnahiaTIckIJczKjb/ixoWKW//BnX9j2s5CWw9cODnnXnLviVNGlfa8b2dufmFxabm0Ul5dW9/YdLe275TIJCYBFkzIZowUYTQhgaaakWYqCeIxI414cDnyGw9EKiqSWz1MSchRL6FdipG2UuTuNSJzcQR5ZETk5zC/NzIyxSvPI7fiVb0x4CzxC1IBBeqR+9XuCJxxkmjMkFIt30t1aJDUFDOSl9uZIinCA9QjLUsTxIkKzfiMHB5YpQO7QtpKNByrvycM4koNeWw7OdJ9Ne2NxP+8Vqa7Z6GhSZppkuDJR92MQS3gKBPYoZJgzYaWICyp3RXiPpIIa5tc2YbgT588S4Lj6nnVvzmp1K6LNEpgF+yDQ+CDU1ADV6AOAoDBI3gGr+DNeXJenHfnY9I65xQzO+APnM8fTteYow==</latexit><latexit sha1_base64="iJwv7B/8dh/Y1fnz6gNS8tSGnig=">AAACBXicbVDLSgMxFM34rPU16lKEYBFcSJkRwceq4EZwU8GxhXYcMmnahiaTIckIJczKjb/ixoWKW//BnX9j2s5CWw9cODnnXnLviVNGlfa8b2dufmFxabm0Ul5dW9/YdLe275TIJCYBFkzIZowUYTQhgaaakWYqCeIxI414cDnyGw9EKiqSWz1MSchRL6FdipG2UuTuNSJzcQR5ZETk5zC/NzIyxSvPI7fiVb0x4CzxC1IBBeqR+9XuCJxxkmjMkFIt30t1aJDUFDOSl9uZIinCA9QjLUsTxIkKzfiMHB5YpQO7QtpKNByrvycM4koNeWw7OdJ9Ne2NxP+8Vqa7Z6GhSZppkuDJR92MQS3gKBPYoZJgzYaWICyp3RXiPpIIa5tc2YbgT588S4Lj6nnVvzmp1K6LNEpgF+yDQ+CDU1ADV6AOAoDBI3gGr+DNeXJenHfnY9I65xQzO+APnM8fTteYow==</latexit> + + … + W rmo2 :,mo2<latexit sha1_base64="opdz4oAh5Obm9hxV3ZKKD/28H70=">AAACBXicbVDLSgMxFM3UV62vUZciBIvgQspMEXysCm4ENxUcW2jHIZNm2tDMZEgyQgmzcuOvuHGh4tZ/cOffmLaz0NYDF07OuZfce8KUUakc59sqLSwuLa+UVytr6xubW/b2zp3kmcDEw5xx0Q6RJIwmxFNUMdJOBUFxyEgrHF6O/dYDEZLy5FaNUuLHqJ/QiGKkjBTY+61AXxzDONA8qOcwv9ci0MUrzwO76tScCeA8cQtSBQWagf3V7XGcxSRRmCEpO66TKl8joShmJK90M0lShIeoTzqGJigm0teTM3J4aJQejLgwlSg4UX9PaBRLOYpD0xkjNZCz3lj8z+tkKjrzNU3STJEETz+KMgYVh+NMYI8KghUbGYKwoGZXiAdIIKxMchUTgjt78jzx6rXzmntzUm1cF2mUwR44AEfABaegAa5AE3gAg0fwDF7Bm/VkvVjv1se0tWQVM7vgD6zPH1HxmKU=</latexit><latexit sha1_base64="opdz4oAh5Obm9hxV3ZKKD/28H70=">AAACBXicbVDLSgMxFM3UV62vUZciBIvgQspMEXysCm4ENxUcW2jHIZNm2tDMZEgyQgmzcuOvuHGh4tZ/cOffmLaz0NYDF07OuZfce8KUUakc59sqLSwuLa+UVytr6xubW/b2zp3kmcDEw5xx0Q6RJIwmxFNUMdJOBUFxyEgrHF6O/dYDEZLy5FaNUuLHqJ/QiGKkjBTY+61AXxzDONA8qOcwv9ci0MUrzwO76tScCeA8cQtSBQWagf3V7XGcxSRRmCEpO66TKl8joShmJK90M0lShIeoTzqGJigm0teTM3J4aJQejLgwlSg4UX9PaBRLOYpD0xkjNZCz3lj8z+tkKjrzNU3STJEETz+KMgYVh+NMYI8KghUbGYKwoGZXiAdIIKxMchUTgjt78jzx6rXzmntzUm1cF2mUwR44AEfABaegAa5AE3gAg0fwDF7Bm/VkvVjv1se0tWQVM7vgD6zPH1HxmKU=</latexit><latexit sha1_base64="opdz4oAh5Obm9hxV3ZKKD/28H70=">AAACBXicbVDLSgMxFM3UV62vUZciBIvgQspMEXysCm4ENxUcW2jHIZNm2tDMZEgyQgmzcuOvuHGh4tZ/cOffmLaz0NYDF07OuZfce8KUUakc59sqLSwuLa+UVytr6xubW/b2zp3kmcDEw5xx0Q6RJIwmxFNUMdJOBUFxyEgrHF6O/dYDEZLy5FaNUuLHqJ/QiGKkjBTY+61AXxzDONA8qOcwv9ci0MUrzwO76tScCeA8cQtSBQWagf3V7XGcxSRRmCEpO66TKl8joShmJK90M0lShIeoTzqGJigm0teTM3J4aJQejLgwlSg4UX9PaBRLOYpD0xkjNZCz3lj8z+tkKjrzNU3STJEETz+KMgYVh+NMYI8KghUbGYKwoGZXiAdIIKxMchUTgjt78jzx6rXzmntzUm1cF2mUwR44AEfABaegAa5AE3gAg0fwDF7Bm/VkvVjv1se0tWQVM7vgD6zPH1HxmKU=</latexit><latexit sha1_base64="opdz4oAh5Obm9hxV3ZKKD/28H70=">AAACBXicbVDLSgMxFM3UV62vUZciBIvgQspMEXysCm4ENxUcW2jHIZNm2tDMZEgyQgmzcuOvuHGh4tZ/cOffmLaz0NYDF07OuZfce8KUUakc59sqLSwuLa+UVytr6xubW/b2zp3kmcDEw5xx0Q6RJIwmxFNUMdJOBUFxyEgrHF6O/dYDEZLy5FaNUuLHqJ/QiGKkjBTY+61AXxzDONA8qOcwv9ci0MUrzwO76tScCeA8cQtSBQWagf3V7XGcxSRRmCEpO66TKl8joShmJK90M0lShIeoTzqGJigm0teTM3J4aJQejLgwlSg4UX9PaBRLOYpD0xkjNZCz3lj8z+tkKjrzNU3STJEETz+KMgYVh+NMYI8KghUbGYKwoGZXiAdIIKxMchUTgjt78jzx6rXzmntzUm1cF2mUwR44AEfABaegAa5AE3gAg0fwDF7Bm/VkvVjv1se0tWQVM7vgD6zPH1HxmKU=</latexit> W rmoi 1 :,moi 1<latexit sha1_base64="vDWHlwlOmdQjyoSWksi1NThDM8A=">AAACDnicbVDLSgMxFM3UV62vUZdugkVxoWVGBB+rghvBTQXHFtpxyKRpG5pJhiQjlNA/cOOvuHGh4ta1O//GtB1Qq+dy4XDOvST3xCmjSnvep1OYmZ2bXygulpaWV1bX3PWNGyUyiUmABROyESNFGOUk0FQz0kglQUnMSD3un4/8+h2Rigp+rQcpCRPU5bRDMdJWitzdemTO9mESGREZeuAPoa1bIyPzLVlEbtmreGPAv8TPSRnkqEXuR6stcJYQrjFDSjV9L9WhQVJTzMiw1MoUSRHuoy5pWspRQlRoxvcM4Y5V2rAjpG2u4Vj9uWFQotQgie1kgnRPTXsj8T+vmenOSWgoTzNNOJ481MkY1AKOwoFtKgnWbGAJwpLav0LcQxJhbSMs2RD86ZP/kuCwclrxr47K1cs8jSLYAttgD/jgGFTBBaiBAGBwDx7BM3hxHpwn59V5m4wWnHxnE/yC8/4F64qcOQ==</latexit><latexit sha1_base64="vDWHlwlOmdQjyoSWksi1NThDM8A=">AAACDnicbVDLSgMxFM3UV62vUZdugkVxoWVGBB+rghvBTQXHFtpxyKRpG5pJhiQjlNA/cOOvuHGh4ta1O//GtB1Qq+dy4XDOvST3xCmjSnvep1OYmZ2bXygulpaWV1bX3PWNGyUyiUmABROyESNFGOUk0FQz0kglQUnMSD3un4/8+h2Rigp+rQcpCRPU5bRDMdJWitzdemTO9mESGREZeuAPoa1bIyPzLVlEbtmreGPAv8TPSRnkqEXuR6stcJYQrjFDSjV9L9WhQVJTzMiw1MoUSRHuoy5pWspRQlRoxvcM4Y5V2rAjpG2u4Vj9uWFQotQgie1kgnRPTXsj8T+vmenOSWgoTzNNOJ481MkY1AKOwoFtKgnWbGAJwpLav0LcQxJhbSMs2RD86ZP/kuCwclrxr47K1cs8jSLYAttgD/jgGFTBBaiBAGBwDx7BM3hxHpwn59V5m4wWnHxnE/yC8/4F64qcOQ==</latexit><latexit sha1_base64="vDWHlwlOmdQjyoSWksi1NThDM8A=">AAACDnicbVDLSgMxFM3UV62vUZdugkVxoWVGBB+rghvBTQXHFtpxyKRpG5pJhiQjlNA/cOOvuHGh4ta1O//GtB1Qq+dy4XDOvST3xCmjSnvep1OYmZ2bXygulpaWV1bX3PWNGyUyiUmABROyESNFGOUk0FQz0kglQUnMSD3un4/8+h2Rigp+rQcpCRPU5bRDMdJWitzdemTO9mESGREZeuAPoa1bIyPzLVlEbtmreGPAv8TPSRnkqEXuR6stcJYQrjFDSjV9L9WhQVJTzMiw1MoUSRHuoy5pWspRQlRoxvcM4Y5V2rAjpG2u4Vj9uWFQotQgie1kgnRPTXsj8T+vmenOSWgoTzNNOJ481MkY1AKOwoFtKgnWbGAJwpLav0LcQxJhbSMs2RD86ZP/kuCwclrxr47K1cs8jSLYAttgD/jgGFTBBaiBAGBwDx7BM3hxHpwn59V5m4wWnHxnE/yC8/4F64qcOQ==</latexit><latexit sha1_base64="vDWHlwlOmdQjyoSWksi1NThDM8A=">AAACDnicbVDLSgMxFM3UV62vUZdugkVxoWVGBB+rghvBTQXHFtpxyKRpG5pJhiQjlNA/cOOvuHGh4ta1O//GtB1Qq+dy4XDOvST3xCmjSnvep1OYmZ2bXygulpaWV1bX3PWNGyUyiUmABROyESNFGOUk0FQz0kglQUnMSD3un4/8+h2Rigp+rQcpCRPU5bRDMdJWitzdemTO9mESGREZeuAPoa1bIyPzLVlEbtmreGPAv8TPSRnkqEXuR6stcJYQrjFDSjV9L9WhQVJTzMiw1MoUSRHuoy5pWspRQlRoxvcM4Y5V2rAjpG2u4Vj9uWFQotQgie1kgnRPTXsj8T+vmenOSWgoTzNNOJ481MkY1AKOwoFtKgnWbGAJwpLav0LcQxJhbSMs2RD86ZP/kuCwclrxr47K1cs8jSLYAttgD/jgGFTBBaiBAGBwDx7BM3hxHpwn59V5m4wWnHxnE/yC8/4F64qcOQ==</latexit>
  14. 14. 14 FIGURES OF THE MODEL + W rmo1 :,mo1<latexit sha1_base64="iJwv7B/8dh/Y1fnz6gNS8tSGnig=">AAACBXicbVDLSgMxFM34rPU16lKEYBFcSJkRwceq4EZwU8GxhXYcMmnahiaTIckIJczKjb/ixoWKW//BnX9j2s5CWw9cODnnXnLviVNGlfa8b2dufmFxabm0Ul5dW9/YdLe275TIJCYBFkzIZowUYTQhgaaakWYqCeIxI414cDnyGw9EKiqSWz1MSchRL6FdipG2UuTuNSJzcQR5ZETk5zC/NzIyxSvPI7fiVb0x4CzxC1IBBeqR+9XuCJxxkmjMkFIt30t1aJDUFDOSl9uZIinCA9QjLUsTxIkKzfiMHB5YpQO7QtpKNByrvycM4koNeWw7OdJ9Ne2NxP+8Vqa7Z6GhSZppkuDJR92MQS3gKBPYoZJgzYaWICyp3RXiPpIIa5tc2YbgT588S4Lj6nnVvzmp1K6LNEpgF+yDQ+CDU1ADV6AOAoDBI3gGr+DNeXJenHfnY9I65xQzO+APnM8fTteYow==</latexit><latexit sha1_base64="iJwv7B/8dh/Y1fnz6gNS8tSGnig=">AAACBXicbVDLSgMxFM34rPU16lKEYBFcSJkRwceq4EZwU8GxhXYcMmnahiaTIckIJczKjb/ixoWKW//BnX9j2s5CWw9cODnnXnLviVNGlfa8b2dufmFxabm0Ul5dW9/YdLe275TIJCYBFkzIZowUYTQhgaaakWYqCeIxI414cDnyGw9EKiqSWz1MSchRL6FdipG2UuTuNSJzcQR5ZETk5zC/NzIyxSvPI7fiVb0x4CzxC1IBBeqR+9XuCJxxkmjMkFIt30t1aJDUFDOSl9uZIinCA9QjLUsTxIkKzfiMHB5YpQO7QtpKNByrvycM4koNeWw7OdJ9Ne2NxP+8Vqa7Z6GhSZppkuDJR92MQS3gKBPYoZJgzYaWICyp3RXiPpIIa5tc2YbgT588S4Lj6nnVvzmp1K6LNEpgF+yDQ+CDU1ADV6AOAoDBI3gGr+DNeXJenHfnY9I65xQzO+APnM8fTteYow==</latexit><latexit sha1_base64="iJwv7B/8dh/Y1fnz6gNS8tSGnig=">AAACBXicbVDLSgMxFM34rPU16lKEYBFcSJkRwceq4EZwU8GxhXYcMmnahiaTIckIJczKjb/ixoWKW//BnX9j2s5CWw9cODnnXnLviVNGlfa8b2dufmFxabm0Ul5dW9/YdLe275TIJCYBFkzIZowUYTQhgaaakWYqCeIxI414cDnyGw9EKiqSWz1MSchRL6FdipG2UuTuNSJzcQR5ZETk5zC/NzIyxSvPI7fiVb0x4CzxC1IBBeqR+9XuCJxxkmjMkFIt30t1aJDUFDOSl9uZIinCA9QjLUsTxIkKzfiMHB5YpQO7QtpKNByrvycM4koNeWw7OdJ9Ne2NxP+8Vqa7Z6GhSZppkuDJR92MQS3gKBPYoZJgzYaWICyp3RXiPpIIa5tc2YbgT588S4Lj6nnVvzmp1K6LNEpgF+yDQ+CDU1ADV6AOAoDBI3gGr+DNeXJenHfnY9I65xQzO+APnM8fTteYow==</latexit><latexit sha1_base64="iJwv7B/8dh/Y1fnz6gNS8tSGnig=">AAACBXicbVDLSgMxFM34rPU16lKEYBFcSJkRwceq4EZwU8GxhXYcMmnahiaTIckIJczKjb/ixoWKW//BnX9j2s5CWw9cODnnXnLviVNGlfa8b2dufmFxabm0Ul5dW9/YdLe275TIJCYBFkzIZowUYTQhgaaakWYqCeIxI414cDnyGw9EKiqSWz1MSchRL6FdipG2UuTuNSJzcQR5ZETk5zC/NzIyxSvPI7fiVb0x4CzxC1IBBeqR+9XuCJxxkmjMkFIt30t1aJDUFDOSl9uZIinCA9QjLUsTxIkKzfiMHB5YpQO7QtpKNByrvycM4koNeWw7OdJ9Ne2NxP+8Vqa7Z6GhSZppkuDJR92MQS3gKBPYoZJgzYaWICyp3RXiPpIIa5tc2YbgT588S4Lj6nnVvzmp1K6LNEpgF+yDQ+CDU1ADV6AOAoDBI3gGr+DNeXJenHfnY9I65xQzO+APnM8fTteYow==</latexit> + c<latexit sha1_base64="8xJmIOcF0OzDWOLwsax2XAaqU70=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL4KXCsYW2lA2m027dLMbdzdCCfkTXjyoePX3ePPfuG1z0NYHA4/3ZpiZF6acaeO6305lZXVtfaO6Wdva3tndq+8fPGiZKUJ9IrlU3RBrypmgvmGG026qKE5CTjvh+Hrqd56o0kyKezNJaZDgoWAxI9hYqdsPJY9yUgzqDbfpzoCWiVeSBpRoD+pf/UiSLKHCEI617nluaoIcK8MIp0Wtn2maYjLGQ9qzVOCE6iCf3VugE6tEKJbKljBopv6eyHGi9SQJbWeCzUgvelPxP6+XmfgyyJlIM0MFmS+KM46MRNPnUcQUJYZPLMFEMXsrIiOsMDE2opoNwVt8eZn4Z82rpnd33mjdlmlU4QiO4RQ8uIAW3EAbfCDA4Rle4c15dF6cd+dj3lpxyplD+APn8we9zo//</latexit><latexit sha1_base64="8xJmIOcF0OzDWOLwsax2XAaqU70=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL4KXCsYW2lA2m027dLMbdzdCCfkTXjyoePX3ePPfuG1z0NYHA4/3ZpiZF6acaeO6305lZXVtfaO6Wdva3tndq+8fPGiZKUJ9IrlU3RBrypmgvmGG026qKE5CTjvh+Hrqd56o0kyKezNJaZDgoWAxI9hYqdsPJY9yUgzqDbfpzoCWiVeSBpRoD+pf/UiSLKHCEI617nluaoIcK8MIp0Wtn2maYjLGQ9qzVOCE6iCf3VugE6tEKJbKljBopv6eyHGi9SQJbWeCzUgvelPxP6+XmfgyyJlIM0MFmS+KM46MRNPnUcQUJYZPLMFEMXsrIiOsMDE2opoNwVt8eZn4Z82rpnd33mjdlmlU4QiO4RQ8uIAW3EAbfCDA4Rle4c15dF6cd+dj3lpxyplD+APn8we9zo//</latexit><latexit sha1_base64="8xJmIOcF0OzDWOLwsax2XAaqU70=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL4KXCsYW2lA2m027dLMbdzdCCfkTXjyoePX3ePPfuG1z0NYHA4/3ZpiZF6acaeO6305lZXVtfaO6Wdva3tndq+8fPGiZKUJ9IrlU3RBrypmgvmGG026qKE5CTjvh+Hrqd56o0kyKezNJaZDgoWAxI9hYqdsPJY9yUgzqDbfpzoCWiVeSBpRoD+pf/UiSLKHCEI617nluaoIcK8MIp0Wtn2maYjLGQ9qzVOCE6iCf3VugE6tEKJbKljBopv6eyHGi9SQJbWeCzUgvelPxP6+XmfgyyJlIM0MFmS+KM46MRNPnUcQUJYZPLMFEMXsrIiOsMDE2opoNwVt8eZn4Z82rpnd33mjdlmlU4QiO4RQ8uIAW3EAbfCDA4Rle4c15dF6cd+dj3lpxyplD+APn8we9zo//</latexit><latexit sha1_base64="8xJmIOcF0OzDWOLwsax2XAaqU70=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL4KXCsYW2lA2m027dLMbdzdCCfkTXjyoePX3ePPfuG1z0NYHA4/3ZpiZF6acaeO6305lZXVtfaO6Wdva3tndq+8fPGiZKUJ9IrlU3RBrypmgvmGG026qKE5CTjvh+Hrqd56o0kyKezNJaZDgoWAxI9hYqdsPJY9yUgzqDbfpzoCWiVeSBpRoD+pf/UiSLKHCEI617nluaoIcK8MIp0Wtn2maYjLGQ9qzVOCE6iCf3VugE6tEKJbKljBopv6eyHGi9SQJbWeCzUgvelPxP6+XmfgyyJlIM0MFmS+KM46MRNPnUcQUJYZPLMFEMXsrIiOsMDE2opoNwVt8eZn4Z82rpnd33mjdlmlU4QiO4RQ8uIAW3EAbfCDA4Rle4c15dF6cd+dj3lpxyplD+APn8we9zo//</latexit> + … + W rmo2 :,mo2<latexit sha1_base64="opdz4oAh5Obm9hxV3ZKKD/28H70=">AAACBXicbVDLSgMxFM3UV62vUZciBIvgQspMEXysCm4ENxUcW2jHIZNm2tDMZEgyQgmzcuOvuHGh4tZ/cOffmLaz0NYDF07OuZfce8KUUakc59sqLSwuLa+UVytr6xubW/b2zp3kmcDEw5xx0Q6RJIwmxFNUMdJOBUFxyEgrHF6O/dYDEZLy5FaNUuLHqJ/QiGKkjBTY+61AXxzDONA8qOcwv9ci0MUrzwO76tScCeA8cQtSBQWagf3V7XGcxSRRmCEpO66TKl8joShmJK90M0lShIeoTzqGJigm0teTM3J4aJQejLgwlSg4UX9PaBRLOYpD0xkjNZCz3lj8z+tkKjrzNU3STJEETz+KMgYVh+NMYI8KghUbGYKwoGZXiAdIIKxMchUTgjt78jzx6rXzmntzUm1cF2mUwR44AEfABaegAa5AE3gAg0fwDF7Bm/VkvVjv1se0tWQVM7vgD6zPH1HxmKU=</latexit><latexit sha1_base64="opdz4oAh5Obm9hxV3ZKKD/28H70=">AAACBXicbVDLSgMxFM3UV62vUZciBIvgQspMEXysCm4ENxUcW2jHIZNm2tDMZEgyQgmzcuOvuHGh4tZ/cOffmLaz0NYDF07OuZfce8KUUakc59sqLSwuLa+UVytr6xubW/b2zp3kmcDEw5xx0Q6RJIwmxFNUMdJOBUFxyEgrHF6O/dYDEZLy5FaNUuLHqJ/QiGKkjBTY+61AXxzDONA8qOcwv9ci0MUrzwO76tScCeA8cQtSBQWagf3V7XGcxSRRmCEpO66TKl8joShmJK90M0lShIeoTzqGJigm0teTM3J4aJQejLgwlSg4UX9PaBRLOYpD0xkjNZCz3lj8z+tkKjrzNU3STJEETz+KMgYVh+NMYI8KghUbGYKwoGZXiAdIIKxMchUTgjt78jzx6rXzmntzUm1cF2mUwR44AEfABaegAa5AE3gAg0fwDF7Bm/VkvVjv1se0tWQVM7vgD6zPH1HxmKU=</latexit><latexit sha1_base64="opdz4oAh5Obm9hxV3ZKKD/28H70=">AAACBXicbVDLSgMxFM3UV62vUZciBIvgQspMEXysCm4ENxUcW2jHIZNm2tDMZEgyQgmzcuOvuHGh4tZ/cOffmLaz0NYDF07OuZfce8KUUakc59sqLSwuLa+UVytr6xubW/b2zp3kmcDEw5xx0Q6RJIwmxFNUMdJOBUFxyEgrHF6O/dYDEZLy5FaNUuLHqJ/QiGKkjBTY+61AXxzDONA8qOcwv9ci0MUrzwO76tScCeA8cQtSBQWagf3V7XGcxSRRmCEpO66TKl8joShmJK90M0lShIeoTzqGJigm0teTM3J4aJQejLgwlSg4UX9PaBRLOYpD0xkjNZCz3lj8z+tkKjrzNU3STJEETz+KMgYVh+NMYI8KghUbGYKwoGZXiAdIIKxMchUTgjt78jzx6rXzmntzUm1cF2mUwR44AEfABaegAa5AE3gAg0fwDF7Bm/VkvVjv1se0tWQVM7vgD6zPH1HxmKU=</latexit><latexit sha1_base64="opdz4oAh5Obm9hxV3ZKKD/28H70=">AAACBXicbVDLSgMxFM3UV62vUZciBIvgQspMEXysCm4ENxUcW2jHIZNm2tDMZEgyQgmzcuOvuHGh4tZ/cOffmLaz0NYDF07OuZfce8KUUakc59sqLSwuLa+UVytr6xubW/b2zp3kmcDEw5xx0Q6RJIwmxFNUMdJOBUFxyEgrHF6O/dYDEZLy5FaNUuLHqJ/QiGKkjBTY+61AXxzDONA8qOcwv9ci0MUrzwO76tScCeA8cQtSBQWagf3V7XGcxSRRmCEpO66TKl8joShmJK90M0lShIeoTzqGJigm0teTM3J4aJQejLgwlSg4UX9PaBRLOYpD0xkjNZCz3lj8z+tkKjrzNU3STJEETz+KMgYVh+NMYI8KghUbGYKwoGZXiAdIIKxMchUTgjt78jzx6rXzmntzUm1cF2mUwR44AEfABaegAa5AE3gAg0fwDF7Bm/VkvVjv1se0tWQVM7vgD6zPH1HxmKU=</latexit> W rmoi 1 :,moi 1<latexit sha1_base64="vDWHlwlOmdQjyoSWksi1NThDM8A=">AAACDnicbVDLSgMxFM3UV62vUZdugkVxoWVGBB+rghvBTQXHFtpxyKRpG5pJhiQjlNA/cOOvuHGh4ta1O//GtB1Qq+dy4XDOvST3xCmjSnvep1OYmZ2bXygulpaWV1bX3PWNGyUyiUmABROyESNFGOUk0FQz0kglQUnMSD3un4/8+h2Rigp+rQcpCRPU5bRDMdJWitzdemTO9mESGREZeuAPoa1bIyPzLVlEbtmreGPAv8TPSRnkqEXuR6stcJYQrjFDSjV9L9WhQVJTzMiw1MoUSRHuoy5pWspRQlRoxvcM4Y5V2rAjpG2u4Vj9uWFQotQgie1kgnRPTXsj8T+vmenOSWgoTzNNOJ481MkY1AKOwoFtKgnWbGAJwpLav0LcQxJhbSMs2RD86ZP/kuCwclrxr47K1cs8jSLYAttgD/jgGFTBBaiBAGBwDx7BM3hxHpwn59V5m4wWnHxnE/yC8/4F64qcOQ==</latexit><latexit sha1_base64="vDWHlwlOmdQjyoSWksi1NThDM8A=">AAACDnicbVDLSgMxFM3UV62vUZdugkVxoWVGBB+rghvBTQXHFtpxyKRpG5pJhiQjlNA/cOOvuHGh4ta1O//GtB1Qq+dy4XDOvST3xCmjSnvep1OYmZ2bXygulpaWV1bX3PWNGyUyiUmABROyESNFGOUk0FQz0kglQUnMSD3un4/8+h2Rigp+rQcpCRPU5bRDMdJWitzdemTO9mESGREZeuAPoa1bIyPzLVlEbtmreGPAv8TPSRnkqEXuR6stcJYQrjFDSjV9L9WhQVJTzMiw1MoUSRHuoy5pWspRQlRoxvcM4Y5V2rAjpG2u4Vj9uWFQotQgie1kgnRPTXsj8T+vmenOSWgoTzNNOJ481MkY1AKOwoFtKgnWbGAJwpLav0LcQxJhbSMs2RD86ZP/kuCwclrxr47K1cs8jSLYAttgD/jgGFTBBaiBAGBwDx7BM3hxHpwn59V5m4wWnHxnE/yC8/4F64qcOQ==</latexit><latexit sha1_base64="vDWHlwlOmdQjyoSWksi1NThDM8A=">AAACDnicbVDLSgMxFM3UV62vUZdugkVxoWVGBB+rghvBTQXHFtpxyKRpG5pJhiQjlNA/cOOvuHGh4ta1O//GtB1Qq+dy4XDOvST3xCmjSnvep1OYmZ2bXygulpaWV1bX3PWNGyUyiUmABROyESNFGOUk0FQz0kglQUnMSD3un4/8+h2Rigp+rQcpCRPU5bRDMdJWitzdemTO9mESGREZeuAPoa1bIyPzLVlEbtmreGPAv8TPSRnkqEXuR6stcJYQrjFDSjV9L9WhQVJTzMiw1MoUSRHuoy5pWspRQlRoxvcM4Y5V2rAjpG2u4Vj9uWFQotQgie1kgnRPTXsj8T+vmenOSWgoTzNNOJ481MkY1AKOwoFtKgnWbGAJwpLav0LcQxJhbSMs2RD86ZP/kuCwclrxr47K1cs8jSLYAttgD/jgGFTBBaiBAGBwDx7BM3hxHpwn59V5m4wWnHxnE/yC8/4F64qcOQ==</latexit><latexit sha1_base64="vDWHlwlOmdQjyoSWksi1NThDM8A=">AAACDnicbVDLSgMxFM3UV62vUZdugkVxoWVGBB+rghvBTQXHFtpxyKRpG5pJhiQjlNA/cOOvuHGh4ta1O//GtB1Qq+dy4XDOvST3xCmjSnvep1OYmZ2bXygulpaWV1bX3PWNGyUyiUmABROyESNFGOUk0FQz0kglQUnMSD3un4/8+h2Rigp+rQcpCRPU5bRDMdJWitzdemTO9mESGREZeuAPoa1bIyPzLVlEbtmreGPAv8TPSRnkqEXuR6stcJYQrjFDSjV9L9WhQVJTzMiw1MoUSRHuoy5pWspRQlRoxvcM4Y5V2rAjpG2u4Vj9uWFQotQgie1kgnRPTXsj8T+vmenOSWgoTzNNOJ481MkY1AKOwoFtKgnWbGAJwpLav0LcQxJhbSMs2RD86ZP/kuCwclrxr47K1cs8jSLYAttgD/jgGFTBBaiBAGBwDx7BM3hxHpwn59V5m4wWnHxnE/yC8/4F64qcOQ==</latexit> tanh
  15. 15. 15 FIGURES OF THE MODEL V1 b1 b1 moi<latexit sha1_base64="khFMxWwj1mfcXUdBB7rNQr8Otg8=">AAAB83icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbQyb7bZduh9xd1MoIb/DiwcVr/4Zb/4bt20O2vpg4PHeDDPz4oRRbTzv2ymtrK6tb5Q3K1vbO7t71f2DBy1ThUmAJZOqHSNNGBUkMNQw0k4UQTxmpBWPrqd+a0yUplLcm0lCQo4GgvYpRsZKYfzoRxmPMhnRPI+qNa/uzeAuE78gNSjQjKpf3Z7EKSfCYIa07vheYsIMKUMxI3mlm2qSIDxCA9KxVCBOdJjNjs7dE6v03L5UtoRxZ+rviQxxrSc8tp0cmaFe9Kbif14nNf3LMKMiSQ0ReL6onzLXSHeagNujimDDJpYgrKi91cVDpBA2NqeKDcFffHmZBGf1q7p/d15r3BZplOEIjuEUfLiABtxAEwLA8ATP8Apvzth5cd6dj3lrySlmDuEPnM8fb2OSHA==</latexit><latexit sha1_base64="khFMxWwj1mfcXUdBB7rNQr8Otg8=">AAAB83icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbQyb7bZduh9xd1MoIb/DiwcVr/4Zb/4bt20O2vpg4PHeDDPz4oRRbTzv2ymtrK6tb5Q3K1vbO7t71f2DBy1ThUmAJZOqHSNNGBUkMNQw0k4UQTxmpBWPrqd+a0yUplLcm0lCQo4GgvYpRsZKYfzoRxmPMhnRPI+qNa/uzeAuE78gNSjQjKpf3Z7EKSfCYIa07vheYsIMKUMxI3mlm2qSIDxCA9KxVCBOdJjNjs7dE6v03L5UtoRxZ+rviQxxrSc8tp0cmaFe9Kbif14nNf3LMKMiSQ0ReL6onzLXSHeagNujimDDJpYgrKi91cVDpBA2NqeKDcFffHmZBGf1q7p/d15r3BZplOEIjuEUfLiABtxAEwLA8ATP8Apvzth5cd6dj3lrySlmDuEPnM8fb2OSHA==</latexit><latexit sha1_base64="khFMxWwj1mfcXUdBB7rNQr8Otg8=">AAAB83icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbQyb7bZduh9xd1MoIb/DiwcVr/4Zb/4bt20O2vpg4PHeDDPz4oRRbTzv2ymtrK6tb5Q3K1vbO7t71f2DBy1ThUmAJZOqHSNNGBUkMNQw0k4UQTxmpBWPrqd+a0yUplLcm0lCQo4GgvYpRsZKYfzoRxmPMhnRPI+qNa/uzeAuE78gNSjQjKpf3Z7EKSfCYIa07vheYsIMKUMxI3mlm2qSIDxCA9KxVCBOdJjNjs7dE6v03L5UtoRxZ+rviQxxrSc8tp0cmaFe9Kbif14nNf3LMKMiSQ0ReL6onzLXSHeagNujimDDJpYgrKi91cVDpBA2NqeKDcFffHmZBGf1q7p/d15r3BZplOEIjuEUfLiABtxAEwLA8ATP8Apvzth5cd6dj3lrySlmDuEPnM8fb2OSHA==</latexit><latexit sha1_base64="khFMxWwj1mfcXUdBB7rNQr8Otg8=">AAAB83icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbQyb7bZduh9xd1MoIb/DiwcVr/4Zb/4bt20O2vpg4PHeDDPz4oRRbTzv2ymtrK6tb5Q3K1vbO7t71f2DBy1ThUmAJZOqHSNNGBUkMNQw0k4UQTxmpBWPrqd+a0yUplLcm0lCQo4GgvYpRsZKYfzoRxmPMhnRPI+qNa/uzeAuE78gNSjQjKpf3Z7EKSfCYIa07vheYsIMKUMxI3mlm2qSIDxCA9KxVCBOdJjNjs7dE6v03L5UtoRxZ+rviQxxrSc8tp0cmaFe9Kbif14nNf3LMKMiSQ0ReL6onzLXSHeagNujimDDJpYgrKi91cVDpBA2NqeKDcFffHmZBGf1q7p/d15r3BZplOEIjuEUfLiABtxAEwLA8ATP8Apvzth5cd6dj3lrySlmDuEPnM8fb2OSHA==</latexit> V1 moi ,:<latexit sha1_base64="6/wzl1Pz+AiS40dOdQg4tQAE2hI=">AAAB/3icbVDLSsNAFJ34rPUVdeHCzWARXEhJRPCxKrgR3FQwbaGNYTKZtENnMmFmIpSQjb/ixoWKW3/DnX/jtM1CWw9cOJxzL/feE6aMKu0439bC4tLyymplrbq+sbm1be/stpTIJCYeFkzITogUYTQhnqaakU4qCeIhI+1weD32249EKiqSez1Kic9RP6ExxUgbKbD3e6FgUd4qHtwg50EuAlqcwKsisGtO3ZkAzhO3JDVQohnYX71I4IyTRGOGlOq6Tqr9HElNMSNFtZcpkiI8RH3SNTRBnCg/nzxQwCOjRDAW0lSi4UT9PZEjrtSIh6aTIz1Qs95Y/M/rZjq+8HOapJkmCZ4uijMGtYDjNGBEJcGajQxBWFJzK8QDJBHWJrOqCcGdfXmeeKf1y7p7d1Zr3JZpVMABOATHwAXnoAFuQBN4AIMCPINX8GY9WS/Wu/UxbV2wypk98AfW5w/RaZYg</latexit><latexit sha1_base64="6/wzl1Pz+AiS40dOdQg4tQAE2hI=">AAAB/3icbVDLSsNAFJ34rPUVdeHCzWARXEhJRPCxKrgR3FQwbaGNYTKZtENnMmFmIpSQjb/ixoWKW3/DnX/jtM1CWw9cOJxzL/feE6aMKu0439bC4tLyymplrbq+sbm1be/stpTIJCYeFkzITogUYTQhnqaakU4qCeIhI+1weD32249EKiqSez1Kic9RP6ExxUgbKbD3e6FgUd4qHtwg50EuAlqcwKsisGtO3ZkAzhO3JDVQohnYX71I4IyTRGOGlOq6Tqr9HElNMSNFtZcpkiI8RH3SNTRBnCg/nzxQwCOjRDAW0lSi4UT9PZEjrtSIh6aTIz1Qs95Y/M/rZjq+8HOapJkmCZ4uijMGtYDjNGBEJcGajQxBWFJzK8QDJBHWJrOqCcGdfXmeeKf1y7p7d1Zr3JZpVMABOATHwAXnoAFuQBN4AIMCPINX8GY9WS/Wu/UxbV2wypk98AfW5w/RaZYg</latexit><latexit sha1_base64="6/wzl1Pz+AiS40dOdQg4tQAE2hI=">AAAB/3icbVDLSsNAFJ34rPUVdeHCzWARXEhJRPCxKrgR3FQwbaGNYTKZtENnMmFmIpSQjb/ixoWKW3/DnX/jtM1CWw9cOJxzL/feE6aMKu0439bC4tLyymplrbq+sbm1be/stpTIJCYeFkzITogUYTQhnqaakU4qCeIhI+1weD32249EKiqSez1Kic9RP6ExxUgbKbD3e6FgUd4qHtwg50EuAlqcwKsisGtO3ZkAzhO3JDVQohnYX71I4IyTRGOGlOq6Tqr9HElNMSNFtZcpkiI8RH3SNTRBnCg/nzxQwCOjRDAW0lSi4UT9PZEjrtSIh6aTIz1Qs95Y/M/rZjq+8HOapJkmCZ4uijMGtYDjNGBEJcGajQxBWFJzK8QDJBHWJrOqCcGdfXmeeKf1y7p7d1Zr3JZpVMABOATHwAXnoAFuQBN4AIMCPINX8GY9WS/Wu/UxbV2wypk98AfW5w/RaZYg</latexit><latexit sha1_base64="6/wzl1Pz+AiS40dOdQg4tQAE2hI=">AAAB/3icbVDLSsNAFJ34rPUVdeHCzWARXEhJRPCxKrgR3FQwbaGNYTKZtENnMmFmIpSQjb/ixoWKW3/DnX/jtM1CWw9cOJxzL/feE6aMKu0439bC4tLyymplrbq+sbm1be/stpTIJCYeFkzITogUYTQhnqaakU4qCeIhI+1weD32249EKiqSez1Kic9RP6ExxUgbKbD3e6FgUd4qHtwg50EuAlqcwKsisGtO3ZkAzhO3JDVQohnYX71I4IyTRGOGlOq6Tqr9HElNMSNFtZcpkiI8RH3SNTRBnCg/nzxQwCOjRDAW0lSi4UT9PZEjrtSIh6aTIz1Qs95Y/M/rZjq+8HOapJkmCZ4uijMGtYDjNGBEJcGajQxBWFJzK8QDJBHWJrOqCcGdfXmeeKf1y7p7d1Zr3JZpVMABOATHwAXnoAFuQBN4AIMCPINX8GY9WS/Wu/UxbV2wypk98AfW5w/RaZYg</latexit> b1 moi + V1 moi ,:h ⇣ rmo<i ⌘ <latexit sha1_base64="LHhK5VPbhS1xg2EMwUUBfHHGwMc=">AAACP3icbVBLSwMxGMzWV62vVY9egkVQlLIrgg88FLwIXirYB7R1yabZNjS7WZJvhbLsT/PiT/Dm2YsHFa/eTNs9+BoITGa+4UvGjwXX4DhPVmFmdm5+obhYWlpeWV2z1zcaWiaKsjqVQqqWTzQTPGJ14CBYK1aMhL5gTX94Mfabd0xpLqMbGMWsG5J+xANOCRjJs5v+reuloZdKj2cZ3scdX4pe2si+yQf4LMv1gSGCBbCb31WWT6Xn43iGcUfx/gD2PLvsVJwJ8F/i5qSMctQ8+7HTkzQJWQRUEK3brhNDNyUKOBUsK3USzWJCh6TP2oZGJGS6m04KyPCOUXo4kMqcCPBE/Z5ISaj1KPTNZEhgoH97Y/E/r51AcNJNeRQnwCI6XRQkAoPE4zZxjytGQYwMIVRx81ZMB0QRCqbzkinB/f3lv6R+WDmtuNdH5epV3kYRbaFttItcdIyq6BLVUB1RdI+e0St6sx6sF+vd+piOFqw8s4l+wPr8AsyNsCo=</latexit><latexit sha1_base64="LHhK5VPbhS1xg2EMwUUBfHHGwMc=">AAACP3icbVBLSwMxGMzWV62vVY9egkVQlLIrgg88FLwIXirYB7R1yabZNjS7WZJvhbLsT/PiT/Dm2YsHFa/eTNs9+BoITGa+4UvGjwXX4DhPVmFmdm5+obhYWlpeWV2z1zcaWiaKsjqVQqqWTzQTPGJ14CBYK1aMhL5gTX94Mfabd0xpLqMbGMWsG5J+xANOCRjJs5v+reuloZdKj2cZ3scdX4pe2si+yQf4LMv1gSGCBbCb31WWT6Xn43iGcUfx/gD2PLvsVJwJ8F/i5qSMctQ8+7HTkzQJWQRUEK3brhNDNyUKOBUsK3USzWJCh6TP2oZGJGS6m04KyPCOUXo4kMqcCPBE/Z5ISaj1KPTNZEhgoH97Y/E/r51AcNJNeRQnwCI6XRQkAoPE4zZxjytGQYwMIVRx81ZMB0QRCqbzkinB/f3lv6R+WDmtuNdH5epV3kYRbaFttItcdIyq6BLVUB1RdI+e0St6sx6sF+vd+piOFqw8s4l+wPr8AsyNsCo=</latexit><latexit sha1_base64="LHhK5VPbhS1xg2EMwUUBfHHGwMc=">AAACP3icbVBLSwMxGMzWV62vVY9egkVQlLIrgg88FLwIXirYB7R1yabZNjS7WZJvhbLsT/PiT/Dm2YsHFa/eTNs9+BoITGa+4UvGjwXX4DhPVmFmdm5+obhYWlpeWV2z1zcaWiaKsjqVQqqWTzQTPGJ14CBYK1aMhL5gTX94Mfabd0xpLqMbGMWsG5J+xANOCRjJs5v+reuloZdKj2cZ3scdX4pe2si+yQf4LMv1gSGCBbCb31WWT6Xn43iGcUfx/gD2PLvsVJwJ8F/i5qSMctQ8+7HTkzQJWQRUEK3brhNDNyUKOBUsK3USzWJCh6TP2oZGJGS6m04KyPCOUXo4kMqcCPBE/Z5ISaj1KPTNZEhgoH97Y/E/r51AcNJNeRQnwCI6XRQkAoPE4zZxjytGQYwMIVRx81ZMB0QRCqbzkinB/f3lv6R+WDmtuNdH5epV3kYRbaFttItcdIyq6BLVUB1RdI+e0St6sx6sF+vd+piOFqw8s4l+wPr8AsyNsCo=</latexit><latexit sha1_base64="LHhK5VPbhS1xg2EMwUUBfHHGwMc=">AAACP3icbVBLSwMxGMzWV62vVY9egkVQlLIrgg88FLwIXirYB7R1yabZNjS7WZJvhbLsT/PiT/Dm2YsHFa/eTNs9+BoITGa+4UvGjwXX4DhPVmFmdm5+obhYWlpeWV2z1zcaWiaKsjqVQqqWTzQTPGJ14CBYK1aMhL5gTX94Mfabd0xpLqMbGMWsG5J+xANOCRjJs5v+reuloZdKj2cZ3scdX4pe2si+yQf4LMv1gSGCBbCb31WWT6Xn43iGcUfx/gD2PLvsVJwJ8F/i5qSMctQ8+7HTkzQJWQRUEK3brhNDNyUKOBUsK3USzWJCh6TP2oZGJGS6m04KyPCOUXo4kMqcCPBE/Z5ISaj1KPTNZEhgoH97Y/E/r51AcNJNeRQnwCI6XRQkAoPE4zZxjytGQYwMIVRx81ZMB0QRCqbzkinB/f3lv6R+WDmtuNdH5epV3kYRbaFttItcdIyq6BLVUB1RdI+e0St6sx6sF+vd+piOFqw8s4l+wPr8AsyNsCo=</latexit> … … s1 moi ⇣ rmo<i ⌘ <latexit sha1_base64="dEuQ73sOOGlAJordFhBO/PRTXnQ=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu63vHS0MvlR7PMuwKFkAVu74U/VRluZNeTLwMY1fxwRCOvXLFrtlT4EXi5KSCcjS98qfblzQJWQRUEK27jh1DLyUKOBUsK7mJZjGhIzJgXUMjEjLdS6ffZfjIKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPVSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaMiE48y8vktZJ7bzm3NQrjes8jSI6QIeoihx0ihroCjVRC1H0iJ7RK3qznqwX6936mLUWrHxmH/2B9fUDSeiiQw==</latexit><latexit sha1_base64="dEuQ73sOOGlAJordFhBO/PRTXnQ=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu63vHS0MvlR7PMuwKFkAVu74U/VRluZNeTLwMY1fxwRCOvXLFrtlT4EXi5KSCcjS98qfblzQJWQRUEK27jh1DLyUKOBUsK7mJZjGhIzJgXUMjEjLdS6ffZfjIKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPVSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaMiE48y8vktZJ7bzm3NQrjes8jSI6QIeoihx0ihroCjVRC1H0iJ7RK3qznqwX6936mLUWrHxmH/2B9fUDSeiiQw==</latexit><latexit sha1_base64="dEuQ73sOOGlAJordFhBO/PRTXnQ=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu63vHS0MvlR7PMuwKFkAVu74U/VRluZNeTLwMY1fxwRCOvXLFrtlT4EXi5KSCcjS98qfblzQJWQRUEK27jh1DLyUKOBUsK7mJZjGhIzJgXUMjEjLdS6ffZfjIKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPVSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaMiE48y8vktZJ7bzm3NQrjes8jSI6QIeoihx0ihroCjVRC1H0iJ7RK3qznqwX6936mLUWrHxmH/2B9fUDSeiiQw==</latexit><latexit sha1_base64="dEuQ73sOOGlAJordFhBO/PRTXnQ=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu63vHS0MvlR7PMuwKFkAVu74U/VRluZNeTLwMY1fxwRCOvXLFrtlT4EXi5KSCcjS98qfblzQJWQRUEK27jh1DLyUKOBUsK7mJZjGhIzJgXUMjEjLdS6ffZfjIKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPVSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaMiE48y8vktZJ7bzm3NQrjes8jSI6QIeoihx0ihroCjVRC1H0iJ7RK3qznqwX6936mLUWrHxmH/2B9fUDSeiiQw==</latexit> sk moi ⇣ rmo<i ⌘ <latexit sha1_base64="8yFvGakMoAgUOhWppwqPpdRpbLM=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu6/uRl4ZeKj2eZdgVLIAqdn0p+qnKcie9mHgZxq7igyEce+WKXbOnwIvEyUkF5Wh65U+3L2kSsgioIFp3HTuGXkoUcCpYVnITzWJCR2TAuoZGJGS6l06/y/CRUfo4kMpUBHiq/p5ISaj1OPRNZ0hgqOe9ifif100gOOulPIoTYBGdLQoSgUHiSVS4zxWjIMaGEKq4uRXTIVGEggm0ZEJw5l9eJK2T2nnNualXGtd5GkV0gA5RFTnoFDXQFWqiFqLoET2jV/RmPVkv1rv1MWstWPnMPvoD6+sHrLCifQ==</latexit><latexit sha1_base64="8yFvGakMoAgUOhWppwqPpdRpbLM=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu6/uRl4ZeKj2eZdgVLIAqdn0p+qnKcie9mHgZxq7igyEce+WKXbOnwIvEyUkF5Wh65U+3L2kSsgioIFp3HTuGXkoUcCpYVnITzWJCR2TAuoZGJGS6l06/y/CRUfo4kMpUBHiq/p5ISaj1OPRNZ0hgqOe9ifif100gOOulPIoTYBGdLQoSgUHiSVS4zxWjIMaGEKq4uRXTIVGEggm0ZEJw5l9eJK2T2nnNualXGtd5GkV0gA5RFTnoFDXQFWqiFqLoET2jV/RmPVkv1rv1MWstWPnMPvoD6+sHrLCifQ==</latexit><latexit sha1_base64="8yFvGakMoAgUOhWppwqPpdRpbLM=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu6/uRl4ZeKj2eZdgVLIAqdn0p+qnKcie9mHgZxq7igyEce+WKXbOnwIvEyUkF5Wh65U+3L2kSsgioIFp3HTuGXkoUcCpYVnITzWJCR2TAuoZGJGS6l06/y/CRUfo4kMpUBHiq/p5ISaj1OPRNZ0hgqOe9ifif100gOOulPIoTYBGdLQoSgUHiSVS4zxWjIMaGEKq4uRXTIVGEggm0ZEJw5l9eJK2T2nnNualXGtd5GkV0gA5RFTnoFDXQFWqiFqLoET2jV/RmPVkv1rv1MWstWPnMPvoD6+sHrLCifQ==</latexit><latexit sha1_base64="8yFvGakMoAgUOhWppwqPpdRpbLM=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu6/uRl4ZeKj2eZdgVLIAqdn0p+qnKcie9mHgZxq7igyEce+WKXbOnwIvEyUkF5Wh65U+3L2kSsgioIFp3HTuGXkoUcCpYVnITzWJCR2TAuoZGJGS6l06/y/CRUfo4kMpUBHiq/p5ISaj1OPRNZ0hgqOe9ifif100gOOulPIoTYBGdLQoSgUHiSVS4zxWjIMaGEKq4uRXTIVGEggm0ZEJw5l9eJK2T2nnNualXGtd5GkV0gA5RFTnoFDXQFWqiFqLoET2jV/RmPVkv1rv1MWstWPnMPvoD6+sHrLCifQ==</latexit> sK moi ⇣ rmo<i ⌘ <latexit sha1_base64="TEHfIFWYNFymBdy4hOMMgNi9FCk=">AAACHHicbVBNS8NAEN34WetX1aOXxSLUS0mkoIKHghehlwrWFpoaNttNu3STDbsToYT8ES/+FS8eVLx4EPw3btsctPXBwOO9GWbm+bHgGmz721paXlldWy9sFDe3tnd2S3v7d1omirIWlUKqjk80EzxiLeAgWCdWjIS+YG1/dDXx2w9MaS6jWxjHrBeSQcQDTgkYySvV9H3DS0MvlR7PMuwKFkAFu74U/VRluZNeTrwMY1fxwRBOvFLZrtpT4EXi5KSMcjS90qfblzQJWQRUEK27jh1DLyUKOBUsK7qJZjGhIzJgXUMjEjLdS6ffZfjYKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPdSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaNCE48y8vktZp9aLq3NTK9UaeRgEdoiNUQQ46Q3V0jZqohSh6RM/oFb1ZT9aL9W59zFqXrHzmAP2B9fUDdjCiXQ==</latexit><latexit sha1_base64="TEHfIFWYNFymBdy4hOMMgNi9FCk=">AAACHHicbVBNS8NAEN34WetX1aOXxSLUS0mkoIKHghehlwrWFpoaNttNu3STDbsToYT8ES/+FS8eVLx4EPw3btsctPXBwOO9GWbm+bHgGmz721paXlldWy9sFDe3tnd2S3v7d1omirIWlUKqjk80EzxiLeAgWCdWjIS+YG1/dDXx2w9MaS6jWxjHrBeSQcQDTgkYySvV9H3DS0MvlR7PMuwKFkAFu74U/VRluZNeTrwMY1fxwRBOvFLZrtpT4EXi5KSMcjS90qfblzQJWQRUEK27jh1DLyUKOBUsK7qJZjGhIzJgXUMjEjLdS6ffZfjYKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPdSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaNCE48y8vktZp9aLq3NTK9UaeRgEdoiNUQQ46Q3V0jZqohSh6RM/oFb1ZT9aL9W59zFqXrHzmAP2B9fUDdjCiXQ==</latexit><latexit sha1_base64="TEHfIFWYNFymBdy4hOMMgNi9FCk=">AAACHHicbVBNS8NAEN34WetX1aOXxSLUS0mkoIKHghehlwrWFpoaNttNu3STDbsToYT8ES/+FS8eVLx4EPw3btsctPXBwOO9GWbm+bHgGmz721paXlldWy9sFDe3tnd2S3v7d1omirIWlUKqjk80EzxiLeAgWCdWjIS+YG1/dDXx2w9MaS6jWxjHrBeSQcQDTgkYySvV9H3DS0MvlR7PMuwKFkAFu74U/VRluZNeTrwMY1fxwRBOvFLZrtpT4EXi5KSMcjS90qfblzQJWQRUEK27jh1DLyUKOBUsK7qJZjGhIzJgXUMjEjLdS6ffZfjYKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPdSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaNCE48y8vktZp9aLq3NTK9UaeRgEdoiNUQQ46Q3V0jZqohSh6RM/oFb1ZT9aL9W59zFqXrHzmAP2B9fUDdjCiXQ==</latexit><latexit sha1_base64="TEHfIFWYNFymBdy4hOMMgNi9FCk=">AAACHHicbVBNS8NAEN34WetX1aOXxSLUS0mkoIKHghehlwrWFpoaNttNu3STDbsToYT8ES/+FS8eVLx4EPw3btsctPXBwOO9GWbm+bHgGmz721paXlldWy9sFDe3tnd2S3v7d1omirIWlUKqjk80EzxiLeAgWCdWjIS+YG1/dDXx2w9MaS6jWxjHrBeSQcQDTgkYySvV9H3DS0MvlR7PMuwKFkAFu74U/VRluZNeTrwMY1fxwRBOvFLZrtpT4EXi5KSMcjS90qfblzQJWQRUEK27jh1DLyUKOBUsK7qJZjGhIzJgXUMjEjLdS6ffZfjYKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPdSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaNCE48y8vktZp9aLq3NTK9UaeRgEdoiNUQQ46Q3V0jZqohSh6RM/oFb1ZT9aL9W59zFqXrHzmAP2B9fUDdjCiXQ==</latexit>
  16. 16. 16 FIGURES OF THE MODEL V1 b1 b1 moi<latexit sha1_base64="khFMxWwj1mfcXUdBB7rNQr8Otg8=">AAAB83icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbQyb7bZduh9xd1MoIb/DiwcVr/4Zb/4bt20O2vpg4PHeDDPz4oRRbTzv2ymtrK6tb5Q3K1vbO7t71f2DBy1ThUmAJZOqHSNNGBUkMNQw0k4UQTxmpBWPrqd+a0yUplLcm0lCQo4GgvYpRsZKYfzoRxmPMhnRPI+qNa/uzeAuE78gNSjQjKpf3Z7EKSfCYIa07vheYsIMKUMxI3mlm2qSIDxCA9KxVCBOdJjNjs7dE6v03L5UtoRxZ+rviQxxrSc8tp0cmaFe9Kbif14nNf3LMKMiSQ0ReL6onzLXSHeagNujimDDJpYgrKi91cVDpBA2NqeKDcFffHmZBGf1q7p/d15r3BZplOEIjuEUfLiABtxAEwLA8ATP8Apvzth5cd6dj3lrySlmDuEPnM8fb2OSHA==</latexit><latexit sha1_base64="khFMxWwj1mfcXUdBB7rNQr8Otg8=">AAAB83icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbQyb7bZduh9xd1MoIb/DiwcVr/4Zb/4bt20O2vpg4PHeDDPz4oRRbTzv2ymtrK6tb5Q3K1vbO7t71f2DBy1ThUmAJZOqHSNNGBUkMNQw0k4UQTxmpBWPrqd+a0yUplLcm0lCQo4GgvYpRsZKYfzoRxmPMhnRPI+qNa/uzeAuE78gNSjQjKpf3Z7EKSfCYIa07vheYsIMKUMxI3mlm2qSIDxCA9KxVCBOdJjNjs7dE6v03L5UtoRxZ+rviQxxrSc8tp0cmaFe9Kbif14nNf3LMKMiSQ0ReL6onzLXSHeagNujimDDJpYgrKi91cVDpBA2NqeKDcFffHmZBGf1q7p/d15r3BZplOEIjuEUfLiABtxAEwLA8ATP8Apvzth5cd6dj3lrySlmDuEPnM8fb2OSHA==</latexit><latexit sha1_base64="khFMxWwj1mfcXUdBB7rNQr8Otg8=">AAAB83icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbQyb7bZduh9xd1MoIb/DiwcVr/4Zb/4bt20O2vpg4PHeDDPz4oRRbTzv2ymtrK6tb5Q3K1vbO7t71f2DBy1ThUmAJZOqHSNNGBUkMNQw0k4UQTxmpBWPrqd+a0yUplLcm0lCQo4GgvYpRsZKYfzoRxmPMhnRPI+qNa/uzeAuE78gNSjQjKpf3Z7EKSfCYIa07vheYsIMKUMxI3mlm2qSIDxCA9KxVCBOdJjNjs7dE6v03L5UtoRxZ+rviQxxrSc8tp0cmaFe9Kbif14nNf3LMKMiSQ0ReL6onzLXSHeagNujimDDJpYgrKi91cVDpBA2NqeKDcFffHmZBGf1q7p/d15r3BZplOEIjuEUfLiABtxAEwLA8ATP8Apvzth5cd6dj3lrySlmDuEPnM8fb2OSHA==</latexit><latexit sha1_base64="khFMxWwj1mfcXUdBB7rNQr8Otg8=">AAAB83icbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbQyb7bZduh9xd1MoIb/DiwcVr/4Zb/4bt20O2vpg4PHeDDPz4oRRbTzv2ymtrK6tb5Q3K1vbO7t71f2DBy1ThUmAJZOqHSNNGBUkMNQw0k4UQTxmpBWPrqd+a0yUplLcm0lCQo4GgvYpRsZKYfzoRxmPMhnRPI+qNa/uzeAuE78gNSjQjKpf3Z7EKSfCYIa07vheYsIMKUMxI3mlm2qSIDxCA9KxVCBOdJjNjs7dE6v03L5UtoRxZ+rviQxxrSc8tp0cmaFe9Kbif14nNf3LMKMiSQ0ReL6onzLXSHeagNujimDDJpYgrKi91cVDpBA2NqeKDcFffHmZBGf1q7p/d15r3BZplOEIjuEUfLiABtxAEwLA8ATP8Apvzth5cd6dj3lrySlmDuEPnM8fb2OSHA==</latexit> V1 moi ,:<latexit sha1_base64="6/wzl1Pz+AiS40dOdQg4tQAE2hI=">AAAB/3icbVDLSsNAFJ34rPUVdeHCzWARXEhJRPCxKrgR3FQwbaGNYTKZtENnMmFmIpSQjb/ixoWKW3/DnX/jtM1CWw9cOJxzL/feE6aMKu0439bC4tLyymplrbq+sbm1be/stpTIJCYeFkzITogUYTQhnqaakU4qCeIhI+1weD32249EKiqSez1Kic9RP6ExxUgbKbD3e6FgUd4qHtwg50EuAlqcwKsisGtO3ZkAzhO3JDVQohnYX71I4IyTRGOGlOq6Tqr9HElNMSNFtZcpkiI8RH3SNTRBnCg/nzxQwCOjRDAW0lSi4UT9PZEjrtSIh6aTIz1Qs95Y/M/rZjq+8HOapJkmCZ4uijMGtYDjNGBEJcGajQxBWFJzK8QDJBHWJrOqCcGdfXmeeKf1y7p7d1Zr3JZpVMABOATHwAXnoAFuQBN4AIMCPINX8GY9WS/Wu/UxbV2wypk98AfW5w/RaZYg</latexit><latexit sha1_base64="6/wzl1Pz+AiS40dOdQg4tQAE2hI=">AAAB/3icbVDLSsNAFJ34rPUVdeHCzWARXEhJRPCxKrgR3FQwbaGNYTKZtENnMmFmIpSQjb/ixoWKW3/DnX/jtM1CWw9cOJxzL/feE6aMKu0439bC4tLyymplrbq+sbm1be/stpTIJCYeFkzITogUYTQhnqaakU4qCeIhI+1weD32249EKiqSez1Kic9RP6ExxUgbKbD3e6FgUd4qHtwg50EuAlqcwKsisGtO3ZkAzhO3JDVQohnYX71I4IyTRGOGlOq6Tqr9HElNMSNFtZcpkiI8RH3SNTRBnCg/nzxQwCOjRDAW0lSi4UT9PZEjrtSIh6aTIz1Qs95Y/M/rZjq+8HOapJkmCZ4uijMGtYDjNGBEJcGajQxBWFJzK8QDJBHWJrOqCcGdfXmeeKf1y7p7d1Zr3JZpVMABOATHwAXnoAFuQBN4AIMCPINX8GY9WS/Wu/UxbV2wypk98AfW5w/RaZYg</latexit><latexit sha1_base64="6/wzl1Pz+AiS40dOdQg4tQAE2hI=">AAAB/3icbVDLSsNAFJ34rPUVdeHCzWARXEhJRPCxKrgR3FQwbaGNYTKZtENnMmFmIpSQjb/ixoWKW3/DnX/jtM1CWw9cOJxzL/feE6aMKu0439bC4tLyymplrbq+sbm1be/stpTIJCYeFkzITogUYTQhnqaakU4qCeIhI+1weD32249EKiqSez1Kic9RP6ExxUgbKbD3e6FgUd4qHtwg50EuAlqcwKsisGtO3ZkAzhO3JDVQohnYX71I4IyTRGOGlOq6Tqr9HElNMSNFtZcpkiI8RH3SNTRBnCg/nzxQwCOjRDAW0lSi4UT9PZEjrtSIh6aTIz1Qs95Y/M/rZjq+8HOapJkmCZ4uijMGtYDjNGBEJcGajQxBWFJzK8QDJBHWJrOqCcGdfXmeeKf1y7p7d1Zr3JZpVMABOATHwAXnoAFuQBN4AIMCPINX8GY9WS/Wu/UxbV2wypk98AfW5w/RaZYg</latexit><latexit sha1_base64="6/wzl1Pz+AiS40dOdQg4tQAE2hI=">AAAB/3icbVDLSsNAFJ34rPUVdeHCzWARXEhJRPCxKrgR3FQwbaGNYTKZtENnMmFmIpSQjb/ixoWKW3/DnX/jtM1CWw9cOJxzL/feE6aMKu0439bC4tLyymplrbq+sbm1be/stpTIJCYeFkzITogUYTQhnqaakU4qCeIhI+1weD32249EKiqSez1Kic9RP6ExxUgbKbD3e6FgUd4qHtwg50EuAlqcwKsisGtO3ZkAzhO3JDVQohnYX71I4IyTRGOGlOq6Tqr9HElNMSNFtZcpkiI8RH3SNTRBnCg/nzxQwCOjRDAW0lSi4UT9PZEjrtSIh6aTIz1Qs95Y/M/rZjq+8HOapJkmCZ4uijMGtYDjNGBEJcGajQxBWFJzK8QDJBHWJrOqCcGdfXmeeKf1y7p7d1Zr3JZpVMABOATHwAXnoAFuQBN4AIMCPINX8GY9WS/Wu/UxbV2wypk98AfW5w/RaZYg</latexit> b1 moi + V1 moi ,:h ⇣ rmo<i ⌘ <latexit sha1_base64="LHhK5VPbhS1xg2EMwUUBfHHGwMc=">AAACP3icbVBLSwMxGMzWV62vVY9egkVQlLIrgg88FLwIXirYB7R1yabZNjS7WZJvhbLsT/PiT/Dm2YsHFa/eTNs9+BoITGa+4UvGjwXX4DhPVmFmdm5+obhYWlpeWV2z1zcaWiaKsjqVQqqWTzQTPGJ14CBYK1aMhL5gTX94Mfabd0xpLqMbGMWsG5J+xANOCRjJs5v+reuloZdKj2cZ3scdX4pe2si+yQf4LMv1gSGCBbCb31WWT6Xn43iGcUfx/gD2PLvsVJwJ8F/i5qSMctQ8+7HTkzQJWQRUEK3brhNDNyUKOBUsK3USzWJCh6TP2oZGJGS6m04KyPCOUXo4kMqcCPBE/Z5ISaj1KPTNZEhgoH97Y/E/r51AcNJNeRQnwCI6XRQkAoPE4zZxjytGQYwMIVRx81ZMB0QRCqbzkinB/f3lv6R+WDmtuNdH5epV3kYRbaFttItcdIyq6BLVUB1RdI+e0St6sx6sF+vd+piOFqw8s4l+wPr8AsyNsCo=</latexit><latexit sha1_base64="LHhK5VPbhS1xg2EMwUUBfHHGwMc=">AAACP3icbVBLSwMxGMzWV62vVY9egkVQlLIrgg88FLwIXirYB7R1yabZNjS7WZJvhbLsT/PiT/Dm2YsHFa/eTNs9+BoITGa+4UvGjwXX4DhPVmFmdm5+obhYWlpeWV2z1zcaWiaKsjqVQqqWTzQTPGJ14CBYK1aMhL5gTX94Mfabd0xpLqMbGMWsG5J+xANOCRjJs5v+reuloZdKj2cZ3scdX4pe2si+yQf4LMv1gSGCBbCb31WWT6Xn43iGcUfx/gD2PLvsVJwJ8F/i5qSMctQ8+7HTkzQJWQRUEK3brhNDNyUKOBUsK3USzWJCh6TP2oZGJGS6m04KyPCOUXo4kMqcCPBE/Z5ISaj1KPTNZEhgoH97Y/E/r51AcNJNeRQnwCI6XRQkAoPE4zZxjytGQYwMIVRx81ZMB0QRCqbzkinB/f3lv6R+WDmtuNdH5epV3kYRbaFttItcdIyq6BLVUB1RdI+e0St6sx6sF+vd+piOFqw8s4l+wPr8AsyNsCo=</latexit><latexit sha1_base64="LHhK5VPbhS1xg2EMwUUBfHHGwMc=">AAACP3icbVBLSwMxGMzWV62vVY9egkVQlLIrgg88FLwIXirYB7R1yabZNjS7WZJvhbLsT/PiT/Dm2YsHFa/eTNs9+BoITGa+4UvGjwXX4DhPVmFmdm5+obhYWlpeWV2z1zcaWiaKsjqVQqqWTzQTPGJ14CBYK1aMhL5gTX94Mfabd0xpLqMbGMWsG5J+xANOCRjJs5v+reuloZdKj2cZ3scdX4pe2si+yQf4LMv1gSGCBbCb31WWT6Xn43iGcUfx/gD2PLvsVJwJ8F/i5qSMctQ8+7HTkzQJWQRUEK3brhNDNyUKOBUsK3USzWJCh6TP2oZGJGS6m04KyPCOUXo4kMqcCPBE/Z5ISaj1KPTNZEhgoH97Y/E/r51AcNJNeRQnwCI6XRQkAoPE4zZxjytGQYwMIVRx81ZMB0QRCqbzkinB/f3lv6R+WDmtuNdH5epV3kYRbaFttItcdIyq6BLVUB1RdI+e0St6sx6sF+vd+piOFqw8s4l+wPr8AsyNsCo=</latexit><latexit sha1_base64="LHhK5VPbhS1xg2EMwUUBfHHGwMc=">AAACP3icbVBLSwMxGMzWV62vVY9egkVQlLIrgg88FLwIXirYB7R1yabZNjS7WZJvhbLsT/PiT/Dm2YsHFa/eTNs9+BoITGa+4UvGjwXX4DhPVmFmdm5+obhYWlpeWV2z1zcaWiaKsjqVQqqWTzQTPGJ14CBYK1aMhL5gTX94Mfabd0xpLqMbGMWsG5J+xANOCRjJs5v+reuloZdKj2cZ3scdX4pe2si+yQf4LMv1gSGCBbCb31WWT6Xn43iGcUfx/gD2PLvsVJwJ8F/i5qSMctQ8+7HTkzQJWQRUEK3brhNDNyUKOBUsK3USzWJCh6TP2oZGJGS6m04KyPCOUXo4kMqcCPBE/Z5ISaj1KPTNZEhgoH97Y/E/r51AcNJNeRQnwCI6XRQkAoPE4zZxjytGQYwMIVRx81ZMB0QRCqbzkinB/f3lv6R+WDmtuNdH5epV3kYRbaFttItcdIyq6BLVUB1RdI+e0St6sx6sF+vd+piOFqw8s4l+wPr8AsyNsCo=</latexit> … … s1 moi ⇣ rmo<i ⌘ <latexit sha1_base64="dEuQ73sOOGlAJordFhBO/PRTXnQ=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu63vHS0MvlR7PMuwKFkAVu74U/VRluZNeTLwMY1fxwRCOvXLFrtlT4EXi5KSCcjS98qfblzQJWQRUEK27jh1DLyUKOBUsK7mJZjGhIzJgXUMjEjLdS6ffZfjIKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPVSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaMiE48y8vktZJ7bzm3NQrjes8jSI6QIeoihx0ihroCjVRC1H0iJ7RK3qznqwX6936mLUWrHxmH/2B9fUDSeiiQw==</latexit><latexit sha1_base64="dEuQ73sOOGlAJordFhBO/PRTXnQ=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu63vHS0MvlR7PMuwKFkAVu74U/VRluZNeTLwMY1fxwRCOvXLFrtlT4EXi5KSCcjS98qfblzQJWQRUEK27jh1DLyUKOBUsK7mJZjGhIzJgXUMjEjLdS6ffZfjIKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPVSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaMiE48y8vktZJ7bzm3NQrjes8jSI6QIeoihx0ihroCjVRC1H0iJ7RK3qznqwX6936mLUWrHxmH/2B9fUDSeiiQw==</latexit><latexit sha1_base64="dEuQ73sOOGlAJordFhBO/PRTXnQ=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu63vHS0MvlR7PMuwKFkAVu74U/VRluZNeTLwMY1fxwRCOvXLFrtlT4EXi5KSCcjS98qfblzQJWQRUEK27jh1DLyUKOBUsK7mJZjGhIzJgXUMjEjLdS6ffZfjIKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPVSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaMiE48y8vktZJ7bzm3NQrjes8jSI6QIeoihx0ihroCjVRC1H0iJ7RK3qznqwX6936mLUWrHxmH/2B9fUDSeiiQw==</latexit><latexit sha1_base64="dEuQ73sOOGlAJordFhBO/PRTXnQ=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu63vHS0MvlR7PMuwKFkAVu74U/VRluZNeTLwMY1fxwRCOvXLFrtlT4EXi5KSCcjS98qfblzQJWQRUEK27jh1DLyUKOBUsK7mJZjGhIzJgXUMjEjLdS6ffZfjIKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPVSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaMiE48y8vktZJ7bzm3NQrjes8jSI6QIeoihx0ihroCjVRC1H0iJ7RK3qznqwX6936mLUWrHxmH/2B9fUDSeiiQw==</latexit> sk moi ⇣ rmo<i ⌘ <latexit sha1_base64="8yFvGakMoAgUOhWppwqPpdRpbLM=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu6/uRl4ZeKj2eZdgVLIAqdn0p+qnKcie9mHgZxq7igyEce+WKXbOnwIvEyUkF5Wh65U+3L2kSsgioIFp3HTuGXkoUcCpYVnITzWJCR2TAuoZGJGS6l06/y/CRUfo4kMpUBHiq/p5ISaj1OPRNZ0hgqOe9ifif100gOOulPIoTYBGdLQoSgUHiSVS4zxWjIMaGEKq4uRXTIVGEggm0ZEJw5l9eJK2T2nnNualXGtd5GkV0gA5RFTnoFDXQFWqiFqLoET2jV/RmPVkv1rv1MWstWPnMPvoD6+sHrLCifQ==</latexit><latexit sha1_base64="8yFvGakMoAgUOhWppwqPpdRpbLM=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu6/uRl4ZeKj2eZdgVLIAqdn0p+qnKcie9mHgZxq7igyEce+WKXbOnwIvEyUkF5Wh65U+3L2kSsgioIFp3HTuGXkoUcCpYVnITzWJCR2TAuoZGJGS6l06/y/CRUfo4kMpUBHiq/p5ISaj1OPRNZ0hgqOe9ifif100gOOulPIoTYBGdLQoSgUHiSVS4zxWjIMaGEKq4uRXTIVGEggm0ZEJw5l9eJK2T2nnNualXGtd5GkV0gA5RFTnoFDXQFWqiFqLoET2jV/RmPVkv1rv1MWstWPnMPvoD6+sHrLCifQ==</latexit><latexit sha1_base64="8yFvGakMoAgUOhWppwqPpdRpbLM=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu6/uRl4ZeKj2eZdgVLIAqdn0p+qnKcie9mHgZxq7igyEce+WKXbOnwIvEyUkF5Wh65U+3L2kSsgioIFp3HTuGXkoUcCpYVnITzWJCR2TAuoZGJGS6l06/y/CRUfo4kMpUBHiq/p5ISaj1OPRNZ0hgqOe9ifif100gOOulPIoTYBGdLQoSgUHiSVS4zxWjIMaGEKq4uRXTIVGEggm0ZEJw5l9eJK2T2nnNualXGtd5GkV0gA5RFTnoFDXQFWqiFqLoET2jV/RmPVkv1rv1MWstWPnMPvoD6+sHrLCifQ==</latexit><latexit sha1_base64="8yFvGakMoAgUOhWppwqPpdRpbLM=">AAACHHicbVBNS8NAEN3Ur1q/qh69LBahXkoiBRU8FLwIXipYW2hq2Gw37dJNNuxOhBLyR7z4V7x4UPHiQfDfuG1z0NYHA4/3ZpiZ58eCa7Dtb6uwtLyyulZcL21sbm3vlHf37rRMFGUtKoVUHZ9oJnjEWsBBsE6sGAl9wdr+6HLitx+Y0lxGtzCOWS8kg4gHnBIwkleu6/uRl4ZeKj2eZdgVLIAqdn0p+qnKcie9mHgZxq7igyEce+WKXbOnwIvEyUkF5Wh65U+3L2kSsgioIFp3HTuGXkoUcCpYVnITzWJCR2TAuoZGJGS6l06/y/CRUfo4kMpUBHiq/p5ISaj1OPRNZ0hgqOe9ifif100gOOulPIoTYBGdLQoSgUHiSVS4zxWjIMaGEKq4uRXTIVGEggm0ZEJw5l9eJK2T2nnNualXGtd5GkV0gA5RFTnoFDXQFWqiFqLoET2jV/RmPVkv1rv1MWstWPnMPvoD6+sHrLCifQ==</latexit> sK moi ⇣ rmo<i ⌘ <latexit sha1_base64="TEHfIFWYNFymBdy4hOMMgNi9FCk=">AAACHHicbVBNS8NAEN34WetX1aOXxSLUS0mkoIKHghehlwrWFpoaNttNu3STDbsToYT8ES/+FS8eVLx4EPw3btsctPXBwOO9GWbm+bHgGmz721paXlldWy9sFDe3tnd2S3v7d1omirIWlUKqjk80EzxiLeAgWCdWjIS+YG1/dDXx2w9MaS6jWxjHrBeSQcQDTgkYySvV9H3DS0MvlR7PMuwKFkAFu74U/VRluZNeTrwMY1fxwRBOvFLZrtpT4EXi5KSMcjS90qfblzQJWQRUEK27jh1DLyUKOBUsK7qJZjGhIzJgXUMjEjLdS6ffZfjYKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPdSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaNCE48y8vktZp9aLq3NTK9UaeRgEdoiNUQQ46Q3V0jZqohSh6RM/oFb1ZT9aL9W59zFqXrHzmAP2B9fUDdjCiXQ==</latexit><latexit sha1_base64="TEHfIFWYNFymBdy4hOMMgNi9FCk=">AAACHHicbVBNS8NAEN34WetX1aOXxSLUS0mkoIKHghehlwrWFpoaNttNu3STDbsToYT8ES/+FS8eVLx4EPw3btsctPXBwOO9GWbm+bHgGmz721paXlldWy9sFDe3tnd2S3v7d1omirIWlUKqjk80EzxiLeAgWCdWjIS+YG1/dDXx2w9MaS6jWxjHrBeSQcQDTgkYySvV9H3DS0MvlR7PMuwKFkAFu74U/VRluZNeTrwMY1fxwRBOvFLZrtpT4EXi5KSMcjS90qfblzQJWQRUEK27jh1DLyUKOBUsK7qJZjGhIzJgXUMjEjLdS6ffZfjYKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPdSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaNCE48y8vktZp9aLq3NTK9UaeRgEdoiNUQQ46Q3V0jZqohSh6RM/oFb1ZT9aL9W59zFqXrHzmAP2B9fUDdjCiXQ==</latexit><latexit sha1_base64="TEHfIFWYNFymBdy4hOMMgNi9FCk=">AAACHHicbVBNS8NAEN34WetX1aOXxSLUS0mkoIKHghehlwrWFpoaNttNu3STDbsToYT8ES/+FS8eVLx4EPw3btsctPXBwOO9GWbm+bHgGmz721paXlldWy9sFDe3tnd2S3v7d1omirIWlUKqjk80EzxiLeAgWCdWjIS+YG1/dDXx2w9MaS6jWxjHrBeSQcQDTgkYySvV9H3DS0MvlR7PMuwKFkAFu74U/VRluZNeTrwMY1fxwRBOvFLZrtpT4EXi5KSMcjS90qfblzQJWQRUEK27jh1DLyUKOBUsK7qJZjGhIzJgXUMjEjLdS6ffZfjYKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPdSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaNCE48y8vktZp9aLq3NTK9UaeRgEdoiNUQQ46Q3V0jZqohSh6RM/oFb1ZT9aL9W59zFqXrHzmAP2B9fUDdjCiXQ==</latexit><latexit sha1_base64="TEHfIFWYNFymBdy4hOMMgNi9FCk=">AAACHHicbVBNS8NAEN34WetX1aOXxSLUS0mkoIKHghehlwrWFpoaNttNu3STDbsToYT8ES/+FS8eVLx4EPw3btsctPXBwOO9GWbm+bHgGmz721paXlldWy9sFDe3tnd2S3v7d1omirIWlUKqjk80EzxiLeAgWCdWjIS+YG1/dDXx2w9MaS6jWxjHrBeSQcQDTgkYySvV9H3DS0MvlR7PMuwKFkAFu74U/VRluZNeTrwMY1fxwRBOvFLZrtpT4EXi5KSMcjS90qfblzQJWQRUEK27jh1DLyUKOBUsK7qJZjGhIzJgXUMjEjLdS6ffZfjYKH0cSGUqAjxVf0+kJNR6HPqmMyQw1PPeRPzP6yYQnPdSHsUJsIjOFgWJwCDxJCrc54pREGNDCFXc3IrpkChCwQRaNCE48y8vktZp9aLq3NTK9UaeRgEdoiNUQQ46Q3V0jZqohSh6RM/oFb1ZT9aL9W59zFqXrHzmAP2B9fUDdjCiXQ==</latexit> Then, we can calculate by softmax. It can be viewed as single-hidden-layer MLP.
  17. 17. 17 • At fist, the authors used timestamps to make order. • The benchmark datasets (e.g,. Movielens, Netflix) contains timestamps information. • They found that a random order works well in practice. • For fair comparison with other algorithms. • There was no significant difference from using timestamp information. • The order has to be prefixed before training. THE ORDER OF THE RATINGS
  18. 18. 18 • Training phase: • Test phase: • Given a user’s past behavior r = . • The user’s rating of new item m* can be predicted as: • where TEST PHASE FOR BASIC MODEL
  19. 19. CONTENTS 1. Motivation 2. NADE 3. Basic Model of CF-NADE 4. Parameters Sharing 5. Ordinal Cost 6. Deep Models and Augmentation 7. Experiments 8. Future Works and Discussion
  20. 20. 20 • 2 kinds of weight sharing in this paper. • Between users • Between ratings • Between users: • The CF (Collaborative Filtering) has sparsity problem. • The training data of CF problem is too sparse. • W, K, c, b are shared between users. • Between ratings: • Share weighs between different ratings. Why? and How? WEIGHT SHARING
  21. 21. 21 • Why? • Some items may not be rated much. • Items that are less rated are less optimized. • To enhance optimization of items that are less rated • How? using other weights of other ratings. WEIGHT SHARING BETWEEN RATINGS [ Basic model ] [ Weight sharing between ratings ]
  22. 22. 22 • When calculating the parameter of rating k in weight sharing, 
 the parameters of rating 1, ... , k of the basic model are added. • It is a kind of regularization, 
 which encourages the model to use as many parameters as possible to explain the data. WHY WEIGHT SHARING BETWEEN RATINGS WORKS?
  23. 23. CONTENTS 1. Motivation 2. NADE 3. Basic Model of CF-NADE 4. Parameters Sharing 5. Ordinal Cost 6. Deep Models and Augmentation 7. Experiments 8. Future Works and Discussion
  24. 24. 24 • In the basic model, we use softmax function to calculate cost. • We call it as regular cost Creg. • However, rating data has ordinal nature. • If user u want to give the movie m 4-star score, 
 the PMF by ordinal cost is more reasonable compared to that by regular cost. • Ordinal cost could be calculated by ranking loss. REGULAR COST CREG AND ORDINAL COST CORD 0.125 0.25 0.375 0.5 1 2 3 4 5 0.15 0.3 0.45 0.6 1 2 3 4 5 [ PMF by Regular Cost Creg ] [ PMF by Ordinal Cost Cord ]
  25. 25. 25 • What is ordinal nature? • Suppose . • the ranking of preferences over all the possible ratings can be expressed as: • denotes the preference of rating k over k-1. • It is used by multiplying two ranking losses (kind of heuristic I think). • To capture this ordinal nature: ORDINAL COST CORD regular to ordinal
  26. 26. 26 • 1. Ordinal cost is inspired by Xia et al (2008) [4]. • 2. the equation of PMF by ordinal cost is not PMF. • We have to normalize! • 3. the ordinal cost can not always capture ordinal nature. • ex. s1, s2, s3, s4, s5 = 1, 2.7, 2, 3, 1 SOME NOTES FOR ORDINAL COST CORD 0.125 0.25 0.375 0.5 1 2 3 4 5
  27. 27. 27 • Let hybrid cost: • CF-NADE: the basic model. • CF-NADE-S: the basic model with shared parameters. THE EFFECTS OF ORDINAL COST • The model with weight sharing
 outperforms the basic model. • The ordinal cost outperforms
 the regular cost.
  28. 28. CONTENTS 1. Motivation 2. NADE 3. Basic Model of CF-NADE 4. Parameters Sharing 5. Ordinal Cost 6. Deep Models and Augmentation 7. Experiments 8. Future Works and Discussion
  29. 29. 29 • The basic CF-NADE model can be viewed as single-hidden-layer MLP. • To make CF-NADE model be deep,
 all we need is to define the relation between hidden layers. • This architecture is adopted from Uria et al. (2014), which makes original NADE be deep networks [5]. MAKE CF-NADE BE DEEP The basic model of CF-NADE
  30. 30. 30 • Training deep networks is difficult compared to training the basic model. • It has the quite large numbers of free parameters. • This paper suggest 2 tricks to handle the situation training deep networks. • 1. Data augmentation • 2. Reduction of free parameters in the networks. TO TRAIN DEEP NETWORKS
  31. 31. 31 • As I mentioned earlier, a random order works well in practice. • Different order is an different instantiation of CF-NADE for the same user. • This is key to extend CF-NADE to a deep model. • Training all possible orderings: • Given a context , CF-NADE be equally good at modeling . • Cost function is rewritten as: DATA AUGMENTATION C = Eo2O DX i=1 log p ⇣ rmoi |rmo<i , o ⌘ <latexit sha1_base64="L5bgHKhYO1EJQoX09/swzqPbmB0=">AAACaXicbVFdaxQxFM2Mttbth9sKKu3LxaVQoS4zUlDBQqEVhD60gmsLO+uQyWZ2QzPJkNwRlpgH/6Jv/gJf/BHNbhex214IOTnn3pvck6KWwmKS/I7iBw+Xlh+tPG6trq1vPGlvbn21ujGM95iW2lwW1HIpFO+hQMkva8NpVUh+UVwdT/WL79xYodUXnNR8UNGREqVgFAOVt39mFcUxo9IdeziE2ako3EefOw2ZUPBPhzMPHjLbVLkTh6n/dgKvIZN6BHXYeIl7kBVaDp0JtSFH58J7+LFIug+B9vsQuhsxGuOrvN1Jusks4C5I56BD5nGet39lQ82aiitkklrbT5MaB44aFExy38oay2vKruiI9wNUtOJ24GZWedgNzBBKbcJSCDP2/wpHK2snVREyp4PbRW1K3qf1GyzfDZxQdYNcsZuLykYCapj6DkNhOEM5CYAyI8JbgY2poQzD77SCCeniyHdB7033fTf9fNA5Op27sUJ2yEuyR1LylhyRT+Sc9Agjf6L16Fn0PPobb8Uv4u2b1Dia1zwltyLuXANyBLoP</latexit><latexit sha1_base64="L5bgHKhYO1EJQoX09/swzqPbmB0=">AAACaXicbVFdaxQxFM2Mttbth9sKKu3LxaVQoS4zUlDBQqEVhD60gmsLO+uQyWZ2QzPJkNwRlpgH/6Jv/gJf/BHNbhex214IOTnn3pvck6KWwmKS/I7iBw+Xlh+tPG6trq1vPGlvbn21ujGM95iW2lwW1HIpFO+hQMkva8NpVUh+UVwdT/WL79xYodUXnNR8UNGREqVgFAOVt39mFcUxo9IdeziE2ako3EefOw2ZUPBPhzMPHjLbVLkTh6n/dgKvIZN6BHXYeIl7kBVaDp0JtSFH58J7+LFIug+B9vsQuhsxGuOrvN1Jusks4C5I56BD5nGet39lQ82aiitkklrbT5MaB44aFExy38oay2vKruiI9wNUtOJ24GZWedgNzBBKbcJSCDP2/wpHK2snVREyp4PbRW1K3qf1GyzfDZxQdYNcsZuLykYCapj6DkNhOEM5CYAyI8JbgY2poQzD77SCCeniyHdB7033fTf9fNA5Op27sUJ2yEuyR1LylhyRT+Sc9Agjf6L16Fn0PPobb8Uv4u2b1Dia1zwltyLuXANyBLoP</latexit><latexit sha1_base64="L5bgHKhYO1EJQoX09/swzqPbmB0=">AAACaXicbVFdaxQxFM2Mttbth9sKKu3LxaVQoS4zUlDBQqEVhD60gmsLO+uQyWZ2QzPJkNwRlpgH/6Jv/gJf/BHNbhex214IOTnn3pvck6KWwmKS/I7iBw+Xlh+tPG6trq1vPGlvbn21ujGM95iW2lwW1HIpFO+hQMkva8NpVUh+UVwdT/WL79xYodUXnNR8UNGREqVgFAOVt39mFcUxo9IdeziE2ako3EefOw2ZUPBPhzMPHjLbVLkTh6n/dgKvIZN6BHXYeIl7kBVaDp0JtSFH58J7+LFIug+B9vsQuhsxGuOrvN1Jusks4C5I56BD5nGet39lQ82aiitkklrbT5MaB44aFExy38oay2vKruiI9wNUtOJ24GZWedgNzBBKbcJSCDP2/wpHK2snVREyp4PbRW1K3qf1GyzfDZxQdYNcsZuLykYCapj6DkNhOEM5CYAyI8JbgY2poQzD77SCCeniyHdB7033fTf9fNA5Op27sUJ2yEuyR1LylhyRT+Sc9Agjf6L16Fn0PPobb8Uv4u2b1Dia1zwltyLuXANyBLoP</latexit><latexit sha1_base64="L5bgHKhYO1EJQoX09/swzqPbmB0=">AAACaXicbVFdaxQxFM2Mttbth9sKKu3LxaVQoS4zUlDBQqEVhD60gmsLO+uQyWZ2QzPJkNwRlpgH/6Jv/gJf/BHNbhex214IOTnn3pvck6KWwmKS/I7iBw+Xlh+tPG6trq1vPGlvbn21ujGM95iW2lwW1HIpFO+hQMkva8NpVUh+UVwdT/WL79xYodUXnNR8UNGREqVgFAOVt39mFcUxo9IdeziE2ako3EefOw2ZUPBPhzMPHjLbVLkTh6n/dgKvIZN6BHXYeIl7kBVaDp0JtSFH58J7+LFIug+B9vsQuhsxGuOrvN1Jusks4C5I56BD5nGet39lQ82aiitkklrbT5MaB44aFExy38oay2vKruiI9wNUtOJ24GZWedgNzBBKbcJSCDP2/wpHK2snVREyp4PbRW1K3qf1GyzfDZxQdYNcsZuLykYCapj6DkNhOEM5CYAyI8JbgY2poQzD77SCCeniyHdB7033fTf9fNA5Op27sUJ2yEuyR1LylhyRT+Sc9Agjf6L16Fn0PPobb8Uv4u2b1Dia1zwltyLuXANyBLoP</latexit> rmo<i<latexit sha1_base64="u17hOyMmvjjd0W+DvViZwKnRctk=">AAAB/XicbVBNSwMxEM3Wr1q/VsWTl2ARPJVdEVTwUPAieKng2kK7LNls2oZmkyXJCiUs+Fe8eFDx6v/w5r8xbfegrQ8GHu/NMDMvzhhV2vO+ncrS8srqWnW9trG5tb3j7u49KJFLTAIsmJCdGCnCKCeBppqRTiYJSmNG2vHoeuK3H4lUVPB7Pc5ImKIBp32KkbZS5B70YsESI4vIpJERkbmiRVFEbt1reFPAReKXpA5KtCL3q5cInKeEa8yQUl3fy3RokNQUM1LUerkiGcIjNCBdSzlKiQrN9PwCHlslgX0hbXENp+rvCYNSpcZpbDtTpIdq3puI/3ndXPcvQkN5lmvC8WxRP2dQCzjJAiZUEqzZ2BKEJbW3QjxEEmFtE6vZEPz5lxdJcNq4bPh3Z/XmbZlGFRyCI3ACfHAOmuAGtEAAMDDgGbyCN+fJeXHenY9Za8UpZ/bBHzifP9Sflkc=</latexit><latexit sha1_base64="u17hOyMmvjjd0W+DvViZwKnRctk=">AAAB/XicbVBNSwMxEM3Wr1q/VsWTl2ARPJVdEVTwUPAieKng2kK7LNls2oZmkyXJCiUs+Fe8eFDx6v/w5r8xbfegrQ8GHu/NMDMvzhhV2vO+ncrS8srqWnW9trG5tb3j7u49KJFLTAIsmJCdGCnCKCeBppqRTiYJSmNG2vHoeuK3H4lUVPB7Pc5ImKIBp32KkbZS5B70YsESI4vIpJERkbmiRVFEbt1reFPAReKXpA5KtCL3q5cInKeEa8yQUl3fy3RokNQUM1LUerkiGcIjNCBdSzlKiQrN9PwCHlslgX0hbXENp+rvCYNSpcZpbDtTpIdq3puI/3ndXPcvQkN5lmvC8WxRP2dQCzjJAiZUEqzZ2BKEJbW3QjxEEmFtE6vZEPz5lxdJcNq4bPh3Z/XmbZlGFRyCI3ACfHAOmuAGtEAAMDDgGbyCN+fJeXHenY9Za8UpZ/bBHzifP9Sflkc=</latexit><latexit sha1_base64="u17hOyMmvjjd0W+DvViZwKnRctk=">AAAB/XicbVBNSwMxEM3Wr1q/VsWTl2ARPJVdEVTwUPAieKng2kK7LNls2oZmkyXJCiUs+Fe8eFDx6v/w5r8xbfegrQ8GHu/NMDMvzhhV2vO+ncrS8srqWnW9trG5tb3j7u49KJFLTAIsmJCdGCnCKCeBppqRTiYJSmNG2vHoeuK3H4lUVPB7Pc5ImKIBp32KkbZS5B70YsESI4vIpJERkbmiRVFEbt1reFPAReKXpA5KtCL3q5cInKeEa8yQUl3fy3RokNQUM1LUerkiGcIjNCBdSzlKiQrN9PwCHlslgX0hbXENp+rvCYNSpcZpbDtTpIdq3puI/3ndXPcvQkN5lmvC8WxRP2dQCzjJAiZUEqzZ2BKEJbW3QjxEEmFtE6vZEPz5lxdJcNq4bPh3Z/XmbZlGFRyCI3ACfHAOmuAGtEAAMDDgGbyCN+fJeXHenY9Za8UpZ/bBHzifP9Sflkc=</latexit><latexit sha1_base64="u17hOyMmvjjd0W+DvViZwKnRctk=">AAAB/XicbVBNSwMxEM3Wr1q/VsWTl2ARPJVdEVTwUPAieKng2kK7LNls2oZmkyXJCiUs+Fe8eFDx6v/w5r8xbfegrQ8GHu/NMDMvzhhV2vO+ncrS8srqWnW9trG5tb3j7u49KJFLTAIsmJCdGCnCKCeBppqRTiYJSmNG2vHoeuK3H4lUVPB7Pc5ImKIBp32KkbZS5B70YsESI4vIpJERkbmiRVFEbt1reFPAReKXpA5KtCL3q5cInKeEa8yQUl3fy3RokNQUM1LUerkiGcIjNCBdSzlKiQrN9PwCHlslgX0hbXENp+rvCYNSpcZpbDtTpIdq3puI/3ndXPcvQkN5lmvC8WxRP2dQCzjJAiZUEqzZ2BKEJbW3QjxEEmFtE6vZEPz5lxdJcNq4bPh3Z/XmbZlGFRyCI3ACfHAOmuAGtEAAMDDgGbyCN+fJeXHenY9Za8UpZ/bBHzifP9Sflkc=</latexit> rmoi<latexit sha1_base64="ue6ZLKhuXfSzu96sGd7/xMCwn5E=">AAAB+3icbVDLSsNAFJ3UV62vaJduBovgqiQiqLuCG8FNBaOFNoTJZNIOnUeYmQglxF9x40LFrT/izr9x2mahrQcuHM65l3vviTNGtfG8b6e2srq2vlHfbGxt7+zuufsH91rmCpMASyZVL0aaMCpIYKhhpJcpgnjMyEM8vpr6D49EaSrFnZlkJORoKGhKMTJWitzmIJYsKVQZFTwqZETLEkZuy2t7M8Bl4lekBSp0I/drkEiccyIMZkjrvu9lJiyQMhQzUjYGuSYZwmM0JH1LBeJEh8Xs+BIeWyWBqVS2hIEz9fdEgbjWEx7bTo7MSC96U/E/r5+b9CIsqMhyQwSeL0pzBo2E0yRgQhXBhk0sQVhReyvEI6QQNjavhg3BX3x5mQSn7cu2f3vW6txUadTBITgCJ8AH56ADrkEXBACDCXgGr+DNeXJenHfnY95ac6qZJvgD5/MH2PSVHw==</latexit><latexit sha1_base64="ue6ZLKhuXfSzu96sGd7/xMCwn5E=">AAAB+3icbVDLSsNAFJ3UV62vaJduBovgqiQiqLuCG8FNBaOFNoTJZNIOnUeYmQglxF9x40LFrT/izr9x2mahrQcuHM65l3vviTNGtfG8b6e2srq2vlHfbGxt7+zuufsH91rmCpMASyZVL0aaMCpIYKhhpJcpgnjMyEM8vpr6D49EaSrFnZlkJORoKGhKMTJWitzmIJYsKVQZFTwqZETLEkZuy2t7M8Bl4lekBSp0I/drkEiccyIMZkjrvu9lJiyQMhQzUjYGuSYZwmM0JH1LBeJEh8Xs+BIeWyWBqVS2hIEz9fdEgbjWEx7bTo7MSC96U/E/r5+b9CIsqMhyQwSeL0pzBo2E0yRgQhXBhk0sQVhReyvEI6QQNjavhg3BX3x5mQSn7cu2f3vW6txUadTBITgCJ8AH56ADrkEXBACDCXgGr+DNeXJenHfnY95ac6qZJvgD5/MH2PSVHw==</latexit><latexit sha1_base64="ue6ZLKhuXfSzu96sGd7/xMCwn5E=">AAAB+3icbVDLSsNAFJ3UV62vaJduBovgqiQiqLuCG8FNBaOFNoTJZNIOnUeYmQglxF9x40LFrT/izr9x2mahrQcuHM65l3vviTNGtfG8b6e2srq2vlHfbGxt7+zuufsH91rmCpMASyZVL0aaMCpIYKhhpJcpgnjMyEM8vpr6D49EaSrFnZlkJORoKGhKMTJWitzmIJYsKVQZFTwqZETLEkZuy2t7M8Bl4lekBSp0I/drkEiccyIMZkjrvu9lJiyQMhQzUjYGuSYZwmM0JH1LBeJEh8Xs+BIeWyWBqVS2hIEz9fdEgbjWEx7bTo7MSC96U/E/r5+b9CIsqMhyQwSeL0pzBo2E0yRgQhXBhk0sQVhReyvEI6QQNjavhg3BX3x5mQSn7cu2f3vW6txUadTBITgCJ8AH56ADrkEXBACDCXgGr+DNeXJenHfnY95ac6qZJvgD5/MH2PSVHw==</latexit><latexit sha1_base64="ue6ZLKhuXfSzu96sGd7/xMCwn5E=">AAAB+3icbVDLSsNAFJ3UV62vaJduBovgqiQiqLuCG8FNBaOFNoTJZNIOnUeYmQglxF9x40LFrT/izr9x2mahrQcuHM65l3vviTNGtfG8b6e2srq2vlHfbGxt7+zuufsH91rmCpMASyZVL0aaMCpIYKhhpJcpgnjMyEM8vpr6D49EaSrFnZlkJORoKGhKMTJWitzmIJYsKVQZFTwqZETLEkZuy2t7M8Bl4lekBSp0I/drkEiccyIMZkjrvu9lJiyQMhQzUjYGuSYZwmM0JH1LBeJEh8Xs+BIeWyWBqVS2hIEz9fdEgbjWEx7bTo7MSC96U/E/r5+b9CIsqMhyQwSeL0pzBo2E0yRgQhXBhk0sQVhReyvEI6QQNjavhg3BX3x5mQSn7cu2f3vW6txUadTBITgCJ8AH56ADrkEXBACDCXgGr+DNeXJenHfnY95ac6qZJvgD5/MH2PSVHw==</latexit>
  32. 32. 32 • Too many parameters… • • Netflix dataset, M=17700, H=500, K=5, then 89 million free parameters. • Weight decay and dropout is ok, but… • Solution: Factorize W, V by a product of 2 low-rank matrices. • e.g. J=50, M=17700, H=500, K=5, then only 9-million free parameters 
 for Netflix dataset. • Is this the best way? I think there is better way to make scalability. REDUCTION OF FREE PARAMETERS k 2 {1, 2, ..., K}<latexit sha1_base64="mB5r8UlHuNdIEU0I2gHnSZUyPKY=">AAACC3icbVDLSsNAFJ3UV62vqks3g0VwUUJSBHVXcCN0U8FYoQllMp20QyeTMHMjlNAPcOOvuHGh4tYfcOffOG2z0NYDA4dzzuXOPWEquAbH+bZKK6tr6xvlzcrW9s7uXnX/4E4nmaLMo4lI1H1INBNcMg84CHafKkbiULBOOLqa+p0HpjRP5C2MUxbEZCB5xCkBI/WqtRH2ucS+YBH4OXbruFHHtm3XcQv7ig+G4E9MyrGdGfAycQtSQwXaveqX309oFjMJVBCtu66TQpATBZwKNqn4mWYpoSMyYF1DJYmZDvLZMRN8YpQ+jhJlngQ8U39P5CTWehyHJhkTGOpFbyr+53UziC6CnMs0AybpfFGUCQwJnjaD+1wxCmJsCKGKm79iOiSKUDD9VUwJ7uLJy8Rr2Je2e3NWa7aKNsroCB2jU+Sic9RE16iNPETRI3pGr+jNerJerHfrYx4tWcXMIfoD6/MHdu2YUQ==</latexit><latexit sha1_base64="mB5r8UlHuNdIEU0I2gHnSZUyPKY=">AAACC3icbVDLSsNAFJ3UV62vqks3g0VwUUJSBHVXcCN0U8FYoQllMp20QyeTMHMjlNAPcOOvuHGh4tYfcOffOG2z0NYDA4dzzuXOPWEquAbH+bZKK6tr6xvlzcrW9s7uXnX/4E4nmaLMo4lI1H1INBNcMg84CHafKkbiULBOOLqa+p0HpjRP5C2MUxbEZCB5xCkBI/WqtRH2ucS+YBH4OXbruFHHtm3XcQv7ig+G4E9MyrGdGfAycQtSQwXaveqX309oFjMJVBCtu66TQpATBZwKNqn4mWYpoSMyYF1DJYmZDvLZMRN8YpQ+jhJlngQ8U39P5CTWehyHJhkTGOpFbyr+53UziC6CnMs0AybpfFGUCQwJnjaD+1wxCmJsCKGKm79iOiSKUDD9VUwJ7uLJy8Rr2Je2e3NWa7aKNsroCB2jU+Sic9RE16iNPETRI3pGr+jNerJerHfrYx4tWcXMIfoD6/MHdu2YUQ==</latexit><latexit sha1_base64="mB5r8UlHuNdIEU0I2gHnSZUyPKY=">AAACC3icbVDLSsNAFJ3UV62vqks3g0VwUUJSBHVXcCN0U8FYoQllMp20QyeTMHMjlNAPcOOvuHGh4tYfcOffOG2z0NYDA4dzzuXOPWEquAbH+bZKK6tr6xvlzcrW9s7uXnX/4E4nmaLMo4lI1H1INBNcMg84CHafKkbiULBOOLqa+p0HpjRP5C2MUxbEZCB5xCkBI/WqtRH2ucS+YBH4OXbruFHHtm3XcQv7ig+G4E9MyrGdGfAycQtSQwXaveqX309oFjMJVBCtu66TQpATBZwKNqn4mWYpoSMyYF1DJYmZDvLZMRN8YpQ+jhJlngQ8U39P5CTWehyHJhkTGOpFbyr+53UziC6CnMs0AybpfFGUCQwJnjaD+1wxCmJsCKGKm79iOiSKUDD9VUwJ7uLJy8Rr2Je2e3NWa7aKNsroCB2jU+Sic9RE16iNPETRI3pGr+jNerJerHfrYx4tWcXMIfoD6/MHdu2YUQ==</latexit><latexit sha1_base64="mB5r8UlHuNdIEU0I2gHnSZUyPKY=">AAACC3icbVDLSsNAFJ3UV62vqks3g0VwUUJSBHVXcCN0U8FYoQllMp20QyeTMHMjlNAPcOOvuHGh4tYfcOffOG2z0NYDA4dzzuXOPWEquAbH+bZKK6tr6xvlzcrW9s7uXnX/4E4nmaLMo4lI1H1INBNcMg84CHafKkbiULBOOLqa+p0HpjRP5C2MUxbEZCB5xCkBI/WqtRH2ucS+YBH4OXbruFHHtm3XcQv7ig+G4E9MyrGdGfAycQtSQwXaveqX309oFjMJVBCtu66TQpATBZwKNqn4mWYpoSMyYF1DJYmZDvLZMRN8YpQ+jhJlngQ8U39P5CTWehyHJhkTGOpFbyr+53UziC6CnMs0AybpfFGUCQwJnjaD+1wxCmJsCKGKm79iOiSKUDD9VUwJ7uLJy8Rr2Je2e3NWa7aKNsroCB2jU+Sic9RE16iNPETRI3pGr+jNerJerHfrYx4tWcXMIfoD6/MHdu2YUQ==</latexit>
  33. 33. CONTENTS 1. Motivation 2. NADE 3. Basic Model of CF-NADE 4. Parameters Sharing 5. Ordinal Cost 6. Deep Models and Augmentation 7. Experiments 8. Future Works and Discussion
  34. 34. 34 • Metric: • is the i-th true rating and is the predicted rating by the model. • S is the total number of ratings in the test set. • 90% training set and 10% test set. • Benchmark dataset: EXPERIMENTS ri<latexit sha1_base64="bZrBveIyIMjF/b94D20eOjf2Xro=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL4KXisYW2lA22027dLMJuxOhhP4ELx5UvPqPvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdD31W09cG5GoBxynPIjpQIlIMIpWutc90avW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx26oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpn+TvtCcoRxbQpkW9lbChlRThjadig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCAZ3iFN0c6L8678zFvLTnFzCH8gfP5A8e0ja8=</latexit><latexit sha1_base64="bZrBveIyIMjF/b94D20eOjf2Xro=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL4KXisYW2lA22027dLMJuxOhhP4ELx5UvPqPvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdD31W09cG5GoBxynPIjpQIlIMIpWutc90avW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx26oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpn+TvtCcoRxbQpkW9lbChlRThjadig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCAZ3iFN0c6L8678zFvLTnFzCH8gfP5A8e0ja8=</latexit><latexit sha1_base64="bZrBveIyIMjF/b94D20eOjf2Xro=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL4KXisYW2lA22027dLMJuxOhhP4ELx5UvPqPvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdD31W09cG5GoBxynPIjpQIlIMIpWutc90avW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx26oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpn+TvtCcoRxbQpkW9lbChlRThjadig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCAZ3iFN0c6L8678zFvLTnFzCH8gfP5A8e0ja8=</latexit><latexit sha1_base64="bZrBveIyIMjF/b94D20eOjf2Xro=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FL4KXisYW2lA22027dLMJuxOhhP4ELx5UvPqPvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdD31W09cG5GoBxynPIjpQIlIMIpWutc90avW3Lo7A1kmXkFqUKDZq351+wnLYq6QSWpMx3NTDHKqUTDJJ5VuZnhK2YgOeMdSRWNugnx26oScWKVPokTbUkhm6u+JnMbGjOPQdsYUh2bRm4r/eZ0Mo8sgFyrNkCs2XxRlkmBCpn+TvtCcoRxbQpkW9lbChlRThjadig3BW3x5mfhn9au6d3dea9wWaZThCI7hFDy4gAbcQBN8YDCAZ3iFN0c6L8678zFvLTnFzCH8gfP5A8e0ja8=</latexit> ˜ri<latexit sha1_base64="x+b/iaYbr09zkeeDQuYZoNK4MVw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbSibzaZdutkNuxOhhP4MLx5UvPpvvPlv3LY5aPXBwOO9GWbmRZngBj3vy6msrK6tb1Q3a1vbO7t79f2DB6NyTVlAlVC6GxHDBJcsQI6CdTPNSBoJ1onG1zO/88i04Ure4yRjYUqGkiecErRSr49cxKzQ0wEf1Bte05vD/Uv8kjSgRHtQ/+zHiuYpk0gFMabnexmGBdHIqWDTWj83LCN0TIasZ6kkKTNhMT956p5YJXYTpW1JdOfqz4mCpMZM0sh2pgRHZtmbif95vRyTy7DgMsuRSbpYlOTCReXO/ndjrhlFMbGEUM3trS4dEU0o2pRqNgR/+eW/JDhrXjX9u/NG67ZMowpHcAyn4MMFtOAG2hAABQVP8AKvDjrPzpvzvmitOOXMIfyC8/ENKFCRZQ==</latexit><latexit sha1_base64="x+b/iaYbr09zkeeDQuYZoNK4MVw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbSibzaZdutkNuxOhhP4MLx5UvPpvvPlv3LY5aPXBwOO9GWbmRZngBj3vy6msrK6tb1Q3a1vbO7t79f2DB6NyTVlAlVC6GxHDBJcsQI6CdTPNSBoJ1onG1zO/88i04Ure4yRjYUqGkiecErRSr49cxKzQ0wEf1Bte05vD/Uv8kjSgRHtQ/+zHiuYpk0gFMabnexmGBdHIqWDTWj83LCN0TIasZ6kkKTNhMT956p5YJXYTpW1JdOfqz4mCpMZM0sh2pgRHZtmbif95vRyTy7DgMsuRSbpYlOTCReXO/ndjrhlFMbGEUM3trS4dEU0o2pRqNgR/+eW/JDhrXjX9u/NG67ZMowpHcAyn4MMFtOAG2hAABQVP8AKvDjrPzpvzvmitOOXMIfyC8/ENKFCRZQ==</latexit><latexit sha1_base64="x+b/iaYbr09zkeeDQuYZoNK4MVw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbSibzaZdutkNuxOhhP4MLx5UvPpvvPlv3LY5aPXBwOO9GWbmRZngBj3vy6msrK6tb1Q3a1vbO7t79f2DB6NyTVlAlVC6GxHDBJcsQI6CdTPNSBoJ1onG1zO/88i04Ure4yRjYUqGkiecErRSr49cxKzQ0wEf1Bte05vD/Uv8kjSgRHtQ/+zHiuYpk0gFMabnexmGBdHIqWDTWj83LCN0TIasZ6kkKTNhMT956p5YJXYTpW1JdOfqz4mCpMZM0sh2pgRHZtmbif95vRyTy7DgMsuRSbpYlOTCReXO/ndjrhlFMbGEUM3trS4dEU0o2pRqNgR/+eW/JDhrXjX9u/NG67ZMowpHcAyn4MMFtOAG2hAABQVP8AKvDjrPzpvzvmitOOXMIfyC8/ENKFCRZQ==</latexit><latexit sha1_base64="x+b/iaYbr09zkeeDQuYZoNK4MVw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQiqLeCF8FLBWMLbSibzaZdutkNuxOhhP4MLx5UvPpvvPlv3LY5aPXBwOO9GWbmRZngBj3vy6msrK6tb1Q3a1vbO7t79f2DB6NyTVlAlVC6GxHDBJcsQI6CdTPNSBoJ1onG1zO/88i04Ure4yRjYUqGkiecErRSr49cxKzQ0wEf1Bte05vD/Uv8kjSgRHtQ/+zHiuYpk0gFMabnexmGBdHIqWDTWj83LCN0TIasZ6kkKTNhMT956p5YJXYTpW1JdOfqz4mCpMZM0sh2pgRHZtmbif95vRyTy7DgMsuRSbpYlOTCReXO/ndjrhlFMbGEUM3trS4dEU0o2pRqNgR/+eW/JDhrXjX9u/NG67ZMowpHcAyn4MMFtOAG2hAABQVP8AKvDjrPzpvzvmitOOXMIfyC8/ENKFCRZQ==</latexit> dataset #Users #Items #Scales #Ratings Movielens 100k 1000 1700 5 105 Movielens 1M 6040 3952 5 106 Movielens 10M 71567 10681 10 107 Netflix 480189 17770 5 108
  35. 35. 35 • The model described in previous slides are user-based model. • We model rating vectors of user: ru. • Similary, item-based CF-NADE model rating vectors of item: ri. • Some CF papers said item-based CF model outperforms user-based CF. USER-BASED VS. ITEM-BASED
  36. 36. 36 • In Movielens 1M dataset, CF-NADE outperforms all state-of-arts algorithms. • The double hidden layer model outperforms single hidden layer model. • Item based CF model outperforms user based CF model. EXPERIMENT RESULTS - MOVIELENS 1M
  37. 37. 37 • CF-NADE shows the best performance even on the larger data sets. • Now, it is one of the best performing algorithms except MRMA (NIPS 2017). • MRMA model is not deep networks, but a matrix factorization based model. • They didn’t use item based CF-NADE because of the computational issue. • I think that the performance will be better if we use item-based CF-NADE model. EXPERIMENT RESULTS - OTHER DATASETS
  38. 38. 38 • At first, the RBM-CF paper [9] said that item based CF model outperforms user based CF model. • The AAE paper [8],AutoRec [7] and CF-NADE [2] paper presented the experimental results supporting the claim. • There are pros and cons: performance vs computational cost. • Why item based model outperforms user based model? • The AutoRec [7] paper said that • The average number of ratings per item is much more than those per user • It is inconsistent with the experimental results with movie lens 100k data of other papers. • High variance in the number of user ratings leads to less reliable prediction for user-based methods. ITEM-BASED VS USER-BASED
  39. 39. 39 • tSNE of the representations of the movies on Movielens 1M dataset. VISUALIZATION OF THE LEARNED MODEL RED: Documentary BLUE: Children’s
  40. 40. 40 • They picked 5 most similar movies of the left side movies COSINE SIMILARITIES OF THE W VECTOR Star TrekVI:
 The Undiscovered Country Star Trek III:
 The Search for Spock Star Trek:
 Generations Star Trek:
 Insurrection Star Trek:
 First Contact Star Trek IV:
 TheVoyage Home The Lion King Aladdin Beauty and
 the Beast The Little
 Mermaid A Bug’s Life Lady and the Tramp
  41. 41. CONTENTS 1. Motivation 2. NADE 3. Basic Model of CF-NADE 4. Parameters Sharing 5. Ordinal Cost 6. Deep Models and Augmentation 7. Experiments 8. Future Works and Discussion
  42. 42. 42 • CF-NADE: • An efficient and powerful architecture for CF tasks. • Sharing parameters is a good way to improve performance. • Better scalability by factorization. • Can be extended to a deep version and performs well. • Ordinal cost for rating data • Meaningful representations • Source code at: https://github.com/Ian09/CF-NADE CONCLUSION
  43. 43. • Implicit Feedback vs Explicit Feedback • The recommendation using the implicit feedback data (e.g., click or view). • Generally, implicit feedback problem is more difficult than explicit feedback problem. • Because, the view or click can not tell the positive preference. • Implicit CF-NADE [6]: • where t is implicit feedback data and c is confidence level. 43 A FOLLOW-UP STUDY
  44. 44. 44 • [1] Larochelle, Hugo and Murray, Iain.The neural autoregressive distribution estimator. In International Conference on Artificial Intelligence and Statistics, pp. 29–37, 2011. • [2] Zheng,Y.,Tang, B., Ding,W., & Zhou, H. (2016).A neural autoregressive approach to collaborative filtering. arXiv preprint arXiv:1605.09477. • [3] https://www.youtube.com/watch?v=JBP1xUIlH5I&t=4s • [4] Xia, Fen, Liu,Tie-Yan,Wang, Jue, Zhang,Wensheng, and Li, Hang. Listwise approach to learning to rank: theory and algorithm. In Proceedings of the 25th international conference on Machine learning, pp. 1192–1199.ACM, 2008 • [5] Uria, Benigno, Murray, Iain, and Larochelle, Hugo.A deep and tractable density estimator. JMLR: W&CP, 32(1): 467–475, 2014. • [6] Zheng,Y., Liu, C.,Tang, B., & Zhou, H. (2016, September). Neural autoregressive collaborative filtering for implicit feedback. In Proceedings of the 1st workshop on deep learning for recommender systems (pp. 2- 6).ACM. • [7] Sedhain, S., Menon,A. K., Sanner, S., & Xie, L. (2015, May).Autorec:Autoencoders meet collaborative filtering. In Proceedings of the 24th International Conference onWorldWideWeb (pp. 111-112).ACM. • [8] Lee, K., Lee,Y. H., & Suh, C. (2018, June).Alternating autoencoders for matrix completion. In 2018 IEEE Data ScienceWorkshop (DSW) (pp. 130-134). IEEE. • [9] Salakhutdinov, R., Mnih,A., & Hinton, G. (2007, June). Restricted Boltzmann machines for collaborative filtering. In Proceedings of the 24th international conference on Machine learning (pp. 791-798). ACM. REFERENCE

    Be the first to comment

    Login to see the comments

PPT for A Neural Autoregressive Approach to Collaborative Filtering (CF-NADE). I made ppt for explaining the paper. Abstract of the paper: This paper proposes CF-NADE, a neural autoregressive architecture for collaborative filtering (CF) tasks, which is inspired by the Restricted Boltzmann Machine (RBM) based CF model and the Neural Autoregressive Distribution Estimator (NADE). We first describe the basic CF-NADE model for CF tasks. Then we propose to improve the model by sharing parameters between different ratings. A factored version of CF-NADE is also proposed for better scalability. Furthermore, we take the ordinal nature of the preferences into consideration and propose an ordinal cost to optimize CF-NADE, which shows superior performance. Finally, CF-NADE can be extended to a deep model, with only moderately increased computational complexity. Experimental results show that CF-NADE with a single hidden layer beats all previous state-of-the-art methods on MovieLens 1M, MovieLens 10M, and Netflix datasets, and adding more hidden layers can further improve the performance.

Views

Total views

430

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

7

Shares

0

Comments

0

Likes

0

×