IkaLog: Data Collector for Splatoon and Machine Learning
•
1 like•1,521 views
This is a translated version of IkaLog tech presentation, originally in Japanese language. The translation work is still in progress!
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
More from Takeshi HASEGAWA
More from Takeshi HASEGAWA (20)
Recently uploaded
Recently uploaded (20)
IkaLog: Data Collector for Splatoon and Machine Learning
- 3. What is Splatoon? A unique third-person shooter – Paint the battlefield with your ink, and claim turf from the enemy team – Switch between an Inkling and a Squid for different approaches – Several gamemodes in addition to turf war, like KOTH and payload Simple rules, but deep – Various strategies and tactics – Unique weapon classes Shooter, Charger, Roller, Splatling, and Slosher
- 4. Strategy is the key to winning STAGE RECON • There’s more than one way around each map • Know where the enemy is coming from and have a plan VARIETY OF WEAPONS • Certain weapons are beCer suited to some maps and modes over others
- 5. You play, IkaLog collects the data HDMI Capture Device IkaLog
- 6. COLLECT PLAY IkaLog: Data Collector for Splatoon – Play Splatoon as you would normally – Your gameplay footage is analyzed – The data can then be sent to log files, and/or other tools, like stat.ink Game Console Log files IkaLog stat.ink, Speech applica6on, etc. Forward to desired tools PROCESS
- 9. stat.ink The online database provided by @fetus_hina – Submit your battle results and statistics with IkaLog – Review your past gameplay easily using the website
- 10. Review your gameplay on stat.ink SCOREBOARD Review the scoreboard later Filtering allows for careful analysis of your past gameplay TIMELINE The graph shows what happened in the game Displays kills, deaths, special weapons, ranked mode counts/distances and events GLOBAL STATS Sta6s6cs data from all stat.ink users is available See the trends among users
- 12. Timeline for Ranked Battles Status of the Splat Zone Splat Zone count the other team earned Splat Zone count your team earned The game-changing moment
- 13. # of IkaLog + stat.ink users Source hCps://stat.ink/en6re/user Avg: 200+ users, 4500+ matches a day Peak: 370 users a day Processed 15,000 matches
- 14. Statistical data from all players Kill/Death Heat map KO raWo
- 18. 18
- 20. 20
- 24. １）Crop the number from the footage, apply some image filters ２）Generate vertical & horizontal histogram to guess each character’s position ３）Resize the characters to identical size, and make the image binary ４）Classify the image using KNN
- 28. • Bascially same idea with recogni6on of numbers • 30+% of accuracy for single classifica6on. Earn accuracy by inves6ga6ng many frames – IkaLog inves6gates approx. 10 frames per a sec – This example shows IkaLog analyzed 49 frames, and found “96gal_deco” is most likely (18 frames, 36%) -> correct. votes={ 'supershot': 6, 'carbon_deco': 1, 'bucketslosher': 1, 'octoshooter_replica': 1, 'splashshield': 1, 'sshooter_collabo': 5, 'hotblaster': 2, 'pablo': 1, 'nzap89': 6, 'sharp_neo': 3, 'hotblaster_custom': 2, '96gal_deco': 18, '52gal': 1, 'hokusai': 1 }
- 30. 30
- 35. 35
- 39. 39
- 43. • Amount of the data is always jus6ce for machine learning • Use the crowd to collect the data 50 games/day per person 5000 games can be possible by 100 people – Field-generated data has the noises and outliers • Model of HDMI capture units, HDMI re-transmiCers, and soUware configura6on • It’s easier to collec6ng the outliers from field, then train, and test classifiers
- 44. Object storage on the cloud Work VM on IaaS 4TB+ Data (collect for 12months+) IkaLog users stat.ink
- 45. • Accuracy of Machine learning/Deep Learning is affected by pre- processing • Template matching-based algorithm automa6cally fixes broken input images Wrong aspect ra6o offseted reference Image to classify
- 46. Thanks @itoooon
- 48. Pros – BeCer classifica6on accuracy Cons – Too large data size（20MB -> 100MB~400MB） Hard to distribute the weights – Breaks Windows version of IkaLog • 90%+ of users are running IkaLog on Windows Plazorm • Most of deep learning framework breaks Py2EXE (Python script to Windows executable converter) – More compu6ng power needed
- 49. – Input： RGB or HSV color value (47*45*3=6,345 units) – Output： possibility of each class（91 units, apply soUmax） – Connec6on： always use fully connec6on – Let computers the feature automa6cally • In this use case, deep learning will find proper weights automa6cally • It will ignore background colors automa6cally – Target Performance • Calc. 6me: less than 350 ms for each mul6-class classifica6on(91 classes) < 3 seconds per a frame • 99.99+% accuracy against stat.ink posted (Field) data
- 51. 51
- 52. • Tested on Azure ML, before implemen6ng the code – Trained classifiers using small dataset – Input : Pixel values（w/PCA) – Hidden Layer: 1 layer、150 units – Output: Probably 91 (Azure ML manages) • Got good results and decided to move forward – 98+% accuracy (with small dataset) – Enough robustness for any images (e.g. los-res)
- 54. 54
- 55. • Used Chainer (Deep Learning Framework) to train the model – Supervised training using stat.ink data (0.9M images) – Can benefit GPU performance • Trained models are converted to Float16 – 32bit float -> 16bit float – 50% smaller file size
- 56. CPU GPU # of jobs Throughput (images/s) GPU Time /epoch (s) RelaWve perf. (virtual) Core i7-4790S Typ. 3.2GHz NA 1 3,928 234.5s 1X Core i7-4790S Typ. 3.2GHz GeForce GTX760 1 10,371 39.2s 5.9X Core i7-4790S Typ. 3.2GHz GeForce GTX760 4 (Cl) 15,988 25.8s (103 / 4) 9.0X Core i7-4790S Typ. 3.2GHz GeForce GTX 1080 4 (Cl) 30,320 6.25s (25.0s / 4) 37.5X
- 57. 57
- 58. CPU GPU # of jobs Throughput (images/s) GPU Time /epoch (s) RelaWve perf. (virtual) Core i7-4790S Typ. 3.2GHz NA 1 3,928 234.5s 1X Core i7-4790S Typ. 3.2GHz GeForce GTX760 1 10,371 39.2s 5.9X Core i7-4790S Typ. 3.2GHz GeForce GTX760 4 (Cl) 15,988 25.8s (103 / 4) 9.0X Core i7-4790S Typ. 3.2GHz GeForce GTX 1080 4 (Cl) 30,320 6.25s (25.0s / 4) 37.5X 2X E5-2630L v3 Typ. 1.80GHz Tesla P100 4 (Cl) 65,811 2.5s (10.3/4) 93.8X Special Thanks to Typical Linux Box
- 59. Pre-training (Parameter tuning) – Tested various layer configura6on(# of units), balanced model file size and its accuracy – 4 concurrent jobs – Started 03:08am 〜 6:28pm （15h20m） Training – 550 epochs / 24hrs Includes disk IO, cross valida6on test 6me – Adopted the 617th epoch for the first NN-based classifier. Took Approx. 48 GPU Hours using 21GB dataset Special Thanks to
- 60. KNN (Original) Complex NN The new NN Accuracy Low, on certain users 99.99+% 99.99+% Data size 20MB 400MB (AlexNet) 100MB (GoogleNet) 14MB (Float32) 7MB (Float16) Classifica6on 6me @ IvyBridge 2GHz (very fast) ~300ms ~20ms • Improved accuracy of weapon classifica6on • Faster than complex neural network models • Smaller data size makes distribu6on easy Special Thanks to
- 61. • IkaLog includes own neural network implementa6on – Deep Learning Frameworks break Windows version – Re-implemented propaga6on func6on • To make the code simple, the model is kept simple • Compa6ble with LinearFunc6on, and ReLU in Chainer hCps://github.com/hasegaw/IkaLog/commit/3238b67749334a3c4254aa6f25c005f83e210895 – Single run takes 20ms @ IvyBridge 2.0GHz 200ms @ PYNQ-Z1 FPGA board (Cortex-A9 650MHz)
- 62. • Apply netural network-based approach to other classifiers – Gear ability image classifier – Rainmaker touchdown detec6on – Use RNN for “Tide of the baCle” evalua6on func6on • Make the dataset smaller – The dataset (21GB) is obviously redundant – Use PCA-based anomaly detec6on to generate more smaller data?
- 64. • Several marchine-learning approach applied in IkaLog – K Nearest Neighbor for real-6me classifica6on – Neural network for more accurate classifica6on • Auto feature extrac6on, training from 0.9M samples • IkaLog and Deep Learning – Most of Deep Learning framework breaks Py2EXE – Easy to re-implement of propaga6on if the network is simple – Can u6lize exis6ng DNN frameworks and GPUs for training process
- 66. © 07strikers