Mongo DB in gaming industry

MongoDB in KIXEYE Analytics
Ben Goswami

Game Play
BP Video clips ( http://www.youtube.com/watch?v=uUHzppVn8WM)

©2011 Kixeye, Private and Confidential

Agenda

• Data scaling problems at KIXEYE analytics and
what technologies we use
• Example application: Cheater Detection
Engine
• How MongoDB helped us to solve it


Game Play
BP Attack Video (https://www.youtube.com/watch?v=qud_-18D2MU)


Game Play and Data
• Games generate lots of click data

• Click = event
o time of the event
o what happened before or after
o how the events are correlated


Attack: {
attacker : {
user:123
player_level: 31
fleet:
{
1:{
Weapons: {
name:“Ripper Cannon IV”
damage:306
accuracy:55
}
armors:{
}
…
}
2: {
}
_id" : ObjectId("506e23001d4524122c00004b"), "ip" : "10.54.72.206", "dt" :{ "_id" :
ObjectId("506e23001d4524dd2b0000bd"), "ip" : "10.54.72.206",
}
"&attacker_level=42&defender=0&defender_ext=salvage&defender_level=25&attack_type=PVE&attacking_
}
units=%7b%224071%22%3a%5b5%2c3%2c0%5d%7d&defending_units=%7b%221040%22%3a%5b4%2c
defender: {
0%2c4%5d%2c%221060%22%3a%5b1%2c0%2c1%5d%7d&attacking_shipconfig=%7b%22ships%22%3a
}
%5b%7b%22armors%22%3a%5b333%2c333%2c333%2c333%5d%2c%22weapons%22%3a%5b134%2c1
attack_time: 100354563
34%2c193%2c134%2c134%2c134%2c134%2c134%5d%2c%22specials%22%3a%5b542%2c561%2c571
attack_duration:290
%2c501%5d%2c%22hullID%22%3a4071%2c%22tacticalModules%22%3a%5b%5d%2c%22actives%22%3
sector:7
a%7b%22hp%22%3a9317.68457%2c%22rank%22%3a4%2c%22fltp%22%3a5%2c%22bid%22%3a54834
…
79%2c%22f%22%3a0%2c%22id%22%3a110%2c%22flt%22%3a5%2c%22vxp%22%3a10.858466%7d%7
d%2c%7b%22armors%22%3a%5b333%2c333%2c333%2c333.. "stage" : "new" }
}

How much data?

•Multi TBs / month (4-5x in next 6 months)

•Critical : Volume reads with speed

•< 1% data has actionable intelligence


MongoDB in KIXEYE

•Why:
– fast read + writes for high volume
– Document based architecture, schema less
– Indexing , auto failover, auto sharding
•Where:
– Games , KIXEYE Platform
– Analytics, Logging


Data technologies at Analytics

Technology Data Volume Query Type I/O speed

MySQL Small Adhoc + Read fast, only
< 300 GB Pre defined Aggregated data
Mongo Mid Predefined Read + Write speed
< 2 TB fast

Hadoop X-Large Adhoc Read slow
(Hive+HDFS) 2 TB+ writes: only insert
no update


An Example Application

Cheater Detection engine for games:
How mongoDB helped us


A simple backyard at start in BYM

Currency is time or $ or both


Sample record

• 4kb nested jason block
• base = {
base_name: “sweet_mango_db”
resources {
1: { name: "storage silo"
level: 5,
mode:“built"
time:123456902
2: { …
..}
...}
...} ..}

How people cheat?

• Each upgrade or repair
takes pre defined time

• if time+payment < predefined time = cheater
• victims chain create snowball effect


..and why they cheat...


Data volume for cheater detection

• 16Mil * 50 * 10 * 3
*....
• multiple time based
snapshots
• 0.01 % cheaters
• Detect < 5 mins
• That's searching
through lot of data
fast


.. and boss told

• Catch and auto
ban cheaters

• Don't label a good
guy a cheater


..team was ready for the challenge

• Speed+Accuracy of
application

• Technology evalauation


MongoDB fits the requirement

• read writes are fast because memory mapped
files compared to mysql+memcache or hbase
• Unstructured data => need schema less
• indexing on a nested data element


.. more on why mongo

• Auto purging thru TTL
• Auto sharding during traffic spikes
• Binary data support, fast uncompression
• Capped collection: writes faster


Development decision points

• Old map-reduce (prior to 2.2) vs New
Aggregation framework
• Global write locks removal in 2.2
• Lazy and Non lazy inserts


Result

• 100+ cheaters caught
per hour

• Better game balancing

• $$ saved


Wrap up : A good partnership

•Good partnership with KIXEYE and 10gen
•Consulting/Training and MMS service was very
helpful
•More mongoDB usages in future at KIXEYE


{Status: “The End”}

•{ next_action: “Q&A” }

• Like this jason? meet us outside – KIXEYE recruiting
{ Company : “KIXEYE”,
“Engineers/Total _employee” : “65%”,
Culture : [“Fun”, “Innovative”, “Passionate”]
Analytics_engg : { technology : “mongo, hadoop , mysql”,
tag : “big data, geek” }
}


Usage of data

• Data mining and aggregation to
o Optimize User Acquisition
o Optimize game analyzing game play
behavior and
Algo +
Data Optimize
r

Feedback

Mongo - pain points

• Diskspace handling
• No sql syntax support
• Capped collection - no deletes allowed
• No good client GUI based tool


Data related to player

• building has 5-10 levels
• building can have 3 states [built, damaged, repair in progress)
• an average base can have 50+ buildings
• 16 million such bases
• 45k people at any second [i.e base gets updated]
• Each level up or repair has time delay. People pay to reduce delay


Map-reduce in cheater detection

function mapf()
{
emit(this.userid, {userid:this.userid, total_time:this.length, bldg_id:1} ;
}

function reducef(key, values)
{
var r = {userid:key, total_time:0, count:0, avg_time:0};
values.forEach(function(v)
{r.total_time += v.total_time; }); return r;
}

function finalizef(key, value) { ... }


The End

•Q&A

•Love numbers and big data? We have a job for
you at analytics engg at KIXEYE. Talk to our
recruiting team


To scale data handling

• Write speed for large volume data
• Read speed + volume [aggregate+find]
• MongoDB provides good balance compared to other sql+nosql solutions
for certain use cases


Games+Platforms
devices

mysql mongo BI Mongo Cluster

ETL

non Real time >= 1 hr real time 5
API
mins

mySQL Mongo Hadoop mySQL
/mongo

Admin console
Data access tools

Dashboards Hive sql query API Scheduler,

Analytics Components - Birds' Eye View

Mongo DB in gaming industry

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (7)

Similar to Mongo DB in gaming industry

Similar to Mongo DB in gaming industry (20)

More from Dmitry Makarchuk

More from Dmitry Makarchuk (11)

Recently uploaded

Recently uploaded (20)

Mongo DB in gaming industry

Editor's Notes