Submit Search
Upload
2014 11-20 adtechxscala-meetup
•
2 likes
•
4,525 views
欣弘 野村
Follow
2014 11-20 に催された「アドテク×Scala meetup」で発表したスライドです。 http://connpass.com/event/8384/
Read less
Read more
Engineering
Report
Share
Report
Share
1 of 19
Download now
Download to read offline
Recommended
軽量言語メインの 文系エンジニアだった自分が Scalaのシステム開発に携わることになった経緯 @shigemk2
軽量言語メインの 文系エンジニアだった自分が Scalaのシステム開発に携わることになった経緯 @shigemk2
Michihito Shigemura
アドテク×Scala×パフォーマンスチューニング
アドテク×Scala×パフォーマンスチューニング
Yosuke Mizutani
アドテク×Scala @Dynalyst
アドテク×Scala @Dynalyst
Sangwon Han
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
Recommended
軽量言語メインの 文系エンジニアだった自分が Scalaのシステム開発に携わることになった経緯 @shigemk2
軽量言語メインの 文系エンジニアだった自分が Scalaのシステム開発に携わることになった経緯 @shigemk2
Michihito Shigemura
アドテク×Scala×パフォーマンスチューニング
アドテク×Scala×パフォーマンスチューニング
Yosuke Mizutani
アドテク×Scala @Dynalyst
アドテク×Scala @Dynalyst
Sangwon Han
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
Skeleton Culture Code
Skeleton Culture Code
Skeleton Technologies
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
How to have difficult conversations
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
Introduction to Data Science
Introduction to Data Science
Christy Abraham Joy
Time Management & Productivity - Best Practices
Time Management & Productivity - Best Practices
Vit Horky
The six step guide to practical project management
The six step guide to practical project management
MindGenius
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
RachelPearson36
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Applitools
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
GetSmarter
ChatGPT webinar slides
ChatGPT webinar slides
Alireza Esmikhani
More than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike Routes
Project for Public Spaces & National Center for Biking and Walking
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
DevGAMM Conference
More Related Content
Featured
Skeleton Culture Code
Skeleton Culture Code
Skeleton Technologies
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
How to have difficult conversations
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
Introduction to Data Science
Introduction to Data Science
Christy Abraham Joy
Time Management & Productivity - Best Practices
Time Management & Productivity - Best Practices
Vit Horky
The six step guide to practical project management
The six step guide to practical project management
MindGenius
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
RachelPearson36
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Applitools
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
GetSmarter
ChatGPT webinar slides
ChatGPT webinar slides
Alireza Esmikhani
More than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike Routes
Project for Public Spaces & National Center for Biking and Walking
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
DevGAMM Conference
Featured
(20)
Skeleton Culture Code
Skeleton Culture Code
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Getting into the tech field. what next
Getting into the tech field. what next
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
How to have difficult conversations
How to have difficult conversations
Introduction to Data Science
Introduction to Data Science
Time Management & Productivity - Best Practices
Time Management & Productivity - Best Practices
The six step guide to practical project management
The six step guide to practical project management
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
ChatGPT webinar slides
ChatGPT webinar slides
More than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike Routes
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
2014 11-20 adtechxscala-meetup
1.
アドテクを支えるScala Ad Generationの場合
2014-11-20 アドテク×Scala meetup
2.
自己紹介 • 野村 欣弘(のむら よしひろ)
• 株式会社mediba(株式会社ScaleOutに出向中) • scala歴:1年半ぐらい • ネット広告業界歴:2年半ぐらい • 主にAd Generationの開発を担当
3.
アジェンダ • medibaでscalaを採用した理由
• Ad Generationでどのようにscalaが使われて いるか? • sprayを使ったrtb-engineについて • sparkを使ったrtb-engine-logについて • scalaをつかってみて
4.
scala採用した理由 • githubを見るに、2011年ぐらい
から使い始める • medibaadでは配信サーバでPlay、 集計でspark
5.
Ad Generationについて ScaleOut
DSP Ad Generation ScaleOut Ad Platform ScaleOut DMP (SSP) • ScaleOut Ad Platform上に構築されたSSP • スマートフォン(web/app)に注力 • 来月からScaleOutに譲渡されます • RTB関連の処理をscalaで実装
6.
どのようにscalaが使われるのか ScaleOut Ad
Platform adserver RTBサーバ rtb-enigne DSP fluentd 集計サーバ HDFS mesos rtb-enigne-log DSP DSP SDK 枠 データ
7.
どのようにscalaが使われてるのか • rtb-engine
• DSPからの入札をオークションするサー バです。 • rtb-engine-log • アクセス数、DSP通信、オークション状 況(入札、落札)の集計を行います。
8.
rtb-engine • spray1.2.1
• アドサーバと枠データ、オークション結果をやり取 りする • DSPとの入札、応札をやり取りをする • DSPからのレスポンスをまって、オークションする • 集計画面(angularJS)用のAPIも
9.
rtb-engine • 現状10台
• 70億オークション/月 • 250億bid リクエスト/月 • 70億 x 3-4DSP • ピーク時 対アドサーバ : 4,900qps、対DSP : 20,000qps
10.
rtb-engine ピーク時にさばけない ->
configの見直し -> コードの見直し
11.
rtb-engine • configの見直し
• akkaの設定とsprayの設定 • 似たような設定項目が複数ある • APIレイヤーが細かい
12.
rtb-engine • コードの見直し
• ブロックする箇所をFutureを使う ように • 用途ごとにスレッドプールを割り当 てた
13.
rtb-engine-log • spark1.0.0
• mesos上で動かしてる • 7種類のログを集計 • 10分ごと、1時間ごと、1日ごとに集計 • 入力はHDFS、出力はDB
14.
rtb-engine-log
15.
rtb-engine-log • 6cpu,20GB/台
x 4台 • master:2台, slave:4台 • 500M - 1.5G x 7 x 10 / hour • 12 - 40s/job
16.
rtb-engine-log • 集計自体よりI/Oがネック(主にDB)
• ほどよいサイズで • 手軽に試せるし、身構えず書ける • spark-shellが試しやすい • コレクションを操作するのと変わらず書 ける
17.
scalaを使ってみて • 並列処理、並行処理を扱いやすい
• アドサーバやRTBサーバのようなものには向いてい る • 用途を選ばない言語 • 何でもそつなくこなせる • 配信も集計も1つの言語で • やっぱりJava、JVM知ってた方がいい
18.
今後 • チューニング
• この会で勉強しようと思います。 • 売り上げ向上につながる、柔軟に速 やかな分析 • spark SQL、chronosとかを調査中
19.
おわり ScaleOutでは各種エンジニアを募集中です。 •
大規模配信を行いたい方 • 大規模データの解析を行いたい方 • 広告システム,ログ解析システムは絶対面白い!と思う方 • Webアプリ開発大好きでデザパタ厨な方 • Pのつく言語に携わるのに少し疲れた方 • 弊社に興味を持たれた方 http://www.scaleout.jp/about/job-opportunities/
Download now