Big Data Analytics on AWS

•

4 likes•1,663 views

In this session, you will learn how to designed clickstream analytics application and how you can use the same architecture to build your own and be ready to handle the changing world of clickstream data. Dive into how to perform advanced user retention and cohort analysis to make near–real time product and marketing decisions. Learn how to build infrastructure that is fast, easy, and cost-effective with AWS resources such as Amazon Kinesis, Spark on Amazon EMR, Amazon S3, Amazon Redshift, and Amazon Elasticsearch.

Technology

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Dickson Yue, Solutions Architect
17 June 2016
Big Data Analytics on AWS
Digital Innovation & e-Commerce Track

Data Answers
START HERE
WITH A BUSINESS CASE
Revenue Lift
Market acquisition
Product recommendation
Improve user experience
Operation intelligence

Data Answers
Time to Answer (Latency)
Throughput
Cost
Ingest/
Collect
Consume/
visualizeStore
Process/
analyze
1 4
0 9
5

Data Answers
Ingest/
Collect
Consume/
visualizeStore
Process/
analyze
1 4
0 9
5
Amazon S3
Amazon Kinesis
Amazon DynamoDB
Amazon RDS
Amazon EMR
Amazon Redshift
Amazon Machine
Learning
Storage Processing Visualize
ElasticSearch service
QuickSight
ElastiCache

Answer
•  User retention
•  High spending customer
navigation pattern
•  Product recommendation
•  User journey in the shop
•  UX improvement
•  What deal/ad to try
next
Use case
Data source
•  Page
•  Click event
•  Web log
•  Thing event

JavaScript
(Snowplow)
AWS SDK
logstach
Fluentd
Ingest Store
@ 30km/s a.k.a 300 rps
HTTP Post
Amazon
S3
Storage

@ 100km/s
Ingest Store
JavaScript
(Snowplow)
AWS SDK
LOG4J
Flume
Fluentd
HTTP Post
Amazon
Kinesis
Firehose
API Server
Streaming
Buﬀer
24hrs-7days
Web
Servers
Amazon
S3
Storage
Data lake

@ 100km/s
Ingest Store
JavaScript
(Snowplow)
AWS SDK
LOG4J
Flume
Fluentd
HTTP Post
Amazon
S3
Amazon
Kinesis
Firehose
API Gateway
API Server
Streaming
Buﬀer
24hrs-7days
Storage
Data lake

Amazon
S3
Storage
Data lake
Store Process/Analyze
EMR
Redshift
Redshift
EMR
ETL
Visualize
JDBC
ODBC
JDBC
ODBC
QuickSight

Amazon S3
Store Process
EMR
Visualize
JDBC
ODBC
Redshift
Basket
CRM ERP
DBs
Log ﬁle
QuickSight

Day-14 retention over time
User retention and growth
N-day retention

Logstash
AWS SDK
Ingest Store
Bot AWS SDK
App
Crawlers
AWS SDK
Amazon
Kinesis
Firehose
Store
Amazon S3
Data Lake
ElasticSearch
Last 120mins
Analysts
AWS SDK

Why do we need machine learning for this?
The social media stream is high-volume, and most of the
messages are not CS-actionable

Logstash
AWS SDK
Ingest Store
Bot AWS SDK
App
Crawlers
AWS SDK
Amazon
Kinesis
Process
Amazon
Lambda
Analyze
AWS SDK
Machine
learning
Notiﬁcation
Action
Support
issue
Database
Feature
request
Keep training the ML model with new data
Action
Amazon S3

AWS SDK
Ingest Store
Bot AWS SDK
Messenger
Amazon
Kinesis
Process
Amazon
Lambda
Analysts
Machine
learning
Action
Bot
App
Get prediction
Keep training the ML model with new data
Amazon S3

OI from Business view
with custom source

Refrigerator
POS
Door sensor
Water
Camera
Storefront
Kitchen
Lambda
SQS
AWS IoT
SQSPoller
Http Event Collector
Serverless
Architecture

Our Big Data Scale
Total ~25 PB DW on Amazon S3
Read ~10% DW daily
Write ~10% of read data daily
~ 550 billion events daily
~ 350 active platform users

predict what you want to watch before you watch it.

Netflix Prize - best collaborative filtering algorithm

Data Answers
Ingest/
Collect
Consume/
visualizeStore
Process/
analyze
1 4
0 9
5
START
WITH A BUSINESS CASE
MATCH
AVAILABLE DATA
CHOOSE
BEST FIT
Amazon S3
Amazon Kinesis
Amazon DynamoDB
Amazon EMR
Amazon Redshift
Amazon Machine
Learning
Storage Processing Visualize
QuickSight

Source DBs
3rd Party Data
Log Data
Reporting
Analysis
Processing
Data Lake
S3
Source of truth

CRM ERP
DBs
Log ﬁle
AWStats
days
MB
2002
Big bang

<2005
Hello world
Page/Event
tracking
GA
hours
GB

SOLOMO
minutes - hours
TB
<2008
New customer service
New System monitoring
New QA

IoT
O2O
seconds – hours
PB
2016
Fast and big
data driven marketing

Analytics
ETL
Interactive data exploration
Interactive slice & dice
RT analytics & iterative/ML algo and more ...
Different Big Data Processing Needs

What's hot

Industry 4.0: come i servizi IoT e Big Data di AWS rendono Smart il Manufactu...

Amazon Web Services

Intro Presentation at AWS AWSome Day Dublin July 2015

Ian Massingham

Up-front design of your AWS account can be done in a way that creates a reliably secure and controlled environment no matter how the AWS resources are used. This session will focus on "Secure by Design" principles and show how an AWS environment can be configured to provide a reliable operational security control capability to meet the compliance needs across multiple industry verticals (e.g. HIPAA, FISMA, PCI, etc.).

Compliance in the Cloud Using “Security by Design” Principles

Amazon Web Services

Mobile app development is complex and time-consuming. Learn how to accelerate the mobile apps development journey with AWS Mobile Hub & Device Farm. We will demonstrate how AWS Mobile Hub abstracts the undifferentiated heavy lifting by providing a single, integrated experience for discovering, provisioning and configuring AWS cloud resources you need to build, test, and monitor usage of your mobile apps.

Accelerate Your Mobile Development Journey with AWS

Amazon Web Services

Often times, developers and auditors can be at odds. The agile, fast-moving environments that developers enjoy will typically give auditors heartburn. The more controlled and stable environments that auditors prefer to demonstrate and maintain compliance are traditionally not friendly to developers or innovation. In this session, we will look at the intersection where developers and auditors meet by discussing techniques, tools, and concepts that appeal to both. Topics covered will include shared responsibility, using compartmentalization and micro-services for scope control, immutable infrastructure, and continuous security testing.

Keeping Developers and Auditors Happy in the Cloud

Amazon Web Services

Innovation Track AWS Cloud Experience Argentina - Data Lakes & Analytics en AWS

Amazon Web Services LATAM

AWS has a large and growing portfolio of big data management and analytics services, designed to be integrated into solution architectures that meet the needs of your business. In this session, we look at analytics through the eyes of a business intelligence analyst, a data scientist, and an application developer, and we explore how to quickly leverage Amazon Redshift, Amazon QuickSight, RStudio, and Amazon Machine Learning to create powerful, yet straightforward, business solutions. Speaker: Paul Armstrong, Solutions Architect, Amazon Web Services

Analytics on AWS with Amazon Redshift, Amazon QuickSight, and Amazon Machine ...

Amazon Web Services

Amazon QuickSight is a fast BI service that makes it easy for you to build visualizations, perform ad-hoc analysis, and quickly get business insights from your data. QuickSight is built to harness the power and scalability of the cloud, so you can easily run analysis on large datasets, and support hundreds of thousands of users. In this session, we’ll demonstrate how you can easily get started with Amazon QuickSight, uploading files, connecting to S3 and Redshift, and creating analyses from visualizations that are optimized based on the underlying data. Once we’ve built our analysis and dashboard, we’ll show you easy it is to share it with colleagues and stakeholders in just a few seconds.

Visualizing Big Data Insights with Amazon QuickSight

Amazon Web Services

Amazon QuickSight is a fast BI service that makes it easy for you to build visualizations, perform ad-hoc analysis, and quickly get business insights from your data. QuickSight is built to harness the power and scalability of the cloud, so you can easily run analysis on large datasets, and support hundreds of thousands of users. In this session, we’ll demonstrate how you can easily get started with Amazon QuickSight, uploading files, connecting to S3 and Redshift and creating analyses from visualizations that are optimized based on the underlying data. Once we’ve built our analysis and dashboard, we’ll show you easy it is to share it with colleagues and stakeholders in just a few seconds. And with SPICE – QuickSight’s in-memory calculation engine – you can go from data to insights, faster than ever.

Visualizing Big Data Insights with Amazon QuickSight

Amazon Web Services

AWS Cloud School Introductory Presentation

Ian Massingham

Customer Sharing: LnData

Amazon Web Services

Keynote Roberto Delamora - AWS Cloud Experience Argentina

Amazon Web Services LATAM

Trova ed utilizza in modo sicuro nel Cloud il software che ti serve con l'AWS...

Amazon Web Services

AWSome Day Dublin Intro & Closing Slides

Ian Massingham

Amazon Machine Learning 採用的機器學習技術與 Amazon 內部資料科學家社群多年來使用的技術相同，該技術經過驗證且可高度擴展。此服務使用強大的演算法，透過找出現有資料的模式來建立機器學習模型。然後，Amazon Machine Learning 使用這些模型來處理新資料並為應用程式產生預測結果。這一個場次中，除了為各位介紹Amazon Machine Learning服務之外，也準備了一個推薦引擎的範例展示，讓各位瞭解Amazon Machine Learning在實務上的運用方式。

使用Amazon Machine Learning 建立即時推薦引擎

Amazon Web Services

AWSome Day Manchester 2105 - Intro/Close

Ian Massingham

AWS Partner Techshift - Expanding Your Software Business To The Global Market...

Amazon Web Services Korea

AWS Summit Kuala Lumpur - Opening Keynote by Dr. Werner Vogels

Amazon Web Services

Transformation Track AWS Cloud Experience Argentina - Bases de Datos en AWS

Amazon Web Services LATAM

Big Data Analytics on AWS for Digital Marketing

Amazon Web Services

What's hot (20)

Industry 4.0: come i servizi IoT e Big Data di AWS rendono Smart il Manufactu...

Intro Presentation at AWS AWSome Day Dublin July 2015

Compliance in the Cloud Using “Security by Design” Principles

Accelerate Your Mobile Development Journey with AWS

Keeping Developers and Auditors Happy in the Cloud

Innovation Track AWS Cloud Experience Argentina - Data Lakes & Analytics en AWS

Analytics on AWS with Amazon Redshift, Amazon QuickSight, and Amazon Machine ...

Visualizing Big Data Insights with Amazon QuickSight

AWS Cloud School Introductory Presentation

Customer Sharing: LnData

Keynote Roberto Delamora - AWS Cloud Experience Argentina

Trova ed utilizza in modo sicuro nel Cloud il software che ti serve con l'AWS...

AWSome Day Dublin Intro & Closing Slides

使用Amazon Machine Learning 建立即時推薦引擎

AWSome Day Manchester 2105 - Intro/Close

AWS Partner Techshift - Expanding Your Software Business To The Global Market...

AWS Summit Kuala Lumpur - Opening Keynote by Dr. Werner Vogels

Transformation Track AWS Cloud Experience Argentina - Bases de Datos en AWS

Big Data Analytics on AWS for Digital Marketing

Viewers also liked

Managing Online Relationships

Heidi Miller

Learn how Amazon Redshift, our fully managed, petabyte-scale data warehouse, can help you quickly and cost-effectively analyze all of your data using your existing business intelligence tools. Get an introduction to how Amazon Redshift uses massively parallel processing, scale-out architecture, and columnar direct-attached storage to minimize I/O time and maximize performance. Learn how you can gain deeper business insights and save money and time by migrating to Amazon Redshift. Take away strategies for migrating from on-premises data warehousing solutions, tuning schema and queries, and utilizing third party solutions.

Getting Started with Amazon Redshift

Amazon Web Services

Secure your critical workload on AWS

Amazon Web Services

Getting Started with Amazon Aurora

Amazon Web Services

The flexibility and scale of the AWS Cloud and the emergence of DevOps have combined to allow developers to build and deploy applications faster than ever before. Assessing these applications for security risks without slowing down the development process can be a challenge with traditional security assessment tools designed for on-premises infrastructure. Amazon Inspector, an automated security assessment service, addresses this by integrating assessments directly into the development process of environments running on Amazon Elastic Compute Cloud (Amazon EC2). In this session, we will review Amazon Inspector security assessments findings and how best to interpret and take action on them as a seamless part of your DevOps lifecycle. We will review the findings structure for rules packages and run through demos on how to make them findings actionable by injecting them into your operations pipeline through automatic submissions into ticketing and change management systems or directly executing remediation activities. Learning Objectives: • Overview of Amazon Inspector assessment findings and how to interpret these • Learn how to utilize AWS services to automate ticketing and change management submission for findings • Automate remediation based on assessment findings Who Should Attend: • Security Professional, Developers, QA Engineers, Security Assessment / Certification Engineers, System Administrators

Addressing Amazon Inspector Assessment Findings - September 2016 Webinar Series

Amazon Web Services

Data sience coffe

Noboru Kano

MUORO Analytics- Data Science Tool

Vyom Bhardwaj

A Business Case Study of Producing Nano Particles for Anti-Counterfeit

Joseph Man

Future Trends in FSI

Amazon Web Services

Big Data: Getting Smarter

Charles Mok

Open Data AT Chatbot FAQ

Svitlana Vakulenko

Spark Summit East 2016, New York City: Talk by Josef Adersberger (@adersberger, CTO at QAware). Abstract: Users leave thousands of traces per second on a successful ecommerce site. It’s very pragmatic to analyse and react on this trace event stream in realtime. This is called clickstream analysis. In the talk I’ll present a software architecture based on Apache Spark which is able to process thousands of clickstream events per second. A product based on this architecture is in production since mid 2015 and is still performing well. The building blocks of the architecture beside Spark are Kafka to handle the inbound event stream, Spark Streaming for initial stream processing and Parquet as serialization format. I argue why we’ve chosen these technologies and what experiences we had in developing, launching and operating the product.

Clickstream Analysis with Spark - Understanding Visitors in Real Time

QAware GmbH

Big Data Analytics with AWS and AWS Marketplace Webinar

Amazon Web Services

United Creations UI/UX Presentation

United Creations

Indonesia's text message-based virtual assistant company YesBoss has introduced a new platform called Kata.ai to bridge conversations between brands and customers using an artificial intelligence-based auto response system. The new platform will be able to process informal conversations on popular messaging apps such as Line or other social media platforms, and connect consumers who want to complain, chat or buy certain products with the official accounts of corresponding companies. The technology is reportedly more cost-efficient for brands, as it allows them to cut staff budget and offers an alternative approach to understand Indonesian consumers' behaviour. For customers, the system might end the frustration of having to rely on outdated lists of frequently asked questions as they can now get instant feedback from the companies. Kata.ai recently recruited a new tech adviser, Jim Geovedi, an Indonesian IT security expert based in the United Kingdom. Its holding company, YesBoss, has also recently appointed a new chief technology officer, Ahmad Rizqi Meydiarso, previously a system engineer at French aircraft manufacturer Airbus. YesBoss' clients have included tech giant Microsoft, Telkom's telephone directory publisher subsidiary Infomedia Nusantara, and aCommerce — the local arm of a Thai online shopping site.

Kata.ai - Artificial Intelligence in Indonesia through conversational chatbot...

Chris Franke

With the rise of ecommerce and ease of global shipping, shoppers cross physical and geographical boundaries for the best deals during seasonal promotion like Black Friday in US, Singles Day in China. With these new challenges, running a ecommerce platform providing great user experience for worldwide customer become the job zero for all the sellers. In this session, we will cover best practices for being able to handle those traffic spikes. We will also share experiences gained from Amazon.com and one of the biggest "flash sale" case. We also have customer sharing on the managing a multi-tenant ecommerce platform at scale.

Ignite eCommerce growth with AWS

Amazon Web Services

Agencies with FISMA High workloads have struggled to take advantage of the cost savings and flexibility of true cloud offerings. Now you can address your most stringent regulatory compliance requirements with AWS GovCloud (US). AWS GovCloud (US)* has received a Provisional Authority to Operate (P-ATO) from the Joint Authorization Board (JAB) under the Federal Risk and Authorization Management Program (FedRAMP) High baseline

FedRAMP High & AWS GovCloud (US): FISMA High Requirements

Amazon Web Services

Getting Started With AWS Security

Amazon Web Services

AWS provides many services to assist customers with their journey to the cloud. Hybrid solutions offer customers a way to continue leveraging existing investments on-premises, while expanding their footprint into the public cloud. This session covers the different technologies available to support hybrid architectures on AWS. We discuss common patterns and anti-patterns for solving enterprise workloads across a hybrid environment.

AWS re:Invent 2016: Hybrid Architectures: Bridging the Gap to the Cloud( ARC2...

Amazon Web Services

the Elemental Cloud provides OTT video delivery to a number of large scale customers. In this presentation, Vance Cook, senior solutions architect, presents us a number of customer user cases, the Software Defined Video solution, and explains how existing infrastructures can be moved to the cloud. The gains in scale, cost reduction, flexibility are also highlighted. This presentation was delivered at the Media and Entertainment Symposium at AWS Toronto.

ACCELERATING OTT DELIVERY AND MODERNIZING MEDIA LOGISTICS WITH CLOUD BASED VI...

Amazon Web Services

Viewers also liked (20)

Managing Online Relationships

Getting Started with Amazon Redshift

Secure your critical workload on AWS

Getting Started with Amazon Aurora

Addressing Amazon Inspector Assessment Findings - September 2016 Webinar Series

Data sience coffe

MUORO Analytics- Data Science Tool

A Business Case Study of Producing Nano Particles for Anti-Counterfeit

Future Trends in FSI

Big Data: Getting Smarter

Open Data AT Chatbot FAQ

Clickstream Analysis with Spark - Understanding Visitors in Real Time

Big Data Analytics with AWS and AWS Marketplace Webinar

United Creations UI/UX Presentation

Kata.ai - Artificial Intelligence in Indonesia through conversational chatbot...

Ignite eCommerce growth with AWS

FedRAMP High & AWS GovCloud (US): FISMA High Requirements

Getting Started With AWS Security

AWS re:Invent 2016: Hybrid Architectures: Bridging the Gap to the Cloud( ARC2...

ACCELERATING OTT DELIVERY AND MODERNIZING MEDIA LOGISTICS WITH CLOUD BASED VI...

Similar to Big Data Analytics on AWS

Your customers probably want a better experience with your brand. Your different business teams want and need better insights in their decision making. Almost certainly, your finance and operations teams require this to happen at a fraction of the cost of traditional on-premises options. Modern data architectures on AWS help many of our best customers realize all of those goals. Your business data contains critical information about customer behaviors, operational decisions, and many factors that have financial impact on your organization. Increasingly, this data sits beyond your transactional systems, and is too big, too fast, and too complex for existing systems to handle. AWS Data and Analytics services are designed from our customers' requirements to ingest, store, analyze, and consume information at record-breaking scale. In this session you will learn how these services work together to deliver business automation, enhance customer engagement and intelligence.

Driving Business Insights with a Modern Data Architecture AWS Summit SG 2017

Amazon Web Services

Your business data contains critical information about customer behaviors, operational decisions, and many factors that have financial impact on your organisation. Increasingly though, this data is too big, too fast, and too complex for existing systems to handle. AWS Data and Analytics services are designed to ingest, store, analyse, and consume information at record-breaking scale. In this session you will learn how these services work together to deliver business automation, enhance customer engagement and intelligence. Speaker: Craig Stires, APAC Business Development - Big Data & Analytics, Amazon Web Services

Driving Business Outcomes with a Modern Data Architecture - Level 100

Amazon Web Services

Using AWS to design and build your data architecture has never been easier to gain insights and uncover new opportunities to scale and grow your business. Join this workshop to learn how you can gain insights at scale with the right big data applications. Building a real-time analytics solution has never been faster or more cost-efficient. Most organizations are trying to find a way to improve customer experience and respond to business events in real time. Importantly, to do this quickly and at a fraction of the price of traditional approaches. In this session we will look at how to use the AWS services to best meet your real-time analytics needs.

Modern Data Architectures for Business Insights at Scale

Amazon Web Services

Data Lake allows an organisation to store all of their data, structured and unstructured, in one, centralised repository. Since data can be stored as-is, there is no need to convert it to a predefined schema and you no longer need to know what questions you want to ask of your data beforehand. In this session we will explore the architecture of a Data Lake on AWS and cover topics such as storage, processing and security.

Building your Datalake on AWS

Amazon Web Services

Learn more about the tools, techniques and technologies for working productively with data at any scale. This session will introduce the family of data analytics tools on AWS which you can use to collect, compute and collaborate around data, from gigabytes to petabytes. We'll discuss Amazon Elastic MapReduce, Hadoop, structured and unstructured data, and the EC2 instance types which enable high performance analytics.

AWS Summit 2013 | Singapore - Big Data Analytics, Presented by AWS, Intel and...

Amazon Web Services

Batch querying and reporting is no longer enough for many organizations. Reducing time to insight – the time it takes to turn data into actionable insights – is becoming increasingly important to remain competitive. That’s why organizations are quickly evolving their data applications to support a broader set of real-time analytic use cases. In this webinar, we will review some of the common use cases for real-time analytics such as click-stream analysis, event data processing, and real-time analytics. We will show proven architectures for collecting, storing, and processing real-time data using a combination of AWS managed services, including Amazon Kinesis Streams, Amazon Kinesis Firehose, Amazon EMR, and AWS Lambda, as well open source tools, such as Apache Spark. Then, we will discuss common approaches and best practices to incorporate real-time analytics into your existing batch applications. Learning Objectives: • Understand how to incorporate real-time analytics into existing applications • Best practices to combine batch with real-time data flows • Learn common architectures and use cases for real-time analytics

Evolving Your Big Data Use Cases from Batch to Real-Time - AWS May 2016 Webi...

Amazon Web Services

Business intelligence is often described as a set of methodologies and technologies that transform raw data into meaningful and useful information for business purposes. But this simple description hides many technical challenges IT teams struggle with. This session will show how to build business intelligence applications leveraging AWS, from the raw data import, consumption and storage down to the information production. We will also cover best practices for services such as Amazon Redshift or Amazon RDS, and how to use applications such as SAP Hana, Jaspersoft and others.

AWS Summit Stockholm 2014 – B4 – Business intelligence on AWS

Amazon Web Services

AWS Dev Day Kyiv 2019 Track: Analytics & Machine Learning Session: "Building a Modern Data platform in the Cloud" Speaker: Alex Casalboni, AWS Technical Evangelist Level: 300 AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services. Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!  Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine Video: https://youtu.be/HIDnAG9AxZo

"Building a Modern Data platform in the Cloud", Alex Casalboni, AWS Dev Day K...

Provectus

Real-Time Streaming: Intro to Amazon Kinesis

Amazon Web Services

B3 - Business intelligence apps on aws

Amazon Web Services

Big Data and Analytics

Amazon Web Services

Level 200: Visualize Your Data in Data Lake with AWS Athena and AWS Quicksight Nowadays, enterprises are building Data Lake which store lots of structured and unstructured data for data analysis. But it takes lots of time for building the data modeling and infrastructure that is required. How to make quick data queries without servers and databases is the next big question for every enterprises. In this workshop, eCloudvalley, the first and only Premier Consulting Partner in GCR, will demonstrate how to use serverless architecture to visualize your data using Amazon Athena and Amazon Quicksight. You can easily query and visualize the data in your S3, and get business insights with the combination of these two services. Also, you can also build business reports with other tools such as AWS IoT, Amazon Kinesis Firehose. Reason to Attend: Learn how to quickly search for thousands of data on S3 via serverless Amazon's Athena Learn how to use AWS QuickSight to retrieve information from your database quickly and create detailed reports

Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...

Amazon Web Services

Big Data on AWS

Amazon Web Services

Getting Started with Amazon Kinesis

Amazon Web Services

Analysing Data in Real-time

Amazon Web Services

Intro Presentation at AWS AWSome Day London September 2015

Ian Massingham

A data lake is an architectural approach that allows you to store massive amounts of data into a central location, so it's readily available to be categorized, processed, analyzed and consumed by diverse groups within an organization. In this session, we will introduce the Data Lake concept and its implementation on AWS. We will explain the different roles our services play and how they fit into the Data Lake picture.

Big Data on AWS - To infinity and beyond! - Tel Aviv Summit 2018

Amazon Web Services

20141021 AWS Cloud Taekwon - Big Data on AWS

Amazon Web Services Korea

Amazon QuickSight is a fast, cloud-powered business intelligence (BI) service that makes it easy to build visualizations, perform ad-hoc analysis, and quickly get business insights from your data. In this session, we demonstrate how you can point Amazon QuickSight to AWS data stores, flat files, or other third-party data sources and begin visualizing your data in minutes. We also introduce you to SPICE - a Super-fast, Parallel, In-memory, Calculation Engine in Amazon QuickSight, which performs advanced calculations and render visualizations rapidly without requiring any additional infrastructure, SQL programming, or dimensional modeling, so you can seamlessly scale to hundreds of thousands of users and petabytes of data. Lastly, you will see how Amazon QuickSight provides you with smart visualizations and graphs that are optimized for your different data types, to ensure the most suitable and appropriate visualization to conduct your analysis, and how to share these visualization stories using the built-in collaboration tools.

Getting Started with Amazon QuickSight

Amazon Web Services

Getting Started with Amazon QuickSight

Amazon Web Services

Similar to Big Data Analytics on AWS (20)

Driving Business Insights with a Modern Data Architecture AWS Summit SG 2017

Driving Business Outcomes with a Modern Data Architecture - Level 100

Modern Data Architectures for Business Insights at Scale

Building your Datalake on AWS

AWS Summit 2013 | Singapore - Big Data Analytics, Presented by AWS, Intel and...

Evolving Your Big Data Use Cases from Batch to Real-Time - AWS May 2016 Webi...

AWS Summit Stockholm 2014 – B4 – Business intelligence on AWS

"Building a Modern Data platform in the Cloud", Alex Casalboni, AWS Dev Day K...

Real-Time Streaming: Intro to Amazon Kinesis

B3 - Business intelligence apps on aws

Big Data and Analytics

Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...

Big Data on AWS

Getting Started with Amazon Kinesis

Analysing Data in Real-time

Intro Presentation at AWS AWSome Day London September 2015

Big Data on AWS - To infinity and beyond! - Tel Aviv Summit 2018

20141021 AWS Cloud Taekwon - Big Data on AWS

Getting Started with Amazon QuickSight

More from Amazon Web Services

Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS. In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...

Amazon Web Services

La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup. Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti. Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.

Big Data per le Startup: come creare applicazioni Big Data in modalità Server...

Amazon Web Services

Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi

Esegui pod serverless con Amazon EKS e AWS Fargate

Amazon Web Services

Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.

Costruire Applicazioni Moderne con AWS

Amazon Web Services

L’utilizzo dei container è in continua crescita. Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili. I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!

Come spendere fino al 90% in meno con i container e le istanze spot

Amazon Web Services

In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th. Event Agenda : Open banking so far (short recap) • PSD2, OB UK, OB Australia, OB LATAM, OB Israel Intro to Open Finance marketplace • Scope • Features • Tech overview and Demo The role of the Cloud The Future of APIs • Complying with regulation • Monetizing data / APIs • Business models • Time to market One platform for all: a Strategic approach Q&A

Open banking as a service

Amazon Web Services

Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc. AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta. Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.

Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...

Amazon Web Services

Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity. AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet. Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.

OpsWorks Configuration Management: automatizza la gestione e i deployment del...

Amazon Web Services

Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.

Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads

Amazon Web Services

Computer Vision con AWS

Amazon Web Services

Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti. Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.

Database Oracle e VMware Cloud on AWS i miti da sfatare

Amazon Web Services

Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti. Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire. Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito. In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.

Crea la tua prima serverless ledger-based app con QLDB e NodeJS

Amazon Web Services

Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline. Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.

API moderne real-time per applicazioni mobili e web

Amazon Web Services

Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali. In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.

Database Oracle e VMware Cloud™ on AWS: i miti da sfatare

Amazon Web Services

Tools for building your MVP on AWS

Amazon Web Services

How to Build a Winning Pitch Deck

Amazon Web Services

Building a web application without servers

Amazon Web Services

Fundraising Essentials

Amazon Web Services

AWS_HK_StartupDay_Building Interactive websites while automating for efficien...

Amazon Web Services

Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.

Introduzione a Amazon Elastic Container Service

Amazon Web Services

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...

Big Data per le Startup: come creare applicazioni Big Data in modalità Server...

Esegui pod serverless con Amazon EKS e AWS Fargate

Costruire Applicazioni Moderne con AWS

Come spendere fino al 90% in meno con i container e le istanze spot

Open banking as a service

Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...

OpsWorks Configuration Management: automatizza la gestione e i deployment del...

Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads

Computer Vision con AWS

Database Oracle e VMware Cloud on AWS i miti da sfatare

Crea la tua prima serverless ledger-based app con QLDB e NodeJS

API moderne real-time per applicazioni mobili e web

Database Oracle e VMware Cloud™ on AWS: i miti da sfatare

Tools for building your MVP on AWS

How to Build a Winning Pitch Deck

Building a web application without servers

Fundraising Essentials

AWS_HK_StartupDay_Building Interactive websites while automating for efficien...

Introduzione a Amazon Elastic Container Service

Recently uploaded

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Scaling API-first – The story of a global engineering organization

Radu Cotescu

This presentation explores the impact of HTML injection attacks on web applications, detailing how attackers exploit vulnerabilities to inject malicious code into web pages. Learn about the potential consequences of such attacks and discover effective mitigation strategies to protect your web applications from HTML injection vulnerabilities. for more information visit https://bostoninstituteofanalytics.org/category/cyber-security-ethical-hacking/

HTML Injection Attacks: Impact and Mitigation Strategies

Boston Institute of Analytics

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc

A Domino Admins Adventures (Engage 2024)

Gabriella Davis

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

sammart93

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

💉💊+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHABI}}+971581248768 +971581248768 Mtp-Kit (500MG) Prices » Dubai [(+971581248768**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Maya Whatsapp +971581248768 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971581248768''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971581248768' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Cl

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

Partners Life - Insurer Innovation Award 2024

The Digital Insurer

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Drew Madelung

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

apidays

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

MINDCTI Revenue Release Quarter One 2024

MIND CTI

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

Recently uploaded (20)

Exploring the Future Potential of AI-Enabled Smartphone Processors

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

Scaling API-first – The story of a global engineering organization

HTML Injection Attacks: Impact and Mitigation Strategies

The 7 Things I Know About Cyber Security After 25 Years | April 2024

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

A Domino Admins Adventures (Engage 2024)

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

2024: Domino Containers - The Next Step. News from the Domino Container commu...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Partners Life - Insurer Innovation Award 2024

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Boost Fertility New Invention Ups Success Rates.pdf

MINDCTI Revenue Release Quarter One 2024

Automating Google Workspace (GWS) & more with Apps Script

Big Data Analytics on AWS

3. How to get started?

4. Data Answers START HERE WITH A BUSINESS CASE Revenue Lift Market acquisition Product recommendation Improve user experience Operation intelligence

5. Data Answers Time to Answer (Latency) Throughput Cost Ingest/ Collect Consume/ visualizeStore Process/ analyze 1 4 0 9 5

6. Data Answers Ingest/ Collect Consume/ visualizeStore Process/ analyze 1 4 0 9 5 Amazon S3 Amazon Kinesis Amazon DynamoDB Amazon RDS Amazon EMR Amazon Redshift Amazon Machine Learning Storage Processing Visualize ElasticSearch service QuickSight ElastiCache

7. Tracking Clickstream, user retention

8. Answer •  User retention •  High spending customer navigation pattern •  Product recommendation •  User journey in the shop •  UX improvement •  What deal/ad to try next Use case Data source •  Page •  Click event •  Web log •  Thing event

9. JavaScript (Snowplow) AWS SDK logstach Fluentd Ingest Store @ 30km/s a.k.a 300 rps HTTP Post Amazon S3 Storage

10. @ 100km/s Ingest Store JavaScript (Snowplow) AWS SDK LOG4J Flume Fluentd HTTP Post Amazon Kinesis Firehose API Server Streaming Buﬀer 24hrs-7days Web Servers Amazon S3 Storage Data lake

11. @ 100km/s Ingest Store JavaScript (Snowplow) AWS SDK LOG4J Flume Fluentd HTTP Post Amazon S3 Amazon Kinesis Firehose API Gateway API Server Streaming Buﬀer 24hrs-7days Storage Data lake

12. Amazon S3 Storage Data lake Store Process/Analyze EMR Redshift Redshift EMR ETL Visualize JDBC ODBC JDBC ODBC QuickSight

13. Amazon S3 Store Process EMR Visualize JDBC ODBC Redshift Basket CRM ERP DBs Log ﬁle QuickSight

14. Day-14 retention over time User retention and growth N-day retention

15. Social listening Social CRM, Chatbot

16. Answer Campaign performance Customer service automation Building Chatbot Use case Data Brand page activity Post #hashtag User profile

17. Logstash AWS SDK Ingest Store Bot AWS SDK App Crawlers AWS SDK Amazon Kinesis Firehose Store Amazon S3 Data Lake ElasticSearch Last 120mins Analysts AWS SDK

18.

19. Why do we need machine learning for this? The social media stream is high-volume, and most of the messages are not CS-actionable

20. Logstash AWS SDK Ingest Store Bot AWS SDK App Crawlers AWS SDK Amazon Kinesis Process Amazon Lambda Analyze AWS SDK Machine learning Notiﬁcation Action Support issue Database Feature request Keep training the ML model with new data Action Amazon S3

21. AWS SDK Ingest Store Bot AWS SDK Messenger Amazon Kinesis Process Amazon Lambda Analysts Machine learning Action Bot App Get prediction Keep training the ML model with new data Amazon S3

22. OI from Business view with custom source

23. Refrigerator POS Door sensor Water Camera Storefront Kitchen Lambda SQS AWS IoT SQSPoller Http Event Collector Serverless Architecture

24.

25.

26. Our Big Data Scale Total ~25 PB DW on Amazon S3 Read ~10% DW daily Write ~10% of read data daily ~ 550 billion events daily ~ 350 active platform users

27. predict what you want to watch before you watch it.

28. Netflix Prize - best collaborative filtering algorithm

29. Storage Compute Service Tools AWS S3

30. Data Answers Ingest/ Collect Consume/ visualizeStore Process/ analyze 1 4 0 9 5 START WITH A BUSINESS CASE MATCH AVAILABLE DATA CHOOSE BEST FIT Amazon S3 Amazon Kinesis Amazon DynamoDB Amazon EMR Amazon Redshift Amazon Machine Learning Storage Processing Visualize QuickSight

31. Source DBs 3rd Party Data Log Data Reporting Analysis Processing Data Lake S3 Source of truth

32. Remember to complete your evaluations!

33. Thank you

34. CRM ERP DBs Log ﬁle AWStats days MB 2002 Big bang

35. <2005 Hello world Page/Event tracking GA hours GB

36. SOLOMO minutes - hours TB <2008 New customer service New System monitoring New QA

37. IoT O2O seconds – hours PB 2016 Fast and big data driven marketing

38. Analytics ETL Interactive data exploration Interactive slice & dice RT analytics & iterative/ML algo and more ... Different Big Data Processing Needs

Big Data Analytics on AWS

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Big Data Analytics on AWS

Similar to Big Data Analytics on AWS (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Recently uploaded

Recently uploaded (20)

Big Data Analytics on AWS