2. Introduction
“Big Data” may very well be the most over-hyped and
misunderstood trend in IT today. The goal of this
presentation is to help demystify the topic by gaining
a better understanding of how it ought to exploited
and managed.
This briefing is not meant to address key technical
concepts or a “Big Data 101.” We’re going to focus
instead on the real-world challenges associated with
Big Data implementations in the context of the larger
enterprise environment.
Copyright 2014 – Semantech Inc.
2
3. Where Does Big Data Fit ?
All Data-related
technologies fit within the
same Data Continuum –
Data must be managed, it
can be discovered and then
hopefully exploited…
Copyright 2014 – Semantech Inc.
3
4. The Appeal of Big Data
The appeal of Big Data is two-fold;
1) it lies in the implied potential of technology
being able to keep pace with the exponential
growth of data volume and
2) there is an expectation that a leaner approach
to data management will ultimately make the
enterprise easier to run…
These notions are both true and false. Yes, Big Data
technology is designed to both increase volume and
speed – yet it is not a silver bullet and many fail to
recognize that any such capability must reside within
already existing enterprise ecosystems.
Copyright 2014 – Semantech Inc.
4
5. What is Big Data, Really?
Does it refer to size, to volume to velocity, to flexibility in
data types? Does it refer to the types of systems and
algorithms loosely confederated under the Big Data
umbrella – NoSQL, Hadoop, Key Value Pair Databases,
Document Databases, Graph Databases…
Is Big Data Operational or Analytic or both? The more
people try to define it, the more it’s starting to sound like
the all “legacy” data technologies it is supposed to be
eclipsing. The truth is that there simply isn’t one standard
definition that encompasses the variety of Big Data
technology that now exists. It’s evolved past that point
already. The main thing in common with all Big Data
solutions is that they are looking to shift old paradigms.
Copyright 2014 – Semantech Inc.
5
6. Where Does Big Data Fit?
Are traditional Data Management and
Architectures now obsolete with the
oncoming waves of Big Data
technology?
Maybe Not – Maybe Big Data isn’t as revolutionary as we think.
Copyright 2014 – Semantech Inc.
6
7. Things to Keep in Mind
1. Existing database technologies are not going away
anytime soon.
2. All Data is an enterprise asset, thus all Data must be
managed in the context of the larger environment.
3. Technology must serve a purpose – a new technology
can help define new applications – however, to justify
an investment, a valid Use Case (or Use Cases) must
eventually appear.
4. Fast Data and lot’s of Data are of little value without
integrity and context.
5. Databases are systems – systems have architectures
and themselves form parts of larger architectures.
6. All IT solutions require Governance.
Copyright 2014 – Semantech Inc.
7
8. The Real Challenge with Big Data
There are three main challenges that tend to plague
every Big Data project today:
1. Lack of a Strategy, Use Cases and a Value
Proposition that fits not just in the context of one
project, but in the context of the enterprise.
2. Lack of a Design and Governance Framework that
can manage the solution lifecycle of any Big Data
project.
3. Lack of an Integrated Architecture that properly
leverages Big Data capabilities within the larger
ecosystem of enterprise Data systems.
Copyright 2014 – Semantech Inc.
8
9. How Do you Design Big Data?
Traditional elements of the
Data Enterprise are well
understood and generally have
clear expectations for both
design and management.
So, can Big Data fit into a
picture like this? If not , is it
acceptable to operate without
such understanding ?
9
10. “Enterprise Big Data” Defined
When Big Data was limited primarily to Hadoop
databases focused on one or two internet focused Use
Cases driven by Google, the idea of Big Data was much
easier to grasp.
Now that Big Data has “grown up,” there is no simple or
standard way to view Big Data, unless we also expand
our scope. Once Big Data jumped from one technology to
dozens, from one Use Case to dozens, from one industry
to dozens and from one prototype system to a production
element within an enterprise – it became something new.
It became “Enterprise Big Data” – and it’s never going
back.
Copyright 2014 – Semantech Inc.
10
11. “Enterprise Big Data” Defined 2
Enterprise Big Data:
The collection of technologies designed to handle the
explosion of data associated with 21st century IT
systems and Internet applications. This technology is
not meant to replace all existing database capability
but rather to supplement it in cases where
performance of large or complex data sets requires
more dynamic and flexible management.
Another key aspect is that Enterprise Big Data is just
that – it is for the Enterprise, not just Google – it is a
collection of technologies designed to serve the
enterprise and ultimately reside within it.
Copyright 2014 – Semantech Inc.
11
12. Understanding the Challenge
Let’s take a look at some of the challenges associated
with deploying Big Data capability in real-world enterprises
12
13. Challenge 1: Strategy
The Challenge:
Demonstrating a technology is one thing –
a relatively easy thing. Demonstrating the
value of that technology within your
organization is something entirely different.
How do you decide when and how to
employ Big Data capability and more
importantly how do you make it relevant?
Typical Problems that Arise:
1. The typical web-focused Use Cases
don’t seem to apply in your org.
2. There isn’t a clear path as to how
the technology will improve
efficiency or fuel growth.
3. The solution seems to be
competing with similar capability The Strategy is missing…
(both new and legacy) with no
clear plan for reconciliation.
Copyright 2014 – Semantech Inc.
13
14. Challenge 2: Governance
The Challenge:
Big Data solutions cannot exist separate
from the rest of the mission and
infrastructure of an enterprise. Yet, there is
no standard Big Data management or
Governance framework in IT.
Typical Problems that Arise:
1. Data Integrity is not evaluated at
all (for Big Data).
2. Solution Lifecycle Management is
absent.
3. There are conflicting views as to
whether it can be governed at all.
4. Big Data solutions are seriously out
of touch with business needs or
representatives.
5. There is no metrics framework in
place to understand value.
Copyright 2014 – Semantech Inc.
14
15. Challenge 3: Enterprise Data Integration
The Challenge:
Both at the Operational and Analytical
level, Big Data represents only part of a
larger picture. And it is no longer easy to
determine just where Big Data fits in that
big picture. In order to actualize any
strategy - data discovery, exploitation and
management must be integrated.
Typical Problems that Arise :
1. There are no standard Big Data
Architecture patterns.
2. There are often no clear design
strategies for integrating Big Data
with ETL, Data Warehouses and
Master Data Management
systems.
3. No one knows how to model Big
Data.
Copyright 2014 – Semantech Inc.
15
17. Top 10 Don’ts
1. Don’t initiate Big Data projects without an enterprise strategy.
2. Don’t let your techies run the project without business input.
3. Don’t assume Big Data can’t be designed, modeled or
architected.
4. Don’t assume that Big Data ought to be limited to Internet or
Social Media data.
5. Don’t assume 1 Big Data technology will support all of your
needs. One size doesn’t fit all.
6. Don’t replace exist technologies too soon.
7. Don’t assume that Big Data will be focused on a narrow set of
data formats.
8. Don’t assume that Big Data can’t be governed (as data or as
systems).
9. Don’t separate management of other data systems from Big
Data solutions. (Don’t reinvent the wheel)
10. Don’t start without defining your Use Cases.
Copyright 2014 – Semantech Inc.
17
18. Top 10 Do’s
1.
Do adopt Big Data Technology, when you’re ready and if it makes sense
– but validate that first.
2. Do integrate Governance and management of Big Data with the rest of
your enterprise data architecture.
3. Do design Big Data solutions – both as systems and as data.
4. Do evaluate All of the available Big Data technologies before deciding
which one/s are the best fit.
5. Do integrate your existing ETL and ESB / Middleware infrastructure
with your Big Data solution from the beginning.
6. Do employ both Semantic modeling and Master Data Management
(MDM) for Big Data – and yes it is possible.
7. Do update and revise enterprise processes to accommodate new
technology and capability when necessary.
8. Do create a security plan as part of the initial Big Data strategy,
especially if that data resides in the Cloud.
9. Do your homework. Do use an Architect to help with your project.
10. Do assess and question your initial assumptions and strategy and
amend after gathering lessons learned.
Copyright 2014 – Semantech Inc.
18
20. A Realization, A Foundation
The first step towards mastering enterprise Big Data
is understanding the realization that regardless of
whether data has a formal structure – like Third
Normal Form (relational), Hierarchy, Schema
Dimensions or little structure (like many Big Data
solutions) – all data can be classified through
Semantics.
Data Classification then facilitates Data
Discovery, Data Management and Data Integration.
Big Data can be classified and modeled within the
context of a larger paradigm.
This is the first step – it is the foundation.
Copyright 2014 – Semantech Inc.
20
21. Step 1: Provide the Foundation
There needs to be a bridge that
can span every data system,
data source and element. This
is our foundation
21
22. Big Data in the Enterprise
We will explain this high level or
Conceptual Architecture in greater
depth in our next presentation
22
23. Conclusion
Semantech Inc. has presented this introductory
topic as the first in a series of briefings on
Enterprise Big Data. The follow-on briefings will
include:
1. How to Architect Enterprise Big Data Solutions
2. How to Model Enterprise Big Data
3. How to Secure Enterprise Big Data Systems
4. How to Govern Enterprise Big Data
5. Enterprise Big Data real-world Scenarios and
Case Studies.
23