Organizations continue to struggle to connect the dots and extract meaningful insight from the growing volume and variety of data in Hadoop.
Our Solution: Data Refinement, Entity Resolution and Analysis: Novetta Entity Analytics unifies the data scattered across your systems to give you a single unified view of the people, organizations, locations, and other entities or “things” and their relationships in your enterprise. By revealing the real-world networks, behaviors, and trends of the entities and relationships that exist within corporate data repositories and data silos, you can connect the dots to do completely new things such as enhance the customer experience, do more targeted marketing and reduce the risk of fraud. Novetta Entity Analytics makes Hadoop data useful to anyone using an adaptive process to unify all types of data – regardless of schema – and allows analysts to look at and connect their data in entirely new ways.
The Benefits:
- Accelerate operational insights by constructing complete 360 degree views of a customer, organization, location, product, event, at any volume from any source whether structured or unstructured
- Improve customer service and retention by identifying dissatisfied customers and service problems found in call details, transactions and other volumes of interaction data and documents
- Increase revenues by creating unified customer profiles and relationships to products and services improving cross-sell/up-sell opportunities
- Detect threat and fraud by connecting the dots between people, organizations and events across data sources including transactional details
-Lower costs by solving large complex data integration and management problems using a predictable, linearly scalable platform
OUR DIFFERENTIATORS
Understands unstructured content in context
Uncovers relationships
Finds the signal within the noise
2. novetta.com 2
The Promise of Hadoop
Leverage ALL data to enhance business
intelligence and data discovery
1.Increase Operational Efficiency
2.Improve Business Decisions
3.Predictive Analysis
Retail
Better understand
consumer buying
triggers and web activity
Healthcare
Create a more effective
care continuum is the
focus in healthcare
Government
Protect citizens by
connecting suspicious
people and their
organizational
relationships
Insurance
Make clearer
connections between
products and customer
needs
Financial
Identifying fraud or
gaining customer
insight
3. novetta.com 3
The Problem for Business Analysts
90%
Searching &
Organizing
10%
Analyzing
Business Analyst
4. novetta.com 4
The Problem for Enterprise IT
ERP, CRM, EDW, CMS
Files, Reports,
Emails
Social Media
Archives
Credit ReportsLogs
FileNet
Enterprise IT
Sharepoint
Splunk,
ocPortal
SnapTrends LexisNexis
Cognos,
Microstrategy
5. novetta.com 5
The Problem – Types of Data
Connecting these data provides
greatest insight
Semi-structured DataStructured Data Unstructured Data
6. novetta.com 6
The Effect
Actionable Intelligence is
costly to retrieve
Gaps in your Enterprise systems
Can’t distinguish
signal from noise
Less trustworthy insight
Inability to scale
Still missing connections
7. novetta.com 7
What are Your Options?
Homegrown Solution Ignore the Problem
Get a Solution That
Connects the Dots
8. novetta.com 8
The Solution – Novetta Entity Analytics
• Provide a true unified view across multiple
systems and sources.
• Uncover hidden insights buried in your data
– including unstructured, semi-structured
and structured data
• Create and manage complex view,
relationships and hierarchies
• Reveal patterns, trends, and relationships
across enterprise data between people
locations, events, and products.
9. novetta.com 9
Map and Bring Together Fragmented
Data
Credit #: 4123-5555-8666-7777
DOB: 1977-09-26
Address: 35 West 15th
St,
Toledo OH, 12345
Email: jones@acme.com
Acct #: 123321
Tran ID: 555999
POSID: 11-56789
Name: Williams Jones
Credit #: 4123-5555-8666-7774
DOB: 1978-01-21
Address: 36 West 15th
St,
Toledo OH, 12345
Address: 125 Main St.
Washington, DC 20001
Email: bjones@abc.com
SSN: 555-11-2222
Phone: 202-123-1234
Name: Barbara Jones
Credit #: 4123-5555-8666-7555
DOB: 1980-04-1677
Address: 53 West 15th
St,
Toledo OH, 12345
Name: Billie Jones
DOB: 1978-01-21
Address: 125 Main St.
Washington, DC 20001
SSN: 555-11-2222
Credit Reporting Agency
Name: Barbara Jones
DOB: 1977-09-26
Credit #: 4123-5555-8666-7777
Address: 35 West 15th
St,
Toledo OH, 12345
Credit Cards
Name: Williams Jones
Credit #: 4123-5555-8666-7555
DOB: 1980-04-1677
Address: 53 West 15th
St,
Toledo OH, 12345
Credit Cards
Name: Billie Jones
DOB: 1978-01-21
Credit #: 4123-5555-8666-7774
Address: 36 West 15th
St.
Toledo OH, 12345
Credit Cards
Name: Barbara Jones
Credit #: 4123-5555-8666-7777
Tran ID: 555999
POSID: 11-56789
RAW
Acct #: 123321
Tran ID: 555999
Address: Toledo OH, 12345
Operational
NOVETTA
Entity Analytics
Acct #: 123321
From: jones@acme.com
To: bjones@abc.com
Phone: 202-123-1234
Address: 35 West 15th
St,
Toledo OH, 12345
Dark Email
10. novetta.com 10
Why Relationships are Important
Smartphones
Computer
Automobile
Role They Play
Purchases
Accidents
“To whom or what are they connected?”
“Who is this?”
“To what organizations are they affiliated?”
“What products do they use?” “How do they shape events?”
11. novetta.com 11
How We Link Unstructured Data
Phone: (202) 123-1234
Name: Barbara Jones
Location: 125 Main St.
DocumentAnnotations
Address: 35 West 15th
St,
Toledo OH, 12345
Address: 125 Main St.
Washington, DC 20001
Email: bjones@abc.com
DOB: 1978-01-21
SSN: 555-11-2222
Phone: 202-123-1234
Name: Barbara Jones
Unstructured Record
Document
Constructs
Golden Enterprise View
13. novetta.com 13
Lead Identification Scenario
Barbara
Financial
Application
Husband
Home Address
Business Address
1. Monitor
Discover home address is also
listed as a business address
4. Act
Make sure personal
banker assists her
with suite of business
products that will add
value to her company
3. Predict
Their business is growing
fast
2. Diagnose
Co-owns business with
husband
14. novetta.com 14
Fraud Detection Scenario
Barbara
Financial
Application
Husband
Home Address
Business Address
1. Monitor
Discover home address is also
listed as a business address
4. Act
Start an investigation
into the relationship
3. Predict
Her home business is a
vendor to your company
2. Diagnose
Barbara is an employee
15. novetta.com 15
What Does This Mean for Your
Organization?
Make more connections,
gain more insight
Gain scalability with big
data analysis
Fulfill the promise of Big
Data make you more
effective, productive,
competitive
Makes downstream
analysis tools more
effective
Get actionable intelligence
with far less effort
90%
Organizing
16. novetta.com 16
Case Study – US Government Agency
Challenge: Needed to detect
• Who is a threat? – from billions of records
• Who is not a threat? – just as important to reduce
workload and protect privacy
• Who is connected to them?
• What events are tied to an individual or group?
• What patterns of behavior identify a particular group or
individual?
Solution: Novetta Entity Analytics
• Who is a threat – from billions of records
• Who is not a threat – just as important to reduce
workload and protect privacy
• Who is connect to them
• What events are tied to an individual or group
• What patterns of behavior identify a particular group or
individual
Value
Enabled ad-hoc analysis of large
EDW data without impacting
operations
Visualize and leverage hidden
relationships – finding patterns of
behavior and threat
Add new data to reporting
faster/agile manner in hours
rather than weeks
Processed 8 billion records on a
128 node cluster in 8 hours
versus 2 years estimate with
existing technology
17. novetta.com 17
Case Study – Oil and Gas
Challenge: Security Threat Assessment
•Need a complete, clear, global perspective of physical risks and
threats posed by individuals, including criminal and terrorist
organizations, to its facilities, assets, employees, and contractors
•Identify the unknown threats and risks being missed through
manual research processes and threat subscription feeds
•Need additional analytics and visualization tools to help Security
Analysts provide rich contextual analysis for timely reporting across
transportation routes, county contacts, geo-political volatility, and
local sentiment
Solution: Novetta Entity Analytics
•Brings in online, subscription, and public source data related to
organizations and individuals of interest
•Decompose and map attributes from external sources and internal
geo-location and security operations sources into entity models of a
person, organization, location or event
•Construct complete, integrated, and clear global profiles of
suspicious individuals, terrorist and criminal threats
•These profiles enable Security Analysts to provide rich, contextual,
up- to-date, reporting of the risks posed to its global operations
Value
Accelerates insights by constructing
complete profiles on individuals and
organizations from external web,
subscription and public data
Correlates profiles with global
operations and location data to better
analyze, monitor, and assess threats
Increases value of security reporting
and analysis with automated retrieval of
sources and past reporting and
advanced, temporal, behavioral, and
link analysis
Enables the use of greater volumes
of internal, third party and external data
at lower costs
18. novetta.com 18
Imagine the Possibilities
Healthcare:
Instill trust and improve
through identification of
inefficiencies and
dissatisfied patients
Financial:
Identify everything
from dissatisfied
customers to fraud
Government:
Increase public trust by
identifying service problems
and dissatisfied citizens
Insurance:
Give brokers a broader
picture of the customer
experience providing an
opportunity to uncover
upsell opportunities
Retail:
Understand affinities and
buzz, gain insight into
influencers beyond
number of connections
19. novetta.com 19
How is Novetta Different?
Scales to enterprise needs
Runs natively on Hadoop
Flexible
Uncovers relationships
Understands unstructured content
20. novetta.com 20
What’s Next?
Raw Data
Non-Disclosure
Agreement
We know your data and
business case is
sensitive
Let us prove it
with your data:
The business capability you are
looking to deliver with Hadoop
We’ll map your business objectives
We’ll employ our engine against your
raw data to discover connections
21. novetta.com 21
Proof of Concept Opportunity
Try Novetta Entity Analytics within your data environment
Novetta offers a brief 4-6 week POC:
•BullsEye and ROI Map – documents detailing how using Novetta Entity Analytics maps to
specifics business objectives and their financial implications
•Data Analytics Report – specific data analytics gathered post Novetta Entity Analytics
processing
•Proposal – Summary of proposed next steps including contemplated services, pricing, and
timeline
•Data and Environment Checklist – a document that details the infrastructure and data
requirements
23. novetta.com 23
Novetta Entity Analytics Augments MDM
SFA ERPPortal
Leverage Master Data with
Novetta Entity Analytics to
unlock new sources of
structured and unstructured
revealing new insights
Leverage Master Data with
Novetta Entity Analytics to
unlock new sources of
structured and unstructured
revealing new insights
Master Data
Repository
Business
Processes/Application
Business
Processes/Application
Social Data
Commercial
Context Data
Public Data
Customer
Data
Operational Data
Raw Data
Dark Data
Master data is limited to
structured information
from core systems
Master data is limited to
structured information
from core systems
Novetta Entity Analytics
360 degree view of Person,
Organization, Location, Event, and
Product
Relationship Analysis
Role Analysis
Behavioral Analysis
Temporal Analysis
Cluster Analysis
Editor's Notes
[Hi, I’m ________, <enter job title>, from Novetta Solutions. Thanks for attending. For over 25 years, Novetta has specialized in applying advanced analytics to organizations’ most complex problems. Today, these organizations benefit from making better data driven decisions.
Now, solving complex problems within an affordable Hadoop environment is the ultimate goal. This is why we’re here today.
In the next half hour or so, I’d like to show you a completely unique approach to rationalizing your data in Hadoop, allowing you to find those critical information and hidden insights within your organization…from a proven solution that’s already being used extensively today.
You need to get value from your Hadoop investment. We’re making that dream a reality.
If you have any questions during the presentation, I’ll be happy to answer them as they arise. But first, let’s talk about the promise Hadoop and big data.]
So what’s the promise of Hadoop?
In a sentence, it’s a single cost-effective place to put all your data where key applications, analysts and business users can leverage this information to enhance business intelligence and data discovery.
Where you can increase operational efficiency. Where you can get a clear picture of your customer. Where you can make better business decisions.
In finance, it might be by identifying fraud or gaining customer insight.
Retailers are looking to better understand consumer buying triggers and web activity.
Integrating clinical care data to create a more effective care continuum is the focus in healthcare.
In government it might protect citizens by connecting suspicious people and their organizational relationships.
For insurance, it could be to make clearer connections between insurance products and customer needs, opening up a world of upsell opportunities.
Big data with Hadoop promises to unlock all of this and more. And it does…sort of. That’s why you’ve invested or are considering investing with Hadoop.]
Thing is, right now the problem is:
Business analysts and other end-users currently find themselves spending 90% of their time organizing and searching for information and only 10% analyzing. In fact, many organizations employ an analyst simply for the process of searching. That’s neither efficient or cost effective. And with data only growing, that problem will only worsen.
[Then there are key applications – your enterprise applications – which are falling short trying to connect datasets that are beyond what they’re designed for. (e.g. Teradata, Data Mirror, Platfora, Cognos, Microstrategy, etc.)
Which means your organization only has partial access to the data. You cannot connect the data across these systems to work effectively, no matter what the application using it. And there’s nothing out there to help your enterprise architect to connect this – making the promised insights and hidden connections you seek more costly and time consuming than expected – if possible at all.
So why is that? Well, for starters, the information still resides in silos. Some of it’s in your ERP, CRM, EDW data. Some of it’s in Content Management Systems and has yet to be organized. Written reports and emails for example in FileNet. Still more can be found through other sources like SharePoint. Or logs in say, ocPortal or Splunk. Then there are external sources like social media archives or credit reporting agencies.]
Then there’s the quality of the data itself.
You’ve got structured data, semi-structured, and unstructured data. Of course it’s the unstructured and semi-structured data that has been the most difficult to get at, both internally and externally. But it’s all three working together that hold your greatest insights. Problem is, how do you link these three sources together in a sound, repeatable way that can deal with dirty and fragmented data?
Just to make things even more confusing, there’s no single way to manage and organize this data from an enterprise perspective.
So what’s that mean to you? Besides making analysts spend way too much time finding actionable intelligence, it means they’re still missing key connections that could help you be more productive, more effective, more competitive.
It also means you can’t fully trust the conclusions the data are leading you to because you can’t fully trust the data.
For instance there’s no single way to manage and organize this data from a business user’s perspective. Which means you have different business analysts getting different results when looking for the same connections. Really, the problem goes even deeper than that. Without a uniform process, the same analyst can get different results doing the same search two different times by simply overlooking a step or omitting search criteria they used before.
It also means you can’t unlock the data on a massive scale to discover the insights within. And respond to market changes at the speed of business.
You need to be able to view true unified data…and trust that data. Isn’t this the promise of Hadoop?
So what’s to be done? Well, you may have even tried to solve this yourself. And if you have, you already know how time consuming and cost prohibitive this approach really is. It’s not your core competence. It’s not where you should be focused.
Or maybe, you just feel like you’re stuck. You’ve made this huge time and money investment and it isn’t delivering ROI. This wasn’t supposed to be so difficult!
[CLICK]
What you need is a solution that identifies the dots and helps your analysts connect them. No matter what the data type.]
That’s where Novetta Entity Analytics comes in. Novetta Entity Analytics finds the dots and connects them so you can finally realize the promise of your Hadoop investment.
We give you a true unified view across multiple systems and sources.
Help you uncover hidden insights buried in your data – including unstructured semi --‐ structured and structured data.
Create and manage complex views, relationships and hierarchies
And reveal patterns trends and relationships across enterprise data between people locations, events, and products.
…all while running natively in Hadoop.
This isn’t more pie in the sky promises.
Developed for the US government and proven in the field by various agencies, this is our core competence, it’s what we do. It’s a central tool being that’s made these agencies more effective and efficient as a result.
Now, we’re making it available to private enterprise so you can benefit the same.
In its simplest form, we’re making connections within and across disparate silos of information, whether it be data locked in your CRM, email, spread sheets – even external information – in effect creating a single, multi-dimensional Enterprise Index from which to operate, interact, and communicate with your many constituents.
Then, we’re integrating the information in unique (previously impossible) ways that make exploring it easy and intuitive.
For those of you who want to know more about just ‘how’ we’re integrating that information, we bring the data together using a flexible resolution model that allows matching/linking to be applied to different data sets so you can deal with fragmented/poor quality data as well as attribute rich data.
It resolves billions of records within hours and links structured with unstructured data, uncovering new connections. By building foundational entity indices of people, organizations, locations, product, and events, organizations have the ability to understand how a person or product is represented in different systems across the enterprise. We’re even going a step further by providing visibility into the relationships and connections between entities.
In addition to being able to make connections. It’s about using those connections to connect the dots and get a clear understanding of that person. Answering questions such as:
Then we build relationships, so you can understand the customer in context, and answer questions like:
Who is an individual (not just an account number) and to whom or what are they connected?
[CLICK]
To who or what are they connected?
[CLICK]
To what organizations are they affiliated?
[CLICK]
What products do they use?
[CLICK]
How do they shape events? Are they a leader or follower? Do others make purchases based on their recommendations?
Based on these relationships, you can answer the more complicated questions that lead to better decisions.
So how do we do it?
We process unstructured data by extracting people, locations, and organizational type information.
[CLICK]
Then we reconstruct that information into ‘fragments’ of likely entities. Finally we run that to see which of those match your structured data.
Basically, we put the unstructured text into context of what matters most, which is our customers products and locations.
This is where it starts to really get good. Because we connect and organize this information, we empower your analysts to do what they do best: ask meaningful questions and find patterns. They can visualize, navigate, sample and group data in ways that allow them monitor, diagnose, predict and act upon the data.
For instance, your analysts can:
Monitor what’s happening – who is my customer – what services, products do they have, what was their most recent interaction?
Diagnose if something has occurred Analyze data for patterns of fraud - or when something has occurred – The interactions that lead up to a customer closing an account.
Predict what could happen – by detecting patterns within the data – such as a poor service with multiple calls, understand that you may lose this customer if something isn’t done.
Act – determine what the next step they should be taking – give a customer a call to offer assistance and/or thank you gift. – give a customer a proactive call in cases of suspected fraud.
Here’s an example. Let’s say you’re a financial organization, and you have a new customer, Barbara who just submitted an application via PDF for a personal banking. Because unstructured data is now so easily incorporated into your analysis, you discover within that document that her home address is also listed as a business address.
[NEXT]
An external site reveals that she co-owns the business with her husband it’s going gang busters. Now, you can make sure that she’s contacted by a personal banker to assist her, not just with her personal loan, but a suite of business products that will add value to her company.
[NEXT]
Now imagine the same connections, but you’re a fraud analyst and Barbara is an employee and you discover her home business is a vendor to your company. Now you can start an investigation into the relationship.
Same Data from the previous example but from a Fraud Perspective. Your new customer, Barbara just submitted an application via PDF for a personal banking. Because unstructured data is now so easily incorporated into your analysis, you discover within that document that, Barbara is also an employee, and her home address business is a vendor to your company. Now you can start an investigation into the relationship
And, note, it’s the same data we just used to solve two very different business use cases in two different parts of the business…BUT, using the same data!!!
So now you’ve empowered you analysts to ask the important questions they need to gain the insights you need. Instead of spending 90% of their time combining and organizing the information, they’re now spending it asking the questions that get results.
Because you’re now getting actionable intelligence with far less effort from analysts, you’re making more connections and gaining more insight. You can trust the insights being made. You can also finally scale it for large amounts of data and discover the hidden possibilities there. It’s even making your downstream tools more effective.
They’re happier, more productive, and far more cost effective.
Finally, you have a true unified view of the information.
With it, you can:
Accelerate operational insights by constructing complete customer and citizen profiles from any volume of data from any application
Improve customer service and retention by identifying dissatisfied customers and service problems
Increase revenues by creating unified customer profiles and relationships to products and services improving cross-sell/up-sell opportunities
Detect threat and fraud by connecting the dots between people, organizations and events across structured and unstructured data sources
Improves business decisions with a fully integrated data quality analysis and measurement workflow
And reduce costs by solving large complex data integration and management problems using a predictable, linearly scalable platform
So you can at last realize the Promise of Big Data.
So let’s look at how NIA is working in action:
One government agency customer – who’s name we can’t mention – needed to sift through billions of records to determine who was a threat and who wasn’t, so they could reduce the workload and protect privacy. For the threat group they wanted to know who was connected to them – what events were tied to them and what groups they were affiliated with. They were also hoping to discover what patterns of behavior identified a particular group or individual.
To do this, they needed to sift through over 8 billion records across over 100 data sources. With their existing technology, they estimated the time for this task required 2 years of processing and analysis.
NIA processed all 8 billion records scaled out to a 128 node cluster – with multi-dimensional support for views of person and organization – in 8 hours. Now that’s ROI.
And we did it all without impacting operations.
Imagine the possibilities in these industries:
For healthcare, by connecting email, documents, charts, reports and social media with relevant events and relationships typically hidden in patient and provider data management systems, healthcare providers can instill trust and improve through identification of inefficiencies and dissatisfied patients.
By correlating data coming in from clickstream, web logs, email and social media with product data warehouse and master data management systems, retailers can understand not only affinities and buzz, but also insight into influencers beyond just number of connections.
Through exposing hidden relationships within email, claims filings, public criminal records or court filings, fraud lists, and social media in conjunction with customer and employee management systems – banks can identify everything from dissatisfied customers to fraud.
When you connect email and social media with agency databases, government organizations can increase public trust by identifying service problems and dissatisfied citizens.
And for insurance, by integrating email, reports, documents and transcripts with customer management systems and external data sources such as public record filings on home purchases, or car title filings, you could give brokers a broader picture of the customer experience – providing an opportunity to uncover upsell opportunities.
What could it do for you?]
So it’s fair to ask what we’re doing that’s so different? (alteration on these points.)
Understand unstructured content – create multi-dimensional views of entities (people, places, products, organizations, events, etc) and their relationships across any and ALL data sources stored on Hadoop. Novetta goes beyond competitors that provide a single 360 view of structured data only. This is an important distinction due to the variety of data stored on a Hadoop cluster or obtained by an organization, where a data source can be structured (e.g., CRM, MDM, ERP, or EDW data), semi-structured (e.g., XML, clickstream, or content metadata), or unstructured (e.g., reports, files, email text, log files, machine data).
Uncover relationships – understand relationships between entity types and across entity types including: familial relationships, organizational hierarchies, shared board members, undisclosed relationships between employees and suppliers, and how events and/or products/services are connected/shared to individuals, organizations and/or locations.
Scale to enterprise needs – support large-scale use cases for constructing single views for an enterprise without needing to store and manage an additional copy in a repository. Novetta Entity Analytics has been successfully deployed on programs that include hundreds of structured and unstructured sources generating tens of billions of records.
Runs natively on HADOOP – turn Hadoop Data Lakes into valuable resources of business opportunity. Its native Hadoop support simplifies deployment and ensures organizations get more out of their big data platform technology. Novetta Entity Analytics is certified to run on Cloudera CDH and Hortonworks.
Flexibility - easily augments your existing business intelligence, master data management and data integration solutions eliminating the need to build out a new platform to support new datasets.
Which brings us to our next step. Let us prove how good this works with your own data. Let’s talk about the business capability you are looking to deliver with Hadoop. Next we’ll map your business objectives to sources of information you’d like to leverage. Finally give us some of your raw data and we’ll employ our engine against it so you can see firsthand with your own information what kinds of connections we can make.
We know info sensitive, so let’s get an NDA so we can speak candidly about these problems.
So give us an opportunity to leverage Novetta Entity Analytics solution (NIA) within your data environment to see the value of our solution and how it maps to the your business problems and desired solutions. It doesn’t take long and at the end, we’ll provide you with an executive summary, data analytics report, and proposal at its conclusion.
For those of you wanting to know more, here’s a few details on the process, including preparation time, execution time, and final presentation. And here’s a list of the roles and responsibilities to make this happen.