A short introduction to Apache Falcon, what is it and what is it used for ? How can it help with Hadoop based data life cycle management ? What is it's architecture and what are the benefits of using it ?

1. Apache Falcon
●
What is it ?
●
Benefits
●
Architecture
●
Example
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

2. Apache Falcon – What is it ?
●
A data life cycle management framework
●
Created for Hadoop
●
Logic based in Falcon rather than apps
●
Simplifies data management
●
Developed by InMobi and HortonWorks
●
Falcon can manage
–
Work flows
–
Replication
–
Provides data abstraction
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

3. Apache Falcon – What is it ?
●
Falcon provides services
–
Data import / replication
–
Scheduling / coordination
–
Lifecycle policies
–
Cluster management
–
SLA Management
●
An enterprise solution for data lifecycle management
●
Currently an Apache incubator project
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

4. Apache Falcon – Benefits
●
Reduce workflow / ETL development time
●
Reduce costs
●
No need to re implement functionality
–
–
●
Already in Falcon
Already tested
Use a single Falcon configuration file to
–
Define replication points
–
Define data processing pipeline
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

5. Apache Falcon – Architecture
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

6. Apache Falcon – BI Example
●
Falcon used to manage work flow
●
Falcon used to manage Cluster data replication
●
BI example
–
Staged and presented data replicated
–
Presented data visible for
Reporting
● Analytics
● See next slide .....
●
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

7. Apache Falcon – BI Example
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

8. Contact Us
●
Feel free to contact us at
–
www.semtech-solutions.co.nz
–
info@semtech-solutions.co.nz
●
We offer IT project consultancy
●
We are happy to hear about your problems
●
You can just pay for those hours that you need
●
To solve your problems