SlideShare a Scribd company logo
1 of 14
Download to read offline
Inkscope
a ceph GUI initiative
Alain DECHORGNAT – Orange Labs
Ghislain CHEVALIER – Orange Labs
September 18, 2014
Orange Labs
2
Agenda
 Ceph at Orange Labs
 Inkscope project
 Demo
 Parallel with Calamari
 Next steps
 Q&A
CephDay Paris – September18,2014
3
 Since 6 years, research on innovative concepts to enrich the cloud storage
functionalities
– Data growth control
– Storage costs reduction
– Bottleneck avoidance
– Data durability insurance
– Ease of data accessibility (ATAWADAC)
– Development of an internal cloud storage solution
 Since June 2013
– CEPH is identified as a real challenger
– Acquire expertise on Ceph
 Study how the innovative concepts can be
implemented into Ceph (erasure codes,
multi-tiering, multi-siting, store-edge, indexation)
 Be an actor in CEPH community
 Be a CEPH tamer
CEPH at Orange Labs
source : Crypto – Kraken - Menvielle/Martin
CephDay Paris – September18,2014
4
 Federation
– Management of heterogeneous storage services
– Storage location regarding costs and performance
 Storage contracts
– content = f(data, contract, metadata)
 Smart data placement
– Storage orchestration (best storage infrastructure)
– Data robustness (replication/erasure code, sharding)
 Multi-tiering
– Right Data, Right Tier
– Implementation in Ceph
– Crushmap management
– Dynamic tiering
 Multi-siting
– Right Data, Right Site
– Implementation in Ceph
– Rados Gateway Federation
 Ceph Monitoring and Administration
– Inkscope initiative
R&D Works
CephDay Paris – September18,2014
5
 Ceph visualization and operation through command lines
 No open-source Ceph GUI available in late 2013
 Breizh Meetup helps us to launch the project
 Our choices:
– Open source
– Web app
– Use of Ceph rest API
– Modularity, simplicity
 Contributors:
– A.Dechorgnat, Ph.Raipin (Orange labs), A.Sall (trainee)
– E. Mourgaya (Arkea)
Why Inkscope ?
CephDay Paris – September18,2014
6
cluster
 inkscopeViz
– Web client
 inkscopeCtrl
– Server part
– Provides an advanced REST API
 inkscopeProbe
– Collects system and ceph infos
– Feeds a mongoDB database
 inkscopeMonitor (not developed)
– Monitoring of Ceph metrics stored in db
– Feeds monitoring tools like Nagios
Inkscope modules
Supervision
framework
(Nagios…)inkscopeProbe
inkscopeCtrl inkscopeMonitor
inkscopeViz
inkscope
server
ceph APIs
inkscope
client
CephDay Paris – September18,2014
7
Inkscope demo
CephDay Paris – September18,2014
8
 Dashboard
Ceph cluster status at a glance
 OSD visualization and details
– Free and used capacity
– Functional or not
 Pools details
– Used capacity
– Parameters
– PG states
 Object storage (S3/swift)
– Users management
– Buckets management
 Crushmap visualization
Inkscope functionalities summary
CephDay Paris – September18,2014
9
 Tiering
– Wring the neck of the crushmap to fulfill tiering requirements
Work in progress on Ceph
From a basic topology…through a hw-based
topology…
to an application-based topology…
CephDay Paris – September18,2014
let’s see it in
inkscope…
10
 Inktank dev
 open source since May 2014
 mailing-list :
ceph-calamari
 presented as diagnostic tool
 4 modules
– calamari-web
– cthulhu
– salt-master
– rest-api
Calamari is “in the wild”
CephDay Paris – September18,2014
11
Parallel Calamari/Inkscope : functionalities
Calamari Inkscope
Visualization Dashboard Yes Yes
OSD Yes Yes
Host Yes Yes
PG Yes Yes
Crushmap No Yes
Object No Yes
Graphs Cluster Yes No
Pools IOPS Yes No
Hosts Yes No
Management Console No Yes
OSD Yes (operation only) Partial (stop, in/out)
Pools Yes (limited parameters) Yes
Cluster settings (flags) Yes No
S3/swift users No Yes
S3/swift buckets No Yes
S3/swift zones/regions No No
Monitoring Alerts No No
Logs Yes No
CephDay Paris – September18,2014
12
Parallel Calamari/Inkscope : technologies
CephDay Paris – September18,2014
Client HTML5 HTML5/AngularJs
Javascript libraries:
•AngularJS
•IndexedDb
•BackboneJS
Javascript libraries:
•AngularJs addons (ngTable, dialog, ui Bootstrap)
•Visualization with D3.js, NVD3, Rickshaw
Compass, Sass CSS (needs gem and Ruby),
Bootstrap 3 CSS
Bootstrap CSS
and also Grunt, NodeJs, make to build the
client
Server - Apache with Django (python scripts) Apache with Flask (python scripts)
- packages: django, flask, python-dev, libcairo,
liqpq5, psutil, gevent, sqlalchemy, pytz,
alembic, zwig, zeromq, m2crypto …
Python packages: flask, pymongo, simples3, python-
dev, psutil
PosgreSql MongoDB
Carbon, Graphite, Diamond Shellinabox (optional)
Saltstack
- Supervisor
- Vagrant
and all their dependancies…
Calamari Inkscope
13
 Planned in Inkscope
– Extend or replace Ceph Rest API request by command lines
powered by Salt
– Improve S3/swift features : zones and regions management
– Improve probes operations
– Implement objects visualization
– Plug the monitoring module to feed Nagios/Shinken
 Other ideas
– Simulation : impact calculation in case of crushmap update
(storage capacity, bandwidth,..)
 “Fusion” with Calamari?
– Contact taken with Calamari team
Next steps for inkscope
CephDay Paris – September18,2014
14
 https://github.com/inkscope/inkscope
 alain.dechorgnat@orange.com
 ghislain.chevalier@orange.com
Q&A
CephDay Paris – September18,2014

More Related Content

Recently uploaded

Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmonyelliciumsolutionspun
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Jaydeep Chhasatia
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxJoão Esperancinha
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntelliSource Technologies
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLAlluxio, Inc.
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Incrobinwilliams8624
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?AmeliaSmith90
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.Sharon Liu
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadIvo Andreev
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilVICTOR MAESTRE RAMIREZ
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native BuildpacksVish Abrams
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIIvo Andreev
 

Recently uploaded (20)

Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptx
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptx
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Inc
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and Bad
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-Council
 
Salesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptxSalesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptx
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native Buildpacks
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AI
 

Featured

Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summarySpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best PracticesVit Horky
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project managementMindGenius
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...RachelPearson36
 
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Applitools
 
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at WorkGetSmarter
 
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...DevGAMM Conference
 
Barbie - Brand Strategy Presentation
Barbie - Brand Strategy PresentationBarbie - Brand Strategy Presentation
Barbie - Brand Strategy PresentationErica Santiago
 
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them wellGood Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them wellSaba Software
 
Introduction to C Programming Language
Introduction to C Programming LanguageIntroduction to C Programming Language
Introduction to C Programming LanguageSimplilearn
 
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...Palo Alto Software
 
9 Tips for a Work-free Vacation
9 Tips for a Work-free Vacation9 Tips for a Work-free Vacation
9 Tips for a Work-free VacationWeekdone.com
 

Featured (20)

Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
 
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
 
ChatGPT webinar slides
ChatGPT webinar slidesChatGPT webinar slides
ChatGPT webinar slides
 
More than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike RoutesMore than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike Routes
 
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
 
Barbie - Brand Strategy Presentation
Barbie - Brand Strategy PresentationBarbie - Brand Strategy Presentation
Barbie - Brand Strategy Presentation
 
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them wellGood Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
 
Introduction to C Programming Language
Introduction to C Programming LanguageIntroduction to C Programming Language
Introduction to C Programming Language
 
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
 
9 Tips for a Work-free Vacation
9 Tips for a Work-free Vacation9 Tips for a Work-free Vacation
9 Tips for a Work-free Vacation
 

Inkscope - a CEPH GUI initiative

  • 1. Inkscope a ceph GUI initiative Alain DECHORGNAT – Orange Labs Ghislain CHEVALIER – Orange Labs September 18, 2014 Orange Labs
  • 2. 2 Agenda  Ceph at Orange Labs  Inkscope project  Demo  Parallel with Calamari  Next steps  Q&A CephDay Paris – September18,2014
  • 3. 3  Since 6 years, research on innovative concepts to enrich the cloud storage functionalities – Data growth control – Storage costs reduction – Bottleneck avoidance – Data durability insurance – Ease of data accessibility (ATAWADAC) – Development of an internal cloud storage solution  Since June 2013 – CEPH is identified as a real challenger – Acquire expertise on Ceph  Study how the innovative concepts can be implemented into Ceph (erasure codes, multi-tiering, multi-siting, store-edge, indexation)  Be an actor in CEPH community  Be a CEPH tamer CEPH at Orange Labs source : Crypto – Kraken - Menvielle/Martin CephDay Paris – September18,2014
  • 4. 4  Federation – Management of heterogeneous storage services – Storage location regarding costs and performance  Storage contracts – content = f(data, contract, metadata)  Smart data placement – Storage orchestration (best storage infrastructure) – Data robustness (replication/erasure code, sharding)  Multi-tiering – Right Data, Right Tier – Implementation in Ceph – Crushmap management – Dynamic tiering  Multi-siting – Right Data, Right Site – Implementation in Ceph – Rados Gateway Federation  Ceph Monitoring and Administration – Inkscope initiative R&D Works CephDay Paris – September18,2014
  • 5. 5  Ceph visualization and operation through command lines  No open-source Ceph GUI available in late 2013  Breizh Meetup helps us to launch the project  Our choices: – Open source – Web app – Use of Ceph rest API – Modularity, simplicity  Contributors: – A.Dechorgnat, Ph.Raipin (Orange labs), A.Sall (trainee) – E. Mourgaya (Arkea) Why Inkscope ? CephDay Paris – September18,2014
  • 6. 6 cluster  inkscopeViz – Web client  inkscopeCtrl – Server part – Provides an advanced REST API  inkscopeProbe – Collects system and ceph infos – Feeds a mongoDB database  inkscopeMonitor (not developed) – Monitoring of Ceph metrics stored in db – Feeds monitoring tools like Nagios Inkscope modules Supervision framework (Nagios…)inkscopeProbe inkscopeCtrl inkscopeMonitor inkscopeViz inkscope server ceph APIs inkscope client CephDay Paris – September18,2014
  • 7. 7 Inkscope demo CephDay Paris – September18,2014
  • 8. 8  Dashboard Ceph cluster status at a glance  OSD visualization and details – Free and used capacity – Functional or not  Pools details – Used capacity – Parameters – PG states  Object storage (S3/swift) – Users management – Buckets management  Crushmap visualization Inkscope functionalities summary CephDay Paris – September18,2014
  • 9. 9  Tiering – Wring the neck of the crushmap to fulfill tiering requirements Work in progress on Ceph From a basic topology…through a hw-based topology… to an application-based topology… CephDay Paris – September18,2014 let’s see it in inkscope…
  • 10. 10  Inktank dev  open source since May 2014  mailing-list : ceph-calamari  presented as diagnostic tool  4 modules – calamari-web – cthulhu – salt-master – rest-api Calamari is “in the wild” CephDay Paris – September18,2014
  • 11. 11 Parallel Calamari/Inkscope : functionalities Calamari Inkscope Visualization Dashboard Yes Yes OSD Yes Yes Host Yes Yes PG Yes Yes Crushmap No Yes Object No Yes Graphs Cluster Yes No Pools IOPS Yes No Hosts Yes No Management Console No Yes OSD Yes (operation only) Partial (stop, in/out) Pools Yes (limited parameters) Yes Cluster settings (flags) Yes No S3/swift users No Yes S3/swift buckets No Yes S3/swift zones/regions No No Monitoring Alerts No No Logs Yes No CephDay Paris – September18,2014
  • 12. 12 Parallel Calamari/Inkscope : technologies CephDay Paris – September18,2014 Client HTML5 HTML5/AngularJs Javascript libraries: •AngularJS •IndexedDb •BackboneJS Javascript libraries: •AngularJs addons (ngTable, dialog, ui Bootstrap) •Visualization with D3.js, NVD3, Rickshaw Compass, Sass CSS (needs gem and Ruby), Bootstrap 3 CSS Bootstrap CSS and also Grunt, NodeJs, make to build the client Server - Apache with Django (python scripts) Apache with Flask (python scripts) - packages: django, flask, python-dev, libcairo, liqpq5, psutil, gevent, sqlalchemy, pytz, alembic, zwig, zeromq, m2crypto … Python packages: flask, pymongo, simples3, python- dev, psutil PosgreSql MongoDB Carbon, Graphite, Diamond Shellinabox (optional) Saltstack - Supervisor - Vagrant and all their dependancies… Calamari Inkscope
  • 13. 13  Planned in Inkscope – Extend or replace Ceph Rest API request by command lines powered by Salt – Improve S3/swift features : zones and regions management – Improve probes operations – Implement objects visualization – Plug the monitoring module to feed Nagios/Shinken  Other ideas – Simulation : impact calculation in case of crushmap update (storage capacity, bandwidth,..)  “Fusion” with Calamari? – Contact taken with Calamari team Next steps for inkscope CephDay Paris – September18,2014
  • 14. 14  https://github.com/inkscope/inkscope  alain.dechorgnat@orange.com  ghislain.chevalier@orange.com Q&A CephDay Paris – September18,2014

Editor's Notes

  1. 1
  2. 2