5. Goals of the Relaunch
● Show how the pieces fit together
○ Projects now discussed with each other in context
● OSS categories mirror internal teams
○ No artificial categories, focal points for each area
● Focus on projects that are core to Netflix
○ Projects mentioned are core and strategic
● Adding project-branded websites
6. High Level Categories
Big Data
Tools and services for (big) data
Build and Delivery Tools
Taking code from desktop to the cloud
Common Runtimes Service & Libraries
Runtime containers, libraries & services that power
microservices
7. High Level Categories
Data Persistence
Storing and serving data in the cloud
Insight, Reliability and Performance
Providing actionable insight at massive scale
12. Fenzo: scheduling optimizations
Speed Accuracy
First fit assignment Optimal assignment
Real world tradeoffs
~ O (1) ~ O (N * M)1
1
Assuming tasks are not reassigned
13. Fenzo: fitness, constraints plugins
● Fitness value (0.0 - 1.0)
○ Degree of fitness - first fit, best fit, worst fit
○ Composable evaluators
○ e.g., bin packing
● Constraints
○ Hard constraints filter appropriate resources
○ Soft constraints specify preferences
○ e.g., zone balancing, instance type preferences
14. Fenzo: bin packing experiment
Bin pack tasks using Fenzo’s built-in CPU bin packer
16. Fenzo: what’s next
● Task management SLAs
● Support for newer Mesos features
● Collaboration
17.
18. Why?
● Easier way for users to troubleshoot
performance issues
● Access to low-level and specialized metrics
● Easier way to visualize and understand
● High-resolution data to detect anomalies
● Real-time and on-demand
● No additional overhead when not in use
● Something easier than SSH
● And simpler than full-fledged monitoring
solution
19. What?
● Is a Performance Monitoring tool
● Host-Level, On-Demand, High-Resolution Metrics (1 second)
● Client-side Application, User-friendly web UI
● Configurable dashboards and widgets
● Leverages Performance Co-Pilot (PCP)
● Stateless and Lightweight Metric Collection
● No persistence
● System Metrics: CPU, Memory, Network, Disk, ...
● Application Metrics*: Java, Memcached, C*, ElasticSearch, Apache
● Extensible. Custom metric agents and widgets.
* Agents are available, but not included by default.
20.
21.
22. What’s Next?
● Interface for different backends
● Better support for containers;
○ With container-specific dashboard and widgets.
● Native flame graph integration;
○ With our d3.js flame graph plugin.
24. Java Mixed-Mode Flame Graphs
● Needs JDK8u60+ with
-XX:+PreserveFramePointer
○ May have some cost
● Lets Linux perf (perf_events)
see Java method frames
● Use with perf-map-agent for
symbols
● http://techblog.netflix.
com/2015/07/java-in-flames.
html
Java
Kernel
JVMGC
28. FIDO - Security Response Orchestration
● Centralize alerts
● Enrich with data
○ User, machine
○ Threat
● Prioritize response
● Automate first
actions
Netflix's FIDO is not a part of or service of the FIDO Alliance