Eron Wright - Flink Security Enhancements

•

2 likes•737 views

http://flink-forward.org/kb_sessions/flink-security-enhancements/ Recent security enhancements to Flink make it easy to access secure data and to protect the associated credentials. In this talk we’ll describe and demonstrate the new features, including Kerberos-based access to HDFS and Kafka, transport security (TLS), and service-level authorization which protects your Flink cluster from unauthorized access.

Data & Analytics

Flink Security
Enhancements
Eron Wright – eron.wright@emc.com
DELL EMC
@eronwright

2 of 11
New Security Features
1. Kerberos Authentication Support
2. Service-Level Authorization
3. Transport Security (SSL/TLS)

3 of 11
Existing Capability
• Hadoop Delegation Token (DT)
• CLI usesKerberosto authenticateto HDFS
• HDFSprovidesa DT, which CLI passesto the Flinkcluster
• Clusteris ableto accessHDFSfilesonbehalfof theuser
• Limitations
• YARN mode only
• Not usefulto non-Hadoopservices,e.g. Kafka.
• Note: Still supported
TM
TM
DATA
AKKA
JM
CLI
WEB
BROWSER
KAFKA HDFSZK
HTTP
Flink
Cluster
delegation token

4 of 11
Kerberos Authentication Support
• “Cluster-Level Kerberos Identity”
• Keytab-based
• Sharedby alljobs, notjob-specific
• Enables Kerberos authentication
• DataSourcesandSinks(HDFS,Kafka…)
• StateBackends(ZooKeeper…)
• Protects state data
• ACL onznodes,files
• Supported in standalone and YARN
deployment modes
TM
TM
DATA
AKKA
JM
CLI
WEB
BROWSER
KAFKA HDFSZK
HTTP
Flink
Cluster
keytab

5 of 11
Service-Level Authorization
• “Restrict access to your Flink cluster”
• Protects all endpoints:
• Akka System(control path)
• Intra-ClusterDataTransfer
• WebUI
• BlobTransfer(JARs…)
• Simple shared secret
• Configuredor generated
• Storedonclient (~/.flink/…)
• Storedincluster
• Supported in standalone and YARN
TM
TM
DATA
AKKA
JM
CLI
WEB
BROWSER
KAFKA HDFSZK
HTTP
Flink
Cluster
keytab secret

6 of 11
Transport-Level Security (SSL/TLS)
• “SSL for all connections”
• May be enabled on a per-endpoint basis
• WebUIis problematic
• Supported in standalone and YARN TM
TM
DATA
AKKA
JM
CLI
WEB
BROWSER
KAFKA HDFSZK
HTTPS
Flink
Cluster
keytab secret TLS cert(s)

8 of 11
Configuration
• Configure Kerberos Identity:
– security.enabled: true
– security.keytab: /path/to/keytab
– security.principal: name@realm
• Configure Service-Level Authorization:
– security.cookie: (secret cookie)
• Configure Transport-Level Security:
– security.ssl.enabled: true
– security.ssl.keystore: /path/to/keystore
– security.ssl.keystore-password: (password)
– security.ssl.key-password: (password)
– security.ssl.truststore: /path/to/truststore
– security.ssl.truststore-password: (password)
TM
TM
DATA
AKKA
JM
CLI
WEB
BROWSER
KAFKA HDFSZK
HTTPS
Flink
Cluster
keytab secret TLS cert(s)

10 of 11
Project Status
• Targeted for: Flink 1.2
• Contributors:
– Vijay Srinivasaraghavan (Dell EMC)
– Suresh Krishnappa (Dell EMC)
• Design Doc: Secure Data Access on Google Docs
• JIRAs:
– FLINK-3929 - Support for Kerberos Authentication with Keytab Credential
– FLINK-3930 - Implement Service-Level Authorization
– FLINK-3931 - Implement Transport Encryption (SSL/TLS)
– FLINK-3932 - Implement State Backend Security
• Code:
– Github: https://github.com/EronWright/flink/tree/feature-flink-security

Eron Wright - Flink Security Enhancements

What's hot

An Overview of the Kafka clients ecosystem. APIs – wire protocol clients – higher level clients (Streams) – REST Languages (with simple snippets – full examples in GitHub) – the most developed clients – Java and C/C++ – the librdkafka wrappers node-rdkafka, python, GO, C# – why use wrappers Shell scripted Kafka ( e.g. custom health checks) kafkacat Platform gotchas (e.g. SASL on Win32) Presented at Kafka Summit SF 2017 by Edoardo Comar and Andrew Schofield, IBM

Kafka Summit SF 2017 - Kafka and the Polyglot Programmer

confluent

Capture the Streams of Database Changes

confluent

When choosing an event streaming platform, Kafka shouldn’t be the only technology you look at. There are a plethora of others in the messaging space today, including open source and proprietary software as well as a range of cloud services. So how do you know you are choosing the right one? A great way to deepen our understanding of event streaming and Kafka is exploring the trade-offs in distributed system design and learning about the choices made by the Kafka project. We’ll look at how Kafka stacks up against other technologies in the space, including traditional messaging systems like Apache ActiveMQ and RabbitMQ as well as more contemporary ones, such as BookKeeper derivatives like Apache Pulsar or Pravega. This talk focuses on the technical details such as difference in messaging models, how data is stored locally as well as across machines in a cluster, when (not) to add tiers to your system, and more. By the end of the talk, you should have a good high-level understanding of how these systems compare and which you should choose for different types of use cases.

Tradeoffs in Distributed Systems Design: Is Kafka The Best? (Ben Stopford and...

HostedbyConfluent

Kafka Summit NYC 2017 Hanging Out with Your Past Self in VR

confluent

Apache Kafka 0.8 basic training (120 slides) covering: 1. Introducing Kafka: history, Kafka at LinkedIn, Kafka adoption in the industry, why Kafka 2. Kafka core concepts: topics, partitions, replicas, producers, consumers, brokers 3. Operating Kafka: architecture, hardware specs, deploying, monitoring, P&S tuning 4. Developing Kafka apps: writing to Kafka, reading from Kafka, testing, serialization, compression, example apps 5. Playing with Kafka using Wirbelsturm Audience: developers, operations, architects Created by Michael G. Noll, Data Architect, Verisign, https://www.verisigninc.com/ Verisign is a global leader in domain names and internet security. Tools mentioned: - Wirbelsturm (https://github.com/miguno/wirbelsturm) - kafka-storm-starter (https://github.com/miguno/kafka-storm-starter) Blog post at: http://www.michael-noll.com/blog/2014/08/18/apache-kafka-training-deck-and-tutorial/ Many thanks to the LinkedIn Engineering team (the creators of Kafka) and the Apache Kafka open source community!

Apache Kafka 0.8 basic training - Verisign

Michael Noll

The last 5 years, Kafka and Flink have become mature technologies that have allowed us to embrace the streaming paradigm. You can bet on them to build reliable and efficient applications. They are active projects backed by companies using them in production. They have a good community contributing, and sharing experience and knowledge. Kafka and Flink are solid choices if you want to build a data platform that your data scientists or developers can use to collect, process, and distribute data. You can put together Kafka Connect, Kafka, Schema Registry, and Flink. First, you will take care of their deployment. Then, for each case, you will setup each part, and of course develop the Flink job so it can integrate easily with the rest. Looks like a challenging but exciting project, isn't it? In this session, you will learn how you can build such data platform, what are the nitty-gritty of each part, how you can plug them together, in particular how to plug Flink in the Kafka ecosystem, what are the common pitfalls to avoid, and what it requires to be deployed on kubernetes. Even if you are not familiar with all the technologies, there will be enough introduction so you can follow. Come and learn how we can actually cross the streams!

Cross the streams thanks to Kafka and Flink (Christophe Philemotte, Digazu) K...

confluent

With Apache Kafka, it's typical to place different events in their own topic. But different event types can be related. Consider customer interactions with an online retailer. The customer searches through the site and clicks on various items before deciding on a final purchase. But businesses can gain insight by processing these events in sequence. Using the event type per topic leaves a lot of work for developers. Is there a better way? Fortunately, there is. Schema Registry now supports having multiple event types in the same topic. By placing various event types in a single topic, you can now handle different related events in-order. In this presentation, I'll introduce Schema Registry then we'll dive into how it handles multiple event types in a single topic, including examples. You will learn how and when to apply the multiple event types per topic pattern. Additionally, you'll learn how schema references work in Schema Registry.

Managing multiple event types in a single topic with Schema Registry | Bill B...

HostedbyConfluent

Kafka Streams is a library for developing applications for processing records from topics in Apache Kafka. It provides high-level Streams DSL and low-level Processor API for describing fault-tolerant distributed streaming pipelines in Java or Scala programming languages. Kafka Streams also offers elaborate API for stateless and stateful stream processing. That’s a high-level view of Kafka Streams. Have you ever wondered how Kafka Streams does all this and what the relationship with Apache Kafka (brokers) is? That’s among the topics of the talk. During this talk we will look under the covers of Kafka Streams and deep dive into Kafka Streams’ Fault-Tolerant Distributed Stream Processing Engine. You will know the role of StreamThreads, TaskManager, StreamTasks, StandbyTasks, StreamsPartitionAssignor, RebalanceListener and few others. The aim of this talk is to get you equipped with knowledge about the internals of Kafka Streams that should help you fine-tune your stream processing pipelines for better performance.

Deep Dive Into Kafka Streams (and the Distributed Stream Processing Engine) (...

confluent

Apache Kafka is the most popular open-source stream-processing software for collecting, processing, storing, and analyzing data at scale. Most known for its excellent performance, low latency, fault tolerance, and high throughput, it's capable of handling thousands of messages per second. For mission-critical applications, how do you ensure that the performance delivered is the performance required? This is especially important as Kafka is written in Java and Scala and runs on the JVM. The JVM is a fantastic platform that delivers on an internet scale. In this session, we'll explore how making changes to the JVM design can eliminate the problems of garbage collection pauses and raise the throughput of applications. For cloud-based Kafka applications, this can deliver both lower latency and reduced infrastructure costs. All without changing a line of code!

Better Kafka Performance Without Changing Any Code | Simon Ritter, Azul

HostedbyConfluent

Apache Pulsar at Yahoo! Japan

StreamNative

Gwen Shapira, Confluent | Kafka Summit 2020 Keynote | Kafka’s New Architecture

confluent

Confluent building a real-time streaming platform using kafka streams and k...

Thomas Alex

Namely is a late-stage startup that builds HR, Payroll and Benefits software for mid-sized businesses. Over the years, we've ended up with a number of monolithic and legacy applications covering overlapping domain concepts, which has limited our ability to deliver new and innovative features to our customers. We need a way to get our data out of the monoliths to decouple our systems and increase our velocity. We've chosen Kafka as our way to liberate our data in a reliable, scalable and maintainable way. This talk covers specific examples of successes and missteps in our move to Kafka as the backbone of our architecture. It then looks to the future - where we are trying to go, and how we plan on getting, both from the short term and long term perspectives. Key Takeaways: - Successful and unsuccessful approaches to gradually introducing Kafka to a large organization in a way that meets the short and long term needs of the business. - Successful and unsuccessful patterns for using Kafka. - Pragmaticism versus purisim: Building Kafka-first systems, and migrating legacy systems to Kafka with Debezium. - Combining event driven systems with RPC based systems. Observability, alerting and testing. - Actionable steps that you can take to your organization to help drive adoption.

Shattering The Monolith(s) (Martin Kess, Namely) Kafka Summit SF 2019

confluent

Kafka is a high-throughput, fault-tolerant, scalable platform for building high-volume near-real-time data pipelines. This presentation is about tuning Kafka pipelines for high-performance. Select configuration parameters and deployment topologies essential to achieve higher throughput and low latency across the pipeline are discussed. Lessons learned in troubleshooting and optimizing a truly global data pipeline that replicates 100GB data under 25 minutes is discussed.

Tuning kafka pipelines

Sumant Tambe

Cloud migration: it's practically a rite of passage for anyone who's built infrastructure on bare metal. When we migrated our 5-year-old Kafka deployment from the datacenter to GCP, we were faced with the task of making our highly mutable server infrastructure more cloud-friendly. This led to a surprising decision: we chose to run our Kafka cluster on Kubernetes. I'll share war stories from our Kafka migration journey, explain why we chose Kubernetes over arguably simpler options like GCP VMs, and present the lessons we learned while making our way toward a stable and self-healing Kubernetes deployment. I'll also go through some improvements in the more recent Kafka releases that make upgrades crucial for any Kafka deployment on immutable and ephemeral infrastructure. You'll learn what happens when you try to run one complex distributed system on top of another, and come away with some handy tricks for automating cloud cluster management, plus some migration pitfalls to avoid. And if you're not sure whether running Kafka on Kubernetes is right for you, our experiences should provide some extra data points that you can use as you make that decision.

Kafka on Kubernetes: Keeping It Simple (Nikki Thean, Etsy) Kafka Summit SF 2019

confluent

Kafka and Spark Streaming

datamantra

A Unified Platform for Real-time Storage and Processing

StreamNative

Both Apache Pulsar and Apache Flink share a similar view on how the data and the computation level of an application can be “streaming-first” with batch as a special case streaming. With Apache Pulsar’s Segmented-Stream storage and Apache Flink’s steps to unify batch and stream processing workloads under one framework, there are numerous ways of integrating the two technologies to provide elastic data processing at massive scale, and build a real streaming warehouse. In this talk, Sijie Guo from the Apache Pulsar community will share the latest integrations between Apache Pulsar and Apache Flink. He will explain how Apache Flink can integrate and leverage Pulsar’s built-in efficient schemas to allow users of Flink SQL query Pulsar streams in realtime.

Query Pulsar Streams using Apache Flink

StreamNative

Cooperative Data Exploration with iPython Notebook

DataWorks Summit/Hadoop Summit

Introduction to Apache Kafka- Part 1

Knoldus Inc.

What's hot (20)

Kafka Summit SF 2017 - Kafka and the Polyglot Programmer

Capture the Streams of Database Changes

Tradeoffs in Distributed Systems Design: Is Kafka The Best? (Ben Stopford and...

Kafka Summit NYC 2017 Hanging Out with Your Past Self in VR

Apache Kafka 0.8 basic training - Verisign

Cross the streams thanks to Kafka and Flink (Christophe Philemotte, Digazu) K...

Managing multiple event types in a single topic with Schema Registry | Bill B...

Deep Dive Into Kafka Streams (and the Distributed Stream Processing Engine) (...

Better Kafka Performance Without Changing Any Code | Simon Ritter, Azul

Apache Pulsar at Yahoo! Japan

Gwen Shapira, Confluent | Kafka Summit 2020 Keynote | Kafka’s New Architecture

Confluent building a real-time streaming platform using kafka streams and k...

Shattering The Monolith(s) (Martin Kess, Namely) Kafka Summit SF 2019

Tuning kafka pipelines

Kafka on Kubernetes: Keeping It Simple (Nikki Thean, Etsy) Kafka Summit SF 2019

Kafka and Spark Streaming

A Unified Platform for Real-time Storage and Processing

Query Pulsar Streams using Apache Flink

Cooperative Data Exploration with iPython Notebook

Introduction to Apache Kafka- Part 1

Viewers also liked

http://flink-forward.org/kb_sessions/streaming-ml-with-flink/ As continuous big data processing is gaining popularity it naturally implies that there is a need to transition many of the distributed machine learning functionality to a streaming backend. The most common use case is to give streaming predictions based on the model learnt in batch, however in some cases it is beneficial to also update the model on the fly. It is not uncommon that streaming learners need different algorithms than their batch counterparts. The talk discusses the common use cases and the pitfalls of the streaming ML transition through the example of recommender systems. It also offer a dive into the implementation of a Scala library augmenting FlinkML with streaming predictors.

Márton Balassi Streaming ML with Flink-

Flink Forward

http://flink-forward.org/kb_sessions/scaling-stream-processing-with-apache-flink-to-very-large-state/ The majority of streaming programs is ‘stateful’: Windowed Aggregations, Sessions, Joins, Complex Event Processing, Tables – they all require to keep some form of state across individual events. With the migration of more and more complex batch jobs or data processing pipelines to streaming applications, some streaming programs need to keep terabytes of state. Apache Flink implements a checkpointing-based recovery mechanism that guarantees exactly-once semantics for state also in the presence of failures. The cost of checkpointing and recovery depends on the size of the program’s state. In this talk, we will discuss the current status of stateful processing in Apache Flink, as well as the ongoing efforts to make Flink’s fault tolerance mechanism scale to very large state sizes, supporting frequent checkpoints and faster recovery of large state, without requiring excessive numbers of machines.

Stephan Ewen - Scaling to large State

Flink Forward

http://flink-forward.org/kb_sessions/streaming-sql/ Streaming is necessary to handle data rates and latency, but SQL is unquestionably the lingua franca of data. Is it possible to combine SQL with streaming, and if so, what does the resulting language look like? Apache Calcite is extending SQL to include streaming, and Apache Flink is using Calcite to support both regular and streaming SQL. In this talk, Julian Hyde describes streaming SQL in detail and shows how you can use streaming SQL in your application. He also describes how Calcite’s planner optimizes queries for throughput and latency.

Julian Hyde - Streaming SQL

Flink Forward

Many use cases in the telecommunication industry require producing counters, quality metrics, and alarms in a streaming fashion with very low latency. Most of this metrics are only valuable when they’re made available as soon as the associated events happened. In our company we are looking for a system able to produce this kind of real-time indicator, which must handle massive amounts of data (400,000 eps) with often peak loads (like New Year’s Eve) or out-of-order events like massive network disorder. Low latency and flexible window management with specific watermark emission are also a must-haves. Heterogeneous format, multiple flow correlation, and the possibility of late data arrival are other challenges. Flink being already widely used at Bouygues Telecom for real-time data integration, its features made it the evident candidate for the future System. In this talk, we'll present a real use case of streaming analytics using Flink, Kafka & HBase along with other legacy systems.

Thomas Lamirault_Mohamed Amine Abdessemed -A brief history of time with Apac...

Flink Forward

http://flink-forward.org/kb_sessions/code-generation-in-serializers-and-comparators-of-apache-flink/ Performance of Big Data applications are often limited by serializers and comparators. In the case of Flink, despite its delicate type system, hand writing serializers leads to a 10% improvement for a simple WordCount job using Pojos. There are applications where the gain can be even more, about 30%. A significant amount of the overhead is the result of using reflection to access the fields of Pojo objects. There is some additional overhead due to the fact that the JVM has a hard time to optimize the current default serializers and comparators because of their dynamic nature. In order to improve the performance, I implemented runtime code generation, to generate specialized code for each Pojo type at runtime to improve the performance. The generated code accesses fields without reflection and easier to optimize. In the talk I will give a detailed overview of the challenges of implementing code generation, the solutions, and some measurements about the performance improvements.

Gábor Horváth - Code Generation in Serializers and Comparators of Apache Flink

Flink Forward

http://flink-forward.org/kb_sessions/apache-zeppelin-a-friendlier-way-to-flink/ In this talk we present Apache Zeppelin, notebook based web-UI for working with several big data platforms, including Flink. In this talk we will show how to leverage several of Zeppelin’s exciting features, using Flink. The talk / demo will be a series of examples of increasing complexity. Starting with the obligatory word count example, we will move on to loading additional jars to perform machine learning tasks using FlinkML, we will do some examples of graph processing in Gelly and visualizing those graphs using AngularJS and d3js (both native to Zeppelin), give examples of how Zeppelin’s “ResourcePools” can be used to share variables between interpreters, leading in to a Flink Streaming Example* where variables a “bound” and visualizations update in real time. *pre-supposes working/stable Flink Streaming in REPL, editors feel free to delete this comment once that has been achieved.

Trevor Grant - Apache Zeppelin - A friendlier way to Flink

Flink Forward

http://flink-forward.org/kb_sessions/flinkspector-taming-the-squirrel/ The costs of logic errors in production for streaming applications are higher than for batch processing systems. Depending on the setup, errors cannot be rectified or have already influenced important decisions. The goal of Flinkspector is to improve the test process of Apache Flink streaming applications in order to detect streaming application logic errors early during development. It features dedicated mechanics for test setup, execution, and evaluation. While Flinkspector’s streamlined API keeps testing overhead small. The framework is able to handle non-terminating and parallelized data flows involving windowing. The lightweight integration-tests enabled by Flinkspector allow Flink applications to be included into the continuous integration and deployment process. The talk introduces the core functionality of Flinkspector. In addition, background concepts of the runtime and the evaluation algorithms are presented. https://github.com/ottogroup/flink-spector

Alexander Kolb - Flinkspector – Taming the squirrel

Flink Forward

http://flink-forward.org/kb_sessions/amidst-toolbox-scalable-probabilistic-machine-learning-with-flink/ In this session we would like to present our AMIDST toolbox for analysis of large-scale data sets using probabilistic machine learning models. AMIDST runs algorithms in a distributed fashion for learning and inference in a wide spectrum of latent variable models such as Gaussian mixtures, (probabilistic) principal component analysis, Hidden Markov Models, Kalman Filters, Latent Dirichlet Allocation, etc. This toolbox is able to perform Bayesian parameter learning on any user-defined probabilistic (graphical) model with billions of nodes using novel distributed message passing algorithms. We plan to give an overview of the AMIDST toolbox (Java open source), some details about the API and the integration with Flink, and an analysis of the scalability of our learning algorithms. All this in the context of a real use case scenario in the financial domain (BCC group), where the profile of millions of customers is analyzed using Flink and the Amazon Web Services.

Ana M Martinez - AMIDST Toolbox- Scalable probabilistic machine learning with...

Flink Forward

http://flink-forward.org/kb_sessions/faster-and-furiouser-flink-drift/ Not long ago, we had the opportunity to test Apache Flink to see just how fast it would go on a moderately realistic task with fast hardware and with a good streaming transport layer underneath. Our goal was not so much careful comparison with other software, but flat-out speed, Flink against Flink. In the process, we learned a lot about what it takes to go fast. Some of the lessons were ones that we had “learned” a number of times before: – the bottleneck isn’t where you thought it was – copying data is expensive – context switches are expensive – measure twice, cut once But there were some real surprises along the way. The really important knobs weren’t quite what people say you should turn. One of the biggest surprises was the degree to which high performance libraries have threading built into them which makes the actual concurrrency much higher than the apparent concurrency. The result was that at least one cluster parameter needed to be adjusted by 30x to get real

Ted Dunning-Faster and Furiouser- Flink Drift

Flink Forward

http://flink-forward.org/kb_sessions/beyond-the-watermark-on-demand-backfilling-in-flink/ Flink has consistency guarantees and efficient checkpointing model which make it a good fit for Uber’s money-related use cases, such as driver incentives. However, Flink’s time-progress model is built around a single watermark, which is incompatible with Uber’s business need for generating aggregates retroactively. The talk covers our solution for on-demand backfilling. It also outlines other abstractions and features we expect Flink to support as it matures.

Maxim Fateev - Beyond the Watermark- On-Demand Backfilling in Flink

Flink Forward

http://flink-forward.org/kb_sessions/keynote-tba/ Apache Flink has come a long way from its academic beginnings. It is now one of the most technically advanced solutions for streaming computation. And batch computation, too. Flink has serious technical advantages when compared with nearly every alternative system. This success ironically means that Apache Flink is right on the cusp of a critical moment. Over the next few months it will be decided whether Flink is the Next Big Thing or if it is a fine technology with limited impact. Right now, what you and I do can make a huge difference. But as business people like to say, what got Flink here isn’t what’s going to get it there. The challenges the Flink community faces now are different from the technical challenges it has met so far. I will talk about what I think will help and how we can all pitch in to take Flink forward.

Ted Dunning - Keynote: How Can We Take Flink Forward?

Flink Forward

http://flink-forward.org/kb_sessions/joining-infinity-windowless-stream-processing-with-flink/ The extensive set of high-level Flink primitives makes it easy to join windowed streams. However, use cases that don’t have windows can prove to be more complicated, making it necessary to leverage operator state and low-level primitives to manually implement a continuous join. This talk will focus on the anomalies that present themselves when performing streaming joins with infinite windows, and the problems encountered operating topologies that back user-facing data. We will describe the approach taken at ResearchGate to implement and maintain a consistent join result of change data capture streams.

Sanjar Akhmedov - Joining Infinity – Windowless Stream Processing with Flink

Flink Forward

http://flink-forward.org/kb_sessions/the-future-of-apache-flinktm/ In this session we will first have a look at the current state of Apache Flink before diving into some of the upcoming features that are either already in development or still in the design phase. Some of the features currently in development that we are going to cover are: – Dynamic Scaling: Adapting a running program to changing workloads. – Queryable State: External querying of internal Flink state. This has the power to replace key/value stores by turning Flink into a key value store that allows for up to date querying of results. – Side Inputs: Having additional data that evolves over time as input to a stream operation. For the glimpse at the far-off future of Apache Flink™ we dare not make any predictions yet. In the session we will look at the latest whisperings and see what the community is currently thinking up as solutions to existing problems and predicted future challenges in the stream processing space.

Aljoscha Krettek - The Future of Apache Flink

Flink Forward

http://flink-forward.org/kb_sessions/advanced-visualization-of-flink-and-spark-jobs/ Understanding the physical plan of a big data application is often crucial for tracking down bottlenecks and faulty behavior. Flink and Spark although offering useful Web UI components for monitoring and understanding the logical plan of the jobs, both lack a tool that helps to understand the physical plan of the scheduler and the possibility to monitor execution at a very low level, along with the communication that occur between parallel vertex instances. We propose a tool that allows users to real-time monitor and later to replay, examine job executions on any cluster currently supported by Flink or Spark. The tool also offers monitoring of the distribution of keys in a data stream and can lead to optimizing data partitioning across parallel subtasks in the future.

Zoltán Zvara - Advanced visualization of Flink and Spark jobs 

Flink Forward

http://flink-forward.org/kb_sessions/robust-stream-processing-with-apache-flink/ In this hands on talk and demonstration I’ll give a very short introduction to stream processing and then dive into writing code and demonstrating the features in Apache Flink that make truly robust stream processing possible. We’ll focus on correctness and robustness in stream processing. During this live demo we’ll be developing a realtime analytics application and modifying it on the fly based on the topics we’re working though. We’ll exercise Flink’s unique features, demonstrate fault-recovery, clearly explain and demonstrate why Event Time is such an important concept in robust stateful stream processing and talk about and demonstrate the features you need in a stream processor in production. Some of the topics covered will be: – Stateful Stream Processing – Event Time vs. Processing Time – Fault tolerance – State management in the face of faults – Savepoints – Data re-processing – Planned downtime and upgrades

Jamie Grier - Robust Stream Processing with Apache Flink

Flink Forward

http://flink-forward.org/kb_sessions/keynote-tba-2/ The past 12 months saw the data streaming ecosystem mature and grow tremendously with new open source projects and products being offered in the market, and more large-scale production applications of streaming data. It is now understood that streaming data is not a fad, but a growing industry that is here to stay. Apache Flink was one of the pioneering communities advocating that stream processing is a great fit for the continuous nature of data production, and that batch processing can be seen and efficiently performed as a special case of stream processing. Flink saw tremendous growth since the last Flink Forward conference, with the project boasting now more than 200 contributors from several companies, several production installations and broad adoption. In this talk, we discuss several large-scale stream processing use cases that we see at data Artisans. Additionally, we discuss what this accelerated growth means for Flink, how we can sustain this growth moving forward, as well as a vision for the next big directions in Flink.

Kostas Tzoumas_Stephan Ewen - Keynote -The maturing data streaming ecosystem ...

Flink Forward

http://flink-forward.org/kb_sessions/no-shard-left-behind-dynamic-work-rebalancing-in-apache-beam/ The Apache Beam (incubating) programming model is designed to support several advanced data processing features such as autoscaling and dynamic work rebalancing. In this talk, we will first explain how dynamic work rebalancing not only provides a general and robust solution to the problem of stragglers in traditional data processing pipelines, but also how it allows autoscaling to be truly effective. We will then present how dynamic work rebalancing works as implemented in Google Cloud Dataflow and which path other Apache Beam runners link Apache Flink can follow to benefit from it.

Malo Denielou - No shard left behind: Dynamic work rebalancing in Apache Beam

Flink Forward

http://flink-forward.org/kb_sessions/declarative-stream-processing-with-streamsql-and-cep/ Complex event processing (CEP) and stream analytics are commonly treated as distinct classes of stream processing applications. While CEP workloads identify patterns from event streams in near real-time, stream analytics queries ingest and aggregate high-volume streams. Both types of use cases have very different requirements which resulted in diverging system designs. CEP systems excel at low-latency processing whereas engines for stream analytics achieve high throughput. Recent advances in open source stream processing yielded systems that can process several millions of events per second at sub-second latency. Systems like Apache Flink enable applications that include typical CEP features as well as heavy aggregations. In this talk we will show how Apache Flink unifies CEP and stream analytics workloads. Guided by examples, we introduce Flink’s CEP-enriched StreamSQL interface and discuss how queries are compiled, optimized, and executed on Flink.

Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...

Flink Forward

http://flink-forward.org/kb_sessions/running-apache-flink-everywhere-standalone-yarn-mesos-docker-kubernetes-etc/ The world of cluster managers and deployment frameworks is getting complicated. There is zoo of tools to deploy and manage data processing jobs, all of which have different resource management and fault tolerance slightly different. Some tools have a only per-job processes (Yarn, Docker/Kubernetes), while others require some long running processes (Mesos, Standalone). In some frameworks, streaming jobs control their own resource allocation (Yarn, Mesos), while for other frameworks, resource management is handled by external tools (Kubernetes). To be broadly usable in a variety of setups, Flink needs to play well with all these frameworks and their paradigms. This talk describes Flink’s new proposed process and deployment model that will make it work together well with the above mentioned frameworks. The new abstraction is designed to cover a variety of use cases, like isolated single job deployments, sessions of multiple short jobs, and multi-tenant setups.

Stephan Ewen - Running Flink Everywhere

Flink Forward

http://flink-forward.org/kb_sessions/connecting-apache-flink-with-the-world-reviewing-the-streaming-connectors/ Getting data in and out of Flink in a reliable fashion is one of the most important tasks of a stream processor. This talk will review the most important and frequently used connectors in Flink. Apache Kafka and Amazon Kinesis Streams both fall into the same category of distributed, high-throughput and durable publish-subscribe messaging systems. The talk will explain how the connectors in Flink for these systems are implemented. In particular we’ll focus on how we ensure exactly-once semantics while consuming data and how offsets/sequence numbers are handled. We will also review two generic tools in Flink for connectors: A message acknowledging source for classical message queues (like those implementing AMQP) and a generic write ahead log sink, using Flink’s state backend abstraction. The objective of the talk is to explain the internals of the streaming connectors, so that people can understand their behavior, configure them properly and implement their own connectors.

Robert Metzger - Connecting Apache Flink to the World - Reviewing the streami...

Flink Forward

Viewers also liked (20)

Márton Balassi Streaming ML with Flink-

Stephan Ewen - Scaling to large State

Julian Hyde - Streaming SQL

Thomas Lamirault_Mohamed Amine Abdessemed -A brief history of time with Apac...

Gábor Horváth - Code Generation in Serializers and Comparators of Apache Flink

Trevor Grant - Apache Zeppelin - A friendlier way to Flink

Alexander Kolb - Flinkspector – Taming the squirrel

Ana M Martinez - AMIDST Toolbox- Scalable probabilistic machine learning with...

Ted Dunning-Faster and Furiouser- Flink Drift

Maxim Fateev - Beyond the Watermark- On-Demand Backfilling in Flink

Ted Dunning - Keynote: How Can We Take Flink Forward?

Sanjar Akhmedov - Joining Infinity – Windowless Stream Processing with Flink

Aljoscha Krettek - The Future of Apache Flink

Zoltán Zvara - Advanced visualization of Flink and Spark jobs 

Jamie Grier - Robust Stream Processing with Apache Flink

Kostas Tzoumas_Stephan Ewen - Keynote -The maturing data streaming ecosystem ...

Malo Denielou - No shard left behind: Dynamic work rebalancing in Apache Beam

Fabian Hueske_Till Rohrmann - Declarative stream processing with StreamSQL an...

Stephan Ewen - Running Flink Everywhere

Robert Metzger - Connecting Apache Flink to the World - Reviewing the streami...

Similar to Eron Wright - Flink Security Enhancements

Big data security

Joey Echeverria

A Journey to Magical Security Creatures' Land

MongoDB

Advanced Security In Hadoop Cluster

Edureka!

Hadoop Operations: How to Secure and Control Cluster Access

Cloudera, Inc.

Securing Your Apache Spark Applications

Cloudera, Inc.

Securing Spark Applications by Kostas Sakellis and Marcelo Vanzin

Spark Summit

Technical tips for secure Apache Hadoop cluster #ApacheConAsia #ApacheCon

Yahoo!デベロッパーネットワーク

SSO with kerberos

Claudia Rosu

Cl116

Juliette Ponnet

When it comes to data security, Uber’s business has unique needs related to scale, use-case, and technical stacks. This talk will discuss how our data platform team addressed specific challenges in deploying Uber's security requirements for Apache Hadoop, including how we leveraged open source building blocks. We'll share insights on how we augmented our Kerberized Hadoop integration with additional authentications mechanisms as well as our approach to supporting custom authentication in Apache Knox. In particular, we will elaborate Uber’s contributions to Apache Knox, specifically a novel pluggable platform for custom validation of any user request. This talk will also cover how we address table, column, and partition-level access control while ensuring improved developer productivity. In particular, we will explain how we translate RBAC policy into HDFS ACL to control data access, our internal audit platform built to detect and analyze the common security infringements, and real-world examples from our experiences in production. Speakers Mohammad Islam, Staff Software Engineer, Uber Wei Han, Manager, Uber

Securing Data in Hadoop at Uber

DataWorks Summit

Protecting your data at rest with Apache Kafka by Confluent and Vormetric

confluent

One of the main sources of concerns when switching to the container paradigm is security. When dealing with big amounts of sensitive customer data it’s very important to be able to guarantee that the data is transported safely between the different components of the system. This talk will focus on how to setup a Flink cluster to run on a Kubernetes environment taking into account all security aspects to ensure secured communication between the nodes of the cluster, state backend and also for job submission, all taking advantage of Kubernetes tools.

Deploying a secured Flink cluster on Kubernetes

Edward Alexander Rojas Clavijo

Apache Kafka® Security Overview

confluent

Hadoop security

shrey mehrotra

Building Open Source Identity Management with FreeIPA

LDAPCon

Flink Forward Berlin 2018: Edward Alexander Rojas Clavijo - "Deploying a secu...

Flink Forward

Securing Kafka

confluent

Kafka 2018 - Securing Kafka the Right Way

Saylor Twift

Dustin Black - Red Hat Storage Server Administration Deep Dive

Gluster.org

Hadoop Security, Cloudera - Todd Lipcon and Aaron Myers - Hadoop World 2010

Cloudera, Inc.

Similar to Eron Wright - Flink Security Enhancements (20)

Big data security

A Journey to Magical Security Creatures' Land

Advanced Security In Hadoop Cluster

Hadoop Operations: How to Secure and Control Cluster Access

Securing Your Apache Spark Applications

Securing Spark Applications by Kostas Sakellis and Marcelo Vanzin

Technical tips for secure Apache Hadoop cluster #ApacheConAsia #ApacheCon

SSO with kerberos

Cl116

Securing Data in Hadoop at Uber

Protecting your data at rest with Apache Kafka by Confluent and Vormetric

Deploying a secured Flink cluster on Kubernetes

Apache Kafka® Security Overview

Hadoop security

Building Open Source Identity Management with FreeIPA

Flink Forward Berlin 2018: Edward Alexander Rojas Clavijo - "Deploying a secu...

Securing Kafka

Kafka 2018 - Securing Kafka the Right Way

Dustin Black - Red Hat Storage Server Administration Deep Dive

Hadoop Security, Cloudera - Todd Lipcon and Aaron Myers - Hadoop World 2010

More from Flink Forward

Apache Flink is a distributed stream processing framework that allows users to process and analyze data in real-time. At LinkedIn, we developed a fully managed stream processing platform on Flink running on K8s to power hundreds of stream processing pipelines in production. This platform is the backbone for other infra systems like Search, Espresso (internal document store) and feature management etc. We provide a rich authoring and testing environment which allows users to create, test, and deploy their streaming jobs in a self-serve fashion within minutes. Users can focus on their business logic, leaving the Flink platform to take care of management aspects such as split deployment, resource provisioning, auto-scaling, job monitoring, alerting, failure recovery and much more. In this talk, we will introduce the overall platform architecture, highlight the unique value propositions that it brings to stream processing at LinkedIn and share the experiences and lessons we have learned.

Building a fully managed stream processing platform on Flink at scale for Lin...

Flink Forward

Flink Forward San Francisco 2022. When running Flink jobs, skew is a common problem that results in wasted resources and limited scalability. In the past years, we have helped our customers and users solve various skew-related issues in their Flink jobs or clusters. In this talk, we will present the different types of skew that users often run into: data skew, key skew, event time skew, state skew, and scheduling skew, and discuss solutions for each of them. We hope this will serve as a guideline to help you reduce skew in your Flink environment. by Jun Qin & Karl Friedrich

Evening out the uneven: dealing with skew in Flink

Flink Forward

Flink Forward San Francisco 2022. To improve Amazon Alexa experiences and support machine learning inference at scale, we built an automated end-to-end solution for incremental model building or fine-tuning machine learning models through continuous learning, continual learning, and/or semi-supervised active learning. Customer privacy is our top concern at Alexa, and as we build solutions, we face unique challenges when operating at scale such as supporting multiple applications with tens of thousands of transactions per second with several dependencies including near-real time inference endpoints at low latencies. Apache Flink helps us transform and discover metrics in near-real time in our solution. In this talk, we will cover the challenges that we faced, how we scale the infrastructure to meet the needs of ML teams across Alexa, and go into how we enable specific use cases that use Apache Flink on Amazon Kinesis Data Analytics to improve Alexa experiences to delight our customers while preserving their privacy. by Aansh Shah

“Alexa, be quiet!”: End-to-end near-real time model building and evaluation i...

Flink Forward

Flink Forward San Francisco 2022. Probably everyone who has written stateful Apache Flink applications has used one of the fault-tolerant keyed state primitives ValueState, ListState, and MapState. With RocksDB, however, retrieving and updating items comes at an increased cost that you should be aware of. Sometimes, these may not be avoidable with the current API, e.g., for efficient event-time stream-sorting or streaming joins where you need to iterate one or two buffered streams in the right order. With FLIP-220, we are introducing a new state primitive: BinarySortedMultiMapState. This new form of state offers you to (a) efficiently store lists of values for a user-provided key, and (b) iterate keyed state in a well-defined sort order. Both features can be backed efficiently by RocksDB with a 2x performance improvement over the current workarounds. This talk will go into the details of the new API and its implementation, present how to use it in your application, and talk about the process of getting it into Flink. by Nico Kruber

Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...

Flink Forward

Flink Forward San Francisco 2022. The Apache Flink Kubernetes Operator provides a consistent approach to manage Flink applications automatically, without any human interaction, by extending the Kubernetes API. Given the increasing adoption of Kubernetes based Flink deployments the community has been working on a Kubernetes native solution as part of Flink that can benefit from the rich experience of community members and ultimately make Flink easier to adopt. In this talk we give a technical introduction to the Flink Kubernetes Operator and demonstrate the core features and use-cases through in-depth examples." by Thomas Weise

Introducing the Apache Flink Kubernetes Operator

Flink Forward

Flink Forward San Francisco 2022. Resource Elasticity is a frequently requested feature in Apache Flink: Users want to be able to easily adjust their clusters to changing workloads for resource efficiency and cost saving reasons. In Flink 1.13, the initial implementation of Reactive Mode was introduced, later releases added more improvements to make the feature production ready. In this talk, we’ll explain scenarios to deploy Reactive Mode to various environments to achieve autoscaling and resource elasticity. We’ll discuss the constraints to consider when planning to use this feature, and also potential improvements from the Flink roadmap. For those interested in the internals of Flink, we’ll also briefly explain how the feature is implemented, and if time permits, conclude with a short demo. by Robert Metzger

Autoscaling Flink with Reactive Mode

Flink Forward

Flink Forward San Francisco 2022. Flink consumers read from Kafka as a scalable, high throughput, and low latency data source. However, there are challenges in scaling out data streams where migration and multiple Kafka clusters are required. Thus, we introduced a new Kafka source to read sharded data across multiple Kafka clusters in a way that conforms well with elastic, dynamic, and reliable infrastructure. In this presentation, we will present the source design and how the solution increases application availability while reducing maintenance toil. Furthermore, we will describe how we extended the existing KafkaSource to provide mechanisms to read logical streams located on multiple clusters, to dynamically adapt to infrastructure changes, and to perform transparent cluster migrations and failover. by Mason Chen

Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...

Flink Forward

Flink Forward San Francisco 2022. Next time you want to integrate with a new destination for a demo, concept or production application, the Async Sink framework will bootstrap development, allowing you to move quickly without compromise. In Flink 1.15 we introduced the Async Sink base (FLIP-171), with the goal to encapsulate common logic and allow developers to focus on the key integration code. The new framework handles things like request batching, buffering records, applying backpressure, retry strategies, and at least once semantics. It allows you to focus on your business logic, rather than spending time integrating with your downstream consumers. During the session we will dive deep into the internals to uncover how it works, why it was designed this way, and how to use it. We will code up a new sink from scratch and demonstrate how to quickly push data to a destination. At the end of this talk you will be ready to start implementing your own Flink sink using the new Async Sink framework. by Steffen Hausmann & Danny Cranmer

One sink to rule them all: Introducing the new Async Sink

Flink Forward

Flink Forward San Francisco 2022. In normal situations, the default Kafka consumer and producer configuration options work well. But we all know life is not all roses and rainbows and in this session we’ll explore a few knobs that can save the day in atypical scenarios. First, we'll take a detailed look at the parameters available when reading from Kafka. We’ll inspect the params helping us to spot quickly an application lock or crash, the ones that can significantly improve the performance and the ones to touch with gloves since they could cause more harm than benefit. Moreover we’ll explore the partitioning options and discuss when diverging from the default strategy is needed. Next, we’ll discuss the Kafka Sink. After browsing the available options we'll then dive deep into understanding how to approach use cases like sinking enormous records, managing spikes, and handling small but frequent updates.. If you want to understand how to make your application survive when the sky is dark, this session is for you! by Olena Babenko

Tuning Apache Kafka Connectors for Flink.pptx

Flink Forward

Flink Forward San Francisco 2022. Pinterest is a visual discovery engine that serves over 433MM users. Stream processing allows us to unlock value from realtime data for pinners. At Pinterest, we adopt Flink as the unified streaming processing engine. In this talk, we will share our journey in building a stream processing platform with Flink and how we onboarding critical use cases to the platform. Pinterest has supported 90+near realtime streaming applications. We will cover the problem statement, how we evaluate potential solutions and our decision to build the framework. by Rainie Li & Kanchi Masalia

Flink powered stream processing platform at Pinterest

Flink Forward

Flink Forward San Francisco 2022. This talk will take you on the long journey of Apache Flink into the cloud-native era. It started all the way from where Hadoop and YARN were the standard way of deploying and operating data applications. We're going to deep dive into the cloud-native set of principles and how they map to the Apache Flink internals and recent improvements. We'll cover fast checkpointing, fault tolerance, resource elasticity, minimal infrastructure dependencies, industry-standard tooling, ease of deployment and declarative APIs. After this talk you'll get a broader understanding of the operational requirements for a modern streaming application and where the current limits are. by David Moravek

Apache Flink in the Cloud-Native Era

Flink Forward

Flinkn Forward San Francisco 2022. In this talk, we will cover various topics around performance issues that can arise when running a Flink job and how to troubleshoot them. We’ll start with the basics, like understanding what the job is doing and what backpressure is. Next, we will see how to identify bottlenecks and which tools or metrics can be helpful in the process. Finally, we will also discuss potential performance issues during the checkpointing or recovery process, as well as and some tips and Flink features that can speed up checkpointing and recovery times. by Piotr Nowojski

Where is my bottleneck? Performance troubleshooting in Flink

Flink Forward

Flink Forward San Francisco 2022. Running natively on Kubernetes, using the new Apache Flink Kubernetes Operator is a great way to deploy and manage Flink application and session deployments. In this presentation, we provide: - A brief overview of Kubernetes operators and their benefits. - Introduce the five levels of the operator maturity model. - Introduce the newly released Apache Flink Kubernetes Operator and FlinkDeployment CRs - Dockerfile modifications you can make to swap out UBI images and Java of the underlying Flink Operator container - Enhancements we're making in: - Versioning/Upgradeability/Stability - Security - Demo of the Apache Flink Operator in-action, with a technical preview of an upcoming product using the Flink Kubernetes Operator. - Lessons learned - Q&A by James Busche & Ted Chang

Using the New Apache Flink Kubernetes Operator in a Production Deployment

Flink Forward

Flink Forward San Francisco 2022. The Table API is one of the most actively developed components of Flink in recent time. Inspired by databases and SQL, it encapsulates concepts many developers are familiar with. It can be used with both bounded and unbounded streams in a unified way. But from afar it can be difficult to keep track of what this API is capable of and how it relates to Flink's other APIs. In this talk, we will explore the current state of Table API. We will show how it can be used as a batch processor, a changelog processor, or a streaming ETL tool with many built-in functions and operators for deduplicating, joining, and aggregating data. By comparing it to the DataStream API we will highlight differences and elaborate on when to use which API. We will demonstrate hybrid pipelines in which both APIs interact with one another and contribute their unique strengths. Finally, we will take a look at some of the most recent additions as a first step to stateful upgrades. by David Andreson

The Current State of Table API in 2022

Flink Forward

Flink Forward San Francisco 2022. Based on the new Flink-Pulsar connector, we implemented Flink's TableAPI and Catalog to help users to interact with the Pulsar cluster via Flink SQL easily. We would like to go through the design and implementation of the SQL connector in the following aspects: 1. Two different modes of use Pulsar as a metadata store 2. Data format transformation and management 3. SQL semantics support within Pulsar context by Sijie Guo & Neng Lu

Flink SQL on Pulsar made easy

Flink Forward

Flink Forward San Francisco 2022. At Bloomberg, we deal with high volumes of real-time market data. Our clients expect to be notified of any anomalies in this market data, which may indicate volatile movements in the markets, notable trades, forthcoming events, or system failures. The parameters for these alerts are always evolving and our clients can update them dynamically. In this talk, we'll cover how we utilized the open source Apache Flink and Siddhi SQL projects to build a distributed, scalable, low-latency and dynamic rule-based, real-time alerting system to solve our clients' needs. We'll also cover the lessons we learned along our journey. by Ajay Vyasapeetam & Madhuri Jain

Dynamic Rule-based Real-time Market Data Alerts

Flink Forward

Flink Forward San Francisco 2022. At Stripe we have created a complete end to end exactly-once processing pipeline to process financial data at scale, by combining the exactly-once power from Flink, Kafka, and Pinot together. The pipeline provides exactly-once guarantee, end-to-end latency within a minute, deduplication against hundreds of billions of keys, and sub-second query latency against the whole dataset with trillion level rows. In this session we will discuss the technical challenges of designing, optimizing, and operating the whole pipeline, including Flink, Kafka, and Pinot. We will also share our lessons learned and the benefits gained from exactly-once processing. by Xiang Zhang & Pratyush Sharma & Xiaoman Dong

Exactly-Once Financial Data Processing at Scale with Flink and Pinot

Flink Forward

Flink Forward San Francisco 2022. What if my data is already in order? Stream Processing has given us an elegant and powerful solution for running analytic queries and logic over high volumes of continuously arriving data. However, in both Apache Flink and Apache Beam, the notion of time-ordering is baked in at a very low level, making it difficult to express computations that are interested in a semantic-, rather than time-ordering of the data. In financial services, what often matters the most about the data moving between systems is not when the data was created, but in what order, to the extent that many institutions engineer a global sequencing over all data entering and produced by their systems to achieve complete determinism. How, then, can financial institutions and others best employ Stream Processing on streams of data that are already ordered? I will cover various techniques that can make this work, as well as seek input from the community on how Flink might be improved to better support these use-cases. by Patrick Lucas

Processing Semantically-Ordered Streams in Financial Services

Flink Forward

Flink Forward San Francisco 2022. In modern data platform architectures, stream processing engines such as Apache Flink are used to ingest continuous streams of data into data lakes such as Apache Iceberg. Streaming ingestion to iceberg tables can suffer by two problems (1) small files problem that can hurt read performance (2) poor data clustering that can make file pruning less effective. To address those two problems, we propose adding a shuffling stage to the Flink Iceberg streaming writer. The shuffling stage can intelligently group data via bin packing or range partition. This can reduce the number of concurrent files that every task writes. It can also improve data clustering. In this talk, we will explain the motivations in details and dive into the design of the shuffling stage. We will also share the evaluation results that demonstrate the effectiveness of smart shuffling. by Gang Ye & Steven Wu

Tame the small files problem and optimize data layout for streaming ingestion...

Flink Forward

Flink Forward San Francisco 2022. Goldman Sachs's Data Lake platform serves as the firm's centralized data platform, ingesting 140K (and growing!) batches per day of Datasets of varying shape and size. Powered by Flink and using metadata configured by platform users, ingestion applications are generated dynamically at runtime to extract, transform, and load data into centralized storage where it is then exported to warehousing solutions such as Sybase IQ, Snowflake, and Amazon Redshift. Data Latency is one of many key considerations as producers and consumers have their own commitments to satisfy. Consumers range from people/systems issuing queries, to applications using engines like Spark, Hive, and Presto to transform data into refined Datasets. Apache Iceberg allows our applications to not only benefit from consistency guarantees important when running on eventually consistent storage like S3, but also allows us the opportunity to improve our batch processing patterns with its scalability-focused features. by Andreas Hailu

Batch Processing at Scale with Flink & Iceberg

Flink Forward

More from Flink Forward (20)

Building a fully managed stream processing platform on Flink at scale for Lin...

Evening out the uneven: dealing with skew in Flink

“Alexa, be quiet!”: End-to-end near-real time model building and evaluation i...

Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...

Introducing the Apache Flink Kubernetes Operator

Autoscaling Flink with Reactive Mode

Dynamically Scaling Data Streams across Multiple Kafka Clusters with Zero Fli...

One sink to rule them all: Introducing the new Async Sink

Tuning Apache Kafka Connectors for Flink.pptx

Flink powered stream processing platform at Pinterest

Apache Flink in the Cloud-Native Era

Where is my bottleneck? Performance troubleshooting in Flink

Using the New Apache Flink Kubernetes Operator in a Production Deployment

The Current State of Table API in 2022

Flink SQL on Pulsar made easy

Dynamic Rule-based Real-time Market Data Alerts

Exactly-Once Financial Data Processing at Scale with Flink and Pinot

Processing Semantically-Ordered Streams in Financial Services

Tame the small files problem and optimize data layout for streaming ingestion...

Batch Processing at Scale with Flink & Iceberg

Recently uploaded

Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...

nirzagarg

5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed

amy56318795

Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...

gajnagarg

Saudi Arabia [ Abortion pills) Jeddah/riaydh/dammam/+966572737505☎️] cytotec tablets uses abortion pills 💊💊 How effective is the abortion pill? 💊💊 +966572737505) "Abortion pills in Jeddah" how to get cytotec tablets in Riyadh " Abortion pills in dammam*💊💊 The abortion pill is very effective. If you’re taking mifepristone and misoprostol, it depends on how far along the pregnancy is, and how many doses of medicine you take:💊💊 +966572737505) how to buy cytotec pills At 8 weeks pregnant or less, it works about 94-98% of the time. +966572737505[ 💊💊💊 At 8-9 weeks pregnant, it works about 94-96% of the time. +966572737505) At 9-10 weeks pregnant, it works about 91-93% of the time. +966572737505)💊💊 If you take an extra dose of misoprostol, it works about 99% of the time. At 10-11 weeks pregnant, it works about 87% of the time. +966572737505) If you take an extra dose of misoprostol, it works about 98% of the time. In general, taking both mifepristone and+966572737505 misoprostol works a bit better than taking misoprostol only. +966572737505 Taking misoprostol alone works to end the+966572737505 pregnancy about 85-95% of the time — depending on how far along the+966572737505 pregnancy is and how you take the medicine. +966572737505 The abortion pill usually works, but if it doesn’t, you can take more medicine or have an in-clinic abortion. +966572737505 When can I take the abortion pill?+966572737505 In general, you can have a medication abortion up to 77 days (11 weeks)+966572737505 after the first day of your last period. If it’s been 78 days or more since the first day of your last+966572737505 period, you can have an in-clinic abortion to end your pregnancy.+966572737505 Why do people choose the abortion pill? Which kind of abortion you choose all depends on your personal+966572737505 preference and situation. With+966572737505 medication+966572737505 abortion, some people like that you don’t need to have a procedure in a doctor’s office. You can have your medication abortion on your own+966572737505 schedule, at home or in another comfortable place that you choose.+966572737505 You get to decide who you want to be with during your abortion, or you can go it alone. Because+966572737505 medication abortion is similar to a miscarriage, many people feel like it’s more “natural” and less invasive. And some+966572737505 people may not have an in-clinic abortion provider close by, so abortion pills are more available to+966572737505 them. +966572737505 Your doctor, nurse, or health center staff can help you decide which kind of abortion is best for you. +966572737505 More questions from patients: Saudi Arabia+966572737505 CYTOTEC Misoprostol Tablets. Misoprostol is a medication that can prevent stomach ulcers if you also take NSAID medications. It reduces the amount of acid in your stomach, which protects your stomach lining. The brand name of this medication is Cytotec®.+966573737505) Unwanted Kit is a combination of two medicin

Abortion pills in Jeddah | +966572737505 | Get Cytotec

Abortion pills in Riyadh +966572737505 get cytotec

原版定制【微信:176555708】【圣地亚哥州立大学毕业证（SDSU毕业证书）】【微信:176555708】（留信学历认证永久存档查询）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。【业务选择办理准则】一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信176555708】文凭即可二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信176555708】即可三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。留信网认证的作用: 1:该专业认证可证明留学生真实身份 2:同时对留学生所学专业登记给予评定 3:国家专业人才认证中心颁发入库证书 4:这个认证书并且可以归档倒地方 5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息 6:个人职称评审加20分 7:个人信誉贷款加10分 8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才留信网服务项目： 1、留学生专业人才库服务（留信分析） 2、国（境）学习人员提供就业推荐信服务 3、留学人员区块链存储服务 → 【关于价格问题（保证一手价格）】我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子我给客户的都是第一手的代理价格，因为我想坦诚对待大家不想跟大家在价格方面浪费时间对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。选择实体注册公司办理，更放心，更安全！我们的承诺：可来公司面谈，可签订合同，会陪同客户一起到教育部认证窗口递交认证材料，客户在教育部官方认证查询网站查询到认证通过结果后付款，不成功不收费！

怎样办理圣地亚哥州立大学毕业证（SDSU毕业证书）成绩单学校原版复制

vexqp

Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...

gajnagarg

PLE-statistics document for primary schs

cnajjemba

Klinik_ Apotek Onlin 085657271886 Solusi Menggugurkan Masalah Kehamilan Anda Jual Obat Aborsi Asli KLINIK ABORSI TERPEECAYA _ Jual Obat Aborsi Cytotec Misoprostol Asli 100% Ampuh Hanya 3 Jam Langsung Gugur || OBAT PENGGUGUR KANDUNGAN AMPUH MANJUR OBAT ABORSI OLINE" APOTIK Jual Obat Cytotec, Gastrul, Gynecoside Asli Ampuh. JUAL ” Obat Aborsi Tuntas | Obat Aborsi Manjur | Obat Aborsi Ampuh | Obat Penggugur Janin | Obat Pencegah Kehamilan | Obat Pelancar Haid | Obat terlambat Bulan | Ciri Obat Aborsi Asli | Obat Telat Bulan | Pil Aborsi Asli | Cara Menggugurkan Konten | Cara Aborsi Tuntas | Harga Obat Aborsi Asli | Pil Aborsi | Jual Obat Aborsi Cytotec | Cara Aborsi Sendiri | Cara Aborsi Usia 1 Bulan | Cara Aborsi Usia 2 Tahun | Cara Aborsi Usia 3 Bulan | Obat Aborsi Usia 4 Bulan | Cara Abrasi Usia 5 Bulan | Cara Menggugurkan Konten | Kandungan Obat Penggugur | Cara Menghitung Usia Konten | Cara Mengatasi Terlambat Bulan | Penjual Obat Aborsi Asli | Obat Aborsi Garansi | Kandungan Obat Peluntur | Obat Telat Datang Bulan | Obat Telat Haid | Obat Aborsi Paling Murah | Klinik Jual Obat Aborsi | Jual Pil Cytotec | Apotik Jual Obat Aborsi | Kandungan Dokter Abrasi | Cara Aborsi Cepat | Jual Obat Aborsi Bergaransi | Jual Obat Cytotec Asli | Obat Aborsi Aman Manjur | Obat Misoprostol Cytotec Asli. "APA ITU ABORSI" “Aborsi Adalah dengan membendung hormon yang di perlukan untuk mempertahankan kehamilan yaitu hormon progesteron, karena hormon ini dibendung, maka jalur kehamilan mulai membuka dan leher rahim menjadi melunak,sehingga mengeluarkan darah yang merupakan tanda bahwa obat telah bekerja || maksimal 1 jam obat diminum || PENJELASAN OBAT ABORSI USIA 1 _7 BULAN Pada usia kandungan ini, pasien akan merasakan sakit yang sedikit tidak berlebihan || sekitar 1 jam ||. namun hanya akan terjadi pada saatdarah keluar merupakan pertanda menstruasi. Hal ini dikarenakan pada usiakandungan 3 bulan,janin sudah terbentuk sebesar kepalan tangan orang dewasa. Cara kerja obat aborsi : JUAL OBAT ABORSI AMPUH dosis 3 bulan secara umum sama dengan cara kerja || DOSIS OBAT ABORSI 2 bulan”, hanya berbedanya selain mengisolasijanin juga menghancurkan janin dengan formula methotrexate dikandungdidalamnya. Formula methotrexate ini sangat ampuh untuk menghancurkan janinmenjadi serpihan-serpihan kecil akan sangat berguna pada saat dikeluarkan nanti. APA ALASAN WANITA MELAKUKAN ABORSI? Aborsi di lakukan wanita hamil baik yang sudah menikah maupun belum menikah dengan berbagai alasan , akan tetapi alasan yang utama adalah alasan-alasan non medis (termasuk aborsi sendiri / di sengaja/ buatan] MELAYANI PEMESANAN OBAT ABORSI SETIAP HARI, SIAP KIRIM KESELURUH KOTA BESAR DI INDONESIA DAN LUAR NEGERI. HUBUNGI PEMESANAN LEBIH NYAMAN VIA WA/: 085657271886

Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...

ZurliaSoop

Building Real-Time Pipelines With FLaNK Timothy Spann, Principal Developer Advocate, Streaming - Cloudera Future of Data meetup, startup grind, AI Camp The combination of Apache Flink, Apache NiFi, and Apache Kafka for building real-time data processing pipelines is extremely powerful, as demonstrated by this case study using the FLaNK-MTA project. The project leverages these technologies to process and analyze real-time data from the New York City Metropolitan Transportation Authority (MTA). FLaNK-MTA demonstrates how to efficiently collect, transform, and analyze high-volume data streams, enabling timely insights and decision-making. Apache NiFi Apache Kafka Apache Flink Apache Iceberg LLM Generative AI Slack Postgresql

DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK

Timothy Spann

Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...

nirzagarg

Lecture_2_Deep_Learning_Overview-newone1

ranjankumarbehera14

Ranking and Scoring Exercises for Research

Rajesh Mondal

Yilin Xia (yilinx2@illinois.edu), Shawn Bowers (bowers@gonzaga.edu), Lan Li (lanl2@illinois.edu), and Bertram Ludäscher (ludaesch@illinois.edu) Presented at IDCC-2024 in Edinburg. ABSTRACT. We propose a new approach for modeling and reconciling conflicting data cleaning actions. Such conflicts arise naturally in collaborative data curation settings where multiple experts work independently and then aim to put their efforts together to improve and accelerate data cleaning. The key idea of our approach is to model conflicting updates as a formal argumentation framework (AF). Such argumentation frameworks can be automatically analyzed and solved by translating them to a logic program PAF whose declarative semantics yield a transparent solution with many desirable properties, e.g., uncontroversial updates are accepted, unjustified ones are rejected, and the remaining ambiguities are exposed and presented to users for further analysis. After motivating the problem, we introduce our approach and illustrate it with a detailed running example introducing both well-founded and stable semantics to help understand the AF solutions. We have begun to develop open source tools and Jupyter notebooks that demonstrate the practicality of our approach. In future work we plan to develop a toolkit for conflict resolution that can be used in conjunction with OpenRefine, a popular interactive data cleaning tool.

Reconciling Conflicting Data Curation Actions: Transparency Through Argument...

Bertram Ludäscher

Digital Transformation Playbook by Graham Ware

Graham Ware

Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models We are available 24*7 Booking Contact Details :- WhatsApp Chat :- +91-7014168258 If you're looking for India Call girls you've come to the right place. You'll find some of the most beautiful call girls in our location with. These ladies have pleasing personalities, hot figures, and a passion for physical pleasure. Call girls in India Lucknow Many men have booked them for their erotic and soul-mixing performances, which are sure to leave you with unforgettable memories. #K09 Escort Service India is available in the city for men and women of all ages. They can satisfy your sexual needs and will make your experience even more enjoyable and memorable. Whether you're looking for a blow-job, stripping, lovemaking, or other dirty acts, you'll be able to find a match for your tastes and budget. These highly trained professionals will help you have an unforgettable night. One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7014168258 We are available 24*7 all days of the year. Call us — 7014168258 Thank you for Visiting.

Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...

nirzagarg

Cytotec in Jeddah+966572737505) get unwanted pregnancy kit Riyadh

Abortion pills in Riyadh +966572737505 get cytotec

Discover Why Less is More in B2B Research

michael115558

原版定制【微信:176555708】【伦敦大学城市学院毕业证（CITY毕业证书）】【微信:176555708】（留信学历认证永久存档查询）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。【业务选择办理准则】一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信176555708】文凭即可二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信176555708】即可三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。留信网认证的作用: 1:该专业认证可证明留学生真实身份 2:同时对留学生所学专业登记给予评定 3:国家专业人才认证中心颁发入库证书 4:这个认证书并且可以归档倒地方 5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息 6:个人职称评审加20分 7:个人信誉贷款加10分 8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才留信网服务项目： 1、留学生专业人才库服务（留信分析） 2、国（境）学习人员提供就业推荐信服务 3、留学人员区块链存储服务 → 【关于价格问题（保证一手价格）】我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子我给客户的都是第一手的代理价格，因为我想坦诚对待大家不想跟大家在价格方面浪费时间对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。选择实体注册公司办理，更放心，更安全！我们的承诺：可来公司面谈，可签订合同，会陪同客户一起到教育部认证窗口递交认证材料，客户在教育部官方认证查询网站查询到认证通过结果后付款，不成功不收费！

怎样办理伦敦大学城市学院毕业证（CITY毕业证书）成绩单学校原版复制

vexqp

This PowerPoint presentation likely explores the journey and evolution of boAt as a brand known for its innovative approach in the consumer electronics industry. The presentation may cover several key aspects: Introduction to boAt: Overview of the company's founding, vision, and initial products. Innovation Highlights: Exploration of boAt's innovative strategies in product design, technology integration, and market positioning. Market Impact: Discussion on how boAt has disrupted the audio electronics market with its unique offerings and consumer-centric approach. Brand Building: Insights into the marketing and branding strategies employed by boAt to create a strong and recognizable brand identity. Growth Trajectory: Analysis of boAt's growth story, including milestones, challenges faced, and strategies for expansion. Future Outlook: Speculation or plans for the future of boAt, including potential areas of innovation and growth opportunities. Lessons Learned: Takeaways and lessons that can be gleaned from boAt's success story, particularly in the context of entrepreneurship and innovation.

The-boAt-Story-Navigating-the-Waves-of-Innovation.pptx

Vivek487417

原版定制【微信:176555708】【伦敦大学毕业证（UoL毕业证书）】【微信:176555708】（留信学历认证永久存档查询）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。【业务选择办理准则】一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信176555708】文凭即可二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信176555708】即可三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。留信网认证的作用: 1:该专业认证可证明留学生真实身份 2:同时对留学生所学专业登记给予评定 3:国家专业人才认证中心颁发入库证书 4:这个认证书并且可以归档倒地方 5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息 6:个人职称评审加20分 7:个人信誉贷款加10分 8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才留信网服务项目： 1、留学生专业人才库服务（留信分析） 2、国（境）学习人员提供就业推荐信服务 3、留学人员区块链存储服务 → 【关于价格问题（保证一手价格）】我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子我给客户的都是第一手的代理价格，因为我想坦诚对待大家不想跟大家在价格方面浪费时间对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。选择实体注册公司办理，更放心，更安全！我们的承诺：可来公司面谈，可签订合同，会陪同客户一起到教育部认证窗口递交认证材料，客户在教育部官方认证查询网站查询到认证通过结果后付款，不成功不收费！

怎样办理伦敦大学毕业证（UoL毕业证书）成绩单学校原版复制

vexqp

Recently uploaded (20)

Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...

5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed

Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...

Abortion pills in Jeddah | +966572737505 | Get Cytotec

怎样办理圣地亚哥州立大学毕业证（SDSU毕业证书）成绩单学校原版复制

Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...

PLE-statistics document for primary schs

Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...

DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK

Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...

Lecture_2_Deep_Learning_Overview-newone1

Ranking and Scoring Exercises for Research

Reconciling Conflicting Data Curation Actions: Transparency Through Argument...

Digital Transformation Playbook by Graham Ware

Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...

Cytotec in Jeddah+966572737505) get unwanted pregnancy kit Riyadh

Discover Why Less is More in B2B Research

怎样办理伦敦大学城市学院毕业证（CITY毕业证书）成绩单学校原版复制

The-boAt-Story-Navigating-the-Waves-of-Innovation.pptx

怎样办理伦敦大学毕业证（UoL毕业证书）成绩单学校原版复制

Eron Wright - Flink Security Enhancements

1. Flink Security Enhancements Eron Wright – eron.wright@emc.com DELL EMC @eronwright

2. 2 of 11 New Security Features 1. Kerberos Authentication Support 2. Service-Level Authorization 3. Transport Security (SSL/TLS)

3. 3 of 11 Existing Capability • Hadoop Delegation Token (DT) • CLI usesKerberosto authenticateto HDFS • HDFSprovidesa DT, which CLI passesto the Flinkcluster • Clusteris ableto accessHDFSfilesonbehalfof theuser • Limitations • YARN mode only • Not usefulto non-Hadoopservices,e.g. Kafka. • Note: Still supported TM TM DATA AKKA JM CLI WEB BROWSER KAFKA HDFSZK HTTP Flink Cluster delegation token

4. 4 of 11 Kerberos Authentication Support • “Cluster-Level Kerberos Identity” • Keytab-based • Sharedby alljobs, notjob-specific • Enables Kerberos authentication • DataSourcesandSinks(HDFS,Kafka…) • StateBackends(ZooKeeper…) • Protects state data • ACL onznodes,files • Supported in standalone and YARN deployment modes TM TM DATA AKKA JM CLI WEB BROWSER KAFKA HDFSZK HTTP Flink Cluster keytab

5. 5 of 11 Service-Level Authorization • “Restrict access to your Flink cluster” • Protects all endpoints: • Akka System(control path) • Intra-ClusterDataTransfer • WebUI • BlobTransfer(JARs…) • Simple shared secret • Configuredor generated • Storedonclient (~/.flink/…) • Storedincluster • Supported in standalone and YARN TM TM DATA AKKA JM CLI WEB BROWSER KAFKA HDFSZK HTTP Flink Cluster keytab secret

6. 6 of 11 Transport-Level Security (SSL/TLS) • “SSL for all connections” • May be enabled on a per-endpoint basis • WebUIis problematic • Supported in standalone and YARN TM TM DATA AKKA JM CLI WEB BROWSER KAFKA HDFSZK HTTPS Flink Cluster keytab secret TLS cert(s)

7. Demo

8. 8 of 11 Configuration • Configure Kerberos Identity: – security.enabled: true – security.keytab: /path/to/keytab – security.principal: name@realm • Configure Service-Level Authorization: – security.cookie: (secret cookie) • Configure Transport-Level Security: – security.ssl.enabled: true – security.ssl.keystore: /path/to/keystore – security.ssl.keystore-password: (password) – security.ssl.key-password: (password) – security.ssl.truststore: /path/to/truststore – security.ssl.truststore-password: (password) TM TM DATA AKKA JM CLI WEB BROWSER KAFKA HDFSZK HTTPS Flink Cluster keytab secret TLS cert(s)

9. Summary

10. 10 of 11 Project Status • Targeted for: Flink 1.2 • Contributors: – Vijay Srinivasaraghavan (Dell EMC) – Suresh Krishnappa (Dell EMC) • Design Doc: Secure Data Access on Google Docs • JIRAs: – FLINK-3929 - Support for Kerberos Authentication with Keytab Credential – FLINK-3930 - Implement Service-Level Authorization – FLINK-3931 - Implement Transport Encryption (SSL/TLS) – FLINK-3932 - Implement State Backend Security • Code: – Github: https://github.com/EronWright/flink/tree/feature-flink-security

Eron Wright - Flink Security Enhancements

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Eron Wright - Flink Security Enhancements

Similar to Eron Wright - Flink Security Enhancements (20)

More from Flink Forward

More from Flink Forward (20)

Recently uploaded

Recently uploaded (20)

Eron Wright - Flink Security Enhancements