Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Alexander Kolb - Flinkspector – Taming the squirrel

862 views

Published on

http://flink-forward.org/kb_sessions/flinkspector-taming-the-squirrel/

The costs of logic errors in production for streaming applications are higher than for batch processing systems. Depending on the setup, errors cannot be rectified or have already influenced important decisions. The goal of Flinkspector is to improve the test process of Apache Flink streaming applications in order to detect streaming application logic errors early during development. It features dedicated mechanics for test setup, execution, and evaluation. While Flinkspector’s streamlined API keeps testing overhead small. The framework is able to handle non-terminating and parallelized data flows involving windowing. The lightweight integration-tests enabled by Flinkspector allow Flink applications to be included into the continuous integration and deployment process. The talk introduces the core functionality of Flinkspector. In addition, background concepts of the runtime and the evaluation algorithms are presented. https://github.com/ottogroup/flink-spector

Published in: Data & Analytics
  • Login to see the comments

Alexander Kolb - Flinkspector – Taming the squirrel

  1. 1. Flinkspector Taming the Squirrel
  2. 2. Alexander Kolb otto group BI @lofifnc Alexander Kolb
 Otto Group BI @lofifnc
  3. 3. Alexander Kolb - 2016 - otto group Agenda • Introduction • Testing Apache Flink Applications • Flinkspector • Concept • Input • Expectations • Execution • Conclusion • Outlook
  4. 4. Alexander Kolb - 2016 - otto group Introduction
  5. 5. Alexander Kolb - 2016 - otto group • Multichannel Retail • Financial Services • Services • 30 countries • 123 companies
  6. 6. Alexander Kolb - 2016 - otto group Problem • Transparency • Traceability • Reproducibility
  7. 7. Alexander Kolb - 2016 - otto group Solutions • Visualisation • Datalineage graph / Message tracing • Unit Testing
  8. 8. Alexander Kolb - 2016 - otto group Testing Apache Flink Applications
  9. 9. Alexander Kolb - 2016 - otto group Apache Flink Application final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 
 DataStream<String> text = env.socketTextStream("localhost", 9999);
 
 
 DataStream<Tuple2<String, Integer>> words = text.flatMap(new WordCount.LineSplitter());
 
 
 DataStream<Tuple2<String, Integer>> aWords = words.filter(new StartsWithAFilter());
 
 
 DataStream<Tuple2<String, Integer>> counts = aWords.keyBy(0).timeWindow(Time.of(2, TimeUnit.MINUTES)).sum(1);
 
 counts.print();
 
 env.execute("Wordcount Example");
  10. 10. Alexander Kolb - 2016 - otto group User Defined Functions public final class StartsWithAFilter
 implements FilterFunction<Tuple2<String,Integer>> { 
 @Override
 public boolean filter(Tuple2<String, Integer> t) throws Exception {
 return t.f0.startsWith("a");
 }
 }
  11. 11. Alexander Kolb - 2016 - otto group User Defined Functions 
 @Test
 public void testAFilter() throws Exception {
 StartsWithAFilter filter = new StartsWithAFilter();
 
 Assert.assertEquals(true, filter.filter(Tuple2.of("aTest",1)));
 Assert.assertEquals(false, filter.filter(Tuple2.of("bTest",1)));
 }
  12. 12. Alexander Kolb - 2016 - otto group Stream Transformations public static DataStream<Tuple2<String,Integer>>
 countAWords(DataStream<Tuple2<String,Integer>> aWords) {
 return aWords
 .keyBy(0)
 .timeWindow(Time.of(2, TimeUnit.MINUTES))
 .sum(1);
 }
  13. 13. Alexander Kolb - 2016 - otto group Stream Transformations @Test
 public void aWordCountTest() {
 StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironment();
 env.setParallelism(1);
 
 DataStream<Tuple2<String, Integer>> words = env.fromCollection(Arrays.asList(
 Tuple2.of("a1", 1),
 Tuple2.of("a2", 1),
 Tuple2.of("a1", 1)
 ));
 
 DataStream<Tuple2<String, Integer>> results = SocketTextStreamWordCount.countAWords(words);
 
 Iterator<Tuple2<String, Integer>> output = DataStreamUtils.collect(results);
 
 
 Assert.assertEquals(output.next(), Tuple2.of("a1", 1));
 Assert.assertEquals(output.next(), Tuple2.of("a2", 2));
 Assert.assertEquals(output.hasNext(), false);
 }
  14. 14. Alexander Kolb - 2016 - otto group Flinkspector
  15. 15. Alexander Kolb - 2016 - otto group Concept 1. Specify input. 2. Aquire data stream from the input specification. 3. Define expectations for the resulting data stream. 4. Apply the expectations to the produced data stream.
  16. 16. Alexander Kolb - 2016 - otto group TestStreamEnvironment Runtime Subscriber Test Source Input Filter KeyedWindowReducer Concept Verifier Test Sink Publisher Trigger (default) paired
  17. 17. Alexander Kolb - 2016 - otto group Expectations
  18. 18. Alexander Kolb - 2016 - otto group Input EventTimeInput<Tuple2<String,Integer>> input =
 EventTimeInputBuilder.startWith(Tuple2.of("a1", 1))
 .emit(Tuple2.of("a2", 1), after(1, minutes))
 .emit(Tuple2.of("a3", 1), before(1, seconds), times(2)) .emit(Tuple3.of(“a4”, 1), intoWindow(4, seconds))
 .repeatAll(after(1, minutes), times(3));
  19. 19. Alexander Kolb - 2016 - otto group List Based ExpectedRecords.create(asList(1,2,3,4)); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().only(); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().sameFrequency(); 
 
 (4,1,2,3,3,5) (4,1,2,3,3,5) (4,1,2,3,3,5)
  20. 20. Alexander Kolb - 2016 - otto group List Based (4,1,2,3,3,5) (4,1,2,3,3,5) (4,1,2,3,3,5) ExpectedRecords.create(asList(1,2,3,4)); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().only(); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().sameFrequency(); 
 

  21. 21. Alexander Kolb - 2016 - otto group List Based (4,1,2,3,3,5) (4,1,2,3,3,5) (4,1,2,3,3,5) ExpectedRecords.create(asList(1,2,3,4)); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().only(); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().sameFrequency(); 
 

  22. 22. Alexander Kolb - 2016 - otto group List Based (4,1,2,3,3,5) (4,1,2,3,3) (4,1,2,3,3,5) ExpectedRecords.create(asList(1,2,3,4)); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().only(); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().sameFrequency(); 
 

  23. 23. Alexander Kolb - 2016 - otto group List Based (4,1,2,3,3,5) (4,1,2,3,3) (4,1,2,3,3,5) ExpectedRecords.create(asList(1,2,3,4)); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().only(); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().sameFrequency(); 
 

  24. 24. Alexander Kolb - 2016 - otto group List Based (4,1,2,3,3,5) (4,1,2,3,3) (4,1,2,3,5) ExpectedRecords.create(asList(1,2,3,4)); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().only(); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().sameFrequency(); 
 

  25. 25. Alexander Kolb - 2016 - otto group List Based ExpectedRecords.create(asList(1,2,3,4))
 .refine().inOrder(notStrict).all(); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().inOrder(strict).from(1); List(4,1,2,3,3,5) List(5,2,3,4,1)
  26. 26. Alexander Kolb - 2016 - otto group List Based List(4,1,2,3,3,5) List(5,2,3,4,1) ExpectedRecords.create(asList(1,2,3,4))
 .refine().inOrder(notStrict).all(); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().inOrder(strict).from(1);
  27. 27. Alexander Kolb - 2016 - otto group List Based List(1,2,3,3,5,4) List(5,2,3,4,1) ExpectedRecords.create(asList(1,2,3,4))
 .refine().inOrder(notStrict).all(); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().inOrder(strict).from(1);
  28. 28. Alexander Kolb - 2016 - otto group List Based List(1,2,3,3,5,4) List(5,2,3,4,1) ExpectedRecords.create(asList(1,2,3,4))
 .refine().inOrder(notStrict).all(); 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine().inOrder(strict).from(1);
  29. 29. Alexander Kolb - 2016 - otto group List Based List(1,2,3,4,1) MatcherBuilder only sameFrequency order 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine() .only() .sameFrequency() .inOrder(strict).from(1);
  30. 30. Alexander Kolb - 2016 - otto group List Based 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine() .only() .sameFrequency() .inOrder(strict).from(1); List(1,2,3,4,1) MatcherBuilder only sameFrequency order
  31. 31. Alexander Kolb - 2016 - otto group List Based List(1,2,3,4,1) MatcherBuilder only sameFrequency order 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine() .only() .sameFrequency() .inOrder(strict).from(1);
  32. 32. Alexander Kolb - 2016 - otto group List Based List(1,2,3,4,1) MatcherBuilder only sameFrequency order 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine() .only() .sameFrequency() .inOrder(strict).from(1);
  33. 33. Alexander Kolb - 2016 - otto group List Based List(2,3,4,1) MatcherBuilder only sameFrequency order 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine() .only() .sameFrequency() .inOrder(strict).from(1);
  34. 34. Alexander Kolb - 2016 - otto group List Based List(2,3,4,1) MatcherBuilder only sameFrequency order 
 
 ExpectedRecords.create(asList(1,2,3,4))
 .refine() .only() .sameFrequency() .inOrder(strict).from(1);
  35. 35. Alexander Kolb - 2016 - otto group Assertion Based new MatchTuples<Tuple2<String,Integer>>("word","count")
 .assertThat("word", startsWith("a"))
 .assertThat("count", greaterThan(3))
 .onEachRecord();
  36. 36. Alexander Kolb - 2016 - otto group Assertion Based new MatchTuples<Tuple2<String,Integer>>("word","count")
 .assertThat("word", startsWith("a"))
 .assertThat("count", greaterThan(3))
 .oneOfThem()
 .onEachRecord();
  37. 37. Alexander Kolb - 2016 - otto group Assertion Based .assertThat(A,…) .assertThat(B,…) .assertThat(C,…) .eachOfThem() .onAnyRecord() A B C
  38. 38. Alexander Kolb - 2016 - otto group Assertion Based A B C .assertThat(A,…) .assertThat(B,…) .assertThat(C,…) .eachOfThem() .onAnyRecord()
  39. 39. Alexander Kolb - 2016 - otto group Assertion Based .assertThat(A,…) .assertThat(B,…) .eachOfThem() .onAnyRecord() A B C
  40. 40. Alexander Kolb - 2016 - otto group Assertion Based .assertThat(A,…) .assertThat(B,…) .assertThat(C,…) .eachOfThem() .onAnyRecord() A B C
  41. 41. Alexander Kolb - 2016 - otto group Assertion Based .assertThat(A,…) .assertThat(A,…) .assertThat(A,…) .atLeastNOfThem(2) .onEach() A B C
  42. 42. Alexander Kolb - 2016 - otto group Assertion Based .assertThat(A,…) .assertThat(B,…) .assertThat(C,…) .atExactlyNOfThem(2) .onEach() A B C
  43. 43. Alexander Kolb - 2016 - otto group Assertion Based public class Each<T> extends UntilCombineMatcher<T> {
 
 public Each(Iterable<Matcher<? super T>> matchers) {
 super(matchers);
 }
 
 @Override
 public String prefix() {
 return "each of";
 }
 
 @Override
 public boolean validWhen(int matches, int possibleMatches) {
 return matches == possibleMatches;
 }
 
 @Factory
 public static <T> Each<T> each(Iterable<Matcher<? super T>> matchers) {
 return new Each<T>(matchers);
 }
 }
  44. 44. Alexander Kolb - 2016 - otto group TestStreamEnvironment Runtime Subscriber Test Source Input Filter KeyedWindowReducer Execution Verifier Test Sink Publisher Trigger (default) paired
  45. 45. Alexander Kolb - 2016 - otto group Execution Test SourceInput Filter KeyedWindowReducer Test Sink Publisher Test Source Input Filter KeyedWindowReducer Test Sink Publisher 1 1 1 1 2 2 2 2 LocalCluster TestStreamEnvironment Runtime Subscriber Verifier Trigger (default)
  46. 46. Alexander Kolb - 2016 - otto group Execution LocalCluster TS I1 F KWR TSi P1 1 1 TS I2 2 F 2 KWR 1 1 2 TSi P2 2 TestStreamEnvironment Runtime Subscriber Verifier Trigger (default) Timeout
  47. 47. Alexander Kolb - 2016 - otto group Execution TestStreamEnvironment Runtime Subscriber Verifier Trigger (default) LocalCluster TS I1 F KWR TSi P1 1 1 TS I2 2 F 2 KWR 1 1 2 TSi P2 2 Timeout
  48. 48. Alexander Kolb - 2016 - otto group Execution TestStreamEnvironment Runtime Subscriber Verifier Trigger (default) LocalCluster TS I1 F KWR TSi P1 1 1 TS I2 2 F 2 KWR 1 1 2 TSi P2 2 Timeout
  49. 49. Alexander Kolb - 2016 - otto group LocalCluster TSI1 F KWR TSiP1 1 1 TSI2 2 F 2 KWR 1 1 2 TSiP2 2 Execution TestStreamEnvironment Runtime Subscriber Verifier Trigger (default) Timeout CLOSE 1 CLOSE 2
  50. 50. Alexander Kolb - 2016 - otto group TSI1 F KWR TSiP1 1 1 TSI2 2 F 2 KWR 1 1 2 TSiP2 2 Execution TestStreamEnvironment Runtime Subscriber Verifier Trigger (default) LocalCluster
  51. 51. Alexander Kolb - 2016 - otto group Execution TestStreamEnvironment Runtime Subscriber Verifier Trigger LocalCluster TS I1 F KWR TSi P1 1 1 TS I2 2 F 2 KWR 1 1 2 TSi P2 2 TimeoutTimeout
  52. 52. Alexander Kolb - 2016 - otto group Execution TestStreamEnvironment Runtime Subscriber Verifier Trigger LocalCluster TS I1 F KWR TSi P1 1 1 TS I2 2 F 2 KWR 1 1 2 TSi P2 2
  53. 53. Alexander Kolb - 2016 - otto group Execution TestStreamEnvironment Runtime Subscriber Verifier Trigger (default) LocalCluster TS I1 F KWR TSi P1 1 1 TS I2 2 F 2 KWR 1 1 2 TSi P2 2 TimeoutTimeout
  54. 54. Alexander Kolb - 2016 - otto group Execution TestStreamEnvironment Runtime Subscriber Verifier Trigger (default) LocalCluster TS I1 F KWR TSi P1 1 1 TS I2 2 F 2 KWR 1 1 2 TSi P2 2 TimeoutTimeout
  55. 55. Alexander Kolb - 2016 - otto group Conclusion
  56. 56. Alexander Kolb - 2016 - otto group Conclusion • Small and concise test cases. • Extensible dedicated runtime. • Integration into existing test platforms.
 • ProcessingTimeWindows. • Input specification too expansive.

  57. 57. Alexander Kolb - 2016 - otto group Outlook
  58. 58. Alexander Kolb - 2016 - otto group Outlook • Scala (Test) support. • Symbolic testing integration. • System tests on the actual cluster.
  59. 59. Alexander Kolb - 2016 - otto group https://github.com/ottogroup/flink-spector
  60. 60. Alexander Kolb - 2016 - otto group ottogroup.com WE ARE HIRING!

×