Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Pipeline parallelism

2,264 views

Published on

Parallelism

Published in: Education, Technology
  • Login to see the comments

Pipeline parallelism

  1. 1. Exploiting Coarse-Grained Task, Data, and Pipeline Parallelism in Stream Programs Dr. C.V. Suresh Babu 1
  2. 2. Multicores Are Here! 512 Picochip PC102 256 Ambric AM2045 Cisco CSR-1 128 Intel Tflops 64 32 # of cores 16 Raw 8 Niagara Broadcom 1480 4 2 1 4004 8080 8086 286 386 486 Pentium 8008 1970 2 Raza XLR 1975 1980 1985 1990 Cavium Octeon Cell Opteron 4P Xeon MP Xbox360 PA-8800 Opteron Tanglewood Power4 PExtreme Power6 Yonah P2 P3 Itanium P4 Athlon Itanium 2 1995 2000 2005 20??
  3. 3. Multicores Are Here! 512 256 128 64 32 # of cores 16 8 4 For uniprocessors, Uniprocessors: C was: C •is the common Portable machine language •High Performance •Composable •Malleable •Maintainable Picochip PC102 Cisco CSR-1 Intel Tflops Raw 1 8086 286 386 486 Broadcom 1480 Pentium 8008 1970 3 8080 1975 1980 1985 1990 Raza XLR Niagara 2 4004 Ambric AM2045 Cavium Octeon Cell Opteron 4P Xeon MP Xbox360 PA-8800 Opteron Tanglewood Power4 PExtreme Power6 Yonah P2 P3 Itanium P4 Athlon Itanium 2 1995 2000 2005 20??
  4. 4. Multicores Are Here! What is the common machine language for multicores? 512 256 128 Picochip PC102 Ambric AM2045 Cisco CSR-1 Intel Tflops 64 32 # of cores 16 Raw 8 Niagara Broadcom 1480 4 2 1 4004 8080 8086 286 386 486 Pentium 8008 1970 4 Raza XLR 1975 1980 1985 1990 Cavium Octeon Cell Opteron 4P Xeon MP Xbox360 PA-8800 Opteron Tanglewood Power4 PExtreme Power6 Yonah P2 P3 Itanium P4 Athlon Itanium 2 1995 2000 2005 20??
  5. 5. Common Machine Languages Uniprocessors: Common Properties Multicores: Common Properties Single flow of control Multiple flows of control Single memory image Multiple local memories Differences: Differences: Number and capabilities of cores Register Allocation Communication Model ISA Instruction Selection Synchronization Model Functional Units Instruction Scheduling Register File von-Neumann languages represent the common properties and abstract away the differences 5 Need common machine language(s) for multicores
  6. 6. Streaming as a Common Machine Language AtoD • Regular and repeating computation FMDemod • Independent filters with explicit communication – Segregated address spaces and multiple program counters Scatter – Producer / Consumer dependencies – Enables powerful, whole-program transformations LPF2 LPF3 HPF1 • Natural expression of Parallelism: LPF1 HPF2 HPF3 Gather Adder Speaker 6
  7. 7. Types of Parallelism Task Parallelism – Parallelism explicit in algorithm – Between filters without producer/consumer relationship Scatter Gather 7 Task Data Parallelism – Peel iterations of filter, place within scatter/gather pair (fission) – parallelize filters with state Pipeline Parallelism – Between producers and consumers – Stateful filters can be parallelized
  8. 8. Types of Parallelism Task Parallelism – Parallelism explicit in algorithm Data Parallel – Between filters without Gather producer/consumer relationship Scatter Pipeline Scatter Gather Data 8 Task Data Parallelism – Between iterations of a stateless filter – Place within scatter/gather pair (fission) – Can’t parallelize filters with state Pipeline Parallelism – Between producers and consumers – Stateful filters can be parallelized
  9. 9. Types of Parallelism Traditionally: Scatter Gather Pipeline Scatter Data Parallelism – Data parallel loop (forall) Gather Data 9 Task Parallelism – Thread (fork/join) parallelism Task Pipeline Parallelism – Usually exploited in hardware
  10. 10. Problem Statement Given: – Stream graph with compute and communication estimate for each filter – Computation and communication resources of the target machine Find: – Schedule of execution for the filters that best utilizes the available parallelism to fit the machine resources 10
  11. 11. Our 3-Phase Solution Coarsen Granularity Data Parallelize Software Pipeline 1. Coarsen: Fuse stateless sections of the graph 2. Data Parallelize: parallelize stateless filters 3. Software Pipeline: parallelize stateful filters Compile to a 16 core architecture – 11 11.2x mean throughput speedup over single core
  12. 12. Outline • StreamIt language overview • Mapping to multicores – Baseline techniques – Our 3-phase solution 12
  13. 13. The StreamIt Project • Applications StreamIt Program – DES and Serpent [PLDI 05] – MPEG-2 [IPDPS 06] – SAR, DSP benchmarks, JPEG, … Front-end • Programmability – StreamIt Language (CC 02) – Teleport Messaging (PPOPP 05) – Programming Environment in Eclipse (P-PHEC 05) Annotated Java • Domain Specific Optimizations – Linear Analysis and Optimization (PLDI 03) – Optimizations for bit streaming (PLDI 05) – Linear State Space Analysis (CASES 05) Simulator (Java Library) Stream-Aware Optimizations • Architecture Specific Optimizations – Compiling for Communication-Exposed Architectures (ASPLOS 02) – Phased Scheduling (LCTES 03) – Cache Aware Optimization (LCTES 05) – Load-Balanced Rendering (Graphics Hardware 05) 13 Uniprocessor backend Cluster backend Raw backend IBM X10 backend C/C++ MPI-like C/C++ C per tile + msg code Streaming X10 runtime
  14. 14. Model of Computation • Synchronous Dataflow [Lee ‘92] A/D – Graph of autonomous filters – Communicate via FIFO channels Band Pass • Static I/O rates – Compiler decides on an order of execution (schedule) Detect – Static estimation of computation LED 14 Duplicate Detect Detect Detect LED LED LED
  15. 15. Example StreamIt Filter 0 1 2 3 4 5 6 7 8 9 10 11 FIR 0 1 output float→float filter FIR (int N, float[N] weights) { work push 1 pop 1 peek N { float result = 0; Stateless for (int i = 0; i < N; i++) { result += weights[i] ∗ peek(i); } pop(); push(result); } } 15 input
  16. 16. Example StreamIt Filter 0 1 2 3 4 5 6 7 8 9 10 11 FIR 0 1 output float→float filter FIR (int N, float[N] weights) { N) { ; Stateful work push 1 pop 1 peek N { float result = 0; weights = adaptChannel(weights); for (int i = 0; i < N; i++) { result += weights[i] ∗ peek(i); } pop(); push(result); } } 16 input
  17. 17. StreamIt Language Overview • StreamIt is a novel language for streaming – Exposes parallelism and communication – Architecture independent – Modular and composable – Simple structures composed to creates complex graphs filter pipeline may be any StreamIt language construct splitjoin splitter parallel computation joiner – Malleable – Change program behavior with small modifications feedback loop joiner 17 splitter
  18. 18. Outline • StreamIt language overview • Mapping to multicores – Baseline techniques – Our 3-phase solution 18
  19. 19. Baseline 1: Task Parallelism • Inherent task parallelism between two processing pipelines Splitter BandPass BandPass Compress Compress Process Process Expand Expand BandStop BandStop Joiner Adder 19 • Task Parallel Model: – Only parallelize explicit task parallelism – Fork/join parallelism • Execute this on a 2 core machine ~2x speedup over single core • What about 4, 16, 1024, … cores?
  20. 20. Throughput Normalized to Single Core StreamIt Evaluation: Task Parallelism Raw Microprocessor Parallelism: Not matched to target! 16 inorder, single-issue cores with D$ and I$ Synchronization: Not matched to with DMA 16 memory banks, each bank target! 19 18 17 16 Cycle accurate simulator 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 20 n M ea da r m et ric R a G eo er oc od V od er G 2D ec T D E P E M t S er pe n F M R ad i o k er ba n F ilt T F F D E S T D C oc lV nn e C ha B it o ni cS or t od e r 0
  21. 21. Baseline 2: Fine-Grained Data Parallelism Splitter Splitter Joiner Splitter BandPass BandPass BandPass BandPass BandPass BandPass BandPass BandPass Splitter Splitter Compress Compress Compress Compress Compress Compress Compress Compress Joiner Joiner Splitter Process Process Process Process Joiner Splitter Splitter Expand Expand Expand Expand BandStop BandStop BandStop BandStop Process Process Process Process Expand Expand Expand Expand Joiner Splitter Splitter Joiner Splitter BandStop BandStop BandStop BandStop Joiner Joiner Splitter – Fiss each stateless filter N ways (N is number of cores) – Remove scatter/gather if possible • We can introduce data parallelism Joiner Joiner – Example: 4 cores • Each fission group occupies entire machine BandStop BandStop BandStop Adder Adder Joiner 21 Joiner • Each of the filters in the example are stateless • Fine-grained Data Parallel Model:
  22. 22. 22 G E et ri c ea n ad ar M R Vo co de r G 2D ec od er eo m PE TD Se rp en t ad io 16 R an k Task Fine-Grained Data FM T ES C FF T D D 17 Vo co de r or t 18 Fi lte rb ha nn el Throughput Normalized to Single Core StreamIt 19 M C Bi to ni cS Evaluation: Fine-Grained Data Parallelism Good Parallelism! Too Much Synchronization! 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
  23. 23. Outline • StreamIt language overview • Mapping to multicores – Baseline techniques – Our 3-phase solution 23
  24. 24. Phase 1: Coarsen the Stream Graph Splitter BandPass Peek BandPass Compress Compress Process Process Expand Expand BandStop Peek Joiner Adder 24 Peek BandStop Peek • Before data-parallelism is exploited • Fuse stateless pipelines as much as possible without introducing state – Don’t fuse stateless with stateful – Don’t fuse a peeking filter with anything upstream
  25. 25. Phase 1: Coarsen the Stream Graph Splitter BandPass Compress Process Expand BandPass Compress Process Expand BandStop BandStop • Before data-parallelism is exploited • Fuse stateless pipelines as much as possible without introducing state – Don’t fuse stateless with stateful – Don’t fuse a peeking filter with anything upstream • Benefits: Joiner Adder 25 – Reduces global communication and synchronization – Exposes inter-node optimization opportunities
  26. 26. Phase 2: Data Parallelize Data Parallelize for 4 cores Splitter BandPass Compress Process Expand BandPass Compress Process Expand BandStop BandStop Joiner Adder Adder Adder Adder Joiner 26 Splitter Fiss 4 ways, to occupy entire chip
  27. 27. Phase 2: Data Parallelize Data Parallelize for 4 cores Splitter Splitter Splitter BandPass BandPass Compress Compress Process Process Expand Expand BandPass BandPass Compress Compress Process Process Expand Expand Joiner Joiner BandStop BandStop Joiner Adder Adder Adder Adder Joiner 27 Splitter Task parallelism! Each fused filter does equal work Fiss each filter 2 times to occupy entire chip
  28. 28. Phase 2: Data Parallelize Data Parallelize for 4 cores Splitter Splitter Splitter BandPass BandPass Compress Compress Process Process Expand Expand BandPass BandPass Compress Compress Process Process Expand Expand Joiner Joiner Splitter – Preserve task parallelism • Benefits: – Reduces global communication and synchronization Splitter BandStop BandStop BandStop BandStop Joiner Joiner Joiner Adder Adder Adder Adder Joiner 28 • Task-conscious data parallelization Splitter Task parallelism, each filter does equal work Fiss each filter 2 times to occupy entire chip
  29. 29. Evaluation: Coarse-Grained Data Parallelism Task Fine-Grained Data Coarse-Grained Task + Data 19 Throughput Normalized to Single Core StreamIt 18 17 Good Parallelism! Low Synchronization! 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 29 n r M ea ad a G eo m et ri c R r co de Vo r ec od e EG 2D TD E M P t rp en Se ad io FM R rb an k Fi l te T FF ES D CT D er lV oc od ha nn e C Bi to ni c So rt 0
  30. 30. Simplified Vocoder Splitter 6 AdaptDFT AdaptDFT 6 Joiner RectPolar 20 Data Parallel Splitter Splitter 2 UnWrap Unwrap 2 1 Diff Diff 1 1 Amplify Amplify 1 1 Accum Accum 1 Data Parallel, but too little work! Joiner Joiner PolarRect 30 20 Data Parallel Target a 4 core machine
  31. 31. Data Parallelize Splitter 6 AdaptDFT AdaptDFT 6 Joiner Splitter RectPolar RectPolar RectPolar RectPolar 20 5 Joiner Splitter Splitter 2 UnWrap Unwrap 2 1 Diff Diff 1 1 Amplify Amplify 1 1 Accum Accum 1 Joiner Joiner Splitter RectPolar RectPolar RectPolar PolarRect 20 5 Joiner 31 Target a 4 core machine
  32. 32. Data + Task Parallel Execution Splitter 6 6 Cores Joiner Splitter 5 Joiner Splitter Splitter 2 2 1 1 1 1 1 1 Time 21 Joiner Joiner Splitter 5 RectPolar Joiner 32 Target 4 core machine
  33. 33. We Can Do Better! Splitter 6 6 Cores Joiner Splitter 5 Joiner Splitter Splitter 2 2 1 1 1 1 1 1 Time 16 Joiner Joiner Splitter 5 RectPolar Joiner 33 Target 4 core machine
  34. 34. Phase 3: Coarse-Grained Software Pipelining Prologue New Steady State RectPolar RectPolar • New steady-state is free of dependencies • Schedule new steady-state using a greedy partitioning 34 RectPolar RectPolar
  35. 35. Greedy Partitioning Cores To Schedule: Time 35 16 Target 4 core machine
  36. 36. M ea n ad ar c et ri co de r Vo R G eo m M PE G 2D ec od er TD E rp en t Se ad io FM R rb an k Fi lte T FF ES D CT ha nn el Vo co de r C 36 Fine-Grained Data Coarse-Grained Task + Data + Software Pipeline Best Parallelism! Lowest Synchronization! D 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Task Coarse-Grained Task + Data Bi to ni cS or t Throughput Normalized to Single Core StreamIt Evaluation: Coarse-Grained Task + Data + Software Pipelining
  37. 37. Generalizing to Other Multicores • Architectural requirements: – Compiler controlled local memories with DMA – Efficient implementation of scatter/gather • To port to other architectures, consider: – Local memory capacities – Communication to computation tradeoff • Did not use processor-to-processor communication on Raw 37
  38. 38. Related Work • Streaming languages: – Brook [Buck et al. ’04] – StreamC/KernelC [Kapasi ’03, Das et al. ’06] – Cg [Mark et al. ‘03] – SPUR [Zhang et al. ‘05] • Streaming for Multicores: – Brook [Liao et al., ’06] • Ptolemy [Lee ’95] • Explicit parallelism: – OpenMP, MPI, & HPF 38
  39. 39. Conclusions • Streaming model naturally exposes task, data, and pipeline parallelism • This parallelism must be exploited at the correct granularity and combined correctly Task Fine-Grained Coarse-Grained Data Task + Data Coarse-Grained Task + Data + Software Pipeline Parallelism Not matched Good Good Best Synchronization Not matched High Low Lowest • Good speedups across varied benchmark suite • Algorithms should be applicable across multicores 39

×