The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
Isda
1. Scaling Genetic Algorithms using MapReduce AbhishekVerma, Xavier Llora, David E. Goldberg, Roy H. Campbell
2. Motivation Genetic Algorithms (GAs) applied to very large scale data-intensiveproblems Current approach: MPI Requires detailed knowledge of h/w architecture Complicated to program, debug, checkpoint Does not scale on commodity clusters MapReduce: simple and scalable abstraction Use MapReduce to scale GAs 2 Intelligent Systems Design and Applications 2009
3. Outline Motivation MapReduce Genetic Algorithm Approach Experimental Results Conclusion 3 Intelligent Systems Design and Applications 2009
5. Genetic Algorithm Initialize population with random individuals. Evaluate fitness value of individuals. Select good solutions by using tournament selection without replacement. Create new individuals by recombining the selected population using uniform crossover. Evaluate the fitness value of all offspring. Repeat steps 3-5 until some convergence criteria are met. 5 Intelligent Systems Design and Applications 2009
6. Genetic Algorithm Initialize population with random individuals. Evaluate fitness value of individuals. Repeat steps 4-5 to 2 until some convergence criteria are met. Select good solutions by using tournament selection without replacement. Create new individuals by recombining the selected population using uniform crossover. 6 Map Reduce Intelligent Systems Design and Applications 2009
8. MapReducing Genetic Algorithm (2) Modifications Mappers write to DFS so that clients can evaluate convergence criteria and control next iteration Random partitioner function Maintain a window of individuals in each reducer Optimizations Create the initial population in 0th MapReduce Compactly represent bits in array of long ints 8 Intelligent Systems Design and Applications 2009
9. Experimental Results 9 Experimental setup 52 nodes: 16GB RAM, 2TB hard drives Each node runs 5 mappers + 3 reducers Population set to nlog(n) Intelligent Systems Design and Applications 2009
10. Scaling GAs to 100 million variables 10 Intelligent Systems Design and Applications 2009
11. Conclusion Modeled GAs in MapReduce Scales on a commodity clusters to 100 million variables Can also use Pthreads(Phoenix), GPUs(Mars), … Future Work Demonstrate scalability for practical applications MapReduce Compact GAs and Extended Compact GAs Comparison with MPI implementation 11 Intelligent Systems Design and Applications 2009
