SlideShare a Scribd company logo
1 of 20
Download to read offline
Adaptation
                          in embodied & situated agents
                                      Author: Claudio Martella
                          Collaborators: Dott. Stefano Nolfi (ISTC - CNR)
                               Prof. N.A. Borghese (AIS Lab - UniMi)

                                          October, 2011


                                                1
Tuesday, October 11, 11
The problem
                It is difficult to build autonomous systems
                through a top-down approach:


                     • the behavior might be too complex for the
                          designer to control
                     • the environment is noisy and not perfect
                     • the world is unpredictable
                                                2
Tuesday, October 11, 11
Evolutionary robotics is a branch of robotics
                       that uses evolutionary methodologies
                  to develop controllers for autonomous robots.

                                        Nolfi, Floreano [2004] - MIT Press




                                        3
Tuesday, October 11, 11
The objective

                            We wanted to analyze the possibility
                                of applying adaptive processes
                               to embodied & situated agents
                                          considering
                          evolutionary, individual and social learning.



                                                4
Tuesday, October 11, 11
E&S agents

                     • Embodied: the agent can exploit the
                          characteristics of the robot (shape,
                          sensors, actuators etc.).
                     • Situated: the solution can exploit the
                          possible interactions that the environments
                          offers.


                                              5
Tuesday, October 11, 11
The methodology
                          E-puck Robot       Simulation




            Problem: categorize 10 objects (Good, Poisonous)
                                         6
Tuesday, October 11, 11
The evolutionary process




                                     7
Tuesday, October 11, 11
1st goal
                   Implement an algorithm for individual learning.

                                  The algorithm should start
                            with one set of candidate parameters
                          and it would modify them by trial & error.

                          Decision: start from Simulated Annealing *


          * "Optimization by Simulated Annealing", Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P. (1983) - Science
                                                             8
Tuesday, October 11, 11
Simulated Annealing
                                       Temperature:

                                       It probabilistically accepts
                                        mutations that decrease
                                               the fitness.

                                       The probability decreases
                                              with time.

                                       It allows the algorithm to
                                       jump out of local minima.
                                   9
Tuesday, October 11, 11
Stochasticity in E&S
                                Evaluation depends on
                            the (random) initial conditions:




                                           10
Tuesday, October 11, 11
The intuition
                            Temperature                                  Stochasticity

             0.9                                         0.9

         0.675                                         0.675

           0.45                                         0.45

         0.225                                         0.225

                0                                         0
                    100   200   300   400   500                10   20       30      40   50

         Probability of accepting negative             Probability of accepting negative
          mutations decreases with the                  mutations decreases with the
                 increase of time                         increase of #evaluations
                                                  11
Tuesday, October 11, 11
Contributions
             Substitute external stochasticity with internal:

             •       Remove Temperature

             •       Start with few evaluations and increase with time


                                       Results
             •       Simplifies the algorithm

             •       Better performance (~10% improvement)

             •       Lighter algorithm (~50% less evaluations for us)

                                               12
Tuesday, October 11, 11
2nd goal
                          Implement an algorithm for social learning.

                            The algorithm should take advantage
                           of the interaction with an expert agent
                               to acquire an adaptive solution
                            that is improved and/or in less time.

                   Decision: apply individual learning to imitation.

                                              13
Tuesday, October 11, 11
Why?

                 Social learning should avoid reinventing the wheel.

                 In principle, when guided, learning is faster & safer.

                          It should be the basis for cultural evolution.




                                                14
Tuesday, October 11, 11
How?
               There are simpler forms of social learning:


                • social facilitation
                • contagious behavior
                • stimulus enhancement

                                       15
Tuesday, October 11, 11
How (technically)?




           Fitness function: student should learn to give
           outputs similar to the agent’s, given the same input.
                                     16
Tuesday, October 11, 11
How (technically)?
               Pure imitation brings to under-fitting individuals.
                      We introduced a hybrid approach.


    f it = f itsoc · (1      ↵) + f itind · ↵

                 c
    ↵=           N




                                          17
Tuesday, October 11, 11
Contributions
             •       Modeled social learning with simple form of imitation

             •       Modeled hybrid social-individual learning approach


                                       Results
             •       Performance on the problem is not improved

             •       Adaptive behavior is acquired faster

             •       More agents acquire an adaptive behavior

                                               18
Tuesday, October 11, 11
Intuitive interpretation
          parameters space                         solutions space




                                  Social learning as a method
                          for promising initial parameters selection.
                                  Social learning as a method
                               for jumping out of local maxima.
                                              19
Tuesday, October 11, 11
Questions?



                              20
Tuesday, October 11, 11

More Related Content

Viewers also liked

Representantes comenius1
Representantes comenius1Representantes comenius1
Representantes comenius1Rosafcid
 
Hadoop: A Hands-on Introduction
Hadoop: A Hands-on IntroductionHadoop: A Hands-on Introduction
Hadoop: A Hands-on IntroductionClaudio Martella
 
Greek trivia2
Greek trivia2Greek trivia2
Greek trivia2sbillia
 
Relay for life
Relay for lifeRelay for life
Relay for lifesbillia
 
Disney trivia2
Disney trivia2Disney trivia2
Disney trivia2sbillia
 
Greek trivia
Greek triviaGreek trivia
Greek triviasbillia
 
Pro sports trivia
Pro sports triviaPro sports trivia
Pro sports triviasbillia
 
Giraph at Hadoop Summit 2014
Giraph at Hadoop Summit 2014Giraph at Hadoop Summit 2014
Giraph at Hadoop Summit 2014Claudio Martella
 

Viewers also liked (10)

Laurel's slide show
Laurel's slide showLaurel's slide show
Laurel's slide show
 
Haikus 1
Haikus 1Haikus 1
Haikus 1
 
Representantes comenius1
Representantes comenius1Representantes comenius1
Representantes comenius1
 
Hadoop: A Hands-on Introduction
Hadoop: A Hands-on IntroductionHadoop: A Hands-on Introduction
Hadoop: A Hands-on Introduction
 
Greek trivia2
Greek trivia2Greek trivia2
Greek trivia2
 
Relay for life
Relay for lifeRelay for life
Relay for life
 
Disney trivia2
Disney trivia2Disney trivia2
Disney trivia2
 
Greek trivia
Greek triviaGreek trivia
Greek trivia
 
Pro sports trivia
Pro sports triviaPro sports trivia
Pro sports trivia
 
Giraph at Hadoop Summit 2014
Giraph at Hadoop Summit 2014Giraph at Hadoop Summit 2014
Giraph at Hadoop Summit 2014
 

Similar to Adaptation in Embodied & Situated Agents

Similar to Adaptation in Embodied & Situated Agents (6)

VBPR 1st seminar
VBPR 1st seminarVBPR 1st seminar
VBPR 1st seminar
 
Why do a designed experiment
Why do a designed experimentWhy do a designed experiment
Why do a designed experiment
 
Adapt or Go extinct
Adapt or Go extinctAdapt or Go extinct
Adapt or Go extinct
 
When Tdd Goes Awry
When Tdd Goes AwryWhen Tdd Goes Awry
When Tdd Goes Awry
 
A Cloud Outage Under the Lens of “Profound Knowledge”
A Cloud Outage Under the Lens of  “Profound Knowledge” A Cloud Outage Under the Lens of  “Profound Knowledge”
A Cloud Outage Under the Lens of “Profound Knowledge”
 
Exploratory Testing in an Agile Context
Exploratory Testing in an Agile ContextExploratory Testing in an Agile Context
Exploratory Testing in an Agile Context
 

Recently uploaded

GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 

Recently uploaded (20)

GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 

Adaptation in Embodied & Situated Agents

  • 1. Adaptation in embodied & situated agents Author: Claudio Martella Collaborators: Dott. Stefano Nolfi (ISTC - CNR) Prof. N.A. Borghese (AIS Lab - UniMi) October, 2011 1 Tuesday, October 11, 11
  • 2. The problem It is difficult to build autonomous systems through a top-down approach: • the behavior might be too complex for the designer to control • the environment is noisy and not perfect • the world is unpredictable 2 Tuesday, October 11, 11
  • 3. Evolutionary robotics is a branch of robotics that uses evolutionary methodologies to develop controllers for autonomous robots. Nolfi, Floreano [2004] - MIT Press 3 Tuesday, October 11, 11
  • 4. The objective We wanted to analyze the possibility of applying adaptive processes to embodied & situated agents considering evolutionary, individual and social learning. 4 Tuesday, October 11, 11
  • 5. E&S agents • Embodied: the agent can exploit the characteristics of the robot (shape, sensors, actuators etc.). • Situated: the solution can exploit the possible interactions that the environments offers. 5 Tuesday, October 11, 11
  • 6. The methodology E-puck Robot Simulation Problem: categorize 10 objects (Good, Poisonous) 6 Tuesday, October 11, 11
  • 7. The evolutionary process 7 Tuesday, October 11, 11
  • 8. 1st goal Implement an algorithm for individual learning. The algorithm should start with one set of candidate parameters and it would modify them by trial & error. Decision: start from Simulated Annealing * * "Optimization by Simulated Annealing", Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P. (1983) - Science 8 Tuesday, October 11, 11
  • 9. Simulated Annealing Temperature: It probabilistically accepts mutations that decrease the fitness. The probability decreases with time. It allows the algorithm to jump out of local minima. 9 Tuesday, October 11, 11
  • 10. Stochasticity in E&S Evaluation depends on the (random) initial conditions: 10 Tuesday, October 11, 11
  • 11. The intuition Temperature Stochasticity 0.9 0.9 0.675 0.675 0.45 0.45 0.225 0.225 0 0 100 200 300 400 500 10 20 30 40 50 Probability of accepting negative Probability of accepting negative mutations decreases with the mutations decreases with the increase of time increase of #evaluations 11 Tuesday, October 11, 11
  • 12. Contributions Substitute external stochasticity with internal: • Remove Temperature • Start with few evaluations and increase with time Results • Simplifies the algorithm • Better performance (~10% improvement) • Lighter algorithm (~50% less evaluations for us) 12 Tuesday, October 11, 11
  • 13. 2nd goal Implement an algorithm for social learning. The algorithm should take advantage of the interaction with an expert agent to acquire an adaptive solution that is improved and/or in less time. Decision: apply individual learning to imitation. 13 Tuesday, October 11, 11
  • 14. Why? Social learning should avoid reinventing the wheel. In principle, when guided, learning is faster & safer. It should be the basis for cultural evolution. 14 Tuesday, October 11, 11
  • 15. How? There are simpler forms of social learning: • social facilitation • contagious behavior • stimulus enhancement 15 Tuesday, October 11, 11
  • 16. How (technically)? Fitness function: student should learn to give outputs similar to the agent’s, given the same input. 16 Tuesday, October 11, 11
  • 17. How (technically)? Pure imitation brings to under-fitting individuals. We introduced a hybrid approach. f it = f itsoc · (1 ↵) + f itind · ↵ c ↵= N 17 Tuesday, October 11, 11
  • 18. Contributions • Modeled social learning with simple form of imitation • Modeled hybrid social-individual learning approach Results • Performance on the problem is not improved • Adaptive behavior is acquired faster • More agents acquire an adaptive behavior 18 Tuesday, October 11, 11
  • 19. Intuitive interpretation parameters space solutions space Social learning as a method for promising initial parameters selection. Social learning as a method for jumping out of local maxima. 19 Tuesday, October 11, 11
  • 20. Questions? 20 Tuesday, October 11, 11