The AI of Horizon Zero Dawn
•
8 likes•6,776 views
Download the full presentation here: https://www.guerrilla-games.com/read/the-ai-of-horizon-zero-dawn Abstract: For Guerrilla, Horizon Zero Dawn was a massive change in direction. Transitioning from the level-based, tactical, first person shooter genre to open world, action RPG required us to develop many new systems and to redesign existing ones. In this session we will describe the challenges we faced when developing the AI for Horizon Zero Dawn and the solutions we chose. Some of the subjects we will cover are navigation for agents of varying size in a dynamic environment, how we use a combination of HTN planning and utility based decision making, and the systems we developed to coordinate groups of agents.
Recommended
Recommended
More Related Content
More from Guerrilla
More from Guerrilla (18)
Recently uploaded
Recently uploaded (20)
The AI of Horizon Zero Dawn
- 1. The AI of Horizon Zero Dawn
- 4. Machine Ecology • Machines coexist in mixed groups • Each class serves a function – Acquisition
- 5. Machine Ecology • Machines coexist in mixed groups • Each class serves a function – Acquisition – Transport
- 6. Machine Ecology • Machines coexist in mixed groups • Each class serves a function – Acquisition – Transport – Scavengers
- 7. Machine Ecology • Machines coexist in mixed groups • Each class serves a function – Acquisition – Transport – Scavengers – Reconnaissance
- 8. Machine Ecology • Machines coexist in mixed groups • Each class serves a function – Acquisition – Transport – Scavengers – Reconnaissance – Combat • Composition affects behavior
- 10. The Rockbreaker
- 11. Individual Agent Hierarchy • Machine herds organized hierarchically – Previously: strategic, tactical, individual – Now: flexible, dynamic hierarchy • Individual agent – Controls a character – Observes world through perception model • Group agent – No physical manifestation – Coordinates agents Group Individual Group
- 12. Blackboard Individual Blackboard Agent Hierarchy • Machine herds organized hierarchically – Previously: strategic, tactical, individual – Now: flexible, dynamic hierarchy • Individual agent – Controls a character – Observes world through perception model • Group agent – No physical manifestation – Coordinates agents • Communication – Blackboard Group Individual Group
- 13. Individual Agent Hierarchy • Machine herds organized hierarchically – Previously: strategic, tactical, individual – Now: flexible, dynamic hierarchy • Individual agent – Controls a character – Observes world through perception model • Group agent – No physical manifestation – Coordinates agents • Communication – Blackboard – Direct messaging Group Individual Group
- 14. The Agent • Groups and individuals use same core decision process HTN planner World state Plan Daemons
- 15. The Agent • Groups and individuals use same core decision process • Daemons provide knowledge, e.g. – Individual: body state, perception – Group: hierarchy – Common: messaging, memory • On-demand queries, such as path finding – Parameters only apparent during planning HTN planner World state Plan Daemons On-demand queries e.g. path finding Functions Memory Facts
- 16. The Agent • Groups and individuals use same core decision process • Daemons provide knowledge • World state consists of facts – Flexible data representation – Tuple of variable type elements, e.g. • Identifier • Integer, floating point • Object pointer • Lists, can be nested HTN planner World state Plan Daemons Facts (position (1 2 3)) (event impact …) …
- 17. The Agent • Groups and individuals use same core decision process • Daemons provide knowledge • World state consists of facts • Hierarchical Task Network makes a plan – Solves a problem by hierarchical decomposition of an abstract task into concrete actions HTN planner World state Plan Daemons Actions
- 18. HTN in brief Method Branch Abstract task Branch Branch (eat fruit) Method: (eat ?type) Branches: In possession Get from house Buy in store Methods recursively refine abstract tasks into concrete tasks a variable a parameter
- 19. SolutionA Solution B HTN in brief Method Branch Abstract task Branch Branch Branch Preconditions Tasks Solution C (eat fruit) Method: (eat ?type) Branches: In possession Get from house Buy in store Branch: in possession Preconditions: (item ?x) (edible ?x) (type ?x ?type) Methods recursively refine abstract tasks into concrete tasks Problem solver Problem solver finds possible solutions to branch preconditions (item wallet) (edible wallet) (type wallet fruit) (item sandwich) (edible sandwich) (type sandwich fruit) (item banana) (edible banana) (type banana fruit) World state
- 20. Branch tasks HTN in brief Method Branch Abstract task Branch Branch Branch Preconditions Tasks Solution (eat fruit) Method: (eat ?type) Branches: In possession Get from house Buy in store Branch: in possession Preconditions: (item ?x) (edible ?x) (type ?x ?type) Methods recursively refine abstract tasks into concrete tasks Problem solver Problem solver finds possible solutions to branch preconditions Task Task … Branch tasks are instantiated using the variables from the preconditions Replacement tasks: (!take banana) (prepare banana) (!eat banana) (item banana) (edible banana) (type banana fruit) concrete task abstract task
- 21. The Agent • Groups and individuals use same core decision process • Daemons provide knowledge • World state consists of facts • Hierarchical Task Network makes a plan • Execute sequence of actions – Individuals: move, look, speak, etc. – Groups: communicate, make subgroup, etc. HTN planner World state Plan Daemons Actions
- 22. The Collective • Collective agent is the root group • All new individuals added to it • Creates groups for compatible individuals • Individuals provide passport – Opaque tuple – Equality determines compatibility Individual Collective Passports: Individual A : (machine …) Individual B : (machine …) Individual C : (hacked machine …)
- 23. The Collective • Collective agent is the root group • All new individuals added to it • Creates groups for compatible individuals • Individuals provide passport – Opaque tuple – Equality determines compatibility • Creates main groups • Assigns individuals Individual Collective Group Group Individual Passports: Individual A : (machine …) Individual B : (machine …) Individual C : (hacked machine …)
- 24. Individual The Collective • Collective agent is the root group • All new individuals added to it • Creates groups for compatible individuals • Individuals provide passport – Opaque tuple – Equality determines compatibility • Creates main groups • Assigns individuals • Reassigns if passport changes Collective Group Group Passports: Individual A : (machine …) Individual B : (hacked machine …) Individual C : (hacked machine …)
- 25. A Relaxed Herd • Mix of acquisition and combat machines • Acquisition group will take center • Defense split in sections around outside • Patrol paths are generated – Avoid passing through stealth vegetation – But pass nearby, allow player opportunity to hack or trap • Systemic solutions help fill a large open world Herd Group AcquisitionSector defense Collective
- 27. Going To Combat • Individuals are eyes and ears of group • In principle limited world knowledge – Take out machine before it alerts the group – Don’t know what you can’t see – Not strictly enforced if it benefits gameplay • When alerted create specialized groups – Acquisition machines flee – Combat group per enemy • Specialized groups limit complexity • Multiple roles exist within group Herd Flee group Collective GroupCombat group Combat coordinator
- 28. Distributor Role Distribution • Role defined by identifier and context – Examples: attacker, scavenger, searcher, etc. – Context is a tuple, contents specific to role • Roles are requested by groups • Optional distributor component handles assignment – Assigns roles for itself and recursively • Goal: find the right individual for the job – Distribute combat machines over enemies – Who will scavenge, who will patrol – For some roles many factors weigh in Group Group Group Group
- 29. Role Distribution Process Priority Limit Role Context 0.9 1 attacker player … 0.8 1 attacker enemy_a … 0.6 0.5 inf combatant enemy_a … 0.6 inf combatant player … … Priority Limit Role Context 0.9 1 attacker player … 0.8 1 attacker enemy_a … 0.6 inf combatant enemy_a … 0.6 inf combatant player … … Priority Limit Role Context 0.9 1 attacker player … 0.8 1 attacker enemy_a … 0.6 inf combatant enemy_a … 0.6 inf combatant player … … Priority Limit Role Context 0.9 1 attacker player … 0.8 1 attacker enemy_a … 0.6 inf combatant enemy_a … 0.6 inf combatant player … … Priority Limit Role Context • All groups request roles with priority and limit • For all roles in order of priority – Query all candidate individual’s utility for that role • Solved by individual’s HTN domain • Proximity, type of machine, knowledge of target, being attacked by target – Assign best candidate – If limit reached: next role – If not: group updates priority • Useful for spreading over multiple groups Individual Utility Reasons Watcher A Watcher C assigned Grazer … Individual Utility Reasons Watcher A 0.4 quite close Watcher C 0.7 very close Grazer -1 not a combat machine … Individual Utility Reasons Watcher A Watcher B assigned Watcher C … Individual Utility Reasons Watcher A 0.4 quite close Watcher B 0.6 quite close, angry at target Watcher C -1 knocked down … Individual Utility Reasons Watcher A Watcher B Ravager assigned … Individual Utility Reasons Watcher A 0.5 pretty close, but not in view Watcher B -1 do not know this target Ravager 0.7 in view, bigger machine … Individual Utility Reasons Watcher A Watcher B Ravager …
- 30. Choreographing Combat • Role distribution used to minimize chaos – Spread machines over targets – Choose a fair attacker for the player, consider • Machine awareness and state • Player awareness • Proximity and aggressiveness • Damage received and dealt • Etc. • Attack selection also utility based – Integrated into HTN preconditions – Aims for variety, challenge and opportunities for the player
- 31. Perception Model • Individuals have sensors • Observable objects have stimuli • Sensor and stimuli types – Common: visual, aural, touch – Special: smell, radar, proximity • Observation strength depends on – Sensor sensitivity – Stimulus size / loudness
- 32. Stimuli • Stimuli carry information packets, e.g. – Enemy description – Arrow impact / whizzby – Carcass • Handler interprets information – Configurable per individual – Rabbit could infer less than human – Low observation strength can reduce information
- 33. Navigation • Previous offline process not desirable for open world workflow – Need traversal between all areas – More designers and artists working on same content – Offline navigation graph would often be out of date – Also: cumbersome to deal with dynamic terrain
- 35. Navigation • Previous offline process not desirable for open world workflow – Need traversal between all areas – More designers and artists working on same content – Offline navigation graph would often be out of date – Also: cumbersome to deal with dynamic terrain • Large variety agent sizes and capabilities to consider – From human size to big as a house – Breaking through trees and rocks – Some machines can also swim – Some machines can also fly • Solution: runtime generation of multiple navigation meshes
- 36. Navigation • Runtime generation of navigation mesh – Fast, iterative designer workflow: never out of date – Only depends on collision geometry – Available anywhere, generated in activity bubble – Dynamic environments require no additional effort • Updates when collision changes • Six different meshes – Small / Medium / Large / Extra large machine – Special cases: player mount and swimming machine
- 37. Areas and Obstacles • Hulls represented on navigation mesh • Relevance depends on query cost function • Obstacles – Hard or soft blocking : impassable or undesirable – Depends on machine type and state • Don’t break through trees unless large and angry enough • Restrictions – Confine to activity bubble or designed hull • Stealth vegetation – Avoid unless in combat • Danger areas – Avoid path of machine attacks
- 38. Contact: arjen.beij@guerrilla-games.com The team: Jose Antonio Escribano Ayllon Julian Berteling Ivaylo Ivanov Wouter Josemans Hylke Kleve Robert Morcus Marco Pisanu Carles Ros Martínez Pieter van de Kerkhof Tim Verweij
Editor's Notes
- A thousand years after a global catastrophe, Nature has reclaimed the Earth and humans are no longer the dominant species. The masters of this untamed wilderness are the machines, whose teeming herds, schools, and flocks fill the lands, oceans, and skies. Aloy, a lonely outcast from a remote tribe, must learn to hunt and survive in this dangerous new environment in order to solve the mystery that defines her – the identity of her mother, hidden from her at birth. Her quest will take her across vast and beautiful landscapes and introduce her to exotic new tribes, eventually leading her into the deepest ruins of the ancient world.
- In the fiction of Horizon machines have replaced most of earth’s fauna. Each of the machines serves a particular function in the ecology of the world. Depending on the combination of machines their behavior may be different. For example: an acquisition machine may defend its herd if no combat machines are present.
- For the organization of machines into groups we drew inspiration from the work we did for Killzone multiplayer bots. There we coordinated agents through a three layer structure: strategic, tactical and individual. For Horizon we chose to make this hierarchy more flexible. It can dynamically reshape itself to fit the requirements of the situation. If needed we can add layers that specialize in coordinating specific behaviors.
- Agents can communicate through the group blackboard. Information that is shared on the blackboard is readable and writable to the group and all agents below it in the hierarchy.
- Agents can also communicate directly by sending messages. These messages do not have to adhere to the hierarchy.
- The output of daemons is a collection of facts, which comprise the world state as it is known to this agent. Each fact is a tuple which starts with an identifier followed by an arbitrary number of elements. Each element of the fact is a value with a variable data type such as an identifier, integer, floating point number, object pointer or lists of atoms. Lists can be nested making it a very flexible data representation.
- The planner is given an abstract task. Something that is not complete and descriptive enough to act upon. [CLICK] As a toy example, consider the task: eat fruit. [CLICK] Where fruit is a parameter of the abstract task eat. [CLICK] The planner takes an abstract task and recursively breaks this down into smaller sub-problems until finally you are left with a concrete plan of actions. What we call the planning domain is a collection of methods for breaking down these tasks. A method consists of a number of possible replacements for a task, called branches. The branches are considered in-order from top to bottom. [CLICK] [CLICK] Our example may have branches for eating something in our possession, getting it from our house or buying it in a store.
- Each branch consists of preconditions and a set of replacement tasks. These replacement tasks can be abstract or concrete. The preconditions are stated declaratively. For example our ‘in possession’ branch preconditions may state: I have an item X, such that X is edible and X is of the given type. This example only uses an and statement, but we allow for nested and / or / not statements. [CLICK] The preconditions are evaluated by running a problem solver. The solver assigns values to variables while it is searching for a solution. This is where the facts from the world state come in. [CLICK] In our example the solver may first attempt our wallet and find that it is not edible. [CLICK] Then it could consider our sandwich, but that is not a fruit. [CLICK] Finally it may settle on a banana.
- The planner then replaces the original abstract task with the set of tasks of the branch. These tasks can reference the variables of the preconditions. [CLICK] In our example this could be: take the banana, prepare it for eating and finally eat it. [CLICK] The exclamation mark indicates that this is a concrete task. The preparation task is abstract and requires further refinement. In case of our banana it will need to be peeled, but a sandwich would need to be unwrapped. Now in this case we found a solution. The process will continue recursively. But if no valid solution is found the planner will try the different branches. And if those fail and there is no valid replacement for a task, the planner will backtrack the recursion to where that task was added and try a different solution there. And so on and so forth.
- At the end of the planning process we are left with a sequence of concrete actions to perform. For individuals these actions will mostly be physical; for groups they are organizational and communicative.
- At the root of the hierarchy of agents we have a group agent called the collective. Every individual that is spawned is initially assigned to the collective. The collective takes care of organizing the individuals into groups that are capable of working together. It does this by asking the individuals for their passport. This is a tuple containing information that can be used to determine if two agents can cooperate. The collective does not inspect the contents of the passport, it just checks them for equality.
- The Collective creates a subgroup for each compatible set of agents. We call these the main groups and they are responsible for further organizing the individuals under their care.
- Individuals can change their passport at any time. For example, the player might hack one of the machines to make it fight for her. This is then reflected in the passport of the machine. The collective will notice the incompatibility and reassign the individual.
- This is an example of a main group. For a herd of acquisition and combat machines, the main group will create a subgroup for the acquisition machines and a few defense groups surrounding the acquisition group. The acquisition machines will stay in the center and the combat machines will be divided around them.
- Within each defense section we will generate a patrol route. This patrol route is made to conveniently pass near player hiding areas but without moving through them. This provides the player with opportunities to override or stealth-kill the machines.
- When this herd is alerted it will reconfigure itself with new subgroups that are specialized in combat. The group is alerted as a consequence of an observation and communication by an individual, as group agents cannot perceive the world by themselves. In principle agents have a limited knowledge of the world. The player can take them out before they can alert the group and an individual cannot know where the player is unless it observes her through sight or sound. This limited knowledge model is not strictly enforced, but the majority of information comes from an agent's perception system. The alerted group will create a combat coordinator group which manages the creation of subgroups for fighting specific enemies. It will also create a flee group for the acquisition machines. By creating specialized groups we can limit the complexity of their implementations. They can be built to handle a particular scenario, such as combat or investigation or fleeing. Within a group behavior there may be multiple different roles that need to be fulfilled.
- Role distribution is the process of assigning individual agents to roles within a group hierarchy. The goal of the role distributor is to find the best individual for each role. A role is described by an identifier and context information specific to that role. Examples of role identifiers are attacker, scavenger or searcher. The context is a tuple containing variable type elements. Its contents are dependent on the role type. The assigning of individuals to roles is handled by a role distributor component. Only some groups have a component like this and it distributes roles for itself and all groups below it recursively. Depending on the type of role there can be a lot of contributing factors when deciding which individual is most suitable, e.g. a fair distribution over targets, proximity to an event, or the capabilities of the machine.
- The process of role distribution starts with each of the groups requesting roles. They specify a limit on the number of individuals that can be assigned to the role, if there is any, and a priority value that determines the order in which the roles will be assigned. In this example there are two combat groups. One fighting is the player and one is fighting another enemy. They might both request a single attacker role and an infinite amount of combatants. The combatants will position themselves around the target waiting for their chance to attack. For each role in order of priority, we ask each candidate individual to determine its own utility for that role. This logic is implemented in the HTN domain of the individual. In this case, the Ravager, a medium sized combat machine has the highest score. The distributor assigns it to the role and removes it from the list of candidates. If the assignment limit for this role was reached and the process continues with the next most important role. The third role is a combatant role, which has no limit on the number of assignments. After the distributor assigns the role, it asks for an updated priority. Now that the group already has a combatant it lowers the priority of that role. This is a mechanic we use when we want to spread a variable number of individuals over multiple groups. The process continues until all roles have been assigned, or we run out of candidates.
- We support optional annotations on the navigation mesh in the form of areas. These are hulls that are represented on the mesh and which can be optionally taken into account when path finding and during other queries. For obstacles we distinguish between hard obstacles, which you cannot traverse, and soft obstacles, which you would like to avoid if its not too much trouble. Whether something is a hard or soft obstacle depends on the machine type and its state. For example a small machine cannot break through a tree and a large machine can, but only if it is angry. Obstacles can therefore be assigned a value that indicates how much effort it would take to pass through. And when doing a path query you can specify how these values correspond to being hard, soft or non-blocking.