Apprentissage Par Renforcement Et Evolution Des - Lyon, France - Université Grenoble Alpes

Université Grenoble Alpes

Entreprise vérifiée

Lyon, France

il y a 3 semaines

Posté par:

Sophie Dupont

beBee Recruiter

Description

Apprentissage par renforcement et evolution des connaissances // Knowledge-based reinforcement learning and knowledge evolution:

Réf
ABG-123946

ADUM-57594

Sujet de Thèse 17/05/2024
Université Grenoble Alpes
Lieu de travail
Saint-Martind'Hères
France
Intitulé du sujet
Apprentissage par renforcement et evolution des connaissances // Knowledgebased reinforcement learning and knowledge evolution
Mots clés Évolution culturelle, Représentation des connaissances, Apprentissage par renforcement, Systèmes multiagents
Cultural evolution, Knowledge representation, Reinforcement learning, Multiagent systems
Description du sujet:
Cultural knowledge evolution and multiagent reinforcement learning share some of their prominent features. Putting explicit knowledge at the heart of the reinforcement process may contribute to better explanation and transfer.
Cultural knowledge evolution deals with the evolution of knowledge representation in a group of agents. For that purpose, cooperating agents adapt their knowledge to the situations they are exposed to and the feedback they receive from others. This framework has been considered in the context of evolving natural languages [Steels, 2012]. We have applied it to ontology alignment repair, i.e. the improvement of incorrect alignments [Euzenat, 2017] and ontology evolution [Bourahla et al., 2021]. We have shown that it converges towards successful communication through improving the intrinsic knowledge quality.
Reinforcement learning is a learning mechanism adapting the decision making process for maximising the reward provided by the environment to the actions performed by agents [Sutton and Barto, 1998]. Many multiagent versions of reinforcement learning have also been proposed depending on the agent attitude (cooperative, competitive) and the task structure (homogeneous, heterogeneous) [Bučoniu et al., 2010].
From an external perspective, the two approaches operate in a similar manner: agents perceive their environment, perform an action, receive reward or punishment, adapt their behaviour in consequence.

However, a look into the inner mechanisms reveals important differences:

the emphasis on knowledge quality instead of reward maximisation, the lack of probabilistic or even gradual interpretation, and even the absence of explicit choice in action or adaptation.

Hence these two knowledge acquisition techniques are close enough to suggest replacing one by the other and different enough to cross-fertilise.

This thesis position aims at further exploring the commonalities and differences between experimental cultural knowledge evolution and reinforcement learning. In particular, its purpose is to study which features of one technique may be fruitful in the context of the other and which may not.
For that purpose, one research direction is the introduction of knowledgebased reinforcement learning. In knowledgebased reinforcement learning, the decisionmaking process (the choice of the action to be performed) is obtained through accumulated explicit knowledge. Thus the adaptation performed after reward or punishment will have to directly affect this knowledge. This has the advantage that it allows to explain the decisions made by agents. It will also allow for explicit knowledge exchange among them [Leno da Silva et al., 2018].
This promotes a less utilitarian view of knowledge in which the evaluation of the performance of the system has to be disconnected from reward maximisation but to depend on the quality of the acquired knowledge. Of course, these two aspects need to remain related (the acquired knowledge must be relevant to the environment). This separation between knowledge and reward is useful when agents have to change environment or use their knowledge to perform various tasks.
Another use of reinforcement mechanisms relevant to cultural knowledge evolution is related to the motivation for agents to explore unknown knowledge territories [Colas et al., 2019]. By associating an intrinsic reward to the newly acquired knowledge, agents are able to improve the coverage of their knowledge in a way not guided by the environment. Complementing cultural knowledge evolution with exploration motivation, should make agents more active in their understanding of the environment and knowledge acquisition.
These problems may be treated both theoretically and experimentally

Cultural knowledge evolution and multiagent reinforcement learning share some of their prominent features. Putting explicit knowledge at the heart of the reinforcement process may contribute to better explanation and transfer.

Cultural knowledge evolution deals with the evolution of knowledge representation in a group of agents. For that purpose, cooperating agents adapt their knowledge to the situations they are exposed to and the feedback they receive from others. This framework has been considered in the context of evolving natural languages [St