Large Language Models for Information Extraction - Palaiseau, France - CEA

CEA

Entreprise vérifiée

Palaiseau, France

il y a 3 semaines

Posté par:

Sophie Dupont

beBee Recruiter

Description

Position description

Category:

Mathematics, information, scientific, software

Contract:

Internship

Job title:

Large Language Models for Information Extraction H/F

Subject:

We propose to study the performance of LLMs on Information Extraction tasks.

Contract duration (months):

Job description:

Large Language Models (LLM) have been widely adopted by the Natural Language Processing (NLP) community and have been applied with success to a variety of tasks (Le Scao et al., 2023; Touvron et al., These models have been pretrained in a selfsupervised fashion on large corpora of raw text and have been tested on standardized benchmarks devised by the community. Most of the time, these benchmarks include Natural Language Understanding (NLU) tasks such as reasoning and common sense in a variety of domains (e.g. microeconomics, physics or maths) (Hendrycks et al., 2021; Srivastava et al., Other evaluate the capacities of these models to generate code or to translate a program into another language (Zheng et al., Only a few research efforts concentrate on evaluating these models on information extraction tasks. Among them, Wang et al introduce IE INSTRUCTIONS, a benchmark composed of 32 information extraction datasets that includes Named Entity Recognition (NER), Relation Extraction (RE) and Event Extraction (EE) tasks.

In this context, we propose to further study the performance of LLMs on Information Extraction tasks. Specifically, this study will focus on their few
- and zero-shot capabilities for Named Entity Recognition (NER) in a context where the number of types of entities to identify in texts is very high, which results in a very small volume of annotated data.

Among other tasks, the successful intern will have the following responsibilities:

Perform and maintain an up-to-date literature review on the topic

- and zero-shot setting

Evaluate stateoftheart models in this framework by relying in our computing cluster
Devise, implement, and evaluate new methods for zero

- and few-shot NER using pretrained LLMs

Hendrycks et al "Measuring Massive Multitask Language Undestanding". ICLR
Le Scao et al "

BLOOM:
A 176B-Parameter Open-Access Multilingual Language Model". arXiv

Srivastava et al "Beyond the

Imitation Game:
Quantifying and extrapolating the capabilities of language models". arXiv

Touvron et al "Llama 2: Open Foundation and Fine-Tuned Chat Models". arXiv
Wang et al "Instruct

UIE:
Multi-task Instruction Tuning for Unified Information Extraction"

arXiv
Applicant Profile:

Nonexhaustive list of required skills:
Able to work in a Linux environment
Background in natural language generation and language modeling
Familiarity with pretrained language models and large language models
Familiarity with Python and specifically with pytorch and other AI/NLP related libraries

Position location

Site:

Saclay

Job location:

France, IledeFrance, Essonne (91)

Location:

Palaiseau

Prepared diploma:

Bac+
Diplôme École d'ingénieurs

PhD opportunity:

Requester

Position start date:

01/04/2024

General information

Organisation:

The French Alternative Energies and Atomic Energy Commission (CEA) is a key player in research, development and innovation in four main areas:

defence and security,

- nuclear energy (fission and fusion),
- technological research for industry,
- fundamental research in the physical sciences and life sciences.

Drawing on its widely acknowledged expertise, and thanks to its 16000 technicians, engineers, researchers and staff, the CEA actively participates in collaborative projects with a large number of academic and industrial partners.

The CEA is established in ten centers spread throughout France
Reference :
Description de l'unité:
Based in Paris-Saclay, CEA List is one of the four institutes under CEA Tech, the technological research branch of CEA. Specializing in intelligent digital systems, it contributes to enhancing the competitiveness of businesses through technology development and transfer.
The expertise and skills cultivated by the 800 research engineers and technicians at CEA List enable the institute to support annually over 200 French and international companies in applied research projects. These projects are based on four programs and nine technological platforms. Since 2003, 21 startups have been created as a result of these efforts. Designated as a "Carnot Institute" since 2006, CEA List is currently recognized as the "Digital Technologies Carnot Institute".
The Laboratory of Semantic Analysis of Texts and Images (LASTI) is a team comprising around 25 individuals, including researchers, engineers, and doctoral students. They are engaged in research activities focusing on technologies for de