INRIA

Post-Doctoral Research Visit F/M [Campagne Postdoctorant 2021 - CRI Lille] Learning Adaptive Communication Graphs for Decentralized Federated Learning (BB-FC3F3)

Trouvé dans: Neuvoo FR

Description:

Contexte et atouts du poste

Magnet (Machine Learning in Information Networks) is an Inria project-team located in Inria’s Lille Nord Europe research center and is also part of CRIStAL (UMR CNRS 9189). In recent years, Magnet has developed a strong expertise in federated/decentralized and privacy-preserving machine learning algorithms. Aurélien Bellet (researcher at Inria) and Marc Tommasi (Professor at the University of Lille), who will supervise the postdoctoral researcher, have published several papers at top ML conferences and co-organized several national and international workshops on these topics.


This project will stimulate existing and emerging collaborations with other research groups. For instance, Magnet has ongoing collaborations with the groups of Rachid Guerraoui (distributed algorithms), Martin Jaggi (optimization for ML) and Anne-Marie Kermarrec (scalable systems) at EPFL, the privacy-preserving data analysis group at Alan Turing Institute London, as well as federated learning and privacy researchers at Google Research (e.g., Adrià Gascón).


In terms of concrete applications, we recently started collaborations with Lille University Hospital (CHU) on decentralized machine learning through some joint projects (such as the Exploratory Action FLAMED). These collaborations can provide concrete use-cases to apply the approaches developed during this postdoc, for instance to run multi-centric decentralized medical studies. Beyond medical applications, we also envision potential applications for privacy and decentralized computations in speech, which are key topics in the European H2020 project Comprise and the French project DEEP-Privacy.

Mission confiée

The postdoctoral researcher will conduct research in the area of federated/decentralized machine learning.

Federated Learning (FL) allows a set of data owners to collaboratively train machine learning models while keeping their datasets decentralized. Decentralized FL algorithms, in which participants communicate in a peer-to-peer fashion along the edges of a network graph, are popular choices thanks to their scalability and privacy properties. However, how to construct the communication graph so as to optimize or balance certain criteria (convergence speed, communication cost, generalization performance, privacy guarantees) remains an open question. In this postdoc, we will develop approaches for learning communication graphs in a data-dependent fashion, as well as new decentralized FL algorithms that are able to efficiently adapt the communication graph during training without exchanging raw data.

Principales activités

The topic of this postdoctoral position is to study, both theoretically and empirically, the role of the network topology in decentralized FL and to develop approaches for learning such communication graphs in a data-dependent fashion. Ultimately, the goal is to design new decentralized FL algorithms that are able to efficiently adapt the communication graph during training so as to optimize or balance certain criteria (convergence speed, communication cost, generalization performance, privacy guarantees) without exchanging raw data. We plan to illustrate the relevance of the developed approaches on real-world data from standard benchmark datasets but also concrete applications (e.g., from the medical domain).
In particular, we would like to investigate some of the following questions:

  • How to choose good neighbors adaptively in decentralized SGD, e.g., based on importance sampling strategies;
  • How to infer an optimal communication graph in a decentralized manner based on assumptions on the data distribution across nodes;
  • How to incorporate formal differential privacy constraints, characterizing the role of topology and the associated privacy-utility-efficiency trade-offs;
  • How to extend the above techniques to learning personalized models, with provable generalization guarantees;
  • How the choice of topology can affect the robustness (or lack thereof) of the algorithms to malicious participants.
  • Compétences

    The applicant must hold a PhD in machine learning or related fields. She/he is expected to have strong mathematical skills (e.g., probability, statistics, linear algebra, numerical optimization). Some knowledge in federated learning or distributed algorithms is a plus.

    Avantages

  • Subsidized meals
  • Partial reimbursement of public transport costs
  • Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
  • Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
  • Professional equipment available (videoconferencing, loan of computer equipment, etc.)
  • Social, cultural and sports events and activities
  • Access to vocational training
  • Social security coverage
  • calendar_todayil y a 3 jours

    report

    info CDD

    location_on Villeneuve-d'Ascq, France

    work INRIA

    Postuler:
    J'autorise expressément le Conditions générales

    Emplois similaires