Novel View Rendering with Dust3r - Grenoble, France - NAVER LABS Europe

NAVER LABS Europe

Entreprise vérifiée

Grenoble, France

il y a 3 semaines

Posté par:

Sophie Dupont

beBee Recruiter

StageSHIP

Description

The most common pipeline for NVS is to first reconstruct the scene and cameras in 3D from a collection of images using SfM and MVS, e.g.

COLMAP. However, NVS is extremely dependent on camera parameters estimation, which makes the process constrained and tedious.

To this aim, we wish to leverage the recent advances in Foundation Models for 3D vision, such as DUSt3R [A], that started to showcase the superiority of plain Vision Transformer architectures (ViTs) for multi-view geometric tasks [A,B,C,D] with limited to no information about camera parameters.

This work will aim at the problem of NVS through a similar scope. The applicants to this internship are required to have strong knowledge of the Structure-from-Motion, MVS and NVS pipelines, e.g. [E,F]. Knowledge about Self-Supervised pretraining of ViTs, and 3D Gaussian Splatting [G] are a plus.

Supervisors:
Vincent Leroy and Jerome Revaud
References

[A] DUSt3R:
Geometric 3D Vision Made Easy. arXiv preprint

[B] CroCo:
Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion, Weinzaepfel et al. NeurIPS 2022

[C] CroCo v2:
Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow, Weinzaepfel et al. ICCV 2023

[D] MFOS:
Model-Free & One-Shot Object Pose Estimation, Lee et al. AAAI 2024

[E] COLMAP:
Structure-from-Motion Revisited, Schonberger et al. CVPR 2016

[F] COLMAP:
Pixelwise View Selection for Unstructured Multi-View Stereo, Schonberger et al. ECCV 2016
[G] 3D Gaussian Splatting for Real-Time Radiance Field Rendering, SIGGRAPH'23
Application instructions

Please note that applicants must be registered students at a university or other academic institution and that this establishment will need to sign an 'Internship Convention' with NAVER LABS Europe before the student is accepted.

About NAVER LABS

NAVER is the #1 Internet portal in Korea with activities that span a wide range of businesses including search, commerce, content, financial and cloud platforms.

NAVER LABS, co-located in Korea and France, is the organization dedicated to preparing NAVER's future. NAVER LABS Europe is located in a spectacular setting in Grenoble, in the heart of the French Alps.

Scientists at NAVER LABS Europe are empowered to pursue long-term research problems that, if successful, can have significant impact and transform NAVER.

We take our ideas as far as research can to create the best technology of its kind.

Active participation in the academic community and collaborations with world-class public research groups are, among others, important tools to achieve these goals.

Teamwork, focus and persistence are important values for us.

NAVER LABS Europe is an equal opportunity employer.