page1 page2 page3 page4 page5 page6 # [SILVR: A Synthetic Immersive Large-Volume Plenoptic Dataset](https://idlabmedia.github.io/large-lightfields-dataset)

GitHub stars Papers With Code Papers With Code arXiv BibTeX

We present SILVR, a dataset of light field images for six-degrees-of-freedom navigation in large fully-immersive volumes. The SILVR dataset is short for “Synthetic Immersive Large-Volume Ray” dataset.

Properties

Our dataset exhibits the following properties:

Scenes

We present light field renders with various camera setup configurations for three scenes: Agent 327: Barbershop, Zen Garden, and Lone Monk.

Agent 327: Barbershop

Barbershop Panorama

This scene is taken from the Blender website, under the “demo files” section. It is licensed CC-BY, by Blender Foundation.

Download the original Agent 327: Barbershop scene with light field camera setups here (272MB). Note that our Blender Lightfield Addon is required to open the Blender file with light fields.

Zen Garden

Zen Garden Panorama

This scene is made in-house by IDLab-MEDIA. It is licensed CC-BY 4.0.

Download the Zen Garden scene with light field camera setups here (231MB). Note that our Blender Lightfield Addon is required to open the Blender file with light fields.

Lone Monk

Lone Monk Panorama

This scene is made by Carlo Bergonzini from Monorender, licensed CC-BY. The original scene is also available for download from the Blender website, under the “demo files” section. Applied modifications:

Download the modified Lone Monk scene with light field camera setups here (33MB). Note that our Blender Lightfield Addon is required to open the Blender file with light fields.

Download

All images are provided in OpenEXR format with HDR colors and depth in meters. Files are available for download from our own storage service. All files can be downloaded individually. Below you can find an overview of the different files and their sizes.

 - Barbershop
   - barbershop_LFCuboid_1mx3mx1m.tar       (16    GB)
   - barbershop_LFSphere_e105cm_d145cm.tar  ( 7.5  GB)
   - barbershop_LFSphere_e110cm_d100cm.tar  (    43MB)
   - barbershop_LFCuboid_8panos.tar         (   172MB)
 - Garden
   - garden_LFCuboid_2x2x1.tar              (25    GB)
   - garden_LFSphere_e100cm_d170cm.tar      ( 8.7  GB)
   - garden_LFSphere_e100cm_d50cm.tar       (    51MB)
   - garden_LFCuboid_8panos.tar             (    86MB)
 - Lone Monk
   - lone_monk_LFCuboid_4mx4mx3m.tar        (24    GB)
   - lone_monk_LFSphere_e220cm_d400cm.tar   ( 6.8  GB)
   - lone_monk_LFSphere_e160cm_d220cm.tar   (   500MB)
   - lone_monk_LFCuboid_8panos.tar          (   159MB)

Find the sha256 checksums here. The letters e and d in the filenames are for ‘elevation’ and ‘diameter’.

Update 2022-09-07: Reuploaded garden_LFCuboid_2x2x1.tar after fixing issue #3.
Update 2022-09-07: Reuploaded garden_mmsys2022.blend after fixing issue #4.

Tools

Lens Reproject

As the images are rendered using equisolid fish-eye lenses, we also supply a tool (written in C++) to generate reprojected images with other lens types, as most established light field research assumes rectilinear lenses.

Project page: github.com/IDLabMEDIA/image-lens-reproject

NeRF configuration generator

We provide a Python script generate_NERF_transforms.py that produces the required NeRF configuration to test our scenes in NeRF using instant-ngp.

Script source: github.com/IDLabMEDIA/large-lightfield-dataset/generate_NERF_transforms.py

Example on the spherical rendering configuration of barbershop, lone monk and garden, after reprojecting it using the lens-reproject tool (as instant-ngp only support rectilinear images):

NeRF Barbershop NeRF Zen Garden NeRF Lone Monk

NeRF: How to?

First, we reproject the images (in this example from the scene lone monk) with a rectilinear lens of 18mm focal length on a 36mm sensor, and store them in PNG format with a resolution 1/8th of the original images (i.e.: 256x256), while reducing exposure by one stop and applying Reinhard tone mapping with maximum brightness 5:

mkdir lone_monk_perspective
./reproject --parallel 4 --rectilinear 18,36 --scale 0.125 \
  --png --exposure -1 --reinhard 5 \
  --input-dir lone_monk/LFSphere_e220cm_d400cm/exr \
  --input-cfg lone_monk/LFSphere_e220cm_d400cm/lightfield.json \
  --output-dir lone_monk_perspective \
  --output-cfg lone_monk_perspective/lightfield.json

Now, we generate the transforms.json required by instant-ngp:

python3 generate_NERF_transforms.py \
  --scene lone_monk \
  --dataset-config lone_monk_perspective/lightfield.json \
  --output-transforms lone_monk_perspective/transforms.json

Finally, open the dataset with with instant-ngp:

cd instant-ngp
build/testbed --scene=path/to/lone_monk_perspective/transforms.json

Here, make sure to set the “Near distance” under the Training options to 0. Consider restarting training after you did.

Blender Lightfield Addon

The Blender addon we developed in-house to produce the dataset images is also open-sourced to enable anyone to start producing light field datasets from virtual scenes in Blender.

Project page: github.com/IDLabMEDIA/blender-lightfield-addon

Addon GIF

Credits

To cite this paper:

@inproceedings{courteaux2022silvr,
 title = {{SILVR: A Synthetic Immersive Large-Volume Plenoptic Dataset}},
 author = {Courteaux, Martijn and Artois, Julie and De Pauw, Stijn and Lambert, Peter and Van Wallendael, Glenn},
 year = {2022},
 doi = {10.1145/3524273.3532890},
 publisher = {Association for Computing Machinery},
 url = {https://doi.org/10.1145/3524273.3532890},
 address = {New York, NY, USA},
 month = {jun},
 numpages = {6},
 isbn = {978-1-4503-9283-9/22/06},
 booktitle = {13th ACM Multimedia Systems Conference (MMSys '22)},
 location = {Athlone, Ireland}
}

Dataset and paper by IDLab MEDIA.