Our special session proposal in ACM International Conference on Distributed Smart Cameras (ICDSC’19) has been accepted!
The special session title is “Trends in Free Navigation Technologies”.
Find here the call for papers
It is an open special session, so you can apply directly through the conference website.
Update: this position is no longer available
There is an increasing interest towards the applications that allow Free Navigation Video Services [1], where users can modify the viewpoint on a scene while receiving a video. These services try to provide the user with the so-called Plenoptic function of the scene [2], defined as:
P_f(x,y,z,theta,phi,lambda,t)
It gives the light intensity at each position for any incident angle , for any wavelength and at any time. This doctoral project is focused on three key problems related to the use of the Plenoptic function : its acquisition, synthesis and visualization.
Current tools for acquisition do not allow collecting the whole Plenoptic function; on the contrary, they allow a sampling of it. For example, in Super-MultiView video[3], the plane (z=z_0) is fixed, and only the forward scene, i.e. when the polar angle comprised , is between -pi/2 and pi/2, is acquired. Moreover, the plane is sampled at the position of each camera.
In this project we are interested in the interpolation of the Plenoptic function, i.e. in the synthesis of virtual viewpoints that were not acquired by real cameras. Moreover, we also want to explore the case of irregular sampling position of P_f.
Access to the Plenoptic function would allow new ways to create and consume visual contents. For example, the Fyuse application [4] allows to change the view angle during the reproduction, while the Lytro system [5] allows post-acquisition refocusing.
Several scientific fields are concerned by this approach :
These items interact one with the other : view synthesis is preliminary for virtual cinema and may benefit from visual attention and perception information ; the whole process impacts on the quality and the aesthetics of the resulting image.
Image synthesis plays a key role in the system that we want to implement. We can see the problem as the interpolation of the Plenoptic function from a set of samples [12]. This reconstruction is based on the scene geometry and often uses post-processing for alleviating the synthesis artifacts.
Image synthesis and rendering have been long studied by the Computer Vision community and the Compression community, even outside the context of Plenoptic function interpolation. The first methods only used the images for the synthesis: they fall into the Image-Based Rendering (IBR) [13] family. Disparity estimation and occlusion detection are typical tools used to improve the synthesis for this case[14], and may prove useful in this doctoral project.
When the depth information is also available, we have the Depth Image-Based Rendering (DIBR) [15] family. Even though DIBR is known since the first 2000’s, the quality of synthesis is not fully satisfying yet [16]. Nevertheless, some promising methods have been proposed recently [17]. They combine temporal and inter-view redundancy to improve the synthesis.
Another difficulty may come from the camera positioning [18]. A preliminary calibration and synchronization phase are needed in order to have a high quality synthesis [19] [20] [21]. To this end, feature matching tools could be employed, such as SIFT [22], SURF [23]. This look necessary in order to achieve the 3D scene understanding [1][18] .
This doctoral project will start with a deep and accurate study of the state of the art in the different concerned domains : image synthesis, camera calibration, 3D geometry, feature matching, visual attention. From a practical point of view, the PhD candidate may use the facilities at b<>com to test the acquisition of the Plenoptic function and to perform camera calibration and synchronization.
Then, the PhD candidate will test and implement different synthesis methods, starting from the state of the art, and then proposing more complex and effective solutions. Human vision principles should be integrated into the new approaches.
At the same time, the impact of the synthesis methods on such practical applications as visualization, free navigation, virtual cinema, …, will be taken into account. The final target of the doctoral project is the mastering of the complete system from acquisition to visualization.
Rémi Cozot, Maître de Conférences, Habilité à Diriger des Recherches, IRT b<>com, IRISA/Université de Rennes 1 – cozot@irisa.fr
Marco Cagnazzo, Maître de Conférences, Habilité à Diriger des Recherches, IRT b<>com, Telecom-ParisTech/Institut Mines-Télécom– cagnazzo@telecom-paristech.fr
Three articles have been accepted in IEEE ICIP
Our article “Reference view selection in DIBR-based multiview coding” has been accepted in IEEE Transactions on Image Processing.
Our article about Network Coding for Multiview Video has been accepted in Springer EURASIP Journal on Advances in Signal Processing.
The basic idea is to adapt the scheduler of a multi-view stream (H.264/MVC format) to user preferences, exploiting Network Coding to maximize the PSNR.
The article is in Open Acces on the journal web site and in the Publications/Journals section of this site.
Our article about depth map compression by elastic interpolation of contours has been accepted into QoMEX 2015.
A journal and a conference paper about virtual viewpoint synthesis were recently accepted.
Our article on lossless contour coding has been accepted in APSIPA Transaction in Signal and Information Processing. Congratulations to the first author, our PhD student Marco Calemme.
Sorry, this entry is only available in French.
Notre article sur la vidéo super-multivues a été accepté dans ICIP 2014.