Direction des Relations Internationales (DRI)

Programme INRIA "Equipes Associées"

I. DEFINITION

EQUIPE ASSOCIEE

CIPRus
sélection
2009
Equipe-Projet INRIA : ARTIS Organisme étranger partenaire : Cornell University
Centre de recherche INRIA : Grenoble Rhône-Alpes
Thème INRIA : CogD
Pays : United States of America
 
 
Coordinateur français
Coordinateur étranger
Nom, prénom Holzschuch, Nicolas Bala, Kavita
Grade/statut CR1 INRIA Assistant Professor
Organisme d'appartenance
(précisez le département et/ou le laboratoire)
INRIA Cornell University, department of Computer Science
Adresse postale 655 Avenue de l'Europe
Inovallée
38334 St-Ismier CEDEX
France
4130 Upson Hall
Cornell University
Ithaca, NY 14853-7501
USA
URL  http://artis.inrialpes.fr/  http://www.cs.cornell.edu/~kb
Téléphone +33 476-615-506  +1 607-255-7316 
Télécopie +33 476-615-440  +1 607-255-4428 
Courriel Nicolas.Holzschuch@inria.fr  kb@cs.cornell.edu 


La proposition en bref
 

Titre de la thématique de collaboration (en français et en anglais) :
CIPRus: Challenges In Photorealistic Rendering
(Challenges nouveaux en Rendu Photoréaliste)

Descriptif (environ 10 lignes) :

Photorealistic rendering deals with the production of pictures of virtual worlds that are as close as possible to what a real photograph of this virtual world would look like. Considerable progress has been made in recent years, and photorealistic pictures are being used in severa sectors of the industry: virtual prototyping, special effects for motion picture, video games...

However, truly photorealistic pictures of a virtual world are still difficult to get. There are multiple difficulties to overcome: model acquisition, model representation, scalability, sampling and perceptual issues. Our goal in this project is to address all these issues simultaneously, targetting the production of high-quality photographic like pictures that are capable of passing a "Turing-test": they are impossible to separate from photographs of the real world, with all its complexity.

Our goal in this project is to address the many hard challenges remaining in Photorealistic Rendering, especially dealing with the inclusion of real-world objects in virtual scenes and modelling complex materials, such as low-order scattering or high-reflectance materials. The challenges we selected have two points in common: they're regarded as difficult research challenges, and they would greatly enhance the realism of the pictures generated. Both teams stand to gain from a joint work in this area. This joint work should result in several scientific breakthroughs, with the production of photorealistic pictures of highly complex virtual worlds.

Présentation détaillée de l'Équipe Associée

1. Objectifs scientifiques de la proposition (1 à 2 pages)

Challenges in Photorealistic Rendering

Photorealistic rendering deals with the production of pictures of virtual worlds that are as close as possible to what a real photograph of this virtual world would look like. Considerable progress has been made in recent years, and photorealistic pictures are being used in severa sectors of the industry: virtual prototyping, special effects for motion picture, video games...

However, truly photorealistic pictures of a virtual world are still difficult to get. There are multiple difficulties to overcome: model acquisition, model representation, scalability, sampling and perceptual issues. Our goal in this project is to address all these issues simultaneously, targetting the production of high-quality photographic like pictures that are capable of passing a "Turing-test": they are impossible to separate from photographs of the real world, with all its complexity.

Our goal in this project is to address the many hard challenges remaining in Photorealistic Rendering, especially dealing with the inclusion of real-world objects in virtual scenes and modelling complex materials, such as low-order scattering or high-reflectance materials. The challenges we selected have two points in common: they're regarded as difficult research challenges, and they would greatly enhance the realism of the pictures generated. Both teams stand to gain from a joint work in this area. This joint work should result in several scientific breakthroughs, with the production of photorealistic pictures of highly complex virtual worlds.

There are several areas of photorealistic rendering that are currently regarded as significant research challenges by the graphics community:

All these issues are deeply inter-connected; for example, building a hierarchical representation for a complex object can be done by replacing micro-scale surface details by macro-scale and a complex material model, in a manner similar to the definition of the Torrance BRDF model. Such a hierarchical representation can be validated by perceptual criteria, and replacing a complex object with a higher level definition is a way to deal with scalability issues. Similarly, the inclusion of real world objects in virtual scenes is currently a difficult problem because the acquisition process gives highly detailed objects, with sub-mm details. Building a hierarchical representation for these objects is a prerequisite for including them in global illumination simulations.

These issues are being regarded by the research community as interesting, but too complex to be solved with existing algorithms and technologies. Our common ambition is to target all these issues, with the long term goal of producing photorealistic pictures of really complex worlds.

Work-program for the duration of the EA

Complex materials and Hierarchical representations: First, we will focus on the twin problems of hierarchical representations for complex objets and dealing with complex material properties. Taking as input a complex object, e.g. obtained through 3D scanners, highly detailed at the micro level, our goal is to build a lower level representation, with geometric details being averaged. However, we wish to keep the light-reflecting properties of the object, both for its visual aspect and for its interaction with the virtual scene. This requires building a reflectance model that behaves, at a larger scale, like the average of the micro-scale details in the most accurate geometry. Simultaneously, we will be working on complex material models, including sub-surface scattering and inhomogeneous materials. These two research themes are deeply inter-connected: the algorithms developed through the latter will help in illumination simulations for the reflectance models created with the former.

Scalability and Perceptual issues: Next, we will target the connected issues of scalability and perceptual rendering. Our goal is to design illumination simulation algorithms that are capable to handle really large scenes, with millions of polygons and thousands of light sources. Our work will build on existing work on Frequency Analysis of Light Transport (by the ARTIS team) and multi-dimensional light cuts (by Cornell).

High-Reflectance Materials Finally, we will focus on the specific questions of convergence of existing methods in the presence of highly reflective environments, or surfaces in close proximity to one another. In these environments, existing simulation methods converge slowly, with a large variange, or even diverge. Hence the need to work out new simulation methods that are able to handle these complex cases.

2. Présentation des partenaires (1 page environ par partenaire)
Présentez les différentes équipes participantes ;
Donnez, pour chaque partenaire, la liste des chercheurs impliqués dans la proposition ainsi qu'un bref CV du responsable ;
Remarque : pour chaque partenaire, la proposition peut être coordonnée par un jeune chercheur à condition de démontrer l'investissement de l'équipe à laquelle il appartient ;
Indiquez, pour chaque partenaire, les étudiants impliqués dans la proposition. Donnez une estimation de leur nombre et précisez si des thèses en cotutelle sont prévues ;
Présentez l'historique de la collaboration entre les équipes ;
Insérez des liens vers les pages des personnes, laboratoires, organismes....

"ARTIS" (Acquisition, Representation and Transformations for Image Synthesis):

ARTIS is both a project-team of INRIA Rhône-Alpes and a team of the LJK laboratory, a joint research unit of CNRS, INPG and UJF.

The project was created in January, 2003, based on the observation that classical image synthesis methods appear limited with respect to the variety of current applications. In particular, the typical approach consisting in separately modeling a 3D geometry and a set of photometric properties (reflectance, lighting conditions), then in computing their interaction to produce a picture, is too restrictive. First, this approach severely limits the adaptability to particular constraints or freedom allowed in each application (such as precision, real-time, interactivity, uncertainty about input data...); second, it restricts the possible image classes and does not easily lend itself to new uses in a more "expressive" way, such as for illustration, where a form of hierarchy on image constituents must be constructed.

One of the goals of the project is the definition of a more generic framework for the creation of synthetic images, integrating elements of 3D geometry, of 2D geometry (built from these), of appearance (photometry, textures...), of rendering style, and of importance or relevance for a given application. The ARTIS project therefore deals with multiple aspects of image synthesis: model creation from various sources of data, transformations between these models, rendering and imaging algorithms, and the adaptation of the models and algorithms to various constraints or application contexts.

The manager for the EA will be Nicolas Holzschuch. He is a CR1 researcher at INRIA. He has done his Ph. D. in the iMAGIS/GRAVIR research team in 1996. He has recently completed a sabbatical stay of 12 month at Cornell University. His work concern various aspects of global illumination simulation, using hierarchical finite element methods, such as wavelets, as well as exploiting possibilities of GPU for real-time realism. He has several contributions on using Frequency Analysis for more efficient computations of global illumination. He organised the 2007 Eurographics Symposium on Rendering in Grenoble, and was a member of the Program Committee of Eurographics Symposium on Rendering 2006 to 2008 and Eurographics 2004 and 2009.

Inside ARTIS, Cyril Soler (CR1 INRIA), Fabrice Neyret (CR1 CNRS) and Kartic Subr (post-doc INRIA) will also play a major role in the collaboration. The following 5 doctoral students are also involved: Charles de Rousiers, Nassim Jibaï, Cyril Crassin, Hedlena Bezerra and Adrien Bousseau.

Cornell University:

The Program of Computer Graphics, established in 1973, has been one of the pioneering leaders in computer graphics research in the world. Since its inception, the Program has been well-known for pioneering research in computer graphics using a rigorous scientific basis combined with experimental verification. The program combines a very strong interdisciplinary team of faculty, from the fields of physics, computer science, mechanical and electrical engineering, applied mathematics, and perception psychology, all who bring a wide range of experience and innovative perspectives to address complex research computer graphics problems.

In the recent past, one of their major goals has been to create realistic simulations which are physically accurate and perceptually indistinguishable from real-world scenes. Their research spans a broad range of topics to achieve this goal. They work on all three areas of physically-based rendering: the physics of light-reflection models, the computation and thermodynamics of energy propagation, and the creation of models of the human visual perceptual systems.

Their collective publishing record is among the strongest in the field, and they have led many workshops at ACM SIGGRAPH and at Cornell in the areas of measurement, rendering, and perception. Many of our alums have continued in academia and have held professorial appointments at CalTech, Utah, Georgia Tech, MIT, Stanford, Carnegie Mellon, University of Oregon, and UC Berkeley, to name a few. Others have gone on to become leaders in the fields of computer graphics, computer animation, and computer-aided-design in Architecture. Former students from the Program of Computer Graphics have now won five of Hollywood's Technical "Oscars", one actual "Oscar", and five have won the prestigious SIGGRAPH Achievement Award.

Kavita Bala is an Assistant Professor in the Computer Science Department and Program of Computer Graphics at Cornell University. She received her S.M. and Ph.D. from the Massachusetts Institute of Technology (MIT), and her B.Tech. from the Indian Institute of Technology (IIT, Bombay). Bala leads research projects in scalable rendering, perceptually-based rendering, interactive global illumination, parallelization, and texture synthesis. She has co-authored the graduate-level textbook "Advanced Global Illumination", now in second edition. In 2005 she co-chaired the Eurographics Symposium on Rendering (EGSR). She has served on more than 25 program committees including SIGGRAPH, SIGGRAPH Asia, Eurographics, Eurographics Symposium on Rendering, Interactive Ray Tracing, Interactive 3D Graphics, Graphics Interface. She is also the associate editor of IEEE TVCG Transaction on Visualization and Graphics. Bala has received the NSF CAREER award, and Cornell's College of Engineering James and Mary Tien Excellence in Teaching Award.

Bruce Walter is a research associate at Cornell University. He spent 12 months as a post-doctoral researcher with the iMAGIS team, a predecessor of ARTIS, in Grenoble, France in 1998-99. His research areas include global illumination, material modeling, and perceptual rendering, and has served on the program committes for SIGGRAPH (2008), Eurographics Rendering Symposium (2003/5/6/7/9) and IEEE Symposium on Interactive Ray Tracing (2006/7/8).

Historique de la collaboration entre les équipes:

3. Impact (1 page maximum)
Indiquez l'impact de cette collaboration sur :
- les objectifs scientifiques des équipes participantes ;
- les relations entre les partenaires et entre les instituts (par exemple discutez de la complémentarité, de la similarité pour un effet de masse critique, de la répartition des tâches pour un gros développement, etc.)

ARTIS and Cornell University are both working at the foreground of research on photorealistic rendering, and have a long experience on the subject. Their skills are complementary. This Associate Research Team targets hard and difficult research areas that are probably out of reach for each team separately. Joining forces will allow them to effectively target these areas, and attain significant results faster. We are expecting that this collaboration will result in regular publications, every year. Some of these cooperations are already under way, as a consequence of Nicolas Holzschuch's sabbatical stay at Cornell, but the potentials for synergy are much larger.

4. Divers : toute autre information que vous jugerez utile d'ajouter.


II. PREVISIONS 2009

Programme de travail

Description du programme scientifique de travail (1 à 2 pages maximum)

Our focus for the first year will be mostly on the subject of complex materials for scalable rendering. This includes two separate lines of work: first, accurate and scalable rendering of low order scattering in media, and second, hierarchical representations for complex objects.

Accurate and scalable rendering of low order scattering in media: this problem includes a wide range of effects from the focusing of light, called caustics, seen in a pool of water, to the distinctive appearance of minerals like amber. We are developing scalable algorithms for rendering refractive media like amber, water, glass as well as subsurface materials such as marble and skin. Currently such media are not correctly handled in interactive graphics due to their computational complexity. Our approach is to analytically find visually important light paths that contribute significantly to the appearance of these materials. This work will be a joint research between Kavita Bala (Cornell), Bruce Walter (Cornell) and Nicolas Holzschuch (ARTIS).

Complex materials and Hierarchical representations: We're taking as input a complex object, e.g. obtained through 3D scanners, highly detailed at the micro level. Our goal is to build a lower level representation, with geometric details being averaged. However, we wish to keep the light-reflecting properties of the object, both for its visual aspect and for its interaction with the virtual scene. This requires building a reflectance model that behaves, at a larger scale, like the average of the micro-scale details in the most accurate geometry. Simultaneously, we will be working on complex material models, including sub-surface scattering and inhomogeneous materials. These two research themes are deeply inter-connected: the algorithms developed through the latter will help in illumination simulations for the reflectance models created with the former. This work will be a joint research between Kavita Bala (Cornell), Bruce Walter (Cornell), Nicolas Holzschuch (ARTIS) and Charles de Rousiers (ARTIS, doctoral student).

Long term research: both previous research themes are expected to carry significant results within the year 2009. In parallel, we plan to start exploratory work on more long term research projects, including the design of new illumination simulation algorithms and the behavior of illumination simulation algorithms in the presence of difficult conditions: high-reflectance environments, aliasing issues. This work will include Kavita Bala (Cornell), Bruce Walter (Cornell), Nicolas Holzschuch (ARTIS), Fabrice Neyret (ARTIS), Cyril Soler (ARTIS) and Kartic Subr (ARTIS).

 

Programme d'échanges avec budget prévisionnel

1. Echanges

Décrivez les échanges prévus dans les deux sens : invitations de chercheurs de votre partenaire et missions INRIA vers votre partenaire ;
Précisez s'il s'agit de chercheurs confirmés ou de juniors (stagiaires, doctorants, post-doctorants) ;
Motivez, si possible, les raisons scientifiques (travail commun, workshop,..) et précisez la durée prévue ;
Résumez ensuite ces informations dans les tableaux 1 et 2 ci-dessous en faisant une estimation budgétaire :

 1. ESTIMATION DES DÉPENSES EN MISSIONS INRIA VERS LE PARTENAIRE
Nombre de personnes
Coût estimé
Chercheurs confirmés
2
5000 €
Post-doctorants
1
1700 €
Doctorants
3
5800 €

Stagiaires

   
Autre (précisez) :
   
   Total
6
12500 €

 

 2. ESTIMATION DES DÉPENSES EN INVITATIONS DES PARTENAIRES
Nombre de personnes
Coût estimé
Chercheurs confirmés
2
5000 €
Post-doctorants
1
2500 €
Doctorants    

Stagiaires

   
Autre (précisez) :
   
   Total
3
7500 €

2. Cofinancement

Cette coopération bénéficie-t-elle déjà d'un soutien financier de la part de l'INRIA, de l'organisme étranger partenaire ou d'un organisme tiers (projet européen, NSF, ...) ?
Indiquez ces éléments et donnez les montants associés. Dans le cas où votre proposition serait retenue, vous parait-il probable d'obtenir de l'organisme étranger partenaire un soutien financier symétrique ?
De quel montant ?

3. Demande budgétaire

Indiquez, dans le tableau ci-dessous, le coût global estimé de la proposition et le budget demandé à la DRI dans le cadre de cette Equipe Associée (maximum 20 K€).

Commentaires
Montant
A. Coût global de la proposition (total des tableaux 1 et 2 : invitations, missions, ...)
20.000 €
B. Cofinancements utilisés (financements autres que Equipe Associée)  
Financement "Équipe Associée" demandé (A.-B.)
(maximum 20 K€)
20.000 €

 

 

© INRIA - mise à jour le 11/08/2008