ENGINEERING

The eyes are among the most important human senses. They make it possible to capture detailed information about spatial relations and to recognize motions. Computers use cameras, but they still have problems interpreting a real environment with the aid of visual input alone. Capturing geometry and motion in scenes is even more difficult for them.

Christian Theobalt, leader of the group "Graphics, Vision and Video" at the Max Planck Institute for Computer Science in Saarbrücken, is dealing with this matter. On the basis of video camera input, he wants to enable computers to reconstruct motions within scenes, as well as surface characteristics of the filmed objects. "Although computer graphics and computer vision have made substantial progress towards this goal, many methods are in the early stages and are bound to tight constraints," says Theobalt, who also is a professor of computer science at Saarland University. "The results so far are not advanced enough that they can be compared with the level of detail of the real world," according to Theobalt. Through his project "CapReal -- Performance Capture of the Real World in Motion," he hopes to change this. The European Research Council (ERC) has honored him for this project with an ERC Starting Grant. Thousands of scientists from Europe apply for the grant every year, but only a few are chosen. With this program, the European Union intends to support excellent junior researchers to break new ground and map out future research directions in their areas. Thus Theobalt will receive a research funding of 1.48 million Euro for five years.

Theobalt wants to establish theoretical bases for whole new methods of dynamic scene reconstruction. In a heretofore unknown level of detail, computers should be able to capture geometry, motions and material characteristics (for example deformation behavior) in complex, real scenes -- independent of whether these scenes take place in studios or outdoors. In the outdoor case, the complexity of illumination, motion and geometry in scenes increases significantly. "Instead of working with a sensitive and dense installed system, we want to get by with fewer cameras, so that in the end a few cameras on mobile phones will be enough," Theobalt explains.

To achieve this, his research group has to review basic concepts of computer vision. "Light creates what we see in a scene. Its multi-facetted interaction with objects in a real scene also leads to very complex appearance effects, which we need to understand. Current methods make starkly simplifying assumptions about this. To reconstruct scenes even under such circumstances, our programs need to do a better job of assessing the lighting conditions, and they must be able to describe them more accurately," Theobalt says. To do this, the computer scientists from Saarbrücken will extend computer graphics methods traditionally used to generate photo-realistic pictures such that they can be used for reconstruction. Furthermore, they will work on new methods to calculate material and surface characteristics from objects that are moving in the scene. But these are not the only challenges in the framework of computer graphics and computer vision, that the researchers have to face. They also need to develop novel approaches to compute the resulting mathematical problems efficiently.

To commercialize research results from their earlier research, the members of his group in Saarbrücken are founding the corporation "The Captury". Their technology allows capturing people?s motions without special equipment, just by the aid of conventional video cameras. That simplifies, for instance, the production of novel visual effects. It is not only film and TV productions that will benefit from the computer scientists? Biomechanics could analyze the motions of a professional athlete on the court in a much higher level of detail. Doctors could not only measure the motions of their patients with inexpensive cameras, but furthermore could watch the deformation of their muscular system simultaneously. In this manner they would be better able to judge whether a therapy is having a positive effect. Moreover, new procedures for how computers might be controlled via gestures could be considered. Within industry, engineers would be able to assess how a material reacts to a certain pressure with fewer cameras and less effort than today. Even autonomous vehicles and robotic systems might be able to understand more of the real world.