ACADEMIA

ARLINGTON, VA -- Your rich uncle dies and leaves you a painted masterpiece he's had hidden away for years. But, it's scratched, torn, and much of the paint has flaked away. You could take it to a painting restorationist, but this can take months and in any case, restoration is very subjective. What to do? You call a mathematician of course. They call it partial differential equation-based interpolation of lost image regions, but don't let that scare you. It's just inpainting – the art of modifying an image in an undetectable form – done by mathematical algorithm instead of conservator subjectivity. Scientists funded by the Office of Naval Research have developed computer techniques that can now automate a large part of a formerly laborious process, and eliminate the subjectivity most conservators working by hand bring to the job. The scientists hard at work on this are Guillermo Sapiro (University of Minnesota) and Andrea Bertozzi (Duke University) and a few of their students and collaborators. Sapiro was interested in image processing in general and found the problem of inpainting – restoring works of art – curiously relevant to his work He watched restorationists at the Minneapolis Institute of Arts, and it occurred to him that he could capture what they were doing in a computer algorithm if he could find the right partial differential equations. Sapiro was familiar with Bertozzi's work in fluid dynamics, and encouraged by ONR's Wen Masters, science officer overseeing the research, they found that the inpainting equations Sapiro was using were connected with work being done by Bertozzi with Navier-Stokes equations. "The novelty of our method," says Sapiro, "is that we automatically fill-in the lost pieces of the image by using information from neighboring available regions. The connection we found between art, image processing, applied mathematics, and fluid dynamics, is not just surprising but fascinating." And don't think they didn't do their homework. Sapiro and Bertozzi tried many sets of equations and flow methods (of color, shape, line, shadow, shade, texture, geometry, etc) before finding algorithms, connected with classical problems in a completely different area, that can perform this quite difficult task of image inpainting. The difference between their method and what is used today even on digital images is that even with current sophisticated computer software, the process remains essentially a manual effort. The beauty of this mathematical method is that it is automatic and based on fundamental theory. The U.S. Navy is not in the business of restoring damaged masterpieces, but it is in the business of surveillance – both in still imagery and in video. Problems always arise in transmitting surveillance images when bandwidth is limited and the channels are noisy (images will be blurry or speckled), or just downright spotty, as when pixels are lost when compressed and then transmitted. Sapiro's group has shown that they can effectively use their inpainting algorithms to recover this lost information. "As long as the image isn't completely lost," says Sapiro, "we can do something with it quickly and computationally. We want to get away from the old method of retransmission query protocols." "A radical new way of encoding images for transmission could be developed based on the ideas of image inpainting," says Masters, "The basic idea is to transmit just enough information for outlines of blocks of an image, then 'fill in the missing parts' at the receiving end. Another application is image enhancement, which is of direct interest in processing of surveillance images. Using image inpainting, one could affect super resolution in processed images."