ABOUT        RESEARCH        PUBLICATIONS       PERSONAL       CV
 

Todor Georgiev

Sr. Research Scientist, Photoshop Group, Adobe Systems
tgeorgie at adobe dot com

Computational Photography (mathematical section)

In 2007 I taught the mathematical section of the SIGGRAPH Course on Computational Photography:

Lecture notes: Mathematical Analysis of Light-Field Cameras (ppt). Small file (pdf).
Homework problems: Now accepting solutions and questions by e-mail.

It presents complete lightfield theory that gives us the tools to solve old and new problems. For example, understanding Dappled Photography, the MERL "heterodyning" camera.

Outline

The physical quantity recored in a light-field camera is the radiance. It's well understood by science. So, if we want to build a rigorous theory of light-fields, we need to study what's already known about radiance in radiometry, optics and theoretical physics:

(1) In the course we use "plane-angle" parametrization of rays. This is a more flexible "differential" form of the familiar two plane parametrization.

(2) Ray transforms are represented in the formalism of matrix optics. Relations to Hamiltonian mechanics are shown. Rays satisfy Liouville's theorem due to conservation of a natural symplectic dot product that exists in ray space.

(3) Radiance conservation is derived. The explicit form of radiance transforms ("lightfield transforms") follows directly from that. This is shown both in spatioangular, and in frequency representation.

(4) Cameras described mathematically as examples of how the method works. This presentation is based on our Adobe Tech Report of April 2007.

- Pinhole camera (nontrivial derivation), and "2F" camera.

- Ives' camera, invented back in 1903, now presented with a new theory. Very interesting!

- Lippmann's "reversible prints" (and the plenoptic camera). His original 1908 paper: Epreuves Reversibles. Photographies Integrales. English translation (Fredo Durand): Reversible Prints. Integral Photographs.

- MERL "heterodyning" camera.

- Adobe camera (arrays of lenses and prisms). See cool presentation at macnews.de

- I showed some cool lightfield cameras and lenses. Also, watch the video of a more recent, and better demonstration of our 3D lens technology: Array of lenses and prisms

- Several examples are created with our Adobe frequency multiplexing camera ("dappled" or "heterodyning" camera in MERL terminology), and with our Adobe lens-prism camera.

(5) Additional links to very good and educational reading material:

Lumigraph

Hamiltonian mechanics and Liouville's theorem.

Plenoptic sampling

A frequency analysis of light transport

Integral history: 100 Years Light-Field

Plenoptic camera and Hand-held plenoptic camera

Stanford camera array

Adobe camera.

Below you see what could be the first real-life "heterodyning" picture! It was taken with our frequency multiplexing camera during the Photoshop CS3 shipping party (back in April 2007). The F/number of the main lens was a bit too low for the mask, resulting in aliasing artifacts. Still, the intended motion of the background is clearly visible. Do we also see reflections moving on the glasses?...

Below is a picture I took during the days of SIGGRAPH 2007, on Coronado Island. Trying to focus on a speeding boat. Taking pictures at the beach is much more rewarding than attending SIGGRAPH talks!

This is again during our CS3 shipping party in April:

This is a very recent picture, "heterodyning" with a microlens array (instead of mask). Trying to refocus. Angular resolution is a bit too low...

This is better. Again microlens array, only bigger and no "heterodyning". Now it's Fall, the leaves are getting yellow.