Topics in Image Processing and Computational Photography

Spring Semester, 2010

---

Course Description
This course covers the exciting new area of "computational photography" which refers to a broad group of imaging and processing techniques that enhance or extend the capabilities of digital photography to produce new photographs that could not have been taken by a traditional camera. Computational photography promises to change the way we think about photography.

See also: What is Computational Camera?, from Shree Nayar 

Location/Time
Room 2443, E3-1 CS Building, 16:00-17:15pm on Tues and Thur

Lecturer
Dr. Yu-Wing Tai (web)
Email: yuwing@cs.kaist.ac.kr / yuwing@gmail.com
Office hours: please arrange appointment via email.

Assessment:
This course is 100% course work (three assignments, 20% each and one final project, presentation and report, 40%).
No quiz, no mid-term, no final exam !

Text
There is no required text. Various lecture notes and assigned readings of relevant research papers will be made available.

Prerequisites:
N/A, but some basic knowledge on image processing are expected.

Computational Photography in Other Universities:
MIT, CMU, Stanford, Georgia Tech, UNC, NYU, Columbia, UWashington, etc

Schedule and Notes

Lecture	Date	Description	Extra Readings	Assignments	Materials
1	2 Feb	Course overview and Image formation			PDF PDF
2	4 Feb	Colors	Color harmonization [Siggraph'06]		PDF
3	9 Feb	Image Processing			PDF (Part 1)
4	11 Feb	Image Processing			PDF (Part 2)
	16 Feb	No Class, School Founding Anniversary
5	18 Feb	Interactive Image Segmentation	Image snapping [Siggraph'95], Intelligent scissors [Siggraph'95], Lazy snapping [Siggraph'04], Grabcut [Siggraph'04], Paint Selection [Siggraph'09], Video Snap Cut [Siggraph'09]	A1 is out Task Library GUI Report Template pie.cpp cfm.h cfm.cpp	PDF
6	23 Feb	Gradient image processing	Poisson Image Editing [Siggraph'03], Interactive Digital Photomontage [Siggraph'04], Drag-and-Drop Pasting [Siggraph'06], Real-Time Gradient-Domain Painting [Siggraph'08], Moving Gradients: A Path-Based Method for Plausible Image Interpolation [Siggraph'09] Coordinates for Instant Image Cloning [Siggraph'09]		PDF
7	25 Feb	Matting	Blue Screen Matting [Siggraph'95], A Bayesian Approach to Digital Matting [CVPR'01], Poisson Matting [Siggraph'04], Defocus Video Matting [Siggraph'05], A Closed Form Solution to Natural Image Matting [CVPR'06], Flash Matting [Siggraph'06], Natural Video Matting using Camera Arrays [Siggraph'06], Soft Scissors : An Interactive Tool for Realtime High Quality Matting [Siggraph'07]		PDF
8	2 Mar	Markov random fields	Probabilistic inference in graphical models		PDF
9	4 Mar	Image Warping/Morphing	Image Warping By Radial Basis Functions: Application to Facial Expressions		PDF
10	9 Mar	Structure-aware Image Manipulation	Feature-Aware Texturing [EGSR’06] 2D Shape Deformation Using Nonlinear Least Squares Optimization [IJCG’06] Seam Carving [Siggraph'07] Improved Seam Carving for Video Retargeting [Siggraph’08] Optimized scale-and-stretch for image resizing [Siggraph Asia'08] Content-Preserving Warps for 3D Video Stabilization [Siggraph’09] Multi-operator Media Retargeting [Siggraph’09] iWIRES: An Analyze-and-Edit Approach to Shape Manipulation [Siggraph’09]	A1 Program Due / A2 is out Task GUI	PDF
11	11Mar	Panoramic imaging	Creating Full View Panoramic Image Mosaics and Environment Maps [Siggraph'97] Recognising Panoramas [CVPR'03] Seamless Image Stitching in the Gradient Domain [ECCV'04] Dynamosaics: Video Mosaics with Non-Chronological Time [CVPR'05] Photographing long scenes with multi-viewpoint panoramas [Siggraph'06]	A1 Report Due	PDF
12	16Mar	Image restoration and Motion Deblurring	Motion-Based Motion Deblurring [PAMI’04] Fluttered Shutter Camera [Siggraph’06] Image Deblurring with Blurred/Noisy Image Pairs [Siggraph'07] Image/Video Deblurring using a Hybrid Camera [CVPR'08] Robust Dual Image Deblurring [CVPR’08] Motion Invariant Photography [Siggraph’08] Coded Exposure Deblurring: Optimized Codes for PSF Estimation and Invertibility [CVPR’09] Understanding and Evaluating Blind Deconvolution Algorithm [CVPR'09] Invertible Motion Blur in Video [Siggraph’09]		PDF
13	18 Mar	Depth and defocus	Scene Geometry from Projector Blur [Siggraph'06] Image and Depth from a Conventional Camera with a Coded Aperture [Siggraph'07] Active Refocusing of Images and Videos [Siggraph’07] Multi-aperture photography [Siggraph'07] Programmable Aperture [Siggraph'08] 4D Frequency Analysis of Computational Cameras for Depth of Field Extension [Siggraph’09]		PDF
	23 Mar	mid-term exam period
	25 Mar	mid-term exam period
14	30 Mar	Computational illumination	Flash Photography Enhancement via Intrinsic Relighting [Siggraph'04] Digital Photography with Flash and No-Flash Image Pairs [Siggraph'04] Recovering high dynamic range radiance maps from photographs [Siggraph'97] High Dynamic Range Imaging: Spatially Varying Pixel Exposures [CVPR'00] Removing Photography Artifacts using Gradient Projection and Flash-Exposure Sampling [Siggraph'05] A Lighting Reproduction Approach to Live-Action Compositing [Siggraph'02] Light Waving: Estimating Light Positions From Photographs Alone [EG'05] Image Fusion for Context Enhancement [NPAR'04]	A2 Program Due / A3 is out Task	PDF
15	1Apr	Temporal sequence re-rendering	Video Texture [Siggraph'00] Dynamic Texture [IJCV’03] Flow-based Video Synthesis and Editing [Siggraph’04] Video Matching [Siggraph’04] Making a Long Video Short: Dynamic Video Synopsis [CVPR’06] Computational Time-Lapse Video [Siggraph’07] Factored Time-Lapse Video [Siggraph’07]	A2 Report Due Choose topic of final report Max. two students in one group. You can use your own research projects for this course. If you don't have any idea, come to my office for discussions.	PDF
16	6Apr	Non-photorealistic rendering	NPR Course note [Siggraph'99] Image Analogies [Siggraph'01] Image Quilting for Texture Synthesis and Transfer [Siggraph'01] Learning Style Translation for the Lines of a Drawing [ToG'03] Non-photorealistic Camera [Siggraph'04] Real-Time Video Abstraction [Siggraph'06]		PDF
17	8 Apr	Photometric Stereo	Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs [PAMI'05] Dense Photometric Stereo: A Markov Random Field Approach [PAMI’06] ShapePalettes: Interactive Normal Transfer via Sketching [Siggraph’07] Non-rigid Photometric Stereo with Colored Lights [ICCV’07] A Photometric Approach for Estimating Normals and Tangents [Siggraph Asia’08] A Hand-held Photometric Stereo Camera for 3-D Modeling, ICCV’09		PDF
18	13 Apr	Misc	Lensless Imaging with a Controllable Aperture [CVPR'06] Dual Photography [Siggraph'05] Jitter Camera: High Resolution Video from a Low Resolution Detector [CVPR'04] Fast Separation of Direct and Global Components of a Scene using High Frequency Illumination [Siggraph'06]	A3 Program due	PDF
19	15 Apr	Plenoptic function and light fields	Light Field Rendering [Siggraph'96] The Lumigraph [Siggraph'96] Light Field Photography with a Hand-held Plenoptic Camera [Siggraph'05]	A3 Report due	PDF
20	20 Apr	Future cameras		1 page proposal of final project due Expected to see abstract, problem definition and project goal.	PDF
21	22 Apr	Human Computer Interaction			PDF
22	27 Apr	Taking great pictures			PDF
	29 Apr	Break for final project
23	4 May	Students' presentations
24	6 May	Students' presentations		Final Report Due on 11 May Expected to see a full paper submission.