Image Alignment

Due Thursday Sept. 22

In this project, you'll automatically create color images from the black and white photographs of Sergei Mikhailovich Prokudin-Gorskii (1863-1944). The download directory contains scanned negatives of the Prokudin-Gorskii images from the Library of Congress.

Programming [20 points]

You should write a single function that takes a grayscale Prokudin-Gorskii image as input, and returns a high quality RGB image as output. Your function should be completely automatic, i.e. it should require no help from the user! Note that the order of the color channels, from top to bottom, in the Prokudin-Gorskii images is BGR, not RGB.

Write a simple brute-force alignment algorithm that first splits the image into 3 parts. Use the G channel as the anchor channel, and find the best displacements for the R and B channels. Once you have the displacements, you can construct the RGB image by put them into corresponding color channels and display one the screen. How do you determine which is the best displacement? Use a simple image comparison metric such as the sum of squared difference (SSD) or the normalized cross correlation (NCC). NCC is the dot product between two normalized vectors. You may want to do normalization on small images blocks instead of the whole image. You also may want to exclude the border from the comparison. Consider using the demo code in our class for localizing objects as a basis for this project. The codes are available at the note section of the course web site (http://www.cs.bc.edu/~hjiang/cs374/notes.html).

You'll notice that if the matching kernel size D is large or if the images are large, then this solution is prohibitively slow: The time complexity is O(n*D^2), where n is the number of pixels in the image. Discuss possible ways to speed up the computation.

Matlab Tips

  1. Keep the images as uint8 to save memory, but beware of using uint8 data in computations: Integer arithmetic in matlab is saturating! For most computations, you'll want to convert the image to single or double if you can afford the memory.

  2. Don't use loops for things like SSD and NCC. SSD is easily computed as sum((a(:)-b(:)).^2), and NCC as dot(a(:),b(:))/norm(a(:))/norm(b(:)).

  3. Matlab functions that may be useful for this assignment:
    • help - as always!
    • cell - create an empty cell array (array of null pointers)
    • tic, toc - timer
    • circshift - one easy way to deal with displacement
    • cat - for creating the RGB image from the 3 parts
    • imfilter - image filtering (convolution) with helpful boundary options
    • conv2 - raw 2D convolution
    • imresize - resize an image
    • im2col - extracting image blocks and put them into columns
    • norm, dot - basic vector operations
    • sprintf - for creating filenames

Writeup [10 points]

Show the aligned RGB image for each input image in the download directory. If you cannot align an image, explain why. Also, for each image, report the time your code took.

What to Hand In

Submit your code and writeup electronically to the black board system. You do not need to hand in a printout.

The images and the idea for the project is from Alyosha Efros. This web page is modified from David Martin's 2009S course project.