CS 335 Principles of Multimedia Systems -- Course Notes

Course Notes

Lecture 1: Introduction

Lecture 2: Digital Audio, Image and Video

Lecture 3: Tutorial on Matlab, OpenCV and SDL

Lecture 4: Lossless Compression - I

Lecture 5: Lossless Compression - II

Lecture 6: Lossless Compression in Multimedia Data Representation

Lecture 7: Lossy Compression

Lecture 8: JPEG

Lecture 9: Cameras and Projectors

Lecture 10: Finding Objects in Images

In this class, we talked about template matching (the naive and multiscale approach), Chamfer matching (a special case of template matching for binary images) and Hough transform (a voting based scheme).
Here is the matlab example of template matching and its multi-scale version, and the matlab example of general Hough transform .
All the above codes and test images can also be downloaded as a tar ball .
A just-for-fun package for reconstructing 3D shapes from multiple view images based on maximum flow image matching.

Lecture 11: Speech in Multimedia

Lecture 12: Audio

Lecture 13: Motion

Motion estimation using image block local search can be found in Li&Drew Ch10.2 and 10.3.
Motion estimation sample images: Desk image 1, Desk image 2, the block search motion estimation result and convex relaxation method result; House image 1, House image 2, the block search motion estimation result and convex relaxation method result.

Lecture 14: Optical Flow, Global Motion and Video Stabilization

Here is the reference paper for optical flow.
Global motion uses a transformation with very few parameters to define the motion field. The transformation can be similar, affine, or projective. Estimating global motion can be done by finding feature points correspondence and then solving a set of linear equations. I have shown an example application, the video stabilization, in class. You can try the video stabilization matlab function. You first need to download the SIFT demo code from Prof. David Lowe's webpage and copy the "stabled" function into the demo directory. You need another function to precompute all the sift features in each video frame. The testing data and results are here.

Lecture 15-16: Video Compression and Standards

Lecture 17: MPEG-7

Lecture 18: Multimedia Over IP Networks -- I

Slides: [ppt].
Code (camera.c and camera.h) to extract video from tcp network cameras can be found in class 3's sample codes.