CS-GY-6643 Computer Vision (class #23724), Spring 2016

Computing properties of our 3-D world from passive and active sensors

Syllabus, Guido Gerig (home)

Goal and Objectives:

To introduce the fundamental problems of 3D computer vision.
To introduce the main concepts and techniques used to solve those.
To enable participants to implement solutions for reasonably complex problems.
To enable participants to understand basic methodology that is discussed in the computer vision literature.

General Information:

Lecture:        Tue 6 - 8pm, Rogers Hall, Rm 215
Instructor:    Guido Gerig (gerig at nyu.edu)
                   Office: 2 MetroTech Center, 10.094. office hours Mon 2-5pm
TA:            Rupanta Rwiteej Dutta, (rrd300 at nyu.edu)
                   Office hours: Tuesday 2 to 5pm, 2 MetroTech Center, 10th floor, cubicle space 10.098E (outside Gerig office)

Material:
Primary Textbook, to be purchased by students: Computer vision: a modern approach, by Forsyth and Ponce. Prentice Hall, 2011 (or original 2002 version).
(Secondary Textbook: not to be purchased by students Book by Rick Szeliski).
The class will make use of MATLAB for projects. Students can also use of C++.

Prerequisites:
This course will focus on the 3D aspect of Computer Vision and thus focus on geometry and algorithms to computer the 3D scene from multiple images. We will use basic image processing but the necessary background will be introduced in class. This course introduces the many techniques and applications of computer vision and scene analysis. | Prerequisites: Graduate standing, CS-GY 5403 and MA-UY 2012, or equivalents, or instructor’s permission.

Syllabus (pdf document)

This course will use NYUClasses NYUClasses for organization of materials, exams, assignments, up- and downloads.

Schedule (DATES AND TOPICS SUBJECT TO CHANGE):

Date	Topic	Slides	Readings (files)	Additional Materials	Assignments
26-Jan-16	Introduction, Organization Image Formation	Introduction slides (pdf)	Textbook Forsyth&Ponce: Book Contents (pdf) Image Formation: Ch1 Cameras (pdf)
02-Feb-16	Image Formation	Image Formation I: Cameras, Lenses, Sensors (slidesI, slidesII, slides-lenses III)	Textbook Forsyth&Ponce: Image Formation: Ch1 Cameras (pdf) Geometric Camera Models CH 2 (pdf)
online	Matlab/Octave Introduction	Preparation Octave: Rupanta Rwiteej Dutta (Credit for Matlab intro: Bo Wang and Avantika Vardhan, U of Utah)	Slides pdf, Matlab code and images archive), Octave code (archive)	Matlab tutorial streaming video: vimeo page directly at http://vimeo.com/105393037
09-Feb-16	Image Formation ctd. Image Transformations, Lenses, Camera Calibration	Image Formation II (pdf) Camera Calibration (pdf)	Geometric Camera Models: Old Book: Read 2.1, 2.2, 3.1-3.3, New Book: 1.2, 1.3 Part I: Image Formation: a) Intrinsic/Extrinsic Parameters, b)	Camera Calibration slides (intrinsic, extrinsic parameters) S.M. Abdullah	Assignment 1 is out: Deadline Wed February 24: (see NYUClasses) see Matlab/Octave tutorials and sample code above calibration pattern: (pdf) see also slides S.M. Abdullah (pdf)
16-Feb-16	Lenses, Distortion Stereo, Epipolar Geometry,Stereo Essential and Fundamental Matrices. Correspondence: Multiview stereo, Triangulation.	Radial Distortion Corr: (pdf) Lenses: slides-lenses III Lenses, depth of field animated (ppt) Depth of field animated slides (pdf) Slides Stereo, Multiple View Geometry I (pdf) Slides Multiple View Geometry II (pdf)	Cameras with Lenses Intro Stereo: New book: 1.13, Old book: 1.2 New book: Ch7 Stereopsis Old Book: Ch10 and 11
23-Feb-16	Multiview Stereo, Rectification, Correspondences Triangulation, 3D Reconstruction	Slides Multiple View Geometry II (pdf)	New book: Ch 7.4/7.5/7.6 Stereopsis Old Book: Ch 11.2/11.3/11.4 Old book: 11.1 and 10.1.4 (motions) New book: Ch7.2 and Problem 7.2		Assignment 2 out (pdf), due Wed 09-Mar midnight Assignment 1 in (NYUClasses), Wed 23-Feb-16 midnight
01-Mar-16	Multiview Stereo, Rectification, Correspondences Triangulation, 3D Reconstruction	Slides Image Rectification (pdf) Slides Correpondences/Correlation (pdf) Stereo: Triangulation (pdf)	see above	Image Rectification (Trucco & Verri Textbook) (pdf)
08-Mar-16	Midterm Exam 60 minutes	Materials covered: 1) Image Formation 1.1, 1.2, 1.3.1) Geometric Camera Calibration 7) Stereopsis: 7.1, 7.2, 7.4.1
08-Mar-16	Multiview Stereo: Wrap-up Correspondences, Rectification	see above	see above	see above
15-Mar-16	NYU Spring Recess
22-Mar-16	Introduction: Reflectance Maps, Shape from Shading Photometric Stereo (ctd) Shape from Shading: Surface Reconstruction	Photometric Stereo and Shape from Shading SfS: Slides G. Gerig Slides introduction (pdf) , SfS (pdf)	New Book: Chapter 2 Old Book:: Chapters 4 and 5 Hands on Shape from Shading, Technical Report, May 2008 by Shireen Y. Elhabian (pdf)	Additional materials/slides Photometric Stereo and Shape fronm Shading (Ohad Ben-Shahar BGU) Slides Wolff JHU (pdf) Link to materials Wolff JHU (link) SfS via curve evolution (Kimmel, Siddiqi, Bruckstein): (pdf)	Assignment 2 in (NYUClasses), Wed 23-Mar-16 midnight Assignment 3 out (pdf) synthetic images sphere images dog-images-png dog-images-tif
29-Mar-16	Shape from Shading continued Optical Flow I	see above Optical Flow (OF) I: Slides G. Gerig (pdf) additional handwritten notes (pdf)	Hands on Shape from Shading, Technical Report, May 2008 by Shireen Y. Elhabian (pdf)		Final Project Examples / Ideas (pdf) Deadline Fri 4/8 midnight for Project Title/Abstract.
05-Apr-16	Optical Flow I Optical Flow II, Structure from Motion	Optical Flow (OF) I: Slides G. Gerig (pdf) Optical Flow II: Slides G. Gerig (pdf)	Optical Flow: Computer Vision Book (pdf) Original Paper Horn & Schunck 1981 (pdf)		Assignment 3 due Fri April 8 noon Assignment 4 out Friday April 8 toycar movie (movie) video-sequence-images b/w video-sequence-images color
12-Apr-16	NO CLASS	Individual work on Assignment 4 and Final Project
19-Apr-16	Optical Flow II, Structure from Motion Optical Flow III	Optical Flow II: Slides G. Gerig (pdf) Optical Flow: Structure from motion (see previous slides), additional handwritten notes on structure from motion (SfM)	Optical Flow: Computer Vision Book T&V (pdf) Original Paper Horn & Schunck 1981 (pdf) Optical Flow Computer Vision Book T&V Ch8 (pdf)	CAP5415 - Computer Vision , Slides: Motion&Optical Flow	Assignment 4 due Fri April 22 midnight (pdf)
26-Apr-16	Structured Lighting I and II	Structured Lighting I: Slides G. Gerig (pdf) Powerpoint slides w. animations (ppt) Structured Lighting II cts. (pdf)	Structured Lighting: Some pages from Computer Vision Book R. Klette (pdf)	Build your own 3D scanner: SIGGRAPH 2009 course: D. Lanman/ G. Taubin Brown University: Notes pdf, Slides Stanford, Levoy et al: Real-time 3D Model Acquisition (link) 3D photography on your desk: Bouget &Perona ICCV 1998: (pdf) *
03-May-16	Project Presentations	List of Projects (pdf)
09-May-16	Final Project Due				Final Project Due

Other Information:

NYU Tandon School of Engineering Honor Code

Students are expected to work on their own, as instructed by the Professor. Students may discuss projects with other individuals either in the class or outside the class, but they may not receive code or results electronically from any source that is not documented in their report. Students must write their own code, conduct their own experiments, write their own reports, and take their own tests. Any use of sources (for projects or tests) that are not specifically given to the student by the Professor or TA, must be discussed with the Professor or TA or documented in the report. Any student who is found to be violating this policy will be given a failing grade for the course and will be reported to the authorities as described in the University's Student Code of Conduct:

Homework
Homework assignments are due at 11:59pm on the given due date. Written assignments should be in
pdf format, while coding assignments should additionally be source files. Coding can be done in MATLAB (using
only the base package, no toolkits), Octave, or C++. The report should clearly identify code developed by the student and pieces of code obtained by external sources.

Grading

Weighted contribution of projects and exams to final grade:

Projects (4), theoretical and practical parts, coding, tests and project report: 55%.
Midterm Exam: 15%.
Final Project: 20%
Participation/Quizzes: 10%. (Attendance to lectures is required to be eligible for office hours by the instructor and the TA. Attendance and regular active participation is required to get the participation grade).
We will have a few short in-class quizzes to check understanding. Scores will be part of the participation grade.
Late policy on projects: 10 percent deduction per day, projects no more accepted after 3 days.*
Please note that for this graduate level class, an A grade means outstanding, whereas A- stands for excellent.

* Exceptions may apply for excused absences documented by officials (e.g. medical problems), requiring additional advanced notice.

Resources:

Matlab Introduction Imaging, special course lecture:

Course Materials Avantika Vardhan and Bo Wang (August 2014) (updated slides pdf, Matlab code and images archive)
Matlab tutorial streaming video: vimeo page http://vimeo.com/sciinstitute and directly at http://vimeo.com/105393037