CS-GY-6643 Computer Vision (class #23724), Spring 2017

Computing properties of our 3-D world from passive and active sensors

Syllabus, Guido Gerig (home)

Goal and Objectives:

General Information:

Lecture:        Tue 6 - 8.30pm,  Rogers Hall, Rm 215
Instructor:    Guido Gerig (gerig at nyu.edu)
                        Office: 2 MetroTech Center, 10.094. office hours Mon 3.30 to 5.30pm
TA:                 Rupanta Rwiteej Dutta (rrd300 at nyu.edu)
                        Guangqi Chen (gc1779 at nyu.edu)
                         Office hours: 2 MetroTech Center, 10th floor, cubicle space 10.098E (outside Gerig office)
                        Tuesdays  3-5pm (Rupanta Dutta), Thursdays 2-4pm (Guangqi Chen).


Material:
Primary Textbook, to be purchased by students:  Computer vision: a modern approach, by Forsyth and Ponce. Prentice Hall, 2011 (or original 2002 version).
(Secondary Textbook:  not to be purchased by students  Book by Rick Szeliski).
The class will make use of MATLAB for projects. Students can also use of C++.

Prerequisites: 
This course will focus on the 3D aspect of Computer Vision and thus focus on geometry and algorithms to computer the 3D scene from multiple images. We will use basic image processing but the necessary background will be introduced in class. This course introduces the many techniques and applications of computer vision and scene analysis. | Prerequisites: Graduate standing, CS-GY 5403 and MA-UY 2012, or equivalents, or instructor’s permission.

Syllabus (pdf document)

This course will use NYUClasses  NYUClasses for organization of materials, exams, assignments, up- and downloads.

Schedule (DATES AND TOPICS SUBJECT TO CHANGE):
Date
 Topic
 Slides
 Readings (files)
 Additional Materials
 Assignments
24-Jan-17
 Introduction, Organization
Image Formation
 Introduction slides (pdf)
 Textbook Forsyth&Ponce:
Book Contents (pdf)
Image Formation: Ch1 Cameras (pdf)

31-Jan-17
 Image Formation
 Image Formation I:
Cameras, Lenses, Sensors (slidesI, slidesII)

 Textbook Forsyth&Ponce:
Image Formation: Ch1 Cameras (pdf)
Geometric Camera Models CH 2 (pdf)


online
 Matlab/Octave Introduction
Preparation Matlab/Octave:
 (Credit for Matlab intro: Bo Wang and Avantika Vardhan, U of Utah), Octave Code: Rupanta Dutta
 Slides pdf,
Matlab code and images archive),
Octave code (archive)
 Matlab tutorial streaming video: vimeo page directly at http://vimeo.com/105393037
07-Feb-17
 Image Formation ctd.
Image Transformations, Lenses, Camera Calibration
 Image Formation II (pdf)
Lenses and Sensors (pdf)
Camera Calibration (pdf)
Radial Distortion Corr: (pdf)

 Geometric Camera Models:
Old Book: Read 2.1, 2.2, 3.1-3.3, New Book: 1.2, 1.3
Part I:  Image Formation: a) Intrinsic/Extrinsic Parameters, b)
Camera Calibration slides (intrinsic, extrinsic parameters) S.M. Abdullah
 Assignment 1 out (see NYUclasses)
Due Wed Feb 22nd
14-Feb-17
 Stereo, Epipolar Geometry,Stereo  Essential and Fundamental Matrices.
Correspondence: Multiview stereo, Triangulation.
 Slides Stereo, Multiple View Geometry I (pdf)
Slides Multiple View Geometry II (pdf)
 Intro Stereo: New book: 1.13, Old book: 1.2
New book: Ch7 Stereopsis
Old Book:  Ch10 and 11

21-Feb-17
 Multiview Stereo, Rectification, Correspondences
Triangulation, 3D Reconstruction
 Slides Multiple View Geometry II (pdf)


 New book:  Ch 7.4/7.5/7.6 Stereopsis
Old Book:  Ch 11.2/11.3/11.4
Old book: 11.1 and 10.1.4 (motions)
New book: Ch7.2 and Problem 7.2
Assignment 2 out Wed 22-Feb
(see NYUclasses)
due Thu Mar 09
28-Feb-17
 Multiview Stereo, Rectification, Correspondences
Triangulation, 3D Reconstruction
 Slides Image Rectification (pdf)
Slides Correpondences/Correlation (pdf)
Stereo: Triangulation (pdf)
 see above
 Image Rectification (Trucco & Verri Textbook) (pdf)

07-Mar-17
 Midterm Exam
60 minutes

POTENTIAL PROJECTS (slides)
 Materials covered:
1) Image Formation 1.1, 1.2,
1.3.1) Geometric Camera Calibration
7) Stereopsis:  7.1, 7.2, 7.4.1


13-17 Mar 2017
 NYU Spring Recess



21-Mar-17
 Introduction: Reflectance Maps, Shape from Shading

Photometric Stereo (ctd)

Shape from Shading: Surface Reconstruction
 Photometric Stereo and Shape from Shading SfS: Slides G. Gerig
Slides introduction (pdf) , SfS (pdf)
 New Book: Chapter 2
Old Book:: Chapters 4 and 5

Hands on Shape from Shading, Technical Report, May 2008 by Shireen Y. Elhabian (pdf)
 Additional materials/slides Photometric Stereo and Shape fronm Shading (Ohad Ben-Shahar BGU)
Slides Wolff JHU (pdf)
Link to materials Wolff JHU (link)

SfS via curve evolution (Kimmel, Siddiqi, Bruckstein): (pdf)
28-Mar-17
 Shape from Shading continued

Optical Flow I
 see above

Optical Flow (OF) I: Slides G. Gerig (pdf)
additional handwritten notes (pdf)
 Hands on Shape from Shading, Technical Report, May 2008 by Shireen Y. Elhabian (pdf)
Assignment 3 out Tue 28-March,
due Tue 11-April
04-Apr-17
 Optical Flow I

Optical Flow II, Structure from Motion
 Optical Flow (OF) I: Slides G. Gerig (pdf)

Optical Flow II:  Slides G. Gerig (pdf)
 Optical Flow:  Computer Vision Book (pdf)
Original Paper Horn & Schunck 1981 (pdf)

11-Apr-17
 Optical Flow II, Structure from Motion

Optical Flow III
 Optical Flow II:  Slides G. Gerig (pdf)


Optical Flow: Structure from motion (see previous slides), additional handwritten notes
on structure from motion (SfM)
 Optical Flow:  Computer Vision Book T&V (pdf)
Original Paper Horn & Schunck 1981 (pdf)
Optical Flow Computer Vision Book T&V Ch8 (pdf)
CAP5415 - Computer Vision ,    
Slides: Motion&Optical Flow
 Assignment 4 out Tue 11-April,
due Tue 15-April
18-Apr-17
 Structured Lighting I and II
 Structured Lighting I: Slides G. Gerig (pdf)
Powerpoint slides w. animations (ppt)

Structured Lighting II  cts.  (pdf)
 Structured Lighting: Some pages from Computer Vision  Book R. Klette (pdf)
 Build your own 3D scanner: SIGGRAPH 2009 course: D. Lanman/ G. Taubin Brown University: Notes pdf,   Slides
Stanford, Levoy et al: Real-time 3D Model Acquisition (link)
3D photography on your desk: Bouget &Perona ICCV 1998: (pdf) *
25-Apr-17
 Shape from Silhouettes
 Shape from Silhouettes I: Slides G. Gerig (pdf)
Shape from Silhouettes II (Visual Hulls): Slides G. Gerig (pdf)
Shape from Silhouettes III (Forbes): Slides G. Gerig (pdf)
 Marching Intersections (M. Tarini (pdf))
Exact Polyhedral Methods (W. Matusik pdf )
Image-based Visual Hulls IBVH (W. Matusik, pdf)
 K. Forbes: Consistency Constraints (pdf)
K. Forbes: Uncalibrated, two planar mirrors (pdf)
02-May-17
 Last Class: Final Project Presentations
Attendance of all for whole time required !
 List of Projects ()


10-May-17
 Final Project Due


Final Project Report Due

Other Information:


NYU Tandon School of Engineering Honor Code

Students are expected to work on their own, as instructed by the Professor. Students may discuss projects with other individuals either in the class or outside the class, but they may not receive code or results electronically from any source that is not documented in their report. Students must write their own code, conduct their own experiments, write their own reports, and take their own tests. Any use of sources (for projects or tests) that are not specifically given to the student by the Professor or TA, must be discussed with the Professor or TA or documented in the report. Any student who is found to be violating this policy will be given a failing grade for the course and will be reported to the authorities as described in the University's Student Code of Conduct:

Homework
Homework assignments are due at 11:59pm on the given due date. Written assignments should be in
pdf format, while coding assignments should additionally be source files. Coding can be done in MATLAB (using
only the base package, no toolkits), Octave, or C++. The report should clearly identify code developed by the student and pieces of code obtained by external sources.

Grading

Weighted contribution of projects and exams to final grade:

* Exceptions may apply for excused absences documented by officials (e.g. medical problems), requiring additional advanced notice.



Resources:

Matlab Introduction Imaging, special course lecture: