Math for Intelligent Systems, COT5615

COT 5615, Math for Intelligent Systems, Spring 2019

Place:CSE; E222
Time:Tuesday 8,9 (3:00-4:55 p.m.), Thursday 9 (4:05-4:55 p.m.)

Instructor:
Arunava Banerjee
Office: CSE E336.
E-mail: arunava@cise.ufl.edu.
Phone: 505-1556.
Office hours: Wednesday 2:00 p.m.-4:00 p.m. or by appointment.

TA:
Anik Chattopadhyay
TA Office: CSE E309.
Office hours: XXXX p.m.

Catalog Description:

COT 5615: Mathematics for Intelligent Systems

Credits: 3 Grading Scheme: Letter

Prerequisite: MAC 2313, Multivariate Calculus; MAS 3114 or MAS 4105, Linear Algebra; STA 4321, Mathematical Statistics.

Mathematical methods commonly used to develop algorithms for computer systems that exhibit intelligent behavior.

Course Objectives:

The goal of this course is to cover several topics in mathematics that is of general interest to people pursuing a Ph.d in intelligent systems. The course will focus on conceptual clarity.

This course is an official pre-requisite for CAP6610 (Machine learning)

Required Text:

There is no official text book for this course. We will mostly work with material posted online. However, following are four good books to have.

References:
Principles of Mathematical Analysis, W. Rudin
Linear Algebra, K. M. Hoffman, R. Kunze
Probability and Measure, P. Billingsley
Probability: Theory and examples, R. Durrett, can be found online at http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.155.4899

Please return to this page at least once a week to check updates in the table below

Evaluation: The final grade will be based on three midterm exams (25% each) and several assignments (remaining 25%).

Course Policies:

Late assignments: All homework assignments are due before class.
Plagiarism: You are expected to submit your own solutions to the assignments. Feel free to discuss the concepts underlying the questions with friends and classmates.
Attendance: Their is no official attendance requirement. If you find better use of the time spent sitting thru lectures, please feel free to devote such to any occupation of your liking. However, keep in mind that it is your responsibility to stay abreast of the material presented in class.
Cell Phones: Please, no phone calls during class. Please turn off the ringer on your cell phone before coming to class.
University Honesty Policy, Campus Resources and other important information : See here for grad and here for undergrad.

Tentative List of Topics to be covered

Real analysis: Rationals and Reals, Basic point set topology, convergence, continuity etc (approximately Chap 1-4 of Rudin)
Vector spaces and Linear algebra (approximately Chap 1-3, 8 of Hoff. Kun.)
Mathematical Probability theory: Sigma algebra, Random variables etc. (approximately Chap 1. of Durett)
Information Theory: Entropy, Mutual Information, etc.

Important Announcement
Exam Schedule

Midterm I will be held on Feb 26th in class.
Midterm II will be held on Apr 23rd in class.
Each exam will last 2 hrs.
1 letter sized cheat sheat (both sides) allowed. Closed Book, closed notes.
There will be no final exam.

List of Topics covered

Week Topic Additional Reading Assignment

Jan 06 - Jan 12

Introduction
How to prove things
Proof of the Fundamental Theorem of Arithmetic
Robin's Theorem about the Riemann Hypothesis
P versus NP
Johnson Lindenstrauss lemma

Paper on the Riemann Hypothesis
Paper on the Johnson Lindenstrauss lemma

Jan 13 - Jan 19

Peano Axioms
Finished Peano's Axioms
Abstract formulation of Rational Numbers
Proof of Pythogorus Theorem
Sqrt(2) is not rational
Partial and Total order

Peano Axioms
Zermelo Frankel Set Theory

Jan 20 - Jan 26

Least upper bound (Supremum) and Greatest lower bound (infimum)
Least upper bound property.
Rationals do not have the LUB property
Theorem: LUB property IFF GLB property.
Reals as Dedekind cuts
Fields: The rational and real fields
Functions; injective, surjective and bijective
Cardinality of Integers, Rationals and Reals
Countably infinite (proof for rationals)
Reals are uncountable (Cantor's diagonalization)
Cardinality of set versus Power set (Cantor's diagonalization)

Jan 27 - Feb 02

Other applications of diagonalization: Halting Problem
Convergent sequence, Cauchy sequence
Theorem: Every convergent sequence is a Cauchy sequence
Metric spaces
epsilon neighborhoods, limit points, interior points

Feb 03 - Feb 09

Basic point set topology continued
Open sets, Closed sets
Thm: epsilon neighborhood is open
Thm: Arbitrary union, Finite intersection of open sets is open
Thm: Arbitrary intersection, Finite union of closed sets is closed
Thm: Open iff complement is closed
Complete Metric space; Seperable Metric Space
Compact sets; properties and theorems

Feb 10 - Feb 16

Definition: Topological space, trivial topology, discrete topology
Compact sets; properties and theorems continued
Continuity: Limit definition of continuous functions
Weierstrass's definition of continuous fns
Proof of equivalance
Pointwise continuity, Uniform continuity
Dirichlet function, Thomae's function, proofs.

Proof of the WAT
Note that there are typos on page 4. Should be K2 instead of K1 in the first two inequalities.

Feb 17 - Feb 23

Lagrange Polynomials
Bernstein Polynomials; properties.
The C[a,b] metric space
Weierstrass Approximation theorem

Feb 24 - Mar 02

MIDTERM I
Limit supremum Limit infimum
Set theoretic convergence

Mar 03 - Mar 09

SPRING BREAK

Mar 10 - Mar 16

Linear/Vector spaces; Linear maps; examples
Definition of Vector/Linear space on a Field.
Subspace, Span of a set of vectors

Mar 17 - Mar 23

Linear Independence, Basis
Matrix representation of a linear map
Matrix multiplication with vector and relationship to linear maps
Linear algebra of linear transforms
Change of basis as a linear transform

Mar 24 - Mar 30

Matrix product and relationship to Composition
Gaussian elimination
LU decomposition

Mar 31 - Apr 06

Normed (+Complete=Banach) vector spaces, Inner product spaces
Lp Norms
Inner product (+Complete=Hilbert) spaces
Induced norm.
Orthonormal basis; advantages, orthonormal vectors are independent.
Gram Schmidt orthogonalization
QR Decomposition

Apr 07 - Apr 13

QR Decomposition
Fourier series
Riesz–Fischer theorem (w/o proof)
Started eigenvectors/eigenvalues
Real symmetric matrices have real eigen values and orthogonal eigen vectors (Proof).

Apr 14 - Apr 20

Eigen Decomposition
Singular value Decomposition
Mathematical Probability Theory
Probability Space: Sample space, outcome, sigma-algebra of events
Random Variables
Expected value and variance
Bernoulli and Binomial RVs.

Week	Topic	Additional Reading	Assignment
Jan 06 - Jan 12	Introduction How to prove things Proof of the Fundamental Theorem of Arithmetic Robin's Theorem about the Riemann Hypothesis P versus NP Johnson Lindenstrauss lemma	Paper on the Riemann Hypothesis Paper on the Johnson Lindenstrauss lemma
Jan 13 - Jan 19	Peano Axioms Finished Peano's Axioms Abstract formulation of Rational Numbers Proof of Pythogorus Theorem Sqrt(2) is not rational Partial and Total order	Peano Axioms Zermelo Frankel Set Theory
Jan 20 - Jan 26	Least upper bound (Supremum) and Greatest lower bound (infimum) Least upper bound property. Rationals do not have the LUB property Theorem: LUB property IFF GLB property. Reals as Dedekind cuts Fields: The rational and real fields Functions; injective, surjective and bijective Cardinality of Integers, Rationals and Reals Countably infinite (proof for rationals) Reals are uncountable (Cantor's diagonalization) Cardinality of set versus Power set (Cantor's diagonalization)
Jan 27 - Feb 02	Other applications of diagonalization: Halting Problem Convergent sequence, Cauchy sequence Theorem: Every convergent sequence is a Cauchy sequence Metric spaces epsilon neighborhoods, limit points, interior points
Feb 03 - Feb 09	Basic point set topology continued Open sets, Closed sets Thm: epsilon neighborhood is open Thm: Arbitrary union, Finite intersection of open sets is open Thm: Arbitrary intersection, Finite union of closed sets is closed Thm: Open iff complement is closed Complete Metric space; Seperable Metric Space Compact sets; properties and theorems
Feb 10 - Feb 16	Definition: Topological space, trivial topology, discrete topology Compact sets; properties and theorems continued Continuity: Limit definition of continuous functions Weierstrass's definition of continuous fns Proof of equivalance Pointwise continuity, Uniform continuity Dirichlet function, Thomae's function, proofs.	Proof of the WAT Note that there are typos on page 4. Should be K2 instead of K1 in the first two inequalities.
Feb 17 - Feb 23	Lagrange Polynomials Bernstein Polynomials; properties. The C[a,b] metric space Weierstrass Approximation theorem
Feb 24 - Mar 02	MIDTERM I Limit supremum Limit infimum Set theoretic convergence
Mar 03 - Mar 09	SPRING BREAK
Mar 10 - Mar 16	Linear/Vector spaces; Linear maps; examples Definition of Vector/Linear space on a Field. Subspace, Span of a set of vectors
Mar 17 - Mar 23	Linear Independence, Basis Matrix representation of a linear map Matrix multiplication with vector and relationship to linear maps Linear algebra of linear transforms Change of basis as a linear transform
Mar 24 - Mar 30	Matrix product and relationship to Composition Gaussian elimination LU decomposition
Mar 31 - Apr 06	Normed (+Complete=Banach) vector spaces, Inner product spaces Lp Norms Inner product (+Complete=Hilbert) spaces Induced norm. Orthonormal basis; advantages, orthonormal vectors are independent. Gram Schmidt orthogonalization QR Decomposition
Apr 07 - Apr 13	QR Decomposition Fourier series Riesz–Fischer theorem (w/o proof) Started eigenvectors/eigenvalues Real symmetric matrices have real eigen values and orthogonal eigen vectors (Proof).
Apr 14 - Apr 20	Eigen Decomposition Singular value Decomposition Mathematical Probability Theory Probability Space: Sample space, outcome, sigma-algebra of events Random Variables Expected value and variance Bernoulli and Binomial RVs.