Math for Intelligent Systems, COT5615

COT 5615, Math for Intelligent Systems, Fall 2022

Place:WEIM; 1064
Time:Monday, Wednesday, Friday 8 (3:00-3:50 p.m.)

Instructor:
Arunava Banerjee
Office: CSE E336.
E-mail: arunava@ufl.edu.
Office hours (On Zoom-- 924 861 2325): Tuesday, Thursday, 11:00 a.m.-noon.

TA:
Anik Chattopadhyay
TA Office hours: (On Zoom-- 990 3599 1667): Wednesday, Thursday 1:00-2:00 p.m.

Catalog Description:

COT 5615: Mathematics for Intelligent Systems

Credits: 3 Grading Scheme: Letter

Prerequisite: MAC 2313, Multivariate Calculus; MAS 3114 or MAS 4105, Linear Algebra; STA 4321, Mathematical Statistics.

Mathematical methods commonly used to develop algorithms for computer systems that exhibit intelligent behavior.

Course Objectives:

The goal of this course is to cover several topics in mathematics that is of general interest to people pursuing a Ph.d in intelligent systems. The course will focus on conceptual clarity.

This course is an official pre-requisite for CAP6610 (Machine learning)

Required Text:

There is no official text book for this course. We will mostly work with material posted online. However, following are four good books to have.

References:
Principles of Mathematical Analysis, W. Rudin
Linear Algebra, K. M. Hoffman, R. Kunze
Probability and Measure, P. Billingsley
Probability: Theory and examples, R. Durrett, can be found online at https://services.math.duke.edu/~rtd/PTE/PTE5_011119.pdf

Please return to this page at least once a week to check updates in the table below

Evaluation: The final grade will be based on three midterm exams (25% each) and several assignments (remaining 25%).

Course Policies:

Late assignments: All homework assignments are due before class.
Plagiarism: You are expected to submit your own solutions to the assignments. Feel free to discuss the concepts underlying the questions with friends and classmates.
Attendance: Their is no official attendance requirement. If you find better use of the time spent sitting thru lectures, please feel free to devote such to any occupation of your liking. However, keep in mind that it is your responsibility to stay abreast of the material presented in class.
Cell Phones: Please, no phone calls during class. Please turn off the ringer on your cell phone before coming to class.
University Honesty Policy, Campus Resources and other important information : See here for grad and here for undergrad.

Tentative List of Topics to be covered

Real analysis: Rationals and Reals, Basic point set topology, convergence, continuity etc (approximately Chap 1-4 of Rudin)
Vector spaces and Linear algebra (approximately Chap 1-3, 8 of Hoff. Kun.)
Mathematical Probability theory: Sigma algebra, Random variables etc. (approximately Chap 1. of Durett)
Information Theory: Entropy, Mutual Information, etc.

Important Announcement
Exam Schedule

Midterm I will be held on Sep 30th in class.
Midterm II will be held on Oct 26th in class.
Midterm III will be held on Dec 7th in class.
Each exam will last 1 hr.
1 letter sized cheat sheat (both sides) allowed. Closed Book, closed notes.
There will be no final exam.

List of Topics covered

Week Topic Additional Reading Assignment

Aug 21 - Aug 27

Introduction
How to prove things
Proof of the Fundamental Theorem of Arithmetic (unique factorization)

New York Times article on AI

Aug 28 - Sep 03

Peano Axioms
Field axioms
Natural numbers, Integers
Abstract formulation of Rational Numbers
Proof of Pythogorus Theorem
Sqrt(2) is not rational
Partial and Total order
Least upper bound (Supremum) and Greatest lower bound (infimum)
Least upper bound property.
Proof: Rationals do not have the LUB property

Peano Axioms
Zermelo Frankel Set Theory

Sep 04 - Sep 10

Theorem: LUB property IFF GLB property.
Reals as Dedekind cuts
Fields: The rational and real fields
Functions; injective, surjective and bijective
Cardinality of Integers, Rationals and Reals
Countably infinite (proof for rationals)
Reals are uncountable (Cantor's diagonalization)
Cardinality of set versus Power set (Cantor's diagonalization)

More on Dedekind Cuts

Sep 11 - Sep 17

Convergent sequence, Cauchy sequence
Theorem: Every convergent sequence is a Cauchy sequence
Metric spaces
epsilon neighborhoods, limit points, interior points
Open sets, Closed sets
Thm: Open iff complement is closed
Thm: epsilon neighborhood is open
Complete Metric space; Seperable Metric Space

Sep 18 - Sep 24

Thm: Arbitrary union, Finite intersection of open sets is open
Thm: Arbitrary intersection, Finite union of closed sets is closed
Definition: Topological space.
Definition: Topological space, trivial topology, discrete topology
Compact sets; properties and theorems.
Thm: compact set is closed. Closed subset of compact set is compact.
Stmt of Heine–Borel theorem
Continuity: Limit definition of continuous functions
Weierstrass's definition of continuous fns
Proof of equivalance
Pointwise continuity, Uniform continuity

Sep 25 - Oct 01

Compactness and continuity
Proof: Continuous functions map compact sets to compact sets.
C[a,b] as a metric space
Hurrincane Ian.(no classes wednesday and friday)

Oct 02 - Oct 08

MIDTERM 1
Lagrange Polynomials and interpolation
Bernstein Polynomials; properties.
Weierstrass Approximation theorem
UF Homecoming (no classes Friday)

Proof of the WAT
Note that there are typos on page 4. Should be K2 instead of K1 in the first two inequalities.

Oct 09 - Oct 15

Finished proof of WAT
Linear/Vector spaces; examples
Definition of Vector/Linear space on a Field.
Subspace, how to prove
Span of a set of vectors (non-constructive/constructive definition)

Oct 16 - Oct 22

Linear independence, Basis
Linear maps; examples
Matrix representation of a linear map
Composition of linear map and matrix matrix multiplication.
Inverse of a linear map and matrix inverse
Ax=b, solution via Gaussian elimination.

Steinitz exchange lemma

Oct 23 - Oct 29

Change of basis as a linear transform
Worked out example of Lagrange Polynomials as a basis
MIDTERM II
LU decomposition

Oct 30 - Nov 05

Rank Nullity Theorem, proof
Normed (+Complete=Banach) vector spaces, Inner product spaces
Lp Norms
Inner product (+Complete=Hilbert) spaces
Induced norm.
Projection of a vector onto another
Pythogorean theorem

Nov 06 - Nov 12

Quadratic form
Cauchy Schwartz inequality, proof
Orthonormal basis; advantages, orthonormal vectors are independent.
Gram Schmidt orthogonalization and orthonormalization
QR Decomposition
inner product when the field is complex

Nov 13 - Nov 19

Fourier series
Mathematical Probability Theory
Probability Space: Sample space, outcome, sigma-algebra of events
Probability measure
Set theoretic convergence: lim sup and lim inf

Nov 20 - Nov 26

Thanksgiving break

Nov 27 - Dec 03

Set theoretic convergence (continued): lim sup and lim inf
Thm: If Ai converges to A from below => P(Ai) converges to P(A) from below
Random Variables. Measurable functions.
Sigma algebra "generated" by a set of subsets of Omega.
Borel sigma algebra
Distribution functions and probability density function
Thm: Indicator function is measurable; Indicator random variable.
Sinple functions
Normal density; Multivariate Normal.

Week	Topic	Additional Reading	Assignment
Aug 21 - Aug 27	Introduction How to prove things Proof of the Fundamental Theorem of Arithmetic (unique factorization)	New York Times article on AI
Aug 28 - Sep 03	Peano Axioms Field axioms Natural numbers, Integers Abstract formulation of Rational Numbers Proof of Pythogorus Theorem Sqrt(2) is not rational Partial and Total order Least upper bound (Supremum) and Greatest lower bound (infimum) Least upper bound property. Proof: Rationals do not have the LUB property	Peano Axioms Zermelo Frankel Set Theory
Sep 04 - Sep 10	Theorem: LUB property IFF GLB property. Reals as Dedekind cuts Fields: The rational and real fields Functions; injective, surjective and bijective Cardinality of Integers, Rationals and Reals Countably infinite (proof for rationals) Reals are uncountable (Cantor's diagonalization) Cardinality of set versus Power set (Cantor's diagonalization)	More on Dedekind Cuts
Sep 11 - Sep 17	Convergent sequence, Cauchy sequence Theorem: Every convergent sequence is a Cauchy sequence Metric spaces epsilon neighborhoods, limit points, interior points Open sets, Closed sets Thm: Open iff complement is closed Thm: epsilon neighborhood is open Complete Metric space; Seperable Metric Space
Sep 18 - Sep 24	Thm: Arbitrary union, Finite intersection of open sets is open Thm: Arbitrary intersection, Finite union of closed sets is closed Definition: Topological space. Definition: Topological space, trivial topology, discrete topology Compact sets; properties and theorems. Thm: compact set is closed. Closed subset of compact set is compact. Stmt of Heine–Borel theorem Continuity: Limit definition of continuous functions Weierstrass's definition of continuous fns Proof of equivalance Pointwise continuity, Uniform continuity
Sep 25 - Oct 01	Compactness and continuity Proof: Continuous functions map compact sets to compact sets. C[a,b] as a metric space Hurrincane Ian.(no classes wednesday and friday)
Oct 02 - Oct 08	MIDTERM 1 Lagrange Polynomials and interpolation Bernstein Polynomials; properties. Weierstrass Approximation theorem UF Homecoming (no classes Friday)	Proof of the WAT Note that there are typos on page 4. Should be K2 instead of K1 in the first two inequalities.
Oct 09 - Oct 15	Finished proof of WAT Linear/Vector spaces; examples Definition of Vector/Linear space on a Field. Subspace, how to prove Span of a set of vectors (non-constructive/constructive definition)
Oct 16 - Oct 22	Linear independence, Basis Linear maps; examples Matrix representation of a linear map Composition of linear map and matrix matrix multiplication. Inverse of a linear map and matrix inverse Ax=b, solution via Gaussian elimination.	Steinitz exchange lemma
Oct 23 - Oct 29	Change of basis as a linear transform Worked out example of Lagrange Polynomials as a basis MIDTERM II LU decomposition
Oct 30 - Nov 05	Rank Nullity Theorem, proof Normed (+Complete=Banach) vector spaces, Inner product spaces Lp Norms Inner product (+Complete=Hilbert) spaces Induced norm. Projection of a vector onto another Pythogorean theorem
Nov 06 - Nov 12	Quadratic form Cauchy Schwartz inequality, proof Orthonormal basis; advantages, orthonormal vectors are independent. Gram Schmidt orthogonalization and orthonormalization QR Decomposition inner product when the field is complex
Nov 13 - Nov 19	Fourier series Mathematical Probability Theory Probability Space: Sample space, outcome, sigma-algebra of events Probability measure Set theoretic convergence: lim sup and lim inf
Nov 20 - Nov 26	Thanksgiving break
Nov 27 - Dec 03	Set theoretic convergence (continued): lim sup and lim inf Thm: If Ai converges to A from below => P(Ai) converges to P(A) from below Random Variables. Measurable functions. Sigma algebra "generated" by a set of subsets of Omega. Borel sigma algebra Distribution functions and probability density function Thm: Indicator function is measurable; Indicator random variable. Sinple functions Normal density; Multivariate Normal.