Introduction to Machine Learning

Li M. Fu

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

What Is Learning?

• "Learning denotes changes in a system that enable a system to do the same task more efficiently the next time."--Herbert Simon
• "Learning is making useful changes in our minds."--Marvin Minsky

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Why Machine Learning?

• Task-oriented studies
• Theoretical analysis
• Cognitive understanding

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Elements of a Learning System

• Generalizer
• Critic
• Performer
• Problem or experiment generator

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Performance Evaluation

• Accuracy
• Efficiency
• Time and space complexity
• Quality
• Consistency

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Learning Paradigms

• Rote learning
• Instruction
• Induction
• Discovery
• Analogy
• Genetic algorithms
• Reinforcement

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Types of Learning

• Supervised Learning
• Unsupervised Learning

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

A Definition of Learning Problems

Learn <=> (E, T, P)

• E: Experience
• T: Tasks
• P: Performance measure

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Design of a Learning System

• Task
• Performance measure
• Training experience
• Target function
• Target representation