Comrite Learning

Intermediate
High School Middle School Professional Technology

USAAIO Mathematical Foundations for AI - Linear Algebra, Probability, and Optimization - I

benjamin benjamin
Overview
Curriculum
  • 12 Sections
  • 206 Lessons
  • 14h Duration
Collapse All
Introduction to Math foundation for USAAIO
4 Lessons
  1. Mathematical foundations for AI
  2. ⭐ Linear Algebra, Tensors, and Linear vs Nonlinear Transformations in ML
  3. Advanced topic: How can ML learn complicated nonlinear things
  4. Linear algebra Homework and reference books
Google Colab + Markdown Programming
2 Lessons
  1. Google Colab + Markdown Programming
  2. 📘 Google Colab LaTeX Math Tutorial
Linear-algebra - Vector and Vector Space
8 Lessons
  1. 📘 Vectors, Matrices, and Matrix Multiplication
  2. What is a Vector?
  3. Understanding Vector Addition in 2D
  4. Vector Space
  5. From Vectors to Linear Independence
  6. Subspaces and Direct Sums
  7. Span, Basis, Linear Independence, Dimension & Rank
  8. Example The list (1, 2, −4), (7, −5, 6) is linearly independent in 𝐅3 but is not a basis of 𝐅3
Linear-algebra - Linear Maps/Linear Transformations
26 Lessons
  1. Linear Maps (Linear Transformations) over Vector Spaces
  2. dimension of a vector space (V) ((dim(V)))
  3. Null T, Range T, and Rank–Nullity Theorem
  4. Advanced: How to proof Rank–Nullity Theorem
  5. Example of dim(kerT)
  6. Gaussian Elimination , Row Echelon Form and Finding basis of Image/Kern
  7. Injective and Surjective Linear Maps
  8. How to check matrix (M) (representing a linear map (T)) is injective, surjective, bijective
  9. Matrices from Linear Maps
  10. Definition: Rank of a Matrix
  11. Understanding Matrix and how to use it with example
  12. Transpose of a Matrix
  13. Column Rank, Row Rank, and Column–Row Factorization
  14. 🧭 Tutorial: Understanding Matrix Rank
  15. Invertible Linear Maps
  16. Find inverse : REF method ( find matrix P^-1)
  17. Advanced topic: General method of P^-1
  18. Why Linear Maps Act Like Matrix Multiplication
  19. Identity Matrix & Inverse Matrix (Sqaure Matrix)
  20. Matrix of a Composition of Linear Maps
  21. Change of Basis Formula
  22. Matrix of the Inverse of a Linear Map
  23. Isomorphisms and the Matrix Representation Map M
  24. Another easy to understand proof for L(V, W) =~ F^m,n
  25. Example how to check iosmorphism
  26. Linear map keeps structure
Linear-algebra - Eigenvalues and Eigenvectors
31 Lessons
  1. Linear Operator, Invariant space, Eigenvalue, EigenVector
  2. Example: how to get the matrix of a linear operator 𝑇
  3. Recap: What must a “matrix of 𝑇 ” do?
  4. Example: How to Compute Eigenvalues and Eigenvectors
  5. Diagonalization Theorem A = P D P^{-1}
  6. Minimal Polynomial of T
  7. How to find the minimal polynomial of a linear operator (T)
  8. Eigenvalues are the zeros of the minimal polynomial
  9. Determinant and Invertibility of Matrix
  10. Advanced : Determinant — From Axioms to Computation Formulas
  11. Finding Eigenvalues and Eigenvectors
  12. Misc: T - aI = 0 vs (T - aI)v = 0
  13. 𝑇 not invertible ⟺ constant term of minimal polynomial of 𝑇 is 0
  14. Misc: operators on odd-dimensional vector spaces have eigenvalues
  15. T has no (real) eigenvalues ⟺ T−λI is invertible ( in real vector space)
  16. Upper-Triangular Matrices and Linear Operators
  17. Misc: matrix of (T) upper triangular
  18. Equation satisfied by operator with upper-triangular matrix
  19. Determination of eigenvalues from upper-triangular matrix
  20. Necessary and sufficient condition to have an upper-triangular matrix
  21. Advanced topic: Characteristic Polynomial vs Minimal Polynomial
  22. Every Linear Operator on a Complex Vector Space has Upper-Triangular Matix
  23. Diagonal Matrices and Eigenspaces
  24. Conditions equivalent to diagonalizability
  25. Enough eigenvalues implies diagonalizability
  26. Example: how do we get the diagonal matrix of T in the eigenvector basis? 
  27. example: Using Diagonalization to Compute Powers of a Linear Operator
  28. Necessary and Sufficient Condition for Diagonalizability
  29. Advanced: why minimual polynormal of T 's distinct of zeros will determine the T can diagonal or not
  30. Advanced : Div(V), number of distinct eigenvalue, Rank(T), Minimal Poly degree
  31. Matrix, Rank, Eigenvalues, Trace, and Determinant
Linear-algebra - Inner product space and its operator ( geometric intuition)
25 Lessons
  1. Commuting Operators in Linear Algebra
  2. Inner Product, Norms, Orthogonality, and Related Theorems
  3. Dot Product, Orthogonality, Transpose, Orthogonal Complement, and Least Squares
  4. Inner Product Space
  5. Orthonormal Bases, Gram–Schmidt, and Schur’s Theorem
  6. Examples of Gram-Schmidt orthogonalization
  7. Orthogonal Projection of a Vector onto a Subspace
  8. Linear transformation preserver structure
  9. Example of a Linear Transformation for Rotate, Stretch, Shear, Flip
  10. Operators on Inner Product Spaces: adjoint operator
  11. Real Orthogonal matrix properties
  12. Self-Adjoint Operators, Positive Operators
  13. Misc: Difference between T* and T-bar
  14. Example : how to cacluate S*
  15. Misc: ⟨Tv,w⟩=⟨v,T^∗ w⟩⟺⟨T^∗ w,v⟩=⟨w,Tv⟩
  16. Null space and range of 𝑇^∗ ( e.g range(S)⊥=null(S∗) )
  17. Misc: dim(range(T∗)) =dim( range(T) )
  18. Misc: proof of (TS)^∗=S^∗T^∗
  19. Advanced: how many operator are normal?
  20. Isometries and Unitary Operators
  21. isometry, unitary operator is injective,
  22. Example: Isometry and Unitary Operator
  23. Properties of ( T^*T )
  24. “positive operator” has non-negative eigenvalues
  25. 🌟 What is the Perron–Frobenius Theorem?
Linear-algebra - Factorization and SVD
25 Lessons
  1. Fundamental Rank Factorization Theorem
  2. Spectral Theorem
  3. Unitary Matrices and QR Factorization
  4. QR Factorization
  5. Details on Proof of Uniqueness of QR
  6. Examples of Unitary Matrices and QR Factorization
  7. QR Factorization for Linearly Dependent Columns
  8. QR application: Solving (Ax=b) Using QR Factorization
  9. QR Application: Solving Least Squares Using QR Factorization
  10. QR Application:  QR Algorithm for Eigenvalues (Symmetric)
  11. Symmetric and Positive Semi-Definite Matrices
  12. Symmetric Matrices, Eigenvalues, and Eigenvectors
  13. Cholesky factorization and its proof
  14. Symmetrics matrix eignevalues will determine positive definite
  15. Singular Values of T
  16. ✅ SVD Theorem (Singular Value Decomposition)
  17. Matrix Version of the Singular Value Decomposition (SVD)
  18. Example For SVD
  19. Advanced topic: Modern SVD Algorithms How SVD Is Really Computed
  20. Advanced: Trace ( more details /properties)
  21. Advanced topic: Determinants
  22. Determinant Application: Proof: a square matrix (A) is invertible if and only if (det(A) != 0 )
  23. Determinant Application: : Mv=0 has Nonzero solution exists ⇔ singular ( T is not invertiable)
  24. Determinant Application: misc: (det(A - lambda I) = 0 ) gives eigenvalues of A
  25. Advanced topic: LU Factorization
Linear Algebra Application for AI/ML
11 Lessons
  1. 🎓 PCA (Principal Component Analysis) via Eigenvalue Decomposition
  2. ⭐ K-means Clustering (Column-Vector Linear Algebra Version + Intuition)
  3. ⭐ How to Choose (k) for K-means Clustering
  4. 📘 Affine Transformations
  5. Difference between affine transformation and linear transfomration
  6. Does a matrix only handle 2D?
  7. matrix vs. tensor
  8. Linear algebra is NOT only about matrices
  9. From Matrices to Tensors
  10. Advanced topic:tensor algebra
  11. Advanced topic: Nonlinear transformations in math
Probability & Statistics
28 Lessons
  1. Probabilities Book reference and homework
  2. 🎲 Introduction to Probability
  3. Sample Space, Random Variables, Events, and Simulation of Discrete Probability
  4. 🎓 Basic Rules of Probability
  5. Example question for probabilities
  6. What is Conditional Probability?
  7. Advanced topic: joint probability P(A, B) or P( A && B)
  8. Joint and marginal probability
  9. 📘 Bayes’ Rule
  10. 📘 Understanding Bayes’ Theorem
  11. 🎯Advanced topic: Frequentist vs Bayesian Probability / Inference
  12. Probability Distributions (PMF, PDF, and CDF)
  13. 🎲 Uniform Distribution: Discrete vs. Continuous
  14. Bernoulli Distribution
  15. Sums of Random Variables (Discrete Case)
  16. Advanced topic: How Do We Compute the Probability of Sums of Multiple Random Variables?
  17. Binomial Distribution ( Sum of Bernoulli )
  18. Example of continuous distribution: Gaussian Distribution
  19. 📘 Gaussian Distribution & Conditional Distributions
  20. Advanced topic: Distributions of Functions of Random Variables
  21. 📊 : Mean, Variance, and Expectation
  22. 📊 Advanced topic: Covariance and Correlation
  23. Markov’s Inequality
  24. Chebyshev’s Inequality
  25. Law of Large Numbers (LLN)
  26. Hoeffding’s Inequality
  27. Advanced topic: Hoeffding’s Inequality Proof
  28. Central Limit Theorem (CLT)
Derivatives in multi-variable calculus
14 Lessons
  1. Single-Variable Calculus
  2. Example: cost(x) = x^2 + 2x +1, to find the minimum cost
  3. Partial Derivatives
  4. Gradient, Jacobian, and Hessian
  5. ML example: Gradient Descent
  6. 📘 Gradients and Jacobians
  7. Directional Derivatives
  8. Chain Rule: From Single-Variable to Vector Calculus
  9. Hands-On Chain Rule Tutorial: Loss Function Example
  10. How to Deduce the Logistic Regression Loss Function — Step by Step
  11. Advanced topic: Numerical Gradient, Analytical Gradient, and Automatic Differentiation
  12. Taylor Expansion in Multiple Variables
  13. Stationary Points (Critical Points)
  14. First- and Second-Order Conditions in Calculus
Convex optimization
19 Lessons
  1. what is Convex Optimization – A Foundation for Machine Learning
  2. Convex Sets & Convex Functions
  3. Convexity and Concavity for Vector Functions
  4. Why need convext set on domain x
  5. Gradient Descent & Projected Gradient Descent
  6. Newton’s Method (Single & Multiple Variables) and How It Differs from Gradient Descent
  7. Newton’s Method: Root-Finding and Optimization
  8. Maximize or minimize a function with contrait - Lagrange Multipliers Method
  9. What is infimum
  10. Duality and Lagrange Multipliers
  11. 🎯 Advanced: Duality in Optimization - real understanding
  12. Strong Duality in Optimization — Requirements and Intuition
  13. Optimality Conditions -KKT Conditions (Inequality + Equality)
  14. Convergence Guarantees
  15. Practical Algorithms for AI Competitions
  16. 📘 Practice – Code Gradient Descent for MSE Loss
  17. Gradient Descent and Newton’s Method for Learning β
  18. implementation for fit ( using gradient descent and newton's method)
  19. USAAIO authentic problem: proof of (weakly) convex for a Loss function
Python + AI Libraries
13 Lessons
  1. 🧮 NumPy Tutorial: Arrays, Broadcasting, and Dot Product
  2. NumPy Shapes, Axes, Broadcasting, and Common Confusions
  3. 🧮 Matrix Inverse and Linear Regression
  4. Vectorized Fibonacci Computation
  5. forward function for Matrix for Wx + b
  6. how NumPy automatically batches over any leading dimensions?
  7. 📘 Use NumPy to Compute Variance, Covariance Matrix, etc.
  8. 🧼 Pandas Tutorial: Data Cleaning and Loading Datasets
  9. 🐼 Tutorial: Getting Started with pandas – Data Cleaning & Loading
  10. 🧩 pandas Merge and Join Tutorial
  11. One-Hot Encoding
  12. 🎨 Data Visualization with matplotlib and seaborn
  13. What is a Scaler in scikit-learn?

Course Overview

This course is a rigorous, math-focused course designed for students preparing for the USA Artificial Intelligence Olympiad (USAIO) and for anyone who wants a deep theoretical understanding of how modern AI algorithms work.

This course covers the official USAIO Mathematical Foundations for AI, emphasizing derivation, reasoning, and problem-solving rather than programming. Students will build strong mathematical intuition through structured lessons and extensive practice using a dedicated math homework system.

If you want to understand the algorithms behind machine learning—not just use tools or write code—this course lays the essential groundwork.

 

Core Topics

Mathematical Foundations for AI

  • Linear Algebra

    • Vectors and vector spaces

    • Linear and affine transformations

    • Eigenvalues and eigenvectors

    • Matrix factorizations (e.g., QR, SVD)

  • Probability & Statistics

    • Random variables and distributions

    • Expectation and variance

    • Bayes’ rule

    • Concentration inequalities (e.g., Hoeffding’s inequality)

  • Multivariable Calculus

    • Partial derivatives and gradients

    • Geometric interpretation of derivatives

  • Convex Optimization

    • Convex sets and functions

    • Gradient descent (conceptual and mathematical analysis)

    • Duality and optimization intuition in AI models

What You Will Learn

  • Mathematical foundations used in modern machine learning

  • How optimization algorithms are derived mathematically

  • How probability theory supports learning guarantees

  • How linear algebra structures AI models and data representations

  • How to solve competition-style theoretical problems

Course Features

  • Pure Math Focus
    No programming required. Emphasis is on reasoning, derivations, and proofs.

  • Practice-Driven Learning
    Every topic includes targeted problem sets.

  • Math Homework System
    Auto-graded assignments for immediate feedback and mastery.

  • Competition-Oriented
    Problem difficulty and style aligned with USAIO Round 1.

Ideal For

  • Middle and high school students preparing for USAIO Round 1

  • Students strong in math who want to enter AI competitions

  • Olympiad-oriented learners interested in theoretical AI foundations

  • Anyone who wants to understand AI math without coding distractions

Course Duration

  • The whole course may last 2 semester

  • each semester 12–14 weeks

  • Follows Austin RRISD schedule

  • 1 session per week

  • ~1 hour per session + homework practice

Prerequisites

  • Solid algebra background

  • Basic single-variable calculus (limits and derivatives)

Materials Included

  • Lecture notes

  • Worked examples

  • Auto-graded math homework system

  • Competition-style practice problems

Learning Outcomes

By the end of this course, students will:

  • Master the mathematical foundations of AI

  • Confidently solve USAIO Round 1 math problems

  • Be well-prepared for future AI, ML, and advanced mathematics study

  • Develop strong analytical thinking and problem-solving skills

Ready to Start?

Join USAAIO Foundations and build the mathematical depth required for success in AI competitions and advanced AI study.

 

 

 

Deleting Course Review

Are you sure? You can't restore this back

Course Access

This course is password protected. To access it please enter your password below:

Related Courses

Intermediate
USACO Track
High School Middle School

USACO Bronze → Silver Bootcamp

intense training

2 hours/week

recent USACO bronze-silver problems

32h
2
Coming Soon
Intermediate
USACO Track
High School Middle School Professional

USACO Fundamental: Data Structure & Problem Solving in C++ (Course II)

Data Structure ( tree, hash, set, queue, vector)

Tailored for USACO

Learning Policy Guarantee

Buy Now
High School Middle School Professional

USAAIO Mathematical Foundations for AI - Linear Algebra, Probability, and Optimization - II

Tailored for USAAIO

Linear Algebra

Probability & Statistics

Buy Now