Linear Algebra for Data Science

\(\usepackage{amsfonts} \usepackage{cancel} \usepackage{amsmath} \usepackage{systeme} \usepackage{amsthm} \usepackage{xcolor} \usepackage{boldsymbol} \newenvironment{am}[1]{% \left(\begin{array}{@{}*{#1}{c}|c@{}} }{% \end{array}\right) } \newcommand{\bordermatrix}[3]{\begin{matrix} ~ & \begin{matrix} #1 \end{matrix} \\ \begin{matrix} #2 \end{matrix}\hspace{-1em} & #3 \end{matrix}} \newcommand{\eref}[1]{Example~\ref{#1}} \newcommand{\fref}[1]{Figure~\ref{#1}} \newcommand{\tref}[1]{Table~\ref{#1}} \newcommand{\sref}[1]{Section~\ref{#1}} \newcommand{\cref}[1]{Chapter~\ref{#1}} \newtheorem{definition}{Definition} \newtheorem{lemma}{Lemma} \newtheorem{fact}{Fact} \newtheorem{thm}{Theorem} \newtheorem{example}{Example}[section] \newcommand{\To}{\Rightarrow} \newcommand{\del}{\nabla} \renewcommand{\Re}{\mathbb{R}} \renewcommand{\O}{\mathcal{O}} \newcommand{\N}{\mathbb{N}} \newcommand{\ben}{\begin{enumerate}} \newcommand{\een}{\end{enumerate}} \newcommand{\eps}{\epsilon} \newcommand{\cont}{\Rightarrow \Leftarrow} \newcommand{\back}{\backslash} \newcommand{\norm}[1]{\|{#1}\|} \newcommand{\abs}[1]{|{#1}|} \newcommand{\ip}[1]{\langle{#1}\rangle} \newcommand{\bo}{\mathbf} \newcommand{\mean}{\boldsymbol\mu} \newcommand{\cov}{\boldsymbol\Sigma} \newcommand{\wt}{\widetilde} \newcommand{\p}{\textbf{p}} \newcommand{\ff}{\textbf{f}} \newcommand{\aj}{\textbf{a}_j} \newcommand{\ajhat}{\widehat{\textbf{a}_j}} \newcommand{\I}{\textbf{I}} \newcommand{\A}{\textbf{A}} \newcommand{\B}{\textbf{B}} \newcommand{\bL}{\textbf{L}} \newcommand{\bP}{\textbf{P}} \newcommand{\bD}{\textbf{D}} \newcommand{\bS}{\textbf{S}} \newcommand{\bW}{\textbf{W}} \newcommand{\id}{\textbf{I}} \newcommand{\M}{\textbf{M}} \renewcommand{\B}{\textbf{B}} \newcommand{\V}{\textbf{V}} \newcommand{\U}{\textbf{U}} \newcommand{\y}{\textbf{y}} \newcommand{\bv}{\textbf{v}} \renewcommand{\v}{\textbf{v}} \newcommand{\cC}{\mathscr{C}} \newcommand{\e}{\textbf{e}} \newcommand{\w}{\textbf{w}} \newcommand{\h}{\textbf{h}} \renewcommand{\b}{\textbf{b}} \renewcommand{\a}{\textbf{a}} \renewcommand{\u}{\textbf{u}} \newcommand{\C}{\textbf{C}} \newcommand{\D}{\textbf{D}} \newcommand{\cc}{\textbf{c}} \newcommand{\Q}{\textbf{Q}} \renewcommand{\S}{\textbf{S}} \newcommand{\X}{\textbf{X}} \newcommand{\Z}{\textbf{Z}} \newcommand{\z}{\textbf{z}} \newcommand{\Y}{\textbf{Y}} \newcommand{\plane}{\textit{P}} \newcommand{\mxn}{$m\mbox{x}n$} \newcommand{\kmeans}{\textit{k}-means\,} \newcommand{\bbeta}{\boldsymbol\beta} \newcommand{\ssigma}{\boldsymbol\Sigma} \newcommand{\xrow}[1]{\mathbf{X}_{{#1}\star}} \newcommand{\xcol}[1]{\mathbf{X}_{\star{#1}}} \newcommand{\yrow}[1]{\mathbf{Y}_{{#1}\star}} \newcommand{\ycol}[1]{\mathbf{Y}_{\star{#1}}} \newcommand{\crow}[1]{\mathbf{C}_{{#1}\star}} \newcommand{\ccol}[1]{\mathbf{C}_{\star{#1}}} \newcommand{\red}[1]{\textcolor{red}{#1}} \newcommand{\blue}[1]{\textcolor{blue}{#1}} \newcommand{\arow}[1]{\mathbf{A}_{{#1}\star}} \newcommand{\acol}[1]{\mathbf{A}_{\star{#1}}} \newcommand{\brow}[1]{\mathbf{B}_{{#1}\star}} \newcommand{\bcol}[1]{\mathbf{B}_{\star{#1}}} \DeclareMathOperator*{\argmin}{arg\,min} \DeclareMathOperator*{\argmax}{arg\,max} \renewcommand{\t}{ \indent} \newcommand{\nt}{ \indent} \newcommand{\x}{\mathbf{x}} \renewcommand{\Y}{\mathbf{Y}} \newcommand{\ep}{\mathbf{\epsilon}} \renewcommand{\pm}{\left(\begin{matrix}} \renewcommand{\mp}{\end{matrix}\right)} \newcommand{\bm}{\bordermatrix} \usepackage{pdfpages,cancel} \newenvironment{code}{\Verbatim [formatcom=\color{blue}]}{\endVerbatim} \)

Preface

This course is meant to instill a working knowledge of linear algebra terminology and to lay the foundations of advanced data mining techniques like Principal Component Analysis, Factor Analysis, Collaborative Filtering, Correspondence Analysis, Network Analysis, Support Vector Machines and many more. In order to fully comprehend these important tools and techniques, we will need to understand the language in which they are presented: Linear Algebra. This is NOT a rigorous proof-based mathematics course. It is an intuitive introduction to the most important definitions and concepts that are needed to understand and effectively implement these important data mining methodologies. So that we know how to stir the pile…

Image source: https://xkcd.com/1838/

Structure of the book

This project is the fusion of a traditional textbook (with definitions, theorems, examples and exercises) with a collection of interactive programming exercises (designed for in-class demonstration) that was engineered to weave practicality and applied problem solving into the curriculum right from the start.

This is a work in progress; please check back frequently for updates.

About the author

Shaina Race Bennett earned her PhD in Operations Research from NC State in 2014 where she focused on matrix theory and clustering high-dimensional data sets. She was a teaching assistant professor at the Institute for Advanced Analytics from 2014 until 2021 when she joined Fidelity Investments as a Principal Data Scientist. She enjoys bringing linear algebra to life with animations and applications, and would love to hear from you about your experience with this text.

Acknowledgements

The author would like to acknowledge and celebrate the work of her PhD advisor Dr. Carl Meyer who wrote the most thorough and complete proof-based presentation of the material in this book. If you’re looking for more details, we strongly suggest the book that contains all of them [56]:

Meyer, Carl D. Matrix analysis and applied linear algebra. Vol. 71. Siam, 2000.

References

[56]

C. D. Meyer, “Matrix analysis and applied linear algebra,” 2nd ed., SIAM, 2001.