My newfound resource masteringmachinelearning.com has an article A Gentle Introduction to Linear Algebra. At the end of the article, the author challenges you to do your own linear algebra exploration. This post shares my linear algebra exploration.
Linear Algebra Quotes
Linear Algebra is one of the most important basic areas in Mathematics, having at least as great an impact as Calculus, and indeed it provides a significant part of the machinery required to generalise Calculus to vector-valued functions of many variables. Unlike many algebraic systems studied in Mathematics or applied within or outwith it, many of the problems studied in Linear Algebra are amenable to systematic and even algorithmic solutions, and this makes them implementable on computers – this explains why so much calculational use of computers involves this kind of algebra and why it is so widely used. Many geometric topics are studied making use of concepts from Linear Algebra, and the idea of a linear transformation is an algebraic version of geometric transformation. Finally, much of modern abstract algebra builds on Linear Algebra and often provides concrete examples of general ideas.
-Andrew Baker, in Basic Linear Algebra1
What that stuck out to me is Baker’s statement, “many of the problems studied in Linear Algebra are amenable to systematic and even algorithmic solutions”. This relates to why linear algebra can be used for algorithmic applications such as machine learning.
Most mathematicians define Linear Algebra as that branch of mathematics that deals with the study of vectors, vector spaces and linear equations. Modern mathematics also relies upon linear transformations and systems of vector matrix. Analytic geometry utilizes the techniques learned during a study of linear algebra, for analytically computing complex geometrical shapes. In addition to science, engineering and mathematics, linear algebra has extensive applications in the natural as well as the social sciences. Linear algebra today has been extended to consider n-dimensional space. Although it is very difficult to visualize vectors in n-space, such n-dimensional vectors are extremely useful in representing data. One can easily summarize and manipulate data efficiently in this framework, when data are ordered as a list of n components.
-Dr. Mysore Narayanan, inImportance of Linear algebra in Engineering Design Methodology2
Dr. Narayanan describes linear algebra as a tool to handle n-dimensional space. This highlights how linear algebra plays a role in Support Vector Machines, which can be used for supervised learning classification and regression problems with an infinite number of features.
You cannot learn too much linear algebra.
-Benedict Gross, Ph.D, in Abstract Algebra, Lecture 2 at 14:253
The quote speaks for itself.
Example Applications of Linear Algebra in Statistics
- Medical image processing and research4
- Computer graphics – special effects
- Computer graphics – image processing
- Sports team rankings
- Movie recommendations
- Traffic control
- Migrating populations
- Genetic coding
- Climate change
- Economical planning
Terms used in Describing Linear Algebra
- linear equation – an equation between two variables that gives a straight line when plotted on
- matrix – a rectangular array of quantities or expressions in rows and columns that is treated as a single entity and manipulated according to particular rules
- scalar – a number, numerical quantity, or element in a field
- linear map – a mapping between two modules that preserves the operations of addition and scalar multiplication
- matrix determinant – a special number that can be calculated from a square matrix
- dimension – the number of rows and columns of a matrix
- vector space – a collection of vectors, which can be added and multiplied by scalars
- Fundamentals of Linear Algebra by James B. Carrell – (2005, free textbook)
- Linear Algebra in Twenty Five Lectures by UC Davis (ebook)
- Linear Algebra on Khan Academy
- Linear Algebra by MIT Open Courseware
- Abstract Algebra Open Learning Course by Harvard
- Applications of Linear Algebra by Davidson
Read more about my Machine Learning Journey.