Harel Lidar

Concept · Feb 2024

SVD intuition

This is in essence what SVD is, inspecting the last equation closely, one sees that what happens is that x is projected onto each one of the eigenvectors of Σ\Sigma , and then the resulting projection is used to scaled the vector that the eigenvector is mapped to by A.

The Power Method

hence, by applying ATAA^TA to viv_i one gets σi2vi\sigma_i^2v_i. From this follows the theorem leading to the power method: “Let x be a random unit vector and define B=ATAB=A^TA then with high probability, lims→inf⁡Bsxlim_{s\rightarrow \inf}B^sx is in the span of the first singular vector v1v_1.  If we normalize BsxB^sx to a unit vector at each s then furthermore the limit is v1v_1.”