A matrix is said to be positive definite if it is a symmetric matrix and all its eigenvalues are positive. In other words, a real symmetric matrix A is positive definite if and only if for all non-zero vectors x, the quadratic form xTAx is positive. Mathematically, it is represented as:
xTAx > 0
Here, xT denotes the transpose of the vector x, and A is a symmetric matrix.
Positive definite matrices have some important properties. They are always invertible (i.e., have a unique inverse), all their pivots (diagonal elements in the row-echelon form) are positive, and all their eigenvalues are positive.
To better illustrate the concept, let us consider a simple 2 x 2 symmetric matrix A:
$$A =
\begin{bmatrix}
a & b \\
b & c
\end{bmatrix}$$
For A to be positive definite, the following conditions must be satisfied:
1. The determinant of the upper-left corner (a) must be positive:
a > 0
2. The determinant of the whole matrix must be positive:
$$|A| =
\begin{vmatrix}
a & b \\
b & c
\end{vmatrix}
= ac - b^2 > 0$$
Satisfying these conditions guarantees that A is a positive definite matrix. If a matrix is positive definite, it indicates that it’s associated with a convex quadratic form. In the context of optimization problems, this means that the matrix represents a strictly convex energy surface with a unique minimum.