Nonorthogonal basis functions

A function $f(x), x \in \mathbb{R}, f \in \mathbb{R}$, not necessarily periodic, can be approximated over an interval $I$ as a finite linear combination of $N$ independent functions $\sigma_i (x)$:

$f(x) \approx \sum_{i=0}^{N-1} \alpha_i \sigma_i (x)$

The condition for the optimality of this approximation is

$min_{\alpha_k} \int_I (f(x) - \sum_{i=0}^{N-1} \alpha_i \sigma_i (x))^2 dx, k \in 0 .. N-1$

equivalent to the set of equations

$\dfrac{\partial}{\partial \alpha_k} \int_I (f(x) - \sum_{i=0}^{N-1} \alpha_i \sigma_i (x))^2 dx=0, k \in 0 .. N-1$

that is

$\int_I (f(x) - \sum_{i=0}^{N-1} \alpha_i \sigma_i (x)) \sigma_k (x))dx=0, k \in 0 .. N-1$

or

$\sum_{i=0}^{N-1} \alpha_i \int_I \sigma_i (x) \sigma_k (x) dx = \int_I f(x) \sigma_k (x) dx, k \in 0 .. N-1 \label{eq:sistema}$

This is a system of $N$ linear equations in the $N$ unknowns $\alpha_i$ , and it is nonsingular as long as the basis functions $\sigma_i (x)$ are independent; if the basis functions are orthogonal the system is completely uncoupled and its solution is immediately available as

$\alpha_i = \frac{\int_I f(x) \sigma_i (x) dx}{\int_I \sigma_i ^2 (x)dx}$

If, moreover, the basis functions are orthonormal, this reduces to

$\alpha_i = \int_I f(x) \sigma_i (x) dx$

But, even if the $\sigma_i (x)$ are not orthogonal, the full system still determines a set of optimal $\alpha_i$.

An important difference is that, while the $\alpha_i$ computed for orthogonal basis functions, as a consequence of the system being uncoupled, are independent of the order $N$ of the approximation, those computed for nonorthogonal basis functions do change with $N$, even if the difference is relatively small if $N$ is large.

As an example, determine the optimal approximation of a given function as a truncated power series over a finite interval $[a,b]$:

$\sum_{i=0}^{N-1} \alpha_i \int_a^b x^{i+k}dx=\int_a^b f(x) x^k dx$

or

$\sum_{i=0}^{N-1} \alpha_i \frac{b^{i+k+1}-a^{i+k+1}}{i+k+1}=\int_a^b f(x) x^k dx$

To put numbers into the symbols, let $a=-1, b=1$ and $f(x)=\sin(\pi x)$; then

$\int_a^b \sin(\pi x) x^{2q}dx=0$, and $\int_a^b \sin(\pi x) x^{2q+1}dx$ can be computed and introduced into the system; at last, truncating to five terms I find

$\sin(\pi x) \approx x (3.14158 + x^2 (-5.16724 +x^2 (2.54601 + x^2 (-0.586621 + x^2 0.066297))))$

with an error around $10^{-5}$ inside the interval; of course the error gets rapidly worse outside.

Note that $\alpha$s have nothing to do with the Taylor coefficients; Taylor series are valid in the neighborhood of a point, not over a finite interval.