### An optimization problem: theme

Let’s start simple:

Question 1: What is the linear function that minimizes the integral ? In other words, what is the best linear approximation of in the sense?

This is something that can in principle be solved by a high schooler with some calculus training. Here’s how one solution may go:

*Solution*: All linear functions take the form . The integrand is equal to when , and otherwise. So we need to find the points of intersection. This requires solve , which we can solve by the quadratic formula. In the case where is signed, we see that changing we can make the integrand strictly smaller, and hence we cannot attain a minimizer. So we know that at the minimizer there must exist at least one root.

Consider the case where there is only one root in the interval (counted with multiplicity), call the root . We have that the integral to minimize is equal to

each part of which can be computed explicitly to obtain

Since we know that the two terms comes from integrands with different signs, we can combine to get

as the integrand. Now, we cannot just take the partial derivatives of the above expression with respect to and set that to zero and see what we get: the root depends also on the parameters. So what we would do then is to plug in using the expression derived from the quadratic formula, , and then take the partial derivatives. Before that, though, we can simplify a little bit: since from the constraint, the quantity to minimize is now

A long computation taking the now shows that necessarily , which implies that . But for the range of where there is only one root in the interval, the quantity does not achieve a minimum. (The formal minimizer happens at but we see for this case the integrand of the original cost function is signed.

So we are down to the case where there are two roots in the interval. Now we call the roots and , and split the integral into

and proceed as before. (The remainder of the proof is omitted; the reader is encouraged to carry out this computation out by hand to see how tedious it is.) Read the rest of this entry »