Riemann-, Generalized-Riemann-, and Darboux-Stieltjes integrals

(The following is somewhat rough and may have typos.)

Let us begin by setting the notations and recalling what happens without the Stieltjes part.

Defn (Partition)
Let be a closed interval. A partition is a finite collection of closed subintervals such that

1. is finite;
2. covers , i.e. ;
3. is pairwise almost disjoint, i.e. for distinct elements of , their intersection contains at most one point.

We write for the set of all partitions of .

Defn (Refinement)
Fix a closed interval, and two partitions. We say that refines or that if for every there exists such that .

Defn (Selection)
Given a closed interval and a partition, a selection is a mapping that satisfies .

Defn (Size)
Given a closed interval and a partition, the size of is defined as , where is the length of the closed interval .

Remark In the above we have defined two different preorders on the set of all partitions. One is induced by the size: we say that if . The other is given by the refinement . Note that neither are partial orders. (But that the preorder given by refinement can be made into a partial order if we disallow zero-length degenerate closed intervals.) Note also that if we must have .

Now we can define the notions of integrability.

Defn (Integrability)
Let be a closed, bounded interval and be a bounded function. We say that is integrable with integral in the sense of

• Riemann if for every there exists such that for every and every selection we have
  

• Generalised-Riemann if for every there exists such that for every and every selection we have
  

• Darboux if
  

From the definition it is clear that “Riemann integrable” implies “Generalised-Riemann integrable”. Furthermore, we have clearly that for a fixed

and that if we have

so “Darboux integrable” also implies “Generalised-Riemann integrable”. A little bit more work shows that “Generalised-Riemann integrable” also implies “Darboux integrable” (if the suprema and infima are obtained on the intervals , this would follow immediately; using the boundedness of the intervals we can find such that the Riemann sum approximates the upper or lower Darboux sums arbitrarily well.

The interesting part is the following
Theorem
Darboux integrable functions are Riemann integrable. Thus all three notions are equivalent.

Proof. Let be partitions. Let , and let be the number of non-degenerate subintervals in . We have the following estimate

The estimate follows by noting that “most” of the will be proper subsets of , and there can be at most of the that straddles between two different non-degenerate sub-intervals of . To prove the theorem it suffices to choose first a such that the upper and lower Darboux sums well-approximates the integral. Then we can conclude for all with sufficiently small the Riemann sum is almost controlled by the -Darboux sums. Q.E.D.

Now that we have recalled the case of the usual integrability. Let us consider the case of the Stieltjes integrals: instead of integrating against , we integrate against , where is roughly speaking a “cumulative distribution function”: we assume that is a bounded monotonically increasing function.

The definition of the integrals are largely the same, except that at every step we replace the width of the interval by the diameter of , i.e. . The arguments above immediately also imply that

• “Riemann-Stieltjes integrable” implies “Generalised-Riemann-Stieltjes integrable”
• “Darboux-Stieltjes integrable” implies “Generalised-Riemann-Stieltjes integrable”
• “Generalised-Riemann-Stieltjes integrable” implies “Darboux-Stientjes integrable”

However, Darboux-Stieltjes integrable functions need not be Riemann-Stieltjes integrable. The possibility of failure can be seen in the proof of the theorem above, where we used the fact that is allow to be made arbitrarily small. The same estimate, in the case of the Stieltjes version of the integrals, has replaced by , which for arbitrary partitions need to shrink to zero. To have a concrete illustration, we give the following:

Example
Let . Let if and otherwise. Let if and otherwise. Let be the partition . We have that

while

so we have that in particular the pair is Darboux-Stieltjes integrable with integral 0. However, let be any odd integer, consider the partition of into equal portions. Depending on the choice of the selection , we see that the sum can take the values

which shows that the Riemann-Stieltjes condition can never be satisfied.

The example above where both and are discontinuous at the same point is essentially sharp. A easy modification of the previous theorem shows that
Prop
If at least one of is continuous, then Darboux-Stieltjes integrability is equivalent to Riemann-Stieltjes integrability.

Remark The nonexistence of Riemann-Stieltjes integral when and has shared discontinuity points is similar in spirit to the idea in distribution theory where whether the product of two distributions is well-defined (as a distribution) depends on their wave-front sets.

An optimization problem: variation

Examining the theorem proven in the previous post, we are led naturally to ask whether there are higher order generalizations.

Question: Let with . What can we say about the minimizer of where ranges over degree polynomials?

It is pretty easy to see that we expect to intersect at the maximum number of points, which is . We label those points and call and . Then the cost function can be written as

Since we know that values of at the points we can write down the interpolation polynomial explicitly using Sylvester’s formula:

The partial derivatives are now

It remains to compute for . We observe that when

and also

So

Now, we observe that

so after some computation we arrive at

which we can further simplify to

Now, since and cross transversely at , the difference of their derivatives is non-zero. (This harks back to our assumption that .) So we are down, as in the case where , to equations entirely independent of .

More precisely, we see that the stationarity condition becomes the choice of such that the integrals

for each . Since form a basis for the polynomials of degree at most , we have that the function

is orthogonal to every polynomial of degree at most . So in particular the are solutions to the following system of equations

From symmetry considerations we have that . This also kills about half of the equations. For the low we have

1. 
2. 
3. 
4. 
5. 

An optimization problem: theme

Let’s start simple:

Question 1: What is the linear function that minimizes the integral ? In other words, what is the best linear approximation of in the sense?

This is something that can in principle be solved by a high schooler with some calculus training. Here’s how one solution may go:

Solution: All linear functions take the form . The integrand is equal to when , and otherwise. So we need to find the points of intersection. This requires solve , which we can solve by the quadratic formula. In the case where is signed, we see that changing we can make the integrand strictly smaller, and hence we cannot attain a minimizer. So we know that at the minimizer there must exist at least one root.

Consider the case where there is only one root in the interval (counted with multiplicity), call the root . We have that the integral to minimize is equal to

each part of which can be computed explicitly to obtain

Since we know that the two terms comes from integrands with different signs, we can combine to get

as the integrand. Now, we cannot just take the partial derivatives of the above expression with respect to and set that to zero and see what we get: the root depends also on the parameters. So what we would do then is to plug in using the expression derived from the quadratic formula, , and then take the partial derivatives. Before that, though, we can simplify a little bit: since from the constraint, the quantity to minimize is now

A long computation taking the now shows that necessarily , which implies that . But for the range of where there is only one root in the interval, the quantity does not achieve a minimum. (The formal minimizer happens at but we see for this case the integrand of the original cost function is signed.

So we are down to the case where there are two roots in the interval. Now we call the roots and , and split the integral into

and proceed as before. (The remainder of the proof is omitted; the reader is encouraged to carry out this computation out by hand to see how tedious it is.)

Products and expectation values

Let us start with an instructive example (modified from one I learned from Steven Landsburg). Let us play a game:

I show you three identical looking boxes. In the first box there are 3 red marbles and 1 blue one. In the second box there are 2 red marbles and 1 blue one. In the last box there is 1 red marble and 4 blue ones. You choose one at random. What is …

• The expected number of red marbles you will find?
• The expected number of blue marbles you will find?
• The expected number of marbles, irregardless of colour, you will find?
• The expected percentage of red marbles you will find?
• The expected percentage of blue marbles you will find?

Answer below the cut…

Continuity of the infimum

Just realised (two seeks ago, but only gotten around to finish this blog posting now) that an argument used to prove a proposition in a project I am working on is wrong. After reducing the problem to its core I found that it is something quite elementary. So today’s post would be of a different flavour from the ones of recent past.

Question Let be topological spaces. Let be a bounded, continuous function. Is the function continuous?

Intuitively, one may be tempted to say “yes”. Indeed, there are plenty of examples where the answer is in the positive. The simplest one is when we can replace the infimum with the minimum:

Example Let the space be a finite set with the discrete topology. Then is continuous.
Proof left as exercise.

But in fact, the answer to the question is “No”. Here’s a counterexample:

Example Let with the standard topology. Define

which is clearly continuous. But the infimum function is roughly the Heaviside function: if , and if .

So what is it about the first example that makes the argument work? What is the different between the minimum and the infimum? A naive guess maybe that in the finite case, we are taking a minimum, and therefore the infimum is attained. This guess is not unreasonable: there are a lot of arguments in analysis where when the infimum can be assumed to be attained, the problem becomes a lot easier (when we are then allowed to deal with a minimizer instead of a minimizing sequence). But sadly that is not (entirely) the case here: for every , we can certainly find a such that . So attaining the infimum point-wise is not enough.

What we need, here, is compactness. In fact, we have the following

Theorem If are topological spaces and is compact. Then for any continuous , the function is well-defined and continuous.

Proof usually proceeds in three parts. That follows from the fact that for any fixed , is a continuous function defined on a compact space, and hence is bounded (in fact the infimum is attained). Then using that the sets and form a subbase for the topology of , it suffices to check that and are open.

Let be the canonical projection , which we recall is continuous and open. It is easy to see that . So continuity of implies that this set is open. (Note that this part does not depend on compactness of . In fact, a minor modification of this proof shows that for any family of upper semicontinuous functions , the pointwise infimum is also upper semicontinuous, a fact that is very useful in convex analysis. And indeed, the counterexample function given above is upper semicontinuous.)

It is in this last part, showing that is open, that compactness is crucially used. Observe that . In other words an open set. This in particular implies that there exists a “box” neighborhood contained in . Now using compactness of , a finite subset of all these boxes cover . And in particular we have

and hence is open. Q.E.D.

One question we may ask is how sharp is the requirement that is compact. As with most things in topology, counterexamples abound.

Example Let be any uncountably infinite set equipped with the co-countable topology. That is, the collection of open subsets are precisely the empty set and all subsets whose complement is countable. The two interesting properties of this topology are (a) is not compact and (b) is hyperconnected. (a) is easy to see: let be some countably infinite subset of . For each let . This forms an open cover with not finite sub-cover. Hyperconnected spaces are, roughly speaking, spaces in which all open nonempty sets are “large”, in the sense that they mutually overlap a lot. In particular, a continuous map from a hyperconnected space to a Hausdorff space must be constant. In our case we can see this directly: suppose is a continuous map. Fix . Let be open neighborhoods of . Since is continuous, is open and non-empty (by the co-countable assumption). Therefore for any pairs of neighborhoods. Since is Hausdorff, this forces to be the constant map. This implies that for any topological space , a continuous function is constant along , and hence for any , we have is continuous.

One can try to introduce various regularity/separation assumptions on the spaces to see at what level compactness becomes a crucial requirement. As an analyst, however, I really only care about topological manifolds. In which case the second counterexample up top can be readily used. We can slightly weaken the assumptions and still prove the following partial converse in essentially the same way.

Theorem Let be Tychonoff, connected, and first countable, such that contains a non-trivial open subset whose closure is not the entire space; and let be paracompact, Lindelof. Then if is noncompact, there exists a continuous function such that is not continuous.

Remark Connected (nontrivial) topological manifolds automatically satisfy the conditions on and except for non-compactness. The conditions given are not necessary for the theorem to hold; but they more or less capture the topological properties used in the construction of the second counterexample above.

Remark If is such that every open set’s closure is the entire space, we must have that it is hyperconnected (let be a closed set. Suppose is another closed set such that . Then and vice versa, but is open, so . Hence cannot be written as the union of two proper closed subsets). And if it is Tychonoff, then is either the empty-set or the one-point set.

Lemma For a paracompact Lindelof space that is noncompact, there exists a countably infinite open cover and a sequence of points such that if .

Proof: By noncompactness, there exists an open cover that is infinite. By Lindelof, this open cover can be assumed to be countable, which we enumerate by and assume WLOG that . Define and inductively by: and choose .

Proof of theorem: We first construct a sequence of continuous functions on . Let be a non-empty open set such that its closure-complement is a non-empty open set ( exists by assumption). By connectedness , so we can pick in the intersection. Let be a sequence of points converging to , which exists by first countability. Using Tychonoff, we can get a sequence of continuous functions on such that and .

On , choose an open cover and points per the previous Lemma. By paracompactness we have a partition of unity subordinate to , and by the conclusion of the Lemma we have that . Now we define the function

which is continuous, and such that . But by construction , which combined with the fact that shows the desired result. q.e.d.

Conway’s Base 13 Function

(N.b. Credit where credit’s due: I learned about this function from an answer of Robin Chapman’s on MathOverflow, and its measurability from Noah Stein.)

Conway’s base 13-function is a strange beast. It was originally crafted by John Conway as a counterexample to the converse of the intermediate value theorem, and has the property that on any open interval its image contains the entire real line. In addition, its support set also serves as an illustration of a dense, uncountable set of numbers whose Lebesgue measure is 0.

Arrow’s Impossibility Theorem

Partially prompted by Terry’s buzz, I decided to take a look at Arrow’s Impossibility Theorem. The name I have heard before, since I participated in CollegeBowl as an undergraduate, and questions about Arrow’s theorem are perennial favourites. The theorem’s most famous interpretation is in voting theory:

Some definitions

1. Given a set of electors and a finite set of candidates , a preference assigns to each elector an ordering of the set . In particular, we can write for the statement “the elector prefers candidate to candidate “. The set of all possible preferences is denoted .
2. A voting system assigns to each preference an ordering of the set .
3. Given a preference and two candidates , a bloc biased toward is defined as the subset
4. The voting system is said to be
   1. unanimous if, whenever all electors prefer candidate to , the voting system will return as such. In other words, ““.
   2. independent if the voting results comparing candidates and only depend on the individual preferences between them. In particular, whether only depends on . An independent system is said to be monotonic if, in addition, a strictly larger biased bloc will give the same voting result: if and , then necessarily.
   3. dictator-free if there isn’t one elector whose vote always coincides with the end-result. In other words, we define a dictator to be an elector such that for any .
5. A voting system is said to be fair if it is unanimous, independent and monotonic, and has no dictators.

And the theorem states

Arrow’s Impossibility Theorem
In an election consisting of a finite set of electors with at least three candidates , there can be no fair voting system.

As we shall see, finiteness of the set of electors and the lower-bound on the number of candidates are crucial. In the case where there are only two candidates, the simple-majority test is a fair voting system. (Finiteness is more subtle.) It is also easy to see that if we allow dictators, i.e. force the voting results to align with the preference of a particular predetermined individual, then unanimity, independence, and monotonicity are all trivially satisfied.

What’s wrong with the simple majority test in more than three candidates? The problem is that it is not, by definition, a proper voting system: it can create loops! Imagine we have three electors and three candidates . The simple majority test says that if and only if two or more of the electors prefer to . But this causes a problem in the following scenario:

then the voting result will have , , and , a circular situation which implies that the “result” is not an ordering of the candidates! (An ordering of the set requires the comparisons to be transitive.) So the simple-majority test is, in fact, not a valid voting system.

From this first example, we see already that, in the situation of more than 2 candidates, designing a voting system is a non-trivial problem. Making them fair, as we shall see, will be impossible.

Compactness. Part 1: Degrees of freedom

Preamble
This series of posts has its genesis in a couple questions posted on MathOverflow.net. It is taking me a bit longer to write this than expected: I am not entirely sure what level of an audience I should address the post to. Another complication is that it is hard to get ideas organized when I am only writing about this during downtime where I hit a snag in research. The main idea that I want to get to is “a failure of sequential compactness of a domain represents the presence of some sort of infinity.” In this first post I will motivate the definition of compactness in mathematics.

Introduction
We start with the pigeonhole principle: “If one tries to stuff pigeons into pigeonholes, and there are more pigeons than there are pigeonholes, then at least one hole is shared by multiple pigeons.” This can be easily generalized the the case where there are infinitely many pigeons, which is the version we will use

Pigeonhole principle. If one tries to stuff infinitely many pigeons into a finite number of pigeonholes, then at least one hole is occupied by infinitely many pigeons.

Now imagine being the pigeon sorter: I hand you a pigeon, you put it into a pigeonhole. However you sort the pigeons, you are making a (infinite) sequence of assignments of pigeons to pigeonholes. Mathematically, you represent a sequence of points (choices) in a finite set (the list of all holes). In this example, we will say that a point in the set of choices is a cluster point relative to a sequence of choices if that particular point is visited infinitely many times. A mathematical way of formulating the pigeonhole principle then is:

Let be a finite set. Let be a sequence of points in . Then there exist some point such that is a cluster point of .

The key here is the word finite: if you have an infinitely long row of pigeonholes (say running to the right of you), and I keep handing you pigeons, as long as you keep stepping to the right after stuffing a pigeon into a hole, you can keep a “one pigeon per hole (or fewer)” policy.