Bubbles Bad; Ripples Good

… Data aequatione quotcunque fluentes quantitates involvente fluxiones invenire et vice versa …

An optimization problem: theme

Let’s start simple:

Question 1: What is the linear function \ell(x) that minimizes the integral \int_{-1}^1 |x^2  + x - \ell(x)| ~\mathrm{d}x? In other words, what is the best linear approximation of x^2 + x in the L^1([-1,1]) sense?

This is something that can in principle be solved by a high schooler with some calculus training. Here’s how one solution may go:

Solution: All linear functions take the form \ell(x) = ax + b. The integrand is equal to x^2 + x - \ell(x) when x^2 + x \geq \ell(x), and \ell(x) - x - x^2 otherwise. So we need to find the points of intersection. This requires solve x^2 + x - ax - b = 0, which we can solve by the quadratic formula. In the case where x^2 + x - a x - b is signed, we see that changing b we can make the integrand strictly smaller, and hence we cannot attain a minimizer. So we know that at the minimizer there must exist at least one root.

Consider the case where there is only one root in the interval (counted with multiplicity), call the root x_0. We have that the integral to minimize is equal to
\displaystyle \left| \int_{-1}^{x_0} x^2 + x - ax - b~\mathrm{d}x \right| + \left| \int_{x_0}^1 x^2 + x - a x - b ~\mathrm{d}x \right|
each part of which can be computed explicitly to obtain
\displaystyle \left| \frac13 x_0^3  + \frac13 + \frac{1-a}{2} x_0^2 - \frac{1-a}{2} - b x_0 - b \right| + \left| \frac13 - \frac13 x_0^3 + \frac{1-a}{2} - \frac{1-a}{2} x_0^2 - b + b x_0\right|
Since we know that the two terms comes from integrands with different signs, we can combine to get
\displaystyle \left| \frac23 x_0^3 + (1-a) x_0^2 - (1-a) - 2b x_0 \right|
as the integrand. Now, we cannot just take the partial derivatives of the above expression with respect to a,b and set that to zero and see what we get: the root x_0 depends also on the parameters. So what we would do then is to plug in x_0 using the expression derived from the quadratic formula, x_0 = \frac{1}{2} \left( a - 1 \pm \sqrt{ (1-a)^2 + 4b}\right), and then take the partial derivatives. Before that, though, we can simplify a little bit: since x_0^3 + (1-a) x_0^2 - b x_0 = 0 from the constraint, the quantity to minimize is now
\displaystyle \left| - \frac13 x_0^3 - (1-a) - b x_0 \right|
A long computation taking the \partial_b now shows that necessarily x_0 = 0, which implies that b = 0. But for the range of a where there is only one root in the interval, the quantity does not achieve a minimum. (The formal minimizer happens at a = 1 but we see for this case the integrand of the original cost function is signed.

So we are down to the case where there are two roots in the interval. Now we call the roots x_+ and x_-, and split the integral into
\displaystyle \left| \int_{-1}^{x_-} x^2 + x - ax - b~\mathrm{d}x  - \int_{x_-}^{x_+} x^2 + x - ax - b~\mathrm{d}x + \int_{x_+}^1 x^2 + x - ax - b~\mathrm{d}x \right|
and proceed as before. (The remainder of the proof is omitted; the reader is encouraged to carry out this computation out by hand to see how tedious it is.) Read the rest of this entry »

So I will be living in Michigan for the next few years…

This year I have had a little bit of sucess with my job search. At the end I sent in around 45 applications in total (though goodness for MathJobs, though a few applications were dealt with differently), from which I got 5 interviews which resulted in 2 offers, 1 rejection, and 2 “we cannot solve your two body problem so let’s not bother going through the motions”.

Starting mid August I will be affiliated with Michigan State University in East Lansing, Michigan.

All in all, a pretty stressful 6 months (job season starts around mid September, and my decision was sent in mid March), and now a slightly less stressful few months of preparing an international move from Switzerland to the US.

I’ve noticed that it is somewhat fashionable nowadays for early career individuals who have found jobs to post their application material on their websites, to serve as a sample for graduate students and new PhDs who are on the market. I think it is a pretty good idea. This was my fourth time applying for jobs. (Last year I applied also, despite there still being time left on my contract here. In the current market that is something I would recommend.) And looking back on the previous research statements I have written, the ones I wrote for my first two times applying for jobs really weren’t that great: they are both too narrowly focussed and too technical. Of course, a chunk of this has to do with my maturing as a professional mathematician. But some blame must be put on the lack of “models” to which I can compare my writing. Similarly, until this year I think my approach to the teaching statement has been on the naive and flippant side, which again partly has to do with me not knowing any better. (And this is to say nothing of the excuse of a cover letter I wrote 6 years ago.)

So below I share my research and teaching statements from this year (no, you don’t get to see the awful stuff I’ve written in the years prior). If you find it in anyway helpful, don’t hesistate to let me know.

2014 Research Statement (please note that this was written around September/October of 2014; the field of research in which I work has seen already some very interesting developments since then, so don’t treat this in anyway as an up-to-date survey!)

2014 Teaching Statement

Automated annotation in LaTeX using OCG

This is written mostly in response to Qiaochu’s Google+ post. The problem is: is there a way to make easy “cross reference previews”? If so, we can just write


\begin{theorem}
Assuming Assumptions \ref{ass1} and \ref{ass2}, we have blah
\end{theorem}

in our document, and on mouse-over we can see what the assumptions are. Or, as another example,


due to \eqref{eq3}, our equation \eqref{eq4} above implies
\[ E = mc^2\]

and mousing over the two generated equation references will show the equations, without us having to necessarily flip back to the page involved.

As it turns out, there is such a utility available: Fancy-Preview. It uses the fancytooltips package for LaTeX to generate PDF tooltips where the tooltips are clipped from an external PDF file. The upside to using fancy-preview is that the results are very pretty. The downside is that it is extremely non-portable: while I am not certain, it seems to require the PDF viewer to have javascript support, and on Linux the only PDF viewer that stands a chance of working with these tooltips is the official Adobe Acrobat Reader. If you are a Linux user you’ll know why I uttered the last phrase with disgust. (Though rumour has it that both evince and okular may eventually support the type of operations required for fancy-preview, the fact is that right now these tooltips are not very useful for Linux users.)

Instead of tooltips, however, one can get a similar effect by using the Optional Content Group feature of the PDF specification. The OCG basically allows the visibility of certain elements to be changeable by user interaction, and the ocg-p package implements easy access to the OCG layers. However, just the ocg-p package is not enough: the way it detects user interaction still doesn’t work on evince (I haven’t tested other PDF viewers). Luckily, there is the ocgx package which builds on the ocg-p package and does not use Javascript. The code has been tested to run on Acrobat Reader, FoxIt, Evince, and others. So I know at least one Linux reader is capable of using this content.

Now, OCG handling by itself only allows us to toggle the visibility of elements. We still need a way to actually present the annotations in the PDF file. Here’s where it gets tricky: the annotations should be invisible by default, and be shown with certain triggers are clicked. So the idea is to typeset the annotations as zero-size objects, offset properly so that it does not obscure the trigger button. The code I am about to show you below does this… to a certain extent. There are some bugs that I have not yet been able to iron out:

  • In the function \annotatetext, if the text starts too near line-breaking boundaries we get sometimes strange behaviours, one of which is that after every compilation pdflatex will tell us to re-run since the labels may have changed.
  • I originally intended to build also a height-detection code so that if there is not enough space above the line, we can try to type-set the annotation below the trigger. However we run into one of the main limitations of the OCG method: the order of setting the text is the order of the layers, at least with the current version of ocg-p. So the annotation text, when shown, will overwrite text that precedes it in the TeX file, but will be overwritten by text that follows. So below-the-line annotations becomes illegible. (Any suggestions on how I can fix this will be welcome!)

Note also that the code below is not “production”, it has only seen limited testing. So use it at your own peril. (I should note here that to make use of the utils, it is necessary to compile the file using some version of pdfTeX.)

%%%%% Annotation utils %%%%%
% The goal is to provide a clickable
% tool-tip like interface to show cross reference information. 
% This duplicates the function of the 'fancy-preview' and
% 'fancytooltips' packages, and does not support mouse-over events,
% but has the advantage of working in evince also. 

% We need some packages
\usepackage[usenames,dvipsnames]{color}
\usepackage{zref-savepos}
\usepackage{xifthen}
\usepackage{ocgx}
\usepackage{xspace}
% the package ocgx also depends on ocg-p, I think.

% Some configuration stuff
% Default colours, see
% http://en.wikibooks.org/wiki/LaTeX/Colors
\newcommand*\annotatetextcolour{OliveGreen} % For the inline text
\newcommand*\annotateboxbordercolour{Dandelion} % For the annotate box
\newcommand*\annotateboxbkgdcolour{Goldenrod} % Ditto
\newcommand*\annotateboxtextcolour{Black} % Ditto
\newcommand*\annotateboxtextfont{\small} % Other font configuration stuff
\newcommand*\annotateheremarktext{note} % Default text for \annotatehere 
\newcommand*\annotatewithmarkmark{$\Uparrow$} % Default mark for \annotatewithmark

\makeatletter
% Dummy variables
\newcounter{@anntposmark}
\newlength\@annt@oldfboxsep

% Usage: \annotatetext{text}{annotation}
% Note: there can be some bugs when the text to be annotated starts
% near the end of a line. The other two commands seems to behave
% better, but will still need extensive testing!
\newcommand\annotatetext[2]{%
        \stepcounter{@anntposmark}%
        \zsavepos{@annt@pos\the@anntposmark}%
        \hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
        \smash{\raisebox{3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour} #2}}}\end{ocg}}}}%
        \hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%
        \switchocg{anntlayer\the@anntposmark}{{\color{\annotatetextcolour}#1}}%
        \xspace}

% Usage: \annotatehere[note mark text]{annotations}
% The default mark text is set above, the mark text is type-set in a superscript box
\newcommand\annotatehere[2][\annotateheremarktext]{%
        \stepcounter{@anntposmark}%
        \zsavepos{@annt@pos\the@anntposmark}%
        \hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
        \smash{\raisebox{3.3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour}#2}}}\end{ocg}}}}%
        \hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%
        \setlength\@annt@oldfboxsep\fboxsep%
        \setlength\fboxsep{1pt}%
        \switchocg{anntlayer\the@anntposmark}{\raisebox{1ex}{\fcolorbox{\annotatetextcolour}{White}{\color{\annotatetextcolour}\tiny \scshape #1}}}%
        \setlength\fboxsep\@annt@oldfboxsep%
        \xspace}

% Usage: \annotatewithmark[mark]{annotations}
% Very similar to \annotatehere, but the mark is unframed, and no conversion made to small caps
\newcommand\annotatewithmark[2][\annotatewithmarkmark]{%
        \stepcounter{@anntposmark}%
        \zsavepos{@annt@pos\the@anntposmark}%
        \hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
        \smash{\raisebox{3.3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour}#2}}}\end{ocg}}}}%
        \hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%
        \setlength\@annt@oldfboxsep\fboxsep%
        \setlength\fboxsep{1pt}%
        \switchocg{anntlayer\the@anntposmark}{\raisebox{1ex}{{\color{\annotatetextcolour}\tiny #1}}}%
        \setlength\fboxsep\@annt@oldfboxsep%
        \xspace}

% Usage: \annotatelabel{label}{annotations}
% Replacement for \label, where attached to each label there is an annotation 
% text which can be recalled using \annotateref{label}. See below. 
\newcommand\annotatelabel[2]{%
        \label{#1}%
        \global\@namedef{@annt@label@#1}{#2}}
% Usage: \annotateref{label}  and  \annotateeqref{label}
% Replacement for \ref and \eqref, where we insert an \annotatewithmark with 
% the text set with \annotatelabel
\newcommand\annotateref[1]{\ref{#1}\annotatewithmark{\@nameuse{@annt@label@#1}}}
\newcommand\annotateeqref[1]{\eqref{#1}\annotatewithmark{\@nameuse{@annt@label@#1}}}

\AtBeginDocument{\zsavepos{@anntleftmargin}} %This figures out where the left margin is
\makeatother

%%%%% End Annotation utils %%%%%

Let me give an example (to compile, replace the line \input{helper_commands} by the code listing above, or save the code listing above in the file helper_commands.tex

\documentclass{article}
\usepackage[standard]{ntheorem}

\newcommand\eqref[1]{(\ref{#1})} % Defined only because I am not using amsart

\input{helper_commands}

\begin{document}
We can \annotatetext{annotate a piece of text}{Here be an
annotation.}. We can equally well annotate a piece of mathematics:
\[ E = \annotatetext{m}{The is the mass} \times \annotatetext{c^2}{Square of
the speed of light} \]
Instead of using the text itself as a toggle, we can use a
mark.\annotatewithmark{See here!} The mark itself
can\annotatehere[click me!]{Wheee!} be descriptive.

Now we can test some cross referencing. We first write down an
equation
\begin{equation}\annotatelabel{waveq}{The nonlinear wave equation \
$\Box u = B(\partial u,\partial u)$}
- \partial_t^2 u + \triangle u = \sum_{i,j = 0}^3 B^{ij} \partial_i u
  \partial_j u
\end{equation}
with respect to which we define
\begin{definition}\annotatelabel{def:nullcon}{The null condition is
when $B^{ij}\xi_i\xi_j = 0$ for any $\xi$ satisfying $m^{ij}\xi_i\xi_j
= 0$ where $m = \mathrm{diag}(-1,1,1,1)$ is the Minkowski metric.}
We say that the null condition is satisfied for \annotateeqref{waveq} if
the term $B^{ij}$ satisfies $\sum_{i,j = 0}^3 B^{ij}\xi_i
\xi_j = 0$ for every $\xi$ satisfying $- \xi_0^2 + \xi_1^2 + \xi_2^2 +
\xi_3^2 = 0$.
\end{definition}
And perhaps a theorem
\begin{theorem}\annotatelabel{thm:nullcon}{Small data global existence
holds provided null condition (Def. \ref{def:nullcon}) is
satisfied}
Small data global existence hold for \annotateeqref{waveq} provided
that the null condition is satisfied (see Definition
\annotateref{def:nullcon}.)
\end{theorem}

Let us talk a bit more about Theorem \annotateref{thm:nullcon}.
\end{document}

As you can see, one has to specify, in the current version, the annotation text for each of the labels when the labels are defined. It is a slight drawback compared to the fancy-preview script, which extracts the annotation text automatically; but on the other hand, this gives slightly better configurability: in particular, nested annotations don’t currently work, so one would have to avoid \annotateref and \annotateeqref commands within an \annotatelabel (as I did in the example above).

If you want to see what the result looks like without building the LaTeX file above yourself, here’s the results: Annotation Demo. (You should download it and open it in a standalone viewer, and not use the built-in ones for Firefox and Chrome. I am pretty sure the Firefox viewer cannot handle OCG content yet.)

Products and expectation values

Let us start with an instructive example (modified from one I learned from Steven Landsburg). Let us play a game:

I show you three identical looking boxes. In the first box there are 3 red marbles and 1 blue one. In the second box there are 2 red marbles and 1 blue one. In the last box there is 1 red marble and 4 blue ones. You choose one at random. What is …

  • The expected number of red marbles you will find?
  • The expected number of blue marbles you will find?
  • The expected number of marbles, irregardless of colour, you will find?
  • The expected percentage of red marbles you will find?
  • The expected percentage of blue marbles you will find?

Answer below the cut… Read the rest of this entry »

Decay of Waves IV: Numerical Interlude

I offer two videos. In both videos the same colour scheme is used: we have four waves in red, green, blue, and magenta. The four represent the amplitudes of spherically symmetric free waves on four different types of spatial geometries: 1 dimension flat space, 2 dimensional flat space, 3 dimensional flat space, and a 3 dimensional asymptotically flat manifold with “trapping” (has closed geodesics). Can you tell which is which? (Answer below the fold.)

Read the rest of this entry »

What’s wrong with tests

“The asymptotically hyperboloidal is not asymptotically null.”

By way of Roland Donninger, I learned today of the statement above which is apparently well-known in the numerical relativity community.

It may seem intuitively surprising: after all, the archetype of an asymptotically hyperboloidal surface is the hyperboloid as embedded in Minkowski space. Let (t,r, \omega)\in \mathbb{R}\times\mathbb{R}_+ \times \mathbb{S}^{d-1} be the spherical coordinate system for the Minkowski space \mathbb{R}^{1,d}, the hyperboloid embeds in it as the surface t^2 - r^2 = 1. If you draw a picture we see clearly that the surface is asymptotic to the null cone t = |r|

The key, however, lies in the definition. For better or for worse, the definition under which the titular statement makes sense the following:

Definition
Let (M,g) be an asymptotically simple space-time (or one for which one can define a Penrose compactification), and let (\bar{M},\Omega^2 g) be the compactified space-time. We say that a hypersurface \Sigma \subset M is asymptotically null if the \bar{\Sigma}\cap \bar{M} transversely and the tangent space of \bar{\Sigma} is null along \partial\bar{M}.

Now suppose near \partial\bar{M} we can foliate via a double-null foliation (u,v), with \partial\bar{M} = \{ u = 0\}. Let x be a coordinate on \partial\bar{M} so that (u,v,x) form a coordinate system for a neighborhood of \partial\bar{M}. Assume that our surface \Sigma can be written as a graph

v = \phi(u,x)

where \phi is a C^3 function. Then the asymptotically null condition is just that \partial_u \phi |_{u = 0} = 0. Taking a Taylor expansion we have that this means

v \approx \phi_{\infty}(x) + \phi^{(2)}_{\infty}(x) u^2.

For the usual conformal compactification of Minkowski space, we have u = \frac{\pi}{2} - \cot^{-1}\left( \frac{1}{r+t}\right). Hence we require that an asymptotically null surface to have convergence to the null surface at rate O(1/(r+t)^2) (if \phi is sufficiently differentiable; if we relax the differentiability at infinity we see that the above condition allows us to relax all the way to O(1/(r+t)^{1+}), but O(1/(r+t)) is not admissible).

On the other hand, the hyperboloid is given by (r+t)(r-t) = -1 \implies r-t = v = O(1/(r+t)) and so is not asymptotically null. And indeed, we can also check by direct computation that in the usual conformal compactification of Minkowski space, the limit of the hyperboloid at null infinity is space-like.

What is mathematical literacy?

There is a wonderful mathematics joke, usually told about Von Neumann, but sometimes with other mathematicians swapped in, it features the following math problem

A train is traveling west at 70 miles per hour, and a train is traveling east at 80 miles an hour. They started 300 miles apart. A fly decided to challenge itself and, at 100 miles per hour, flew from the west-bound train train to the other, and then back, and then forward, and then back, until the two trains passed each other. (An exercise path that is somewhat akin to what we call “suicides” in high school swim practice.) Question: what is the total distance traveled by the fly?

There are two ways to do this problem.

  1. The brute-force way. The fly and the east-bound train are moving toward each other at 180mph. Starting at 300 miles apart they would meet after 300/180 = 100/60 hours = 100 minutes, during which the fly would have traveled 500/3 miles, and the west bound train 350/3 miles. After the fly turned around, it will be 50 miles (= 500/3 – 350/3) from the west-bound train and they head toward each other at 170 miles per hour, so they would meet after 5/17 hours, during which the fly traveled 500/17 miles and the east-bound train 400/17. Now 100/17 miles apart the fly and the east-bound train head toward each other at 180mph and…

    This eventually leads to an infinite series which one can potentially sum geometrically and arrive at a final answer. Or,

  2. The “smart” way. The two trains move toward each other at 150 miles per hour. It takes 2 hours until they meet. The fly, traveling at 100mph, much have traveled a total of 200 miles during the trip.

The “joke” is usually told with some joker, say dear old Feynman, posing this mathematical problem to some more serious-minded mathematician, often renown for his computational abilities (say Von Neumann). The mathematician would reply instantly 200 miles. The joker would be disappointed and complain that the mathematician had heard this question before and somehow knew “the trick”, upon which the mathematician replies: “What trick? It wasn’t that hard to sum the series.”

I just came back from teaching an exercise course (in Princeton language, a precept), and the official answer to one of the exercises reminded me of the above problem. Here is the question:

You are given a biased coin which lands on tails with probability p. You flip the coin until you get heads, and record the number of tosses. (a) What is the probability that the number of tosses is exactly n? (b) What is the probability that the number of tosses is greater than or equal to n?

Part (a) is standard: for n total tosses, the nth toss is heads and the rest are tails. So the probability is p^{n-1}(1-p), representing n-1 successive tails followed by a single heads.

The official answer to part (b) is this: The probability of number of tosses is at least n is the sum

\displaystyle \sum_{k = n}^\infty P( X = k ) = \sum_{k = n}^{\infty} p^{k-1}(1-p)

which after summing the geometric series we arrive at p^{n-1}.

Here’s what I thought after seeing the question: for at least n total tosses, the first n-1 must be all tails. After which the tosses don’t matter: either one eventually ends up with heads at some finite time, in which case it is counted as some finite k = N event, or one never hits heads, in which cases the probability is vanishingly small. So the probability is exactly the probability of getting exactly n-1 tails in a row, which is p^{n-1}.

Just like the fly problem, one of the solutions brute forces the answer, the other try to “reason” away the computational bits until one is left with a simple computation which can be done quickly in one’s head.

Which of these is mathematical literacy? Read the rest of this entry »

IPython Notebook, Take 2

With the help of Fernando Perez, some aspects of my IPython Notebook code has been improved.

Changelog:

  1. Instead of using .show() and .fullshow() methods to display the content of the objects with dependence on a global toggle for MathJax, now we use the _repr_latex_ method (which hooks into IPython’s display functionality). This allows one to enter just m = math_expr([insert atoms here]) followed by m to display the content of the math expression.
  2. Whether the default display mentioned above shows the atom number labels is now configured via a boolean: one can play with it using the .labeled property
  3. To show the LaTeX code, we now use the .latex property
  4. Operations are no longer in place. Instead they return a new math_expr object. This has the advantage that if you can keep playing with the entry until you get it right, and then afterwords you can do m = m.replace(...). This makes it better for for interactive computations.
  5. By default, on object creation it will print its own content.

The code now looks like

from IPython.core.display import *

def add(x,y): return x+y
def MDPL(string): display(Math(string))
def comp_str(listofstrings): return reduce(add,listofstrings)

class math_expr(object):
   '''''Math Expression object'''''

   def __init__(self,atomslist, labeldefault = True, highlight = []):
      '''''init takes arg: list of atoms, each atom being a compilable chunck of LaTeX expression'''''
      self.listofatoms = atomslist
      self.labels = labeldefault
      MDPL(comp_str(self.__colouratoms(highlight)))

   def _repr_latex_(self):
      '''''Returns a latex expression of the object. Will be parsed by MathJax'''''
      self.__labelatoms()
      latexstring = comp_str(self.labeledatoms) if self.labels else comp_str(self.listofatoms)
      return "$" + latexstring + "$"

   @property
   def labeled(self):
      '''''Tells you whether labelling is turned on by default'''''
      return self.labels

   @labeled.setter
   def label(self,boolean):
      '''''Sets the default labelling'''''
      self.labels = boolean

   @property
   def latex(self):
      '''''Accesses the LaTeX code snip for the expression'''''
      display(Latex(comp_str(self.listofatoms)))

   def __labelatoms(self):
      '''''Label atoms by adding underbraces'''''
      self.labeledatoms = [ "\underbrace{" + self.listofatoms[i] + "}_{" + str(i) + "}" for i in range(len(self.listofatoms)) ]

   def replace(self,pos,newstr):
      '''''Replaces an atom with another atom'''''
      MDPL(comp_str(self.__colouratoms([pos])))
      newstrings = list(self.listofatoms)
      newstrings[pos] = newstr
      return math_expr(newstrings,self.labels,[pos])

   def merge(self,positions):
      '''''Merges atoms: the input is a list of positions. The new atom is placed at the position of the foremost of the positions'''''
      MDPL(comp_str(self.__colouratoms(positions)))
      newstrings = list(self.listofatoms)
      temp = [ newstrings[i] for i in positions ]
      positions.sort()
      positions.reverse()
      for i in positions: del newstrings[i]
      newstrings.insert(positions[-1],comp_str(temp))
      return math_expr(newstrings,self.labels,[positions[-1]])

   def split(self,pos,newatoms):
      '''''Splits atoms: replaces an atom in place with multiple sub atoms'''''
      MDPL(comp_str(self.__colouratoms([pos])))
      newstrings = list(self.listofatoms)
      del newstrings[pos]
      templen = len(newatoms)
      while len(newatoms) > 0:
         newstrings.insert(pos,newatoms.pop())
      return math_expr(newstrings,self.labels,range(pos,pos+templen))

   def cancel(self,positions):
      '''''Cancels a bunch of terms: input a list of positions'''''
      MDPL(comp_str(self.__colouratoms(positions)))
      positions.sort()
      positions.reverse()
      newstrings = list(self.listofatoms)
      for i in positions: del newstrings[i]
      return math_expr(newstrings,self.labels)

   def move(self,posini,posfin):
      '''''Move atom at posini to posfin, pushing all others back'''''
      MDPL(comp_str(self.__colouratoms([posini])))
      newstrings = list(self.listofatoms)
      temp = newstrings.pop(posini)
      newstrings.insert(posfin if posfin < posini else posfin-1, temp)
      return math_expr(newstrings,self.labels,[posfin if posfin < posini else posfin - 1])

   def __colouratoms(self,positions,labelled=False):
      '''''Returns the list of atoms, but with selected terms coloured'''''
      temp = list(self.listofatoms)
      if labelled:
         self.labelatoms()
         temp = list(self.labeledatoms)
      for i in positions: temp[i] = "\color{red}{"+temp[i]+"}"
      return temp

Using IPython Notebook for manual computations

(Edit July 27: please see this update)

One of the hardest organizational tasks I’ve found in the past few years is keeping track of computations. When I started as a graduate student I began by doing computations by hand on scratch paper and copying everything (that is worth keeping) into a spiral-bound notebook. Notebooks being hard to carry, a little bit later I moved on to scanning my notes. Than I decided that my handwriting is quite horrible and I should just TeX everything up instead. Occasionally this has led to me doing my computations directly in LaTeX inside a Vim buffer. Yesterday I discovered a better alternative.

A few of the problems I’ve run into with the various methods of keeping computations include:

  1. In deadtree format, the sheets of paper are easily misplaced or soiled, and are not as easily modifiable.
  2. Doing a hand computation and then typing up the result takes extra time.
  3. Computing directly in LaTeX requires a lot of redundant typing or copying and pasting of parts of the expressions that don’t change from line to line.
  4. Typed notes (and sometimes written notes) don’t easily show (unless you take the extra time to format it) the changes between the lines, and sometimes a few months later the computation becomes hard to follow.

The present solution below has

  1. Storage in electronic format for easy backup and search.
  2. Doing the calculation directly on a computer to avoid the two-step process.
  3. Let the computer do the work of copying unchanged expressions for you.
  4. Let the computer do the formatting; the computation is written in an almost-human-readable code that is self-documentary.

Software needed
What I discovered, is the IPython Notebook (link to project homepage). The “notebook” part of the name, I believe, refers to its semblance to Mathematica/MatLab/Maxima style notebooks where one sequentially and interactively enter commands and observe their output. Unlike the mentioned software, this notebook is for interactively using the Python programming language. Furthermore, it supports inline documentation using MarkDown and MathJax! (This can be had by changing the cell type to Markdown, instead of the default Python code.)

If you want to see for yourself how this works, you will need to install IPython Notebook yourself. (In Gentoo linux, emerge ipython with the notebook use flag set; in Debian or Ubuntu, you can just apt-get the relevant packages.)

Doing computations
With a little bit of (very messy and unprofessional) Python code, the Notebook can be convinced to print certain outputs prettified by MathJax. Here is the set-up I used:

from IPython.core.display import *

global MathJax 
MathJax = True
def MDPL(string): display(Math(string)) if MathJax else display(Latex(string))

def add(x,y): return x+y

def comp_str(listofstrings): return reduce(add,listofstrings)

class math_expr(object):
   '''''Math Expression object'''''

   def __init__(self,arg1):
      '''''init takes arg: list of atoms, each atom being a compilable chunck of LaTeX expression'''''
      self.listofatoms = arg1

   def show(self):
      '''''Displays the content of the expression in mathmode'''''
      MDPL(comp_str(self.listofatoms))

   def replace(self,pos,newstr):
      '''''Replaces an atom with another atom'''''
      MDPL(comp_str(self.colouratoms([pos])))
      self.listofatoms[pos] = newstr
      MDPL(comp_str(self.colouratoms([pos],True)))

   def merge(self,positions):
      '''''Merges atoms: the input is a list of positions. The new atom is placed at the position of the foremost of the positions'''''
      MDPL(comp_str(self.colouratoms(positions)))
      temp = [ self.listofatoms[i] for i in positions ]
      positions.sort()
      positions.reverse()
      for i in positions: del self.listofatoms[i]
      self.listofatoms.insert(positions[-1],comp_str(temp))
      MDPL(comp_str(self.colouratoms([positions[-1]],True)))

   def split(self,pos,newatoms):
      '''''Splits atoms: replaces an atom in place with multiple sub atoms'''''
      MDPL(comp_str(self.colouratoms([pos])))
      del self.listofatoms[pos]
      templen = len(newatoms)
      while len(newatoms) > 0:
         self.listofatoms.insert(pos,newatoms.pop())
      MDPL(comp_str(self.colouratoms(range(pos, pos+templen),True)))

   def cancel(self,positions):
      '''''Cancels a bunch of terms: input a list of positions'''''
      MDPL(comp_str(self.colouratoms(positions)))
      positions.sort()
      positions.reverse()
      for i in positions: del self.listofatoms[i]
      self.fullshow()

   def move(self,posini,posfin):
      '''''Move atom at posini to posfin, pushing all others back'''''
      MDPL(comp_str(self.colouratoms([posini])))
      temp = self.listofatoms.pop(posini)
      self.listofatoms.insert(posfin if posfin < posini else posfin-1, temp)
      MDPL(comp_str(self.colouratoms([posfin if posfin < posini else posfin-1],True)))

   def colouratoms(self,positions,labelled=False):
      '''''Returns the list of atoms, but with selected terms coloured'''''
      temp = list(self.listofatoms)
      if labelled:
         self.labelatoms()
         temp = list(self.labeledatoms)
      for i in positions: temp[i] = "\color{red}{"+temp[i]+"}"
      return temp

   def labelatoms(self):
      '''''Label atoms by adding underbraces'''''
      self.labeledatoms = [ "\underbrace{" + self.listofatoms[i] + "}_{" + str(i) + "}" for i in range(len(self.listofatoms)) ]

   def fullshow(self):
      '''''Shows the content whilst labeling positions'''''
      self.labelatoms()
      MDPL(comp_str(self.labeledatoms))

The entire python code can be copied and saved somewhere on your computer. After launching the Notebook, drag the file and drop it into the first input box. Then hit Shift-Enter to load the code. After that you can do things like:

Loaded preamble, do some computations

After loading the preamble in cell 1, we insert a math expression and display it.

and

Next we do some computations (manipulations of the expression).

After I am done, I just click “Save” and the entire computation is saved, and bits and pieces of it can be easily copied out by switching to non-mathJax output.

The code is still in the early stages: a lot more improvements can be made. But it is already a working system.