Bubbles Bad; Ripples Good

… Data aequatione quotcunque fluentes quantitates involvente fluxiones invenire et vice versa …

Category: Life of a mathematician

TeXLive — on Android

I’ve just done something that is, admittedly, rather silly. And I am somewhat surprised that it actually worked.

I managed to have TeXLive running on my new Samsung Galaxy Tab A, which runs Android.

It is painfully slow (speed is comparable to my old netbook from 2010). But in a pinch, I now know that it works. And that gives me some (minor) peace of mind.

Read the rest of this entry »

Workflow

Recently I started rethinking how I organize my incomplete and under development notes.

I know full well the inherent dangers of such an exercise in terms of my actual productivity. But now that I have completed my newest workflow I think I’ve finally found one that works well. (Fingers crossed.)

Before I describe what I do now, I’d like to document what I used to do, what I changed the last time, and why I am changing it again.

Read the rest of this entry »

In defense of integration by parts

A prominent academic, who happens not to be a mathematician, visited my home institution recently and gave a public address about the role of the university in the modern world. Most of what he said concerning our teaching mission are the usual platitudes about not being stuck in the past and making sure that our curricular content and learning objectives are aligned with what we would expect a 21st century college graduates to need.

It however bugged me to no end that the recurring example this particular individual returns to for something old-fashioned and “ought not be taught” is integration by parts; and he justifies this by mentioning that computer algebra systems (or even just google) can do the integrals faster and better than we humans can.

I don’t generally mind others cracking jokes at mathematicians’ expense. But this particular self-serving strawman uttered by so well-regarded an individual is, to those of us actually in the field teaching calculus to freshmen and sophomores, very damaging and disingenuous.

I happened to have just spent the entirety of last year rethinking how we can best teach calculus to the modern engineering majors. Believe me, students nowadays know perfectly well when we are just asking them to do busywork; they also know perfectly well that computer algebra systems are generally better at finding closed-form integral expressions than we can. Part of the challenge of the redesign that I am involved in is precisely to convince the students that calculus is worth learning in spite of computers. The difficulty is not in dearth of reason; on the contrary, there are many good reasons why a solid grounding of calculus is important to a modern engineering students. To give a few examples:

  1. Taylor series are in fact important because of computers, since they provide a method of compactly encoding an entire function.
  2. Newton’s method for root finding (and its application to, say, numerical optimization) is build on a solid understanding of differential calculus.
  3. The entirety of the finite element method of numerical simulation, which underlies a lot of civil and mechanical engineering applications, are based on a variational formulation of differential equations that, guess what, only make sense when one understand integration by parts.
  4. The notion of Fourier transform which is behind a lot of signal/image processing requires understanding how trigonometric functions behave under integration.

No, the difficulty for me and my collaborator is narrowing down a list of examples that we can not only reasonably explain to undergraduate students, but also have them have some hands-on experience working with.

When my collaborator and I were first plunged into this adventure of designing engineering-specific calculus material, one of the very first things that we did was to seek out inputs from our engineering colleagues. My original impulse was to cut some curricular content in order to give the students a chance to develop deeper understanding of fewer topics. To that end I selected some number of topics which I thought are old-fashioned, out-dated, and no longer used in this day and age. How wrong I was! Even something like “integration by partial fractions” which most practicing mathematicians will defer to a computer to do has its advocates (those who have to teach control theory insists that a lot of fundamental examples in their field can be reduced to evaluating integrals of rational functions, and a good grasp of how such integrals behave is key to developing a general sense of how control theory works).

In short, unlike some individuals will have you believe, math education is not obsolete because we all have calculators. In fact, I would argue the opposite: math education is especially pertinent now that we all have calculators. Long gone was the age where a superficial understanding of mathematics in terms of its rote computations is a valuable skill. A successful scientist or engineer needs to be able to effectively leverage the large toolbox that is available to her, and this requires a much deeper understanding of mathematics, one that goes beyond just the how but also the what and the why.

There are indeed much that can be done to better math education for the modern student. But one thing that shouldn’t be done is getting rid of integration by parts.

Abusing JabRef to manage snipplets of TeX

I use JabRef as my reference manager. In this post, however, I will discuss how we can abuse it to do some other things.

The problem

Let’s start with a concrete example: I keep a “lab notebook”. It is where I document all my miscellaneous thoughts and computations that come up during my research. Some of those are immediately useful and are collected into papers for publication. Some of those are not, and I prefer to keep them for future reference. These computations range over many different subjects. Now and then, I want to share with a collaborator or a student some subset of these notes. So I want a way to quickly search (by keywords/abstract) for relevant notes, and that compile them into one large LaTeX document.

Another concrete example: I am starting to collect a bunch of examples and exercises in analysis for use in my various classes. Again, I want to have them organized for easy search and retrieval, especially to make into exercise sheets.

The JabRef solution

The “correct” way to do this is probably with a database (or a document store), with each document tagged with a list of keywords. But that requires a bit more programming than I want to worry about at the moment.

JabRef, as it turns out, is sort of a metadata database: by defining a customized entry type you can use the BibTeX syntax as a proxy for JSON-style data. So for my lab notebook example, I define a custom type lnbentry in JabRef with

  • Required fields: year, month, day, title, file
  • Optional fields: keywords, abstract

I store each lab notebook entry as an individual TeX file, whose file system address is stored in the file field. The remaining metadata fields’ contents are self-evident.

(Technical note: in my case I actually store the metadata in the TeX file and have a script to parse the TeX files and update the bib database accordingly.)

For generating output, we can use JabRef’s convenient export filter support. In the simplest case we can create a custom export layout with the main layout file containing the single line

\\input{\file}

with appropriate begin and end incantations to make the output a fully-formed TeX file. Then one can simply select the entries to be exported, click on “Export”, and generate the appropriate TeX file on the fly.

(Technical note: JabRef can also be run without a GUI. So one can use this to perform searches through the database on the command line.)

So I will be living in Michigan for the next few years…

This year I have had a little bit of sucess with my job search. At the end I sent in around 45 applications in total (though goodness for MathJobs, though a few applications were dealt with differently), from which I got 5 interviews which resulted in 2 offers, 1 rejection, and 2 “we cannot solve your two body problem so let’s not bother going through the motions”.

Starting mid August I will be affiliated with Michigan State University in East Lansing, Michigan.

All in all, a pretty stressful 6 months (job season starts around mid September, and my decision was sent in mid March), and now a slightly less stressful few months of preparing an international move from Switzerland to the US.

I’ve noticed that it is somewhat fashionable nowadays for early career individuals who have found jobs to post their application material on their websites, to serve as a sample for graduate students and new PhDs who are on the market. I think it is a pretty good idea. This was my fourth time applying for jobs. (Last year I applied also, despite there still being time left on my contract here. In the current market that is something I would recommend.) And looking back on the previous research statements I have written, the ones I wrote for my first two times applying for jobs really weren’t that great: they are both too narrowly focussed and too technical. Of course, a chunk of this has to do with my maturing as a professional mathematician. But some blame must be put on the lack of “models” to which I can compare my writing. Similarly, until this year I think my approach to the teaching statement has been on the naive and flippant side, which again partly has to do with me not knowing any better. (And this is to say nothing of the excuse of a cover letter I wrote 6 years ago.)

So below I share my research and teaching statements from this year (no, you don’t get to see the awful stuff I’ve written in the years prior). If you find it in anyway helpful, don’t hesistate to let me know.

2014 Research Statement (please note that this was written around September/October of 2014; the field of research in which I work has seen already some very interesting developments since then, so don’t treat this in anyway as an up-to-date survey!)

2014 Teaching Statement

Automated annotation in LaTeX using OCG

This is written mostly in response to Qiaochu’s Google+ post. The problem is: is there a way to make easy “cross reference previews”? If so, we can just write


\begin{theorem}
Assuming Assumptions \ref{ass1} and \ref{ass2}, we have blah
\end{theorem}

in our document, and on mouse-over we can see what the assumptions are. Or, as another example,


due to \eqref{eq3}, our equation \eqref{eq4} above implies
\[ E = mc^2\]

and mousing over the two generated equation references will show the equations, without us having to necessarily flip back to the page involved.

As it turns out, there is such a utility available: Fancy-Preview. It uses the fancytooltips package for LaTeX to generate PDF tooltips where the tooltips are clipped from an external PDF file. The upside to using fancy-preview is that the results are very pretty. The downside is that it is extremely non-portable: while I am not certain, it seems to require the PDF viewer to have javascript support, and on Linux the only PDF viewer that stands a chance of working with these tooltips is the official Adobe Acrobat Reader. If you are a Linux user you’ll know why I uttered the last phrase with disgust. (Though rumour has it that both evince and okular may eventually support the type of operations required for fancy-preview, the fact is that right now these tooltips are not very useful for Linux users.)

Instead of tooltips, however, one can get a similar effect by using the Optional Content Group feature of the PDF specification. The OCG basically allows the visibility of certain elements to be changeable by user interaction, and the ocg-p package implements easy access to the OCG layers. However, just the ocg-p package is not enough: the way it detects user interaction still doesn’t work on evince (I haven’t tested other PDF viewers). Luckily, there is the ocgx package which builds on the ocg-p package and does not use Javascript. The code has been tested to run on Acrobat Reader, FoxIt, Evince, and others. So I know at least one Linux reader is capable of using this content.

Now, OCG handling by itself only allows us to toggle the visibility of elements. We still need a way to actually present the annotations in the PDF file. Here’s where it gets tricky: the annotations should be invisible by default, and be shown with certain triggers are clicked. So the idea is to typeset the annotations as zero-size objects, offset properly so that it does not obscure the trigger button. The code I am about to show you below does this… to a certain extent. There are some bugs that I have not yet been able to iron out:

  • In the function \annotatetext, if the text starts too near line-breaking boundaries we get sometimes strange behaviours, one of which is that after every compilation pdflatex will tell us to re-run since the labels may have changed.
  • I originally intended to build also a height-detection code so that if there is not enough space above the line, we can try to type-set the annotation below the trigger. However we run into one of the main limitations of the OCG method: the order of setting the text is the order of the layers, at least with the current version of ocg-p. So the annotation text, when shown, will overwrite text that precedes it in the TeX file, but will be overwritten by text that follows. So below-the-line annotations becomes illegible. (Any suggestions on how I can fix this will be welcome!)

Note also that the code below is not “production”, it has only seen limited testing. So use it at your own peril. (I should note here that to make use of the utils, it is necessary to compile the file using some version of pdfTeX.)

%%%%% Annotation utils %%%%%
% The goal is to provide a clickable
% tool-tip like interface to show cross reference information. 
% This duplicates the function of the 'fancy-preview' and
% 'fancytooltips' packages, and does not support mouse-over events,
% but has the advantage of working in evince also. 

% We need some packages
\usepackage[usenames,dvipsnames]{color}
\usepackage{zref-savepos}
\usepackage{xifthen}
\usepackage{ocgx}
\usepackage{xspace}
% the package ocgx also depends on ocg-p, I think.

% Some configuration stuff
% Default colours, see
% http://en.wikibooks.org/wiki/LaTeX/Colors
\newcommand*\annotatetextcolour{OliveGreen} % For the inline text
\newcommand*\annotateboxbordercolour{Dandelion} % For the annotate box
\newcommand*\annotateboxbkgdcolour{Goldenrod} % Ditto
\newcommand*\annotateboxtextcolour{Black} % Ditto
\newcommand*\annotateboxtextfont{\small} % Other font configuration stuff
\newcommand*\annotateheremarktext{note} % Default text for \annotatehere 
\newcommand*\annotatewithmarkmark{$\Uparrow$} % Default mark for \annotatewithmark

\makeatletter
% Dummy variables
\newcounter{@anntposmark}
\newlength\@annt@oldfboxsep

% Usage: \annotatetext{text}{annotation}
% Note: there can be some bugs when the text to be annotated starts
% near the end of a line. The other two commands seems to behave
% better, but will still need extensive testing!
\newcommand\annotatetext[2]{%
        \stepcounter{@anntposmark}%
        \zsavepos{@annt@pos\the@anntposmark}%
        \hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
        \smash{\raisebox{3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour} #2}}}\end{ocg}}}}%
        \hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%
        \switchocg{anntlayer\the@anntposmark}{{\color{\annotatetextcolour}#1}}%
        \xspace}

% Usage: \annotatehere[note mark text]{annotations}
% The default mark text is set above, the mark text is type-set in a superscript box
\newcommand\annotatehere[2][\annotateheremarktext]{%
        \stepcounter{@anntposmark}%
        \zsavepos{@annt@pos\the@anntposmark}%
        \hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
        \smash{\raisebox{3.3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour}#2}}}\end{ocg}}}}%
        \hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%
        \setlength\@annt@oldfboxsep\fboxsep%
        \setlength\fboxsep{1pt}%
        \switchocg{anntlayer\the@anntposmark}{\raisebox{1ex}{\fcolorbox{\annotatetextcolour}{White}{\color{\annotatetextcolour}\tiny \scshape #1}}}%
        \setlength\fboxsep\@annt@oldfboxsep%
        \xspace}

% Usage: \annotatewithmark[mark]{annotations}
% Very similar to \annotatehere, but the mark is unframed, and no conversion made to small caps
\newcommand\annotatewithmark[2][\annotatewithmarkmark]{%
        \stepcounter{@anntposmark}%
        \zsavepos{@annt@pos\the@anntposmark}%
        \hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
        \smash{\raisebox{3.3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour}#2}}}\end{ocg}}}}%
        \hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%
        \setlength\@annt@oldfboxsep\fboxsep%
        \setlength\fboxsep{1pt}%
        \switchocg{anntlayer\the@anntposmark}{\raisebox{1ex}{{\color{\annotatetextcolour}\tiny #1}}}%
        \setlength\fboxsep\@annt@oldfboxsep%
        \xspace}

% Usage: \annotatelabel{label}{annotations}
% Replacement for \label, where attached to each label there is an annotation 
% text which can be recalled using \annotateref{label}. See below. 
\newcommand\annotatelabel[2]{%
        \label{#1}%
        \global\@namedef{@annt@label@#1}{#2}}
% Usage: \annotateref{label}  and  \annotateeqref{label}
% Replacement for \ref and \eqref, where we insert an \annotatewithmark with 
% the text set with \annotatelabel
\newcommand\annotateref[1]{\ref{#1}\annotatewithmark{\@nameuse{@annt@label@#1}}}
\newcommand\annotateeqref[1]{\eqref{#1}\annotatewithmark{\@nameuse{@annt@label@#1}}}

\AtBeginDocument{\zsavepos{@anntleftmargin}} %This figures out where the left margin is
\makeatother

%%%%% End Annotation utils %%%%%

Let me give an example (to compile, replace the line \input{helper_commands} by the code listing above, or save the code listing above in the file helper_commands.tex

\documentclass{article}
\usepackage[standard]{ntheorem}

\newcommand\eqref[1]{(\ref{#1})} % Defined only because I am not using amsart

\input{helper_commands}

\begin{document}
We can \annotatetext{annotate a piece of text}{Here be an
annotation.}. We can equally well annotate a piece of mathematics:
\[ E = \annotatetext{m}{The is the mass} \times \annotatetext{c^2}{Square of
the speed of light} \]
Instead of using the text itself as a toggle, we can use a
mark.\annotatewithmark{See here!} The mark itself
can\annotatehere[click me!]{Wheee!} be descriptive.

Now we can test some cross referencing. We first write down an
equation
\begin{equation}\annotatelabel{waveq}{The nonlinear wave equation \
$\Box u = B(\partial u,\partial u)$}
- \partial_t^2 u + \triangle u = \sum_{i,j = 0}^3 B^{ij} \partial_i u
  \partial_j u
\end{equation}
with respect to which we define
\begin{definition}\annotatelabel{def:nullcon}{The null condition is
when $B^{ij}\xi_i\xi_j = 0$ for any $\xi$ satisfying $m^{ij}\xi_i\xi_j
= 0$ where $m = \mathrm{diag}(-1,1,1,1)$ is the Minkowski metric.}
We say that the null condition is satisfied for \annotateeqref{waveq} if
the term $B^{ij}$ satisfies $\sum_{i,j = 0}^3 B^{ij}\xi_i
\xi_j = 0$ for every $\xi$ satisfying $- \xi_0^2 + \xi_1^2 + \xi_2^2 +
\xi_3^2 = 0$.
\end{definition}
And perhaps a theorem
\begin{theorem}\annotatelabel{thm:nullcon}{Small data global existence
holds provided null condition (Def. \ref{def:nullcon}) is
satisfied}
Small data global existence hold for \annotateeqref{waveq} provided
that the null condition is satisfied (see Definition
\annotateref{def:nullcon}.)
\end{theorem}

Let us talk a bit more about Theorem \annotateref{thm:nullcon}.
\end{document}

As you can see, one has to specify, in the current version, the annotation text for each of the labels when the labels are defined. It is a slight drawback compared to the fancy-preview script, which extracts the annotation text automatically; but on the other hand, this gives slightly better configurability: in particular, nested annotations don’t currently work, so one would have to avoid \annotateref and \annotateeqref commands within an \annotatelabel (as I did in the example above).

If you want to see what the result looks like without building the LaTeX file above yourself, here’s the results: Annotation Demo. (You should download it and open it in a standalone viewer, and not use the built-in ones for Firefox and Chrome. I am pretty sure the Firefox viewer cannot handle OCG content yet.)

What’s wrong with tests

What is mathematical literacy?

There is a wonderful mathematics joke, usually told about Von Neumann, but sometimes with other mathematicians swapped in, it features the following math problem

A train is traveling west at 70 miles per hour, and a train is traveling east at 80 miles an hour. They started 300 miles apart. A fly decided to challenge itself and, at 100 miles per hour, flew from the west-bound train train to the other, and then back, and then forward, and then back, until the two trains passed each other. (An exercise path that is somewhat akin to what we call “suicides” in high school swim practice.) Question: what is the total distance traveled by the fly?

There are two ways to do this problem.

  1. The brute-force way. The fly and the east-bound train are moving toward each other at 180mph. Starting at 300 miles apart they would meet after 300/180 = 100/60 hours = 100 minutes, during which the fly would have traveled 500/3 miles, and the west bound train 350/3 miles. After the fly turned around, it will be 50 miles (= 500/3 – 350/3) from the west-bound train and they head toward each other at 170 miles per hour, so they would meet after 5/17 hours, during which the fly traveled 500/17 miles and the east-bound train 400/17. Now 100/17 miles apart the fly and the east-bound train head toward each other at 180mph and…

    This eventually leads to an infinite series which one can potentially sum geometrically and arrive at a final answer. Or,

  2. The “smart” way. The two trains move toward each other at 150 miles per hour. It takes 2 hours until they meet. The fly, traveling at 100mph, much have traveled a total of 200 miles during the trip.

The “joke” is usually told with some joker, say dear old Feynman, posing this mathematical problem to some more serious-minded mathematician, often renown for his computational abilities (say Von Neumann). The mathematician would reply instantly 200 miles. The joker would be disappointed and complain that the mathematician had heard this question before and somehow knew “the trick”, upon which the mathematician replies: “What trick? It wasn’t that hard to sum the series.”

I just came back from teaching an exercise course (in Princeton language, a precept), and the official answer to one of the exercises reminded me of the above problem. Here is the question:

You are given a biased coin which lands on tails with probability p. You flip the coin until you get heads, and record the number of tosses. (a) What is the probability that the number of tosses is exactly n? (b) What is the probability that the number of tosses is greater than or equal to n?

Part (a) is standard: for n total tosses, the nth toss is heads and the rest are tails. So the probability is p^{n-1}(1-p), representing n-1 successive tails followed by a single heads.

The official answer to part (b) is this: The probability of number of tosses is at least n is the sum

\displaystyle \sum_{k = n}^\infty P( X = k ) = \sum_{k = n}^{\infty} p^{k-1}(1-p)

which after summing the geometric series we arrive at p^{n-1}.

Here’s what I thought after seeing the question: for at least n total tosses, the first n-1 must be all tails. After which the tosses don’t matter: either one eventually ends up with heads at some finite time, in which case it is counted as some finite k = N event, or one never hits heads, in which cases the probability is vanishingly small. So the probability is exactly the probability of getting exactly n-1 tails in a row, which is p^{n-1}.

Just like the fly problem, one of the solutions brute forces the answer, the other try to “reason” away the computational bits until one is left with a simple computation which can be done quickly in one’s head.

Which of these is mathematical literacy? Read the rest of this entry »

IPython Notebook, Take 2

With the help of Fernando Perez, some aspects of my IPython Notebook code has been improved.

Changelog:

  1. Instead of using .show() and .fullshow() methods to display the content of the objects with dependence on a global toggle for MathJax, now we use the _repr_latex_ method (which hooks into IPython’s display functionality). This allows one to enter just m = math_expr([insert atoms here]) followed by m to display the content of the math expression.
  2. Whether the default display mentioned above shows the atom number labels is now configured via a boolean: one can play with it using the .labeled property
  3. To show the LaTeX code, we now use the .latex property
  4. Operations are no longer in place. Instead they return a new math_expr object. This has the advantage that if you can keep playing with the entry until you get it right, and then afterwords you can do m = m.replace(...). This makes it better for for interactive computations.
  5. By default, on object creation it will print its own content.

The code now looks like

from IPython.core.display import *

def add(x,y): return x+y
def MDPL(string): display(Math(string))
def comp_str(listofstrings): return reduce(add,listofstrings)

class math_expr(object):
   '''''Math Expression object'''''

   def __init__(self,atomslist, labeldefault = True, highlight = []):
      '''''init takes arg: list of atoms, each atom being a compilable chunck of LaTeX expression'''''
      self.listofatoms = atomslist
      self.labels = labeldefault
      MDPL(comp_str(self.__colouratoms(highlight)))

   def _repr_latex_(self):
      '''''Returns a latex expression of the object. Will be parsed by MathJax'''''
      self.__labelatoms()
      latexstring = comp_str(self.labeledatoms) if self.labels else comp_str(self.listofatoms)
      return "$" + latexstring + "$"

   @property
   def labeled(self):
      '''''Tells you whether labelling is turned on by default'''''
      return self.labels

   @labeled.setter
   def label(self,boolean):
      '''''Sets the default labelling'''''
      self.labels = boolean

   @property
   def latex(self):
      '''''Accesses the LaTeX code snip for the expression'''''
      display(Latex(comp_str(self.listofatoms)))

   def __labelatoms(self):
      '''''Label atoms by adding underbraces'''''
      self.labeledatoms = [ "\underbrace{" + self.listofatoms[i] + "}_{" + str(i) + "}" for i in range(len(self.listofatoms)) ]

   def replace(self,pos,newstr):
      '''''Replaces an atom with another atom'''''
      MDPL(comp_str(self.__colouratoms([pos])))
      newstrings = list(self.listofatoms)
      newstrings[pos] = newstr
      return math_expr(newstrings,self.labels,[pos])

   def merge(self,positions):
      '''''Merges atoms: the input is a list of positions. The new atom is placed at the position of the foremost of the positions'''''
      MDPL(comp_str(self.__colouratoms(positions)))
      newstrings = list(self.listofatoms)
      temp = [ newstrings[i] for i in positions ]
      positions.sort()
      positions.reverse()
      for i in positions: del newstrings[i]
      newstrings.insert(positions[-1],comp_str(temp))
      return math_expr(newstrings,self.labels,[positions[-1]])

   def split(self,pos,newatoms):
      '''''Splits atoms: replaces an atom in place with multiple sub atoms'''''
      MDPL(comp_str(self.__colouratoms([pos])))
      newstrings = list(self.listofatoms)
      del newstrings[pos]
      templen = len(newatoms)
      while len(newatoms) > 0:
         newstrings.insert(pos,newatoms.pop())
      return math_expr(newstrings,self.labels,range(pos,pos+templen))

   def cancel(self,positions):
      '''''Cancels a bunch of terms: input a list of positions'''''
      MDPL(comp_str(self.__colouratoms(positions)))
      positions.sort()
      positions.reverse()
      newstrings = list(self.listofatoms)
      for i in positions: del newstrings[i]
      return math_expr(newstrings,self.labels)

   def move(self,posini,posfin):
      '''''Move atom at posini to posfin, pushing all others back'''''
      MDPL(comp_str(self.__colouratoms([posini])))
      newstrings = list(self.listofatoms)
      temp = newstrings.pop(posini)
      newstrings.insert(posfin if posfin < posini else posfin-1, temp)
      return math_expr(newstrings,self.labels,[posfin if posfin < posini else posfin - 1])

   def __colouratoms(self,positions,labelled=False):
      '''''Returns the list of atoms, but with selected terms coloured'''''
      temp = list(self.listofatoms)
      if labelled:
         self.labelatoms()
         temp = list(self.labeledatoms)
      for i in positions: temp[i] = "\color{red}{"+temp[i]+"}"
      return temp

Using IPython Notebook for manual computations

(Edit July 27: please see this update)

One of the hardest organizational tasks I’ve found in the past few years is keeping track of computations. When I started as a graduate student I began by doing computations by hand on scratch paper and copying everything (that is worth keeping) into a spiral-bound notebook. Notebooks being hard to carry, a little bit later I moved on to scanning my notes. Than I decided that my handwriting is quite horrible and I should just TeX everything up instead. Occasionally this has led to me doing my computations directly in LaTeX inside a Vim buffer. Yesterday I discovered a better alternative.

A few of the problems I’ve run into with the various methods of keeping computations include:

  1. In deadtree format, the sheets of paper are easily misplaced or soiled, and are not as easily modifiable.
  2. Doing a hand computation and then typing up the result takes extra time.
  3. Computing directly in LaTeX requires a lot of redundant typing or copying and pasting of parts of the expressions that don’t change from line to line.
  4. Typed notes (and sometimes written notes) don’t easily show (unless you take the extra time to format it) the changes between the lines, and sometimes a few months later the computation becomes hard to follow.

The present solution below has

  1. Storage in electronic format for easy backup and search.
  2. Doing the calculation directly on a computer to avoid the two-step process.
  3. Let the computer do the work of copying unchanged expressions for you.
  4. Let the computer do the formatting; the computation is written in an almost-human-readable code that is self-documentary.

Software needed
What I discovered, is the IPython Notebook (link to project homepage). The “notebook” part of the name, I believe, refers to its semblance to Mathematica/MatLab/Maxima style notebooks where one sequentially and interactively enter commands and observe their output. Unlike the mentioned software, this notebook is for interactively using the Python programming language. Furthermore, it supports inline documentation using MarkDown and MathJax! (This can be had by changing the cell type to Markdown, instead of the default Python code.)

If you want to see for yourself how this works, you will need to install IPython Notebook yourself. (In Gentoo linux, emerge ipython with the notebook use flag set; in Debian or Ubuntu, you can just apt-get the relevant packages.)

Doing computations
With a little bit of (very messy and unprofessional) Python code, the Notebook can be convinced to print certain outputs prettified by MathJax. Here is the set-up I used:

from IPython.core.display import *

global MathJax 
MathJax = True
def MDPL(string): display(Math(string)) if MathJax else display(Latex(string))

def add(x,y): return x+y

def comp_str(listofstrings): return reduce(add,listofstrings)

class math_expr(object):
   '''''Math Expression object'''''

   def __init__(self,arg1):
      '''''init takes arg: list of atoms, each atom being a compilable chunck of LaTeX expression'''''
      self.listofatoms = arg1

   def show(self):
      '''''Displays the content of the expression in mathmode'''''
      MDPL(comp_str(self.listofatoms))

   def replace(self,pos,newstr):
      '''''Replaces an atom with another atom'''''
      MDPL(comp_str(self.colouratoms([pos])))
      self.listofatoms[pos] = newstr
      MDPL(comp_str(self.colouratoms([pos],True)))

   def merge(self,positions):
      '''''Merges atoms: the input is a list of positions. The new atom is placed at the position of the foremost of the positions'''''
      MDPL(comp_str(self.colouratoms(positions)))
      temp = [ self.listofatoms[i] for i in positions ]
      positions.sort()
      positions.reverse()
      for i in positions: del self.listofatoms[i]
      self.listofatoms.insert(positions[-1],comp_str(temp))
      MDPL(comp_str(self.colouratoms([positions[-1]],True)))

   def split(self,pos,newatoms):
      '''''Splits atoms: replaces an atom in place with multiple sub atoms'''''
      MDPL(comp_str(self.colouratoms([pos])))
      del self.listofatoms[pos]
      templen = len(newatoms)
      while len(newatoms) > 0:
         self.listofatoms.insert(pos,newatoms.pop())
      MDPL(comp_str(self.colouratoms(range(pos, pos+templen),True)))

   def cancel(self,positions):
      '''''Cancels a bunch of terms: input a list of positions'''''
      MDPL(comp_str(self.colouratoms(positions)))
      positions.sort()
      positions.reverse()
      for i in positions: del self.listofatoms[i]
      self.fullshow()

   def move(self,posini,posfin):
      '''''Move atom at posini to posfin, pushing all others back'''''
      MDPL(comp_str(self.colouratoms([posini])))
      temp = self.listofatoms.pop(posini)
      self.listofatoms.insert(posfin if posfin < posini else posfin-1, temp)
      MDPL(comp_str(self.colouratoms([posfin if posfin < posini else posfin-1],True)))

   def colouratoms(self,positions,labelled=False):
      '''''Returns the list of atoms, but with selected terms coloured'''''
      temp = list(self.listofatoms)
      if labelled:
         self.labelatoms()
         temp = list(self.labeledatoms)
      for i in positions: temp[i] = "\color{red}{"+temp[i]+"}"
      return temp

   def labelatoms(self):
      '''''Label atoms by adding underbraces'''''
      self.labeledatoms = [ "\underbrace{" + self.listofatoms[i] + "}_{" + str(i) + "}" for i in range(len(self.listofatoms)) ]

   def fullshow(self):
      '''''Shows the content whilst labeling positions'''''
      self.labelatoms()
      MDPL(comp_str(self.labeledatoms))

The entire python code can be copied and saved somewhere on your computer. After launching the Notebook, drag the file and drop it into the first input box. Then hit Shift-Enter to load the code. After that you can do things like:

Loaded preamble, do some computations

After loading the preamble in cell 1, we insert a math expression and display it.

and

Next we do some computations (manipulations of the expression).

After I am done, I just click “Save” and the entire computation is saved, and bits and pieces of it can be easily copied out by switching to non-mathJax output.

The code is still in the early stages: a lot more improvements can be made. But it is already a working system.