Automated annotation in LaTeX using OCG

by Willie Wong

This is written mostly in response to Qiaochu’s Google+ post. The problem is: is there a way to make easy “cross reference previews”? If so, we can just write

Assuming Assumptions \ref{ass1} and \ref{ass2}, we have blah

in our document, and on mouse-over we can see what the assumptions are. Or, as another example,

due to \eqref{eq3}, our equation \eqref{eq4} above implies
\[ E = mc^2\]

and mousing over the two generated equation references will show the equations, without us having to necessarily flip back to the page involved.

As it turns out, there is such a utility available: Fancy-Preview. It uses the fancytooltips package for LaTeX to generate PDF tooltips where the tooltips are clipped from an external PDF file. The upside to using fancy-preview is that the results are very pretty. The downside is that it is extremely non-portable: while I am not certain, it seems to require the PDF viewer to have javascript support, and on Linux the only PDF viewer that stands a chance of working with these tooltips is the official Adobe Acrobat Reader. If you are a Linux user you’ll know why I uttered the last phrase with disgust. (Though rumour has it that both evince and okular may eventually support the type of operations required for fancy-preview, the fact is that right now these tooltips are not very useful for Linux users.)

Instead of tooltips, however, one can get a similar effect by using the Optional Content Group feature of the PDF specification. The OCG basically allows the visibility of certain elements to be changeable by user interaction, and the ocg-p package implements easy access to the OCG layers. However, just the ocg-p package is not enough: the way it detects user interaction still doesn’t work on evince (I haven’t tested other PDF viewers). Luckily, there is the ocgx package which builds on the ocg-p package and does not use Javascript. The code has been tested to run on Acrobat Reader, FoxIt, Evince, and others. So I know at least one Linux reader is capable of using this content.

Now, OCG handling by itself only allows us to toggle the visibility of elements. We still need a way to actually present the annotations in the PDF file. Here’s where it gets tricky: the annotations should be invisible by default, and be shown with certain triggers are clicked. So the idea is to typeset the annotations as zero-size objects, offset properly so that it does not obscure the trigger button. The code I am about to show you below does this… to a certain extent. There are some bugs that I have not yet been able to iron out:

  • In the function \annotatetext, if the text starts too near line-breaking boundaries we get sometimes strange behaviours, one of which is that after every compilation pdflatex will tell us to re-run since the labels may have changed.
  • I originally intended to build also a height-detection code so that if there is not enough space above the line, we can try to type-set the annotation below the trigger. However we run into one of the main limitations of the OCG method: the order of setting the text is the order of the layers, at least with the current version of ocg-p. So the annotation text, when shown, will overwrite text that precedes it in the TeX file, but will be overwritten by text that follows. So below-the-line annotations becomes illegible. (Any suggestions on how I can fix this will be welcome!)

Note also that the code below is not “production”, it has only seen limited testing. So use it at your own peril. (I should note here that to make use of the utils, it is necessary to compile the file using some version of pdfTeX.)

%%%%% Annotation utils %%%%%
% The goal is to provide a clickable
% tool-tip like interface to show cross reference information. 
% This duplicates the function of the 'fancy-preview' and
% 'fancytooltips' packages, and does not support mouse-over events,
% but has the advantage of working in evince also. 

% We need some packages
% the package ocgx also depends on ocg-p, I think.

% Some configuration stuff
% Default colours, see
\newcommand*\annotatetextcolour{OliveGreen} % For the inline text
\newcommand*\annotateboxbordercolour{Dandelion} % For the annotate box
\newcommand*\annotateboxbkgdcolour{Goldenrod} % Ditto
\newcommand*\annotateboxtextcolour{Black} % Ditto
\newcommand*\annotateboxtextfont{\small} % Other font configuration stuff
\newcommand*\annotateheremarktext{note} % Default text for \annotatehere 
\newcommand*\annotatewithmarkmark{$\Uparrow$} % Default mark for \annotatewithmark

% Dummy variables

% Usage: \annotatetext{text}{annotation}
% Note: there can be some bugs when the text to be annotated starts
% near the end of a line. The other two commands seems to behave
% better, but will still need extensive testing!
        \hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
        \smash{\raisebox{3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour} #2}}}\end{ocg}}}}%
        \hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%

% Usage: \annotatehere[note mark text]{annotations}
% The default mark text is set above, the mark text is type-set in a superscript box
        \hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
        \smash{\raisebox{3.3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour}#2}}}\end{ocg}}}}%
        \hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%
        \switchocg{anntlayer\the@anntposmark}{\raisebox{1ex}{\fcolorbox{\annotatetextcolour}{White}{\color{\annotatetextcolour}\tiny \scshape #1}}}%

% Usage: \annotatewithmark[mark]{annotations}
% Very similar to \annotatehere, but the mark is unframed, and no conversion made to small caps
        \hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
        \smash{\raisebox{3.3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour}#2}}}\end{ocg}}}}%
        \hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%
        \switchocg{anntlayer\the@anntposmark}{\raisebox{1ex}{{\color{\annotatetextcolour}\tiny #1}}}%

% Usage: \annotatelabel{label}{annotations}
% Replacement for \label, where attached to each label there is an annotation 
% text which can be recalled using \annotateref{label}. See below. 
% Usage: \annotateref{label}  and  \annotateeqref{label}
% Replacement for \ref and \eqref, where we insert an \annotatewithmark with 
% the text set with \annotatelabel

\AtBeginDocument{\zsavepos{@anntleftmargin}} %This figures out where the left margin is

%%%%% End Annotation utils %%%%%

Let me give an example (to compile, replace the line \input{helper_commands} by the code listing above, or save the code listing above in the file helper_commands.tex


\newcommand\eqref[1]{(\ref{#1})} % Defined only because I am not using amsart


We can \annotatetext{annotate a piece of text}{Here be an
annotation.}. We can equally well annotate a piece of mathematics:
\[ E = \annotatetext{m}{The is the mass} \times \annotatetext{c^2}{Square of
the speed of light} \]
Instead of using the text itself as a toggle, we can use a
mark.\annotatewithmark{See here!} The mark itself
can\annotatehere[click me!]{Wheee!} be descriptive.

Now we can test some cross referencing. We first write down an
\begin{equation}\annotatelabel{waveq}{The nonlinear wave equation \
$\Box u = B(\partial u,\partial u)$}
- \partial_t^2 u + \triangle u = \sum_{i,j = 0}^3 B^{ij} \partial_i u
  \partial_j u
with respect to which we define
\begin{definition}\annotatelabel{def:nullcon}{The null condition is
when $B^{ij}\xi_i\xi_j = 0$ for any $\xi$ satisfying $m^{ij}\xi_i\xi_j
= 0$ where $m = \mathrm{diag}(-1,1,1,1)$ is the Minkowski metric.}
We say that the null condition is satisfied for \annotateeqref{waveq} if
the term $B^{ij}$ satisfies $\sum_{i,j = 0}^3 B^{ij}\xi_i
\xi_j = 0$ for every $\xi$ satisfying $- \xi_0^2 + \xi_1^2 + \xi_2^2 +
\xi_3^2 = 0$.
And perhaps a theorem
\begin{theorem}\annotatelabel{thm:nullcon}{Small data global existence
holds provided null condition (Def. \ref{def:nullcon}) is
Small data global existence hold for \annotateeqref{waveq} provided
that the null condition is satisfied (see Definition

Let us talk a bit more about Theorem \annotateref{thm:nullcon}.

As you can see, one has to specify, in the current version, the annotation text for each of the labels when the labels are defined. It is a slight drawback compared to the fancy-preview script, which extracts the annotation text automatically; but on the other hand, this gives slightly better configurability: in particular, nested annotations don’t currently work, so one would have to avoid \annotateref and \annotateeqref commands within an \annotatelabel (as I did in the example above).

If you want to see what the result looks like without building the LaTeX file above yourself, here’s the results: Annotation Demo. (You should download it and open it in a standalone viewer, and not use the built-in ones for Firefox and Chrome. I am pretty sure the Firefox viewer cannot handle OCG content yet.)