Automated annotation in LaTeX using OCG
by Willie Wong
This is written mostly in response to Qiaochu’s Google+ post. The problem is: is there a way to make easy “cross reference previews”? If so, we can just write
\begin{theorem}
Assuming Assumptions \ref{ass1} and \ref{ass2}, we have blah
\end{theorem}
in our document, and on mouse-over we can see what the assumptions are. Or, as another example,
due to \eqref{eq3}, our equation \eqref{eq4} above implies
\[ E = mc^2\]
and mousing over the two generated equation references will show the equations, without us having to necessarily flip back to the page involved.
As it turns out, there is such a utility available: Fancy-Preview. It uses the fancytooltips package for LaTeX to generate PDF tooltips where the tooltips are clipped from an external PDF file. The upside to using fancy-preview is that the results are very pretty. The downside is that it is extremely non-portable: while I am not certain, it seems to require the PDF viewer to have javascript support, and on Linux the only PDF viewer that stands a chance of working with these tooltips is the official Adobe Acrobat Reader. If you are a Linux user you’ll know why I uttered the last phrase with disgust. (Though rumour has it that both evince and okular may eventually support the type of operations required for fancy-preview, the fact is that right now these tooltips are not very useful for Linux users.)
Instead of tooltips, however, one can get a similar effect by using the Optional Content Group feature of the PDF specification. The OCG basically allows the visibility of certain elements to be changeable by user interaction, and the ocg-p package implements easy access to the OCG layers. However, just the ocg-p package is not enough: the way it detects user interaction still doesn’t work on evince (I haven’t tested other PDF viewers). Luckily, there is the ocgx package which builds on the ocg-p package and does not use Javascript. The code has been tested to run on Acrobat Reader, FoxIt, Evince, and others. So I know at least one Linux reader is capable of using this content.
Now, OCG handling by itself only allows us to toggle the visibility of elements. We still need a way to actually present the annotations in the PDF file. Here’s where it gets tricky: the annotations should be invisible by default, and be shown with certain triggers are clicked. So the idea is to typeset the annotations as zero-size objects, offset properly so that it does not obscure the trigger button. The code I am about to show you below does this… to a certain extent. There are some bugs that I have not yet been able to iron out:
- In the function
\annotatetext, if the text starts too near line-breaking boundaries we get sometimes strange behaviours, one of which is that after every compilation pdflatex will tell us to re-run since the labels may have changed. - I originally intended to build also a height-detection code so that if there is not enough space above the line, we can try to type-set the annotation below the trigger. However we run into one of the main limitations of the OCG method: the order of setting the text is the order of the layers, at least with the current version of ocg-p. So the annotation text, when shown, will overwrite text that precedes it in the TeX file, but will be overwritten by text that follows. So below-the-line annotations becomes illegible. (Any suggestions on how I can fix this will be welcome!)
Note also that the code below is not “production”, it has only seen limited testing. So use it at your own peril. (I should note here that to make use of the utils, it is necessary to compile the file using some version of pdfTeX.)
%%%%% Annotation utils %%%%%
% The goal is to provide a clickable
% tool-tip like interface to show cross reference information.
% This duplicates the function of the 'fancy-preview' and
% 'fancytooltips' packages, and does not support mouse-over events,
% but has the advantage of working in evince also.
% We need some packages
\usepackage[usenames,dvipsnames]{color}
\usepackage{zref-savepos}
\usepackage{xifthen}
\usepackage{ocgx}
\usepackage{xspace}
% the package ocgx also depends on ocg-p, I think.
% Some configuration stuff
% Default colours, see
% http://en.wikibooks.org/wiki/LaTeX/Colors
\newcommand*\annotatetextcolour{OliveGreen} % For the inline text
\newcommand*\annotateboxbordercolour{Dandelion} % For the annotate box
\newcommand*\annotateboxbkgdcolour{Goldenrod} % Ditto
\newcommand*\annotateboxtextcolour{Black} % Ditto
\newcommand*\annotateboxtextfont{\small} % Other font configuration stuff
\newcommand*\annotateheremarktext{note} % Default text for \annotatehere
\newcommand*\annotatewithmarkmark{$\Uparrow$} % Default mark for \annotatewithmark
\makeatletter
% Dummy variables
\newcounter{@anntposmark}
\newlength\@annt@oldfboxsep
% Usage: \annotatetext{text}{annotation}
% Note: there can be some bugs when the text to be annotated starts
% near the end of a line. The other two commands seems to behave
% better, but will still need extensive testing!
\newcommand\annotatetext[2]{%
\stepcounter{@anntposmark}%
\zsavepos{@annt@pos\the@anntposmark}%
\hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
\smash{\raisebox{3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour} #2}}}\end{ocg}}}}%
\hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%
\switchocg{anntlayer\the@anntposmark}{{\color{\annotatetextcolour}#1}}%
\xspace}
% Usage: \annotatehere[note mark text]{annotations}
% The default mark text is set above, the mark text is type-set in a superscript box
\newcommand\annotatehere[2][\annotateheremarktext]{%
\stepcounter{@anntposmark}%
\zsavepos{@annt@pos\the@anntposmark}%
\hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
\smash{\raisebox{3.3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour}#2}}}\end{ocg}}}}%
\hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%
\setlength\@annt@oldfboxsep\fboxsep%
\setlength\fboxsep{1pt}%
\switchocg{anntlayer\the@anntposmark}{\raisebox{1ex}{\fcolorbox{\annotatetextcolour}{White}{\color{\annotatetextcolour}\tiny \scshape #1}}}%
\setlength\fboxsep\@annt@oldfboxsep%
\xspace}
% Usage: \annotatewithmark[mark]{annotations}
% Very similar to \annotatehere, but the mark is unframed, and no conversion made to small caps
\newcommand\annotatewithmark[2][\annotatewithmarkmark]{%
\stepcounter{@anntposmark}%
\zsavepos{@annt@pos\the@anntposmark}%
\hskip\dimexpr - \zposx{@annt@pos\the@anntposmark}sp + \zposx{@anntleftmargin}sp + 3em%
\smash{\raisebox{3.3ex}{\makebox[0pt][l]{\begin{ocg}{Annotation Layer \the@anntposmark}{anntlayer\the@anntposmark}{0}\fcolorbox{\annotateboxbordercolour}{\annotateboxbkgdcolour}{\parbox[b]{\textwidth-6em}{{\annotateboxtextfont\color{\annotateboxtextcolour}#2}}}\end{ocg}}}}%
\hskip\dimexpr + \zposx{@annt@pos\the@anntposmark}sp - \zposx{@anntleftmargin}sp - 3em%
\setlength\@annt@oldfboxsep\fboxsep%
\setlength\fboxsep{1pt}%
\switchocg{anntlayer\the@anntposmark}{\raisebox{1ex}{{\color{\annotatetextcolour}\tiny #1}}}%
\setlength\fboxsep\@annt@oldfboxsep%
\xspace}
% Usage: \annotatelabel{label}{annotations}
% Replacement for \label, where attached to each label there is an annotation
% text which can be recalled using \annotateref{label}. See below.
\newcommand\annotatelabel[2]{%
\label{#1}%
\global\@namedef{@annt@label@#1}{#2}}
% Usage: \annotateref{label} and \annotateeqref{label}
% Replacement for \ref and \eqref, where we insert an \annotatewithmark with
% the text set with \annotatelabel
\newcommand\annotateref[1]{\ref{#1}\annotatewithmark{\@nameuse{@annt@label@#1}}}
\newcommand\annotateeqref[1]{\eqref{#1}\annotatewithmark{\@nameuse{@annt@label@#1}}}
\AtBeginDocument{\zsavepos{@anntleftmargin}} %This figures out where the left margin is
\makeatother
%%%%% End Annotation utils %%%%%
Let me give an example (to compile, replace the line \input{helper_commands} by the code listing above, or save the code listing above in the file helper_commands.tex
\documentclass{article}
\usepackage[standard]{ntheorem}
\newcommand\eqref[1]{(\ref{#1})} % Defined only because I am not using amsart
\input{helper_commands}
\begin{document}
We can \annotatetext{annotate a piece of text}{Here be an
annotation.}. We can equally well annotate a piece of mathematics:
\[ E = \annotatetext{m}{The is the mass} \times \annotatetext{c^2}{Square of
the speed of light} \]
Instead of using the text itself as a toggle, we can use a
mark.\annotatewithmark{See here!} The mark itself
can\annotatehere[click me!]{Wheee!} be descriptive.
Now we can test some cross referencing. We first write down an
equation
\begin{equation}\annotatelabel{waveq}{The nonlinear wave equation \
$\Box u = B(\partial u,\partial u)$}
- \partial_t^2 u + \triangle u = \sum_{i,j = 0}^3 B^{ij} \partial_i u
\partial_j u
\end{equation}
with respect to which we define
\begin{definition}\annotatelabel{def:nullcon}{The null condition is
when $B^{ij}\xi_i\xi_j = 0$ for any $\xi$ satisfying $m^{ij}\xi_i\xi_j
= 0$ where $m = \mathrm{diag}(-1,1,1,1)$ is the Minkowski metric.}
We say that the null condition is satisfied for \annotateeqref{waveq} if
the term $B^{ij}$ satisfies $\sum_{i,j = 0}^3 B^{ij}\xi_i
\xi_j = 0$ for every $\xi$ satisfying $- \xi_0^2 + \xi_1^2 + \xi_2^2 +
\xi_3^2 = 0$.
\end{definition}
And perhaps a theorem
\begin{theorem}\annotatelabel{thm:nullcon}{Small data global existence
holds provided null condition (Def. \ref{def:nullcon}) is
satisfied}
Small data global existence hold for \annotateeqref{waveq} provided
that the null condition is satisfied (see Definition
\annotateref{def:nullcon}.)
\end{theorem}
Let us talk a bit more about Theorem \annotateref{thm:nullcon}.
\end{document}
As you can see, one has to specify, in the current version, the annotation text for each of the labels when the labels are defined. It is a slight drawback compared to the fancy-preview script, which extracts the annotation text automatically; but on the other hand, this gives slightly better configurability: in particular, nested annotations don’t currently work, so one would have to avoid \annotateref and \annotateeqref commands within an \annotatelabel (as I did in the example above).
If you want to see what the result looks like without building the LaTeX file above yourself, here’s the results: Annotation Demo. (You should download it and open it in a standalone viewer, and not use the built-in ones for Firefox and Chrome. I am pretty sure the Firefox viewer cannot handle OCG content yet.)
Bubbles Bad; Ripples Good 
There is a PDF viewer for OS X, called Skim, which does this fancy preview thing automatically for all PDFs. Someone should implement the same feature for evince or okular.
Thanks, I didn’t know about Skim. For those interested: Skim can be found here. It is only for OS-X, and is free. (And it looks like one heck of a PDF viewer!)
But best as I can tell, the preview renders a
fixed (small)user-adjustable, by default small, area around the linkwhich may or may not be sufficient in a scientific (especially mathematics) document. In that regard fancy-preview is actually superior, since it picks up exactly what it needs to show, no more and no less. (And this is unavoidable: what I am trying to describe/accomplish really is something on the content end. A solution purely on the end of the viewer will never be a perfect fit.Thanks. This is very helpful. I was looking for a way to my distribute beamer-based slides for a conference talk in a way such that a viewer can access extra information that I conveyed only in speech.
How can I place \annotatelabel at an arbitrary place, e.g. in the middle of a figure?
I am afraid I don’t understand the question. \annotatelabel is an extension to the usual \label command in LaTeX. Anywhere \label can be used you can use \annotatelabel. The \annotatelabel command should not create something that is reader visible; it is only useful in conjunction with the \annotateref command which replaces the standard \ref command with a version that shows an alt-text describing what the reference is to, using a “note” superscript so as not to destroy any hyperlinking when present.
So I am not sure what you mean by “placing an \annotatelabel at an arbitrary place”, and what can be the problem. If you can clarify what you mean maybe I can be more helpful. Sorry!
My mistake. I meant \annotatewithmark . I’m sorry.
I often want the mark ($\Uparrow$) to appear inside a figure.
These are cases where the content of it is explaining some aspect of the figure.
I was wondering if there is a way to make it appear at an arbitrary position, perhaps specified as coordinates w.r.t. the whole slide, or w.r.t. a figure in the slide. Perhaps I should read a TikZ tutorial.
I see. The horizontal and vertical positions behave differently.
If you read the \annotatewithmark command’s definition, there are two lines that being with “\hskip”; the first line places the left edge of the annotation box, the second one “restores” the position to the position before the annotation box has been processed, so you see it skipping the opposite amount of the first \hskip. Editing this line changes the horizontal coordinate of the left edge of the annotation.
The width of the annotation is defined in the command in between the two \hskip s. It is governed by the length “\textwidth-6em” currently, but you can make it any length you want.
The vertical positioning is a bit trickier. I achieved the effect by raising the box in the line between the \hskip s; it is the argument to the \raisebox command, which is currently 3.3em, that makes the difference.
What you can do is then to modify the definition of \annotatewithmark to take 3 additional arguments, representing the relative horizontal and vertical positions of the top left corner of your annotation box, and also the width. Replace the argument to the two \hskip s with plus and minus the horizontal displacement respectively, and the argument to the \raisebox to the vertical displacement. The width replaces the length in the \parbox.
This should in principle allows for arbitrary relative placement of the annotation relative to the “current position”. This can be slightly problematic if you rewrote your slides and the texts reflow.
Now, absolute positioning of objects is a bit orthogonal to the intended design use of the annotation tools I blogged about. (The whole point is to have relatively positioned annotation text boxes that show up near where the annotated object sits on the page.) So if you want to actually just have absolute positioning of objects, you are better off not using the tools described in my blog post here, and just use OCG layers directly. I highly recommend reading through the package documentation for the OCGX package (https://www.ctan.org/pkg/ocgx?lang=en) Essentially you should just create a bunch of OCG layers with the annotation you want to show placed exactly where you want them, and use OCGX’s built-in toggling functionality to show/hide the content.
Hope this explanation helps.