SEEing LaTeX 11: Typesetting Extended

In installment 10 of this series, I refactored the shell scripts used to compile and view a LaTeX document from SubEthaEdit. To change how a LaTeX document is compiled or viewed, it is now only necessary to change a variable, rather than messing around with the logic of the script. Let's take things a step further.

The relevant variables in the script are LATEX, PRODUCT_TYPE, and VIEWER. Their values in the script shown before were to use latexmk to build a PDF and PDFView to display it. However, other settings are possible. For example, setting

LATEX='pdflatex "$FILE"'
VIEWER='open "$PRODUCT"'
PRODUCT_TYPE=pdf

builds a PDF using pdflatex and shows it in Preview. These settings would be quite suitable for a "vanilla" installation, assuming no extras are installed beyond TeX.

A more flexible approach is to use those vanilla settings as defaults, and allow them to be overridden, giving

LATEX=${SEE_LATEX_COMPILER:-'pdflatex "$FILE"'}
VIEWER=${SEE_LATEX_VIEWER:-'open "$PRODUCT"'}
PRODUCT_TYPE="${SEE_LATEX_PRODUCT_TYPE:-pdf}"

Now, if we run the script from the shell, we can set environment variables to change how the script does its work.

Within SEE, the environment variables will be provided by the AppleScript that runs the script. The change is relatively simple: we extend the string that calls our shell script to also set the environment variables. Since a number of minor changes to the AppleScript have accumulated, I'll show the whole thing:

tell application "SubEthaEdit"
    if exists path of front document then
        if modified of front document then
            try
                save front document
            end try
        end if
        set filePath to path of front document
        set lineNumber to startLineNumber of selection of front document
        set modeResources to resource path of mode of front document
    else
        error "You have to save the document first"
    end if
end tell

set buildScript to prependEnvironment onto (join of {quotedForm for (modeResources & "/Scripts/shell/buildlatex.sh"), quotedForm for filePath, lineNumber} by space)

do shell script buildScript

on seescriptsettings()
    return {displayName:"Typeset and View PDF", shortDisplayName:"Typeset", keyboardShortcut:"@b", toolbarIcon:"ToolbarIconBuildAndRun", inDefaultToolbar:"yes", toolbarTooltip:"Typeset and view the current document", inContextMenu:"no"}
end seescriptsettings

on join of tokenList by delimiter
    set oldTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to delimiter
    set joinedString to tokenList as string
    set text item delimiters of AppleScript to oldTIDs
    return joinedString
end join

on quotedForm for baseString  
    quote & baseString & quote
end quotedForm

to prependEnvironment onto scriptString
    "export SEE_LATEX_COMPILER='latexmk -pdf -quiet \"$FILE\"'; export SEE_LATEX_PRODUCT_TYPE=pdf; export SEE_LATEX_VIEWER='/Applications/Skim.app/Contents/SharedSupport/displayline \"$LINE\" \"$PRODUCT\" \"$FILE\"';" & scriptString
end prependEnvironment

The basic strategy is still to construct a string that calls our shell script and feed it into do shell script. The string is changed in two ways. First, since the shell script now takes a line number as a second argument, I get the current line from SEE and pass it in. Second, and of more immediate interest, I set the environment variables and prepend those to the earlier buildScript. I've simplified the construction by defining join, quotedForm, and prependEnvironment handlers.

PrependEnvironment sets the SEE_LATEX_COMPILER, SEE_LATEX_PRODUCT_TYPE, and SEE_LATEX_VIEWER environment variables, customizing the behavior of the shell script. In the above AppleScript, I have set the environment so that latexmk and Skim are used to build and view a PDF.

SEEing LaTeX 10: Typesetting Refactored

Let's unify the scripts shown in the preceding installment. The approach I'll take is to move everything that I can into shell variables. My earlier script

PATH=/usr/texbin:/usr/local/bin:$PATH
export PATH

cd "$(dirname "$1")"
latexmk -pdf -quiet "$(basename "$1")"
PRODUCT="$(basename "$1" .tex).pdf"

if [ -s "$PRODUCT" ]
then
    open -a PDFView.app "$PRODUCT"
fi

becomes

PATH=/usr/texbin:/usr/local/bin:$PATH
export PATH

LATEX='latexmk -pdf -quiet "$FILE"'
PRODUCT_TYPE=pdf
VIEWER='open -a PDFView.app "$PRODUCT"'

FILE="$(basename "$1")"
DIRNAME="$(dirname "$1")"
LINE="$2"
PRODUCT="$(basename "$1" .tex).$PRODUCT_TYPE"

cd "$DIRNAME"
eval $LATEX
if [ -s "$PRODUCT" ]
then
    eval $VIEWER
fi

All I've really done is to move some things around; note the use of eval to allow shell variables to be referred to before they are defined. The script behaves the same. That's fine, the goal for the moment it just to refactor for some later behavioral changes, not to make those behavioral changes now.

The behavior of the script provided by Al Kasprzyk can be produced by the above script by changing just the line defining VIEWER. That line becomes

VIEWER='/Applications/Skim.app/Contents/SharedSupport/displayline "$LINE" "$PRODUCT" "$FILE"'

Nothing else need be changed; we now can compile our LaTeX file and see the resulting PDF in Skim.

To incorporate this into the SubEthaEdit mode, we do need to have the AppleScript pass the line number to the shell script. For now, I'll omit showing the AppleScript, since the essential changes are minor. I'll take a closer look at the AppleScript next time.

Update: I've renamed the LINENUM variable in the script to LINE. This is just to be more symmetric with how applications like PDFView and Skim work with pdfsync.

SEEing LaTeX 9: Typesetting Revisited

Earlier, I discussed typesetting a LaTeX document from SubEthaEdit and sending it to PDFView. PDFView works fine, but is no longer being developed; the developer recommends Skim.

Replacing PDFView with Skim is something that I've been thinking about for a while, but it hasn't been a high priority. Happily, Al Kasprzyk provided a script to make that replacement in a comment to the earlier post. For convenience, I'll reproduce it here, with syntax highlighting:

cd "`dirname "$1"`"
latexmk -pdf -quiet "`basename "$1"`"
pdfName=$(basename "$1" tex)pdf

if test -s "$pdfName"; then
/Applications/Skim.app/Contents/SharedSupport/displayline $2 "$pdfName" "$1"
fi

Perhaps the greatest difference is that Al uses the displayline script included with Skim, instead of just opening the PDF created from the LaTeX document. Thus, instead of having latexmk open the preview, he handles it himself. behaviorally, the preview is shown in Skim, with the current line selected in SEE being visible in Skim.

We can rewrite my earlier script to take a similar approach. Without either following Al's script exactly or avoiding looking at it, I came up with this:

PATH=/usr/texbin:/usr/local/bin:$PATH
export PATH

cd "$(dirname "$1")"
latexmk -pdf -quiet "$(basename "$1")"
PRODUCT="$(basename "$1" .tex).pdf"

if [ -s "$PRODUCT" ]
then
    open -a PDFView.app "$PRODUCT"
fi

The differences between the two scripts are minor. Apart from some formatting choices, there are just a few differences. First, I set the PATH environment variable. I'm pretty sure Al must have done so as well, but he didn't include it in his comment. Second, I have an extra layer of quoting for what I called the PRODUCT; based on restrictions for LaTeX file names, these are superfluous. Third is the obvious and essential difference of which viewer we call.

Right now, there is no change in behavior between my earlier script and the one I present above. However, I no longer need to define anything in a .latexmkrc file, which is helpful in making a portable LaTeX mode. Better, though, is that the obvious parallels between Al's script and mine can be taken advantage of. Next time.

No Real Surprise

I've registered SubEthaEdit. My trial period was finally at an end, and the decision was a no-brainer. As it turns out, my "thirty day" trial period was close to three months long; each time SEE was updated, the trial period was reset. Very fair of the Coding Monkeys, I think.

On TextExtras

I'm back from a lengthy workshop. There has been quite a delay since my last post, due to the workshop and all the associated tasks needed both before and after. Almost a month, in fact. Let's rectify that now with a quick, easy post to get things started again.

In an earlier post, I wrote that I was unable to download TextExtras. Not too long after that, I was able to reach the site again, and have been using TextExtras since.

TextExtras is, in short, a wonderful extension to every text field in Cocoa applications. There are an almost overwhelming number of options you can set by enabling key bindings. However, I've enabled just one key binding so far, since I don't really want to relearn a big list of keyboard shortcuts and don't really need lots of editing power in every application. All that I did was to bind esc to TextExtras' completion option. This calls the normal Cocoa autocompletion, including whatever extensions are provided by an application (like in SubEthaEdit), but with a different interface. The Cocoa autocompletion system completes the word for a single match and shows a list of possibilities with multiple possibilities. TextExtras' completions insert the first match and cycles through additional matches with additional strokes of the appropriate key. I find this is often more convenient when programming, because you often get the match on the first try with variable names. In this case, the Cocoa approach selects the newly inserted text, requiring you to use the arrow key or some other method to move on. In constrast, TextExtras just puts the cursor at the end of the word, and you can just keep typing. If Cocoa completions always showed the list of possibilities, even in the case of a single match, it would be a lot nicer to use, since you could, e.g., just press the space bar to choose the only match and keep typing. Of course, I still have the usual Cocoa completions using option-esc.

More significant is the "Execute Pipe..." action. This pipes text through a Unix shell command. SubEthaEdit has an AppleScript to do that, but it's pretty limited. TextExtras gives you something more like the pipe action in TextMate. This is great to have when editing, and comes in handy for all sorts of situations. Shell pipelines everywhere!

There are many, many other possibilities with TextExtras. Try it!

SEEing LaTeX 8: Dealing with Auxiliary Files

As noted in part 5 of this series, there were three main features to add to the LaTeX mode for SubEthaEdit (SEE). I've already taken care of the first, typesetting, by taking advantage of latexmk. In this installment, I'll address the second, cleaning up the mess of auxiliary files.

LaTeX produces several sorts of auxiliary file to handle cross-references, tables of contents, bibliographies, and more. The auxiliary files are essential to how LaTeX works, but of little use when, e.g., sending a paper to a co-author or submitting it to a journal. With a bit of bad luck, it is also possible to have a corrupted auxiliary file that prevents you from creating a PDF from correct LaTeX sources; deleting some or all of the auxiliary files solves this problem.

Auxiliary file deletion is thus an infrequent, but necessary task. It would be possible to just ignore the issue and do the needed clean-up in either the Terminal or the Finder, but doing it by hand can be error prone. Also, it is simple to add to SEE. Overall, it seems worth doing.

In an earlier post, I used latexmk to trigger typesetting from SEE. With the right command line options, latexmk can also be used to clean up the auxiliary files! Adding a clean-up feature to SEE then just requires some relatively minor modifications to the scripts used for typesetting.

First, let's look at the shell script. It becomes:

#!/bin/sh

PATH=/usr/texbin:/usr/local/bin:$PATH
export PATH

cd "`dirname "$1"`"
latexmk -c "`basename "$1"`"

The only change is that latexmk is called with a -c option, so that it will clean up the auxiliary files, leaving the output PDF. I saved the script as cleanupaux.sh.

Second, we'll look at the AppleScript. It becomes:

tell application "SubEthaEdit"
    if exists path of front document then
        set latexFilePath to path of front document
        set modeResources to resource path of mode of front document
    else
        --Unsaved document, so LaTeX not run on it and can just return
        return
    end if
end tell

set cleanupScript to quote & modeResources & "/Scripts/shell/cleanupaux.sh" & quote & space & quote & latexFilePath & quote

do shell script cleanupScript

on seescriptsettings()
    return {displayName:"Clean Up Auxiliary Files"}
end seescriptsettings

Again, the changes are minimal. For typesetting, it mattered whether the front document was saved; here, it is only necessary to check whether the document has ever been saved. If it has, we run the shell script to clean up the auxiliary files, but if it hasn't, then LaTeX must not have been run on it so we just quietly do nothing.

The seescriptsettings handler is simplified, as well. All that remains is a displayName, which appears in the mode menu. There seems to be little reason to have either a keyboard shortcut or a toolbar item, since the task is infrequent.

The clean-up done by latexmk is fairly conservative. It only eliminates the auxiliary files from the specifically named document (in our case, the front document in SEE), and not from other files that are included. Further, it doesn't clean up absolutely everything, since different LaTeX packages can create files of their own which latexmk doesn't recognize. My feeling is that being conservative here is appropriate, as we don't want to accidentally delete something important while getting rid of the usual suspects.

Behaving Stupidly in Groups

I was only going to post the once concerning the MacUpdate promotion, but I just can't resist. The current state of the promotion provides such a beautiful illustration of how making good individual choices can lead to perverse large-scale behavior. Thinking it through also leads to such an interesting optimal strategy for purchasing and whether or not to recommend that others purchase the bundle. Keep in mind, this is coming from someone that has already described the bundle as a great deal.

Right now, the promotion can fairly be described as not going well. There are fewer than two days left of a nine-day promotion, and the number of sales is at about 3300. It looks like sales are going to limp along and just fail to reach the unlock point of 4000 sales for Intaglio. There was clearly an expectation of at least 10,000 sales, which is where the final application, TechTool Pro, would have been unlocked. I argued in the previous post that the entire system of unlocks is ill conceived, putting a strong disincentive on buying early, even though early sales are needed. As well, there originally was a system of invitations that benefitted not the sender of the invitation, but the recipient, providing further encouragement to wait before purchasing; the invites have since been fixed to reward the sender and recipient both.

To provide an example of where the problem comes in, consider Intaglio. People have already posted comments on the promotion site to the effect that they will buy once Intaglio is unlocked. How many people are waiting for that? The bundle could be viewed as getting Intaglio, a well-regarded program that costs $89, for $50 with a bunch of other applications thrown in for free. It must be tempting, but, really, why should they buy it without knowing for sure that Intaglio will be unlocked? If enough people make that decision, none of them get the program, even though together they could have provided the needed number of sales. The group behavior seems stupid, but the individual behavior is completely sensible.

Given the state of the sales, I think it worth considering the optimal strategy for those who have already bought the bundle, and for those who are thinking about buying the bundle. The optimal strategy for MacUpdate is indifferent to me; I see no reason to care, except in how it influences the benefits I will get from the promotion.

Let's focus first on those who, like me, have already bought the bundle. We'd like to get as much more added to the bundle as possible, since it's all just free for us at this point. At first, it looks like we should encourage people to buy the bundle, so as to unlock Intaglio, at least. However, Joel Mueller, one of the organizers of the promotion, posted (as MUeller) a comment on 13/7 at 4:04 pm:

We still haven't pulled out our big guns yet. We wanted the community to pull through first. Do not loose [sic] hope. We have every intent to unlock every app in this bundle.

Interesting claim! Those "big guns" must be really something, right? Let's assume that they are enough to cause enough sales to at least unlock Intaglio.

However, it does mean that there are some nice additions to the bundle which they're holding back. This in turn means that encouraging people to buy before those "big guns" are made available is pretty much a fool's game. We should encourage people to wait, hopefully getting both the "big guns" and unlocking Intaglio and TechTool Pro. In passing, I think Intaglio is quite possible, but doubt TechTool Pro will be unlocked.

The same thinking applies to those who have already decided to buy, but haven't yet: they should wait to be sure that MacUpdate makes the "big guns" available. The same goes for those who are waiting to buy until, e.g., Intaglio is unlocked: they should wait, both to be sure that Intagio does unlock and so as to get the "big guns".

I can see no reason for prospective buyers not to wait, at this point. There isn't a sufficient rate of sales to be at all sure that Intaglio will unlock; TechTool Pro is a pipe dream. There are some extras that haven't been offered yet. What appears to be the best case right now would be for sales to tank completely, pretty much forcing MacUpdate to offer some serious incentives with enough time left for Intaglio to unlock.

The combination of supposed incentives in the promotion are pretty tragic. Each one has actually been a disincentive to early purchasing, basically killing any chance of enough sales to unlock the "crown jewel" of TTP. I must admit that I'm puzzled how MacUpdate could have gotten all of this so wrong. Could they actually have believed all the marketing dross about "the Mac community" on the promotion web site? There really isn't a relevant community there, just potential customers who'll make decisions that are right for them, not for some mythical community. It is not exactly a secret that people make choices for individual benefit can lead to perversely contrary results in the large.

I will offer up a thought on what MacUpdate's strategy should be, although I won't put much effort into it. You want to sell as many as possible, so offer those "big guns," do it fast, and make clear that there's nothing else. So long as there is reason to think you're holding back, you're providing incentives to wait, killing sales to those who would buy if Intaglio were unlocked.

As a closing note, do remember this is from someone who's already bought the bundle and thus has every reason to hope for a successful promotion. If I'm concluding that it would really be best for sales to die off, and fast, then there's something quite wrong with the incentive structure.