Monday, December 17, 2007

If I Were Better With bash

There's an interesting question posed on the O'reilly FYI blog: what would you do if you were better at bash? There is a prize for the best response: the deadline is tomorrow, so hurry if you're interested!

I was a little surprised at the responses. The responses fit into two groups. The first group of responses is, roughly, doing specific projects on the job. The second group is about rounding out skill sets, without being overly specific about applications of that improved skill set; while I definitely support self-improvement and continued learning, the responses in the second group don't really seem to answer the question. What surprised me was that no one took the opportunity to dream a little and suggest something out of the ordinary. Is bash really so prosaic?

Here's what I submitted:
If I were better at bash, I'd write a book.

Every year, many graduate students in the sciences are confronted with the fact that they have to use their conceptual knowledge of their field to conduct original research. It's not easy. One needless difficulty is that they need to write programs to support their research, but without having learned about the tools available. Tools like revisions control systems, make, and shells like bash.

Their classmates and advisors generally don't know about those tools, either -- there is a cultural mismatch, and so there is no support. I'd write a book to provide that support and introduce those tools.

Basically, I'd write the book from which I could have so much benefitted when I was a physics graduate student about fifteen years ago.


What would you do?

Tuesday, December 4, 2007

I Guess That Answers It

Back in July, I posted twice about a promotion by MacUpdate. I first obliquely noted their curious incentive structure at the beginning of the promotion, questioning the basic idea. Towards the end of their promotion, I was considerably more direct and more critical.

To summarize, their promotion had a system of "unlocks," where additional -- and higher quality -- applications would be "unlocked" with sufficient sales and added to the promotion. This is, in short, a classically bad idea, because there must be early sales to add the extra applications and attract more customers, but, as a customer, you're better off waiting to make sure the extras are unlocked. Thus, the promotion either needs to be sufficiently attractive without the extra applications, or you need a bunch of (let's be positive) optimists to buy under the assumption that the best applications would in fact be reached.

The approach seemed like a bad idea to me at the start of the promotion, and the way the promotion played out only strengthened that. MacUpdate didn't actually reach their goals. They changed the targets and extended the promotion, so that all the applications were in fact provided. They really had to -- just imagine how poorly it would have reflected on both MacUpdate and the "premium" applications had they failed to unlock everything!

So, MacUpdate has another promotion. They've again got 10 applications, with 7 available at first and 3 more to be unlocked with sufficient sales. The target numbers for the unlocks are much more modest, which seems prudent. However, instead of having the most valuable application being unlocked last, they've gone and reversed it! That's right, the $300 Xmind Pro application is unlocked at 1000 sales, while the final unlock at 5000 sales gives the $45 PulpMotion application. Does this make sense? No, of course not. Really, what should we conclude from the structure? That PulpMotion is the crown jewel of the promotion? Isn't that tantamount to saying that Xmind is overpriced by an order of magnitude? (N.B., I have not used either Xmind or PulpMotion.)

It is obviously an attempt to ensure success, but I think that MacUpdate has drawn entirely the wrong conclusion. Wouldn't a simpler approach just have been to offer all ten applications from the beginning? If nothing else, it would have avoided the absurdly contrived bonus structure that they've produced.

My key point in the posts from July was put in the form of a question: is this really a good idea in the long run? I think MacUpdate has, unintentionally, provided a clear answer: no.

Sales, Sales, Sales!

There sure seem to be a lot of sales on Mac software this month. I don't recall quite so many last year. I've already mentioned MacSanta. There are also Give Good Food to Your Mac and a MacUpdate promotion. Are there others?

Of the three, Give Good Food to Your Mac is definitely the most appealing for me. It allows you to decide on your own bundle of applications, with a discount based on the number of applications. Their approach does lead to some oddities, though. For example, you should never buy between 7 and 9 applications, because it is always cheaper to add inexpensive applications and get 10. Even if you'd never use them, you still pay less, and, hey, I would use, e.g., CoverScout, even if I'd never buy it on its own merits.

MacSanta is more traditional in its structure. Unfortunately. New applications are announced every day, with a 20% discount possible for that day. Later in the month, you can still get the applications, but with only a 10% discount. It's a very annoying set-up. I really can't picture buying anything from it at the 10% level, but the 20% level seems only relevant for something that I'd decided to get anyway. Of course, if I've decided to get an application, I've probably already bought it! That said, I have already bought ShutterBug and may well get MarsEdit, too, so they are definitely doing something right. I guess it works to push me off the fence…

The MacUpdate promotion looks to be a good value, if it is offering applications you want. Which it is not, for me.

Update: There is one other point about Give Good Food to Your Mac that I'd wanted to mention. In their FAQ, there is a clear and unambiguous statement that all licenses purchased are normal licenses, with no restrictions. This is the way it should be done, in my opinion.

In contrast, MacUpdate has "full licenses with normal upgrade paths" except for "Swift Publisher, which will be a paid upgrade." Seriously, why?

MacSanta is structured differently, so the issue doesn't really come up there, as far as I can see. You just get a discount code that you use directly on the developers' sites, rather than a bundle through a third party.

Trying Something New…

I see on the MacSanta site that MarsEdit is on sale for 20% off today. Now, while I'm not above complaining, I haven't really commented on my general dissatisfaction with the editing tools for Blogger. In short, I don't care for the tools: even if they worked properly in Safari (and they don't), they'd still be a typically disappointing web application.

So, the next few posts will be done with MarsEdit—let's hope that is a good thing. They will also be quite a bit lighter in content than my usual posting style—and that's certainly a good thing!

Update: Well, that was a nice start. Composing and uploading went just fine. This update is essentially just a way to test out editing an existing post.

Update 2: Editing an existing post went fine. Editing again to try adding some tags. As a whole, this seems to be working very well, even to the point where one could just work in MarsEdit and assume that it gets to Blogger correctly.

Sunday, November 4, 2007

SEEing LaTeX 14: General Use of Environments

We've now seen how to define and update an environment for SubEthaEdit. The approach is modeled on how Mac OS X applications store their preferences; effectively, I used application preferences as a design pattern. To demonstrate the general applicability of the approach, let's apply it to some additional scripts; specifically, let's revisit viewing the PDF compiled from a LaTeX document and cleaning up the auxiliary files that LaTeX produces. The short version is that the approach works smoothly in both cases, with minimal differences in the AppleScripts used to add behavior to SEE. The long version follows, including the scripts to actually implement it.

First, let's look at viewing the compilation product. Previously, I'd just used the LaTeX file name to define the PDF file name, and sent it to PDFView using AppleScript. With the new approach, I need a shell script defining the default behavior, and an AppleScript invoking the shell script from SEE. The shell script is:
PATH="$PATH:/usr/texbin:/usr/local/bin"
export PATH

VIEWER=${SEE_LATEX_VIEWER:-'open "$PRODUCT"'}
PRODUCT_TYPE="${SEE_LATEX_PRODUCT_TYPE:-pdf}"

FILE="$(basename "$1")"
DIRNAME="$(dirname "$1")"
LINE="$2"
PRODUCT="$(basename "$1" .tex).$PRODUCT_TYPE"

cd "$DIRNAME"
if [ -s "$PRODUCT" ]
then
    eval $VIEWER
fi
Note that the new definition nowhere assumes that we will produce a PDF file as output; it could be used with latex to produce a DVI, for instance.
The AppleScript is:
tell application "SubEthaEdit"
    if exists path of front document then
        set filePath to path of front document
        set lineNumber to startLineNumber of selection of front document
        set activeMode to mode of front document
        set modeResources to resource path of activeMode
    else
        error "You have to save the document first"
    end if
end tell


set viewScript to prependEnvironment for activeMode onto (join of {quotedForm for (modeResources & "/bin/viewproduct.sh"), quotedForm for filePath, lineNumber} by space)

do shell script viewScript


-- SubEthaEdit settings

on seescriptsettings()
    {displayName:"View", shortDisplayName:"View", keyboardShortcut:"^~@o", toolbarIcon:"ToolbarIconRun", inDefaultToolbar:"yes", toolbarTooltip:"View current document in external viewer", inContextMenu:"no"}
end seescriptsettings

on join of tokenList by delimiter
    set oldTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to delimiter
    set joinedString to tokenList as string
    set text item delimiters of AppleScript to oldTIDs
    return joinedString
end join

on quotedForm for baseString  
    quote & baseString & quote
end quotedForm

to prependEnvironment for seeMode onto scriptString
    set envFilePath to (path to preferences from user domain as string) & "de.codingmonkeys.SubEthaEdit." & (name of seeMode) & "_environment.plist"
    (readEnvironment out of envFilePath) & scriptString
end prependEnvironment

to readEnvironment out of plist
    readListPair out of plist
    environmentString from result
end readEnvironment

to readListPair out of plist
    tell application "System Events"
        if exists file plist then
            tell property list file plist
                get {name, value} of every property list item
            end tell
        else
            {{}, {}}
        end if
    end tell
end readPlist

on environmentString from keyValueListPair
    set {plistKeys, plistValues} to keyValueListPair
    set accumulator to {}
    set oldTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to ""
    repeat with i from 1 to number of items in plistKeys
        set tokens to {"export ", item i of plistKeys, "=", item i of plistValues, ";"}
        copy (tokens as string) to the end of the accumulator
    end repeat
    set AppleScript's text item delimiters to space
    set envString to accumulator as string
    set AppleScript's text item delimiters to oldTIDs
    envString
end environmentString


The shell script uses the same SEE_LATEX_VIEWER environment variable used for compiling; I'll adapt the compilation script a little to allow separate viewing behavior for the two cases, defaulting to both using the SEE_LATEX_VIEWER contents. Essentially, this consists of changing just one line, replacing
VIEWER=${SEE_LATEX_VIEWER:-'open "$PRODUCT"'}
with
VIEWER=${SEE_LATEX_COMPILEVIEWER:-${SEE_LATEX_VIEWER:-'open "$PRODUCT"'}}
Note that the AppleScript uses the same code to read from the same plist of environment settings as the compilation script--no changes were needed to accommodate the new settings.

Second, let's examine cleaning up the auxiliary files. The shell script is:
PATH="$PATH:/usr/texbin:/usr/local/bin"
export PATH

CLEANUP=${SEE_LATEX_CLEANUP:-'rm -f $(basename "$FILE" .tex).{aux,bbl,blg,dvi,log,out,ps,pdf,pdfsync,toc}'}

FILE="$(basename "$1")"
DIRNAME="$(dirname "$1")"
PRODUCT="$(basename "$1" .tex).$PRODUCT_TYPE"

cd "$DIRNAME"
eval $CLEANUP
The cleanup behavior can be defined in a SEE_LATEX_CLEANUP variable, used to set the CLEANUP variable. The default value for CLEANUP is to remove files with the same name as the LaTeX file but with different filename extensions. The list of extensions (aux, bbl, blg, dvi, log, out, ps, pdf, pdfsync, toc) is pretty arbitrary, being essentially what were created for my own writings.

The associated AppleScript is:
tell application "SubEthaEdit"
    if exists path of front document then
        set filePath to path of front document
        set activeMode to mode of front document
        set modeResources to resource path of activeMode
    else
        --Unsaved document, so LaTeX not run on it and can just return
        return
    end if
end tell

set cleanupScript to prependEnvironment for activeMode onto (join of {quotedForm for (modeResources & "/bin/cleanupaux.sh"), quotedForm for filePath} by space)

do shell script cleanupScript

on seescriptsettings()
    return {displayName:"Clean Up Auxiliary Files"}
end seescriptsettings

on join of tokenList by delimiter
    set oldTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to delimiter
    set joinedString to tokenList as string
    set text item delimiters of AppleScript to oldTIDs
    return joinedString
end join

on quotedForm for baseString  
    quote & baseString & quote
end quotedForm

to prependEnvironment for seeMode onto scriptString
    set envFilePath to (path to preferences from user domain as string) & "de.codingmonkeys.SubEthaEdit." & (name of seeMode) & "_environment.plist"
    (readEnvironment out of envFilePath) & scriptString
end prependEnvironment

to readEnvironment out of plist
    readListPair out of plist
    environmentString from result
end readEnvironment

to readListPair out of plist
    tell application "System Events"
        if exists file plist then
            tell property list file plist
                get {name, value} of every property list item
            end tell
        else
            {{}, {}}
        end if
    end tell
end readPlist

on environmentString from keyValueListPair
    set {plistKeys, plistValues} to keyValueListPair
    set accumulator to {}
    set oldTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to ""
    repeat with i from 1 to number of items in plistKeys
        set tokens to {"export ", item i of plistKeys, "=", item i of plistValues, ";"}
        copy (tokens as string) to the end of the accumulator
    end repeat
    set AppleScript's text item delimiters to space
    set envString to accumulator as string
    set AppleScript's text item delimiters to oldTIDs
    envString
end environmentString
Again, the bulk of the script is unchanged, with no changes at all to the portions handling the environment settings.

Sunday, October 14, 2007

SEEing LaTeX 13: Accessing the Environment

In the preceding post, I showed a way to use a property list file to define the environment for shell scripts run from within a SubEthaEdit mode. For convenience, we can also use AppleScript to add an item to the mode menu that opens up the appropriate plist for the mode. I'll not discuss the AppleScript at any great length, but instead just give the gist of it.

The basic idea is that we can just open up the plist in whatever application happens to be appropriate; probably, but not certainly, that is the Property List Editor. There are a few ways to accomplish that. I decided, more or less arbitrarily, to use System Events to open the file.

However, we do need to deal with the case when there is no environment property list file. What I did was to put a string representing an empty plist onto the clipboard, and then run a shell script that uses pbpaste to dump that string to an appropriately named file in the preferences folder. This was actually the second approach I took. The first I tried was to just write the file directly using AppleScript, but this associated the resulting file with TextEdit, an undesirable choice for working with a plist. I didn't explore the reason, but assume that AppleScript defines (inappropriate) type and creator codes for the file.

One other odd thing I encountered was that SEE doesn't show an ellipsis "…" correctly in the menu. I just used three dots. Whatever.

Here's the AppleScript:
on seescriptsettings()
    return {displayName:"Mode Environment...", shortDisplayName:"Environment", inContextMenu:"no"}
end seescriptsettings

tell application "SubEthaEdit" to set activeMode to the mode of the front document
openEnvironmentSettings for activeMode

to openEnvironmentSettings for mode
    set envFilePath to (path to preferences from user domain as string) & "de.codingmonkeys.SubEthaEdit." & (name of mode) & "_environment.plist"
    tell application "System Events"
        if not exists file envFilePath
            my writeDefaultEnvironment at envFilePath
        end if
        open file envFilePath
    end tell
end openEnvironment

to writeDefaultEnvironment at envPath
    set savedClipboard to the clipboard
    set the clipboard to "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<!DOCTYPE plist PUBLIC \"-//Apple Computer//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">
<plist version=\"1.0\">
<dict/>
</plist>"
    do shell script "pbpaste > " & (POSIX path of envPath)
    set the clipboard to the savedClipboard
end writeDefaultEnvironment

SEEing LaTeX 12: Setting the Environment

Last time, I reworked the shell script and AppleScript that invokes LaTeX compilation from within SubEthaEdit. In some sense, it was pointless: instead of having a fixed method for compilation defined in a shell script, I now run a flexible shell script within an environment fixed by the AppleScript. The relevant portion of the AppleScript is the prependEnvironment handler:
to prependEnvironment onto scriptString
    "export SEE_LATEX_COMPILER='latexmk -pdf -quiet \"$FILE\"'; export SEE_LATEX_PRODUCT_TYPE=pdf; export SEE_LATEX_VIEWER='/Applications/Skim.app/Contents/SharedSupport/displayline \"$LINE\" \"$PRODUCT\" \"$FILE\"';" & scriptString
end prependEnvironment

The handler abstracts away the details of how the environment is constructed from the rest of the AppleScript. We can thus just focus on the internals of the handler, without worrying about how the rest of the AppleScript will be affected. Put another way, we'll replace the string defining the environment by a function call that returns the string.

The approach I'll take will be to store the environment settings in a property list file, read them into lists representing variables and values, and format the list contents appropriately. This is pretty easy, thanks to the property list suite in System Events. I'll encapsulate reading the environment into a handler of its own:
to readEnvironment out of plist
    readListPair out of plist
    environmentString from result
end readEnvironment

The plist parameter is the path to the property list file containing our environment.

With readEnvironment, the prependEnvironment handler is pretty straightforward. We just define the path to the environment file and let readEnvironment do the work. All that remains is to decide where to store the environment settings. The Preferences folder seems like a natural choice, so let's use a file called de.codingmonkeys.SubEthaEdit.LaTeX_environment.plist, which is similar to how the LaTeX mode settings are treated in TextMate. The handler becomes:
to prependEnvironment onto scriptString
    set envFilePath to (path to preferences from user domain as string) & "de.codingmonkeys.SubEthaEdit.LaTeX_environment.plist"
    (readEnvironment out of envFilePath) & scriptString
end prependEnvironment

To be clear, it is not necessary to have the plist file present at all, since we defined our shell script to use default values when no environment variables are set. We'll handle the case of an absent environment file below.

We now need to provide readListPair and environmentString handlers. The former draws on the property list suite of System Events, returning two lists of equal length. The first list contains the environment variable names, while the second contains the corresponding values. The handler is complicated a bit by checking whether the plist exists, but has a single get at its core:
to readListPair out of plist
    tell application "System Events"
        if exists file plist then
            tell property list file plist
                get {name, value} of every property list item
            end tell
        else
            {{}, {}}
        end if
    end tell
end readPlist



To format the list contents into an appropriate string, we define the environmentString handler:
on environmentString from keyValueListPair
    set {plistKeys, plistValues} to keyValueListPair
    set accumulator to {}
    set oldTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to ""
    repeat with i from 1 to number of items in plistKeys
        set tokens to {"export ", item i of plistKeys, "=", item i of plistValues, ";"}
        copy (tokens as string) to the end of the accumulator
    end repeat
    set AppleScript's text item delimiters to space
    set envString to accumulator as string
    set AppleScript's text item delimiters to oldTIDs
    envString
end environmentString

This is a straightforward, ugly function that just iterates through the lists, formatting the contents. Each variable-value pair is turned into a shell-style export statement and accumulated in a list. The accumulated export statements are then joined together into one big string that defines the environment.

To test all of this out, it is easier to just call the base prependEnvironment handler, instead of working through a SEE mode. I used prependEnvironment onto "buildScript", and ran it with the environment plist both present and absent. It works as expected, so could be simply dropped into the LaTeX mode bundle. However, I prefer to make another, relatively minor change that requires an extra parameter for the prependEnvironment handler. Specifically, I will pass in the active mode from SubEthaEdit:
to prependEnvironment for seeMode onto scriptString
    set envFilePath to (path to preferences from user domain as string) & "de.codingmonkeys.SubEthaEdit." & (name of seeMode) & "_environment.plist"
    (readEnvironment out of envFilePath) & scriptString
end prependEnvironment

Later extensions could make use of more properties of the mode, so I passed in the entire mode, instead of just the name. The advantage of passing the mode as a parameter is that all of the stuff I've shown in this post can be used without change for other scripts and for other modes. It will only be necessary to change the line defining the string holding the shell script and to make appropriate redefinitions of the seescriptsettings handler.

Next time, I'll take a look at how to easily access and make changes to the environment.

For completeness, here's the entire AppleScript:
tell application "SubEthaEdit"
    if exists path of front document then
        if modified of front document then
            try
                save front document
            end try
        end if
        set filePath to path of front document
        set lineNumber to startLineNumber of selection of front document
        set activeMode to mode of front document
        set modeResources to resource path of activeMode
    else
        error "You have to save the document first"
    end if
end tell

set buildScript to prependEnvironment for activeMode onto (join of {quotedForm for (modeResources & "/Scripts/shell/buildlatex.sh"), quotedForm for filePath, lineNumber} by space)

do shell script buildScript

on seescriptsettings()
    return {displayName:"Typeset and View PDF", shortDisplayName:"Typeset", keyboardShortcut:"@b", toolbarIcon:"ToolbarIconBuildAndRun", inDefaultToolbar:"yes", toolbarTooltip:"Typeset and view the current document", inContextMenu:"no"}
end seescriptsettings

on join of tokenList by delimiter
    set oldTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to delimiter
    set joinedString to tokenList as string
    set text item delimiters of AppleScript to oldTIDs
    return joinedString
end join

on quotedForm for baseString  
    quote & baseString & quote
end quotedForm

to prependEnvironment for seeMode onto scriptString
    set envFilePath to (path to preferences from user domain as string) & "de.codingmonkeys.SubEthaEdit." & (name of seeMode) & "_environment.plist"
    (readEnvironment out of envFilePath) & scriptString
end prependEnvironment

to readEnvironment out of plist
    readListPair out of plist
    environmentString from result
end readEnvironment

to readListPair out of plist
    tell application "System Events"
        if exists file plist then
            tell property list file plist
                get {name, value} of every property list item
            end tell
        else
            {{}, {}}
        end if
    end tell
end readPlist

on environmentString from keyValueListPair
    set {plistKeys, plistValues} to keyValueListPair
    set accumulator to {}
    set oldTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to ""
    repeat with i from 1 to number of items in plistKeys
        set tokens to {"export ", item i of plistKeys, "=", item i of plistValues, ";"}
        copy (tokens as string) to the end of the accumulator
    end repeat
    set AppleScript's text item delimiters to space
    set envString to accumulator as string
    set AppleScript's text item delimiters to oldTIDs
    envString
end environmentString