Monday, January 7, 2008

SEEing LaTeX 25: Putting Things in Order

As noted below, the entries in the LaTeX mode menu for SubEthaEdit need to be put into a sensible order. After some thinking, I decided that there are really three groups of scripts: scripts for interacting with the LaTeX system, scripts for simplifying typing, and a script for interacting with the LaTeX mode itself.

Ideally, we'd put the three into groups divided by a horizontal rule. Unfortunately, SEE does not (yet) allow dividing lines to be inserted into the menu. Regardless, let's organize the scripts into the three groups, and - with one exception- just alphabetize within the groups. The exception is for the "Typeset and View" menu item; it strikes me as natural to put that first in the list.

End result is a menu ordered as:

Typeset and View
Clean Up Auxiliary Files
Run BibTeX
View

Complete Citation
Inline Math
Insert Environment...
Un/Comment Selected Lines

Customize Mode...

Although I've broken the groups apart with spacing, they'll just run together in the menu. Perhaps a later version of SEE will allow some more structure to be added.

One final change is that I've renamed the "Mode Environment..." menu item to "Customize Mode...". The two distinct meanings of "environment" struck me as confusing, and "Insert Environment..." is simply too appropriate for LaTeX to change.

I'll add another entry to the menu as soon as possible. It will go into the third group, with a title something like "Mode Help". I still have to write the mode help, first.

Sunday, January 6, 2008

SEEing LaTeX 24: For Completeness, BibTeX

I guess the right solution is to add a menu item for running bibtex, and that's it. Since I don't use makeindex myself, I'll leave that aside, unless someone wants to contribute scripts or just examples of use. I'd guess that the scripts would be easy enough, just adapt the ones I'll present for BibTeX.

Here's the AppleScript:
checkSaveStatus without updating
set bibScript to join of {modeEnvironment(), quotedForm for "$SEE_MODE_RESOURCES/bin/runbibtex.sh", quotedForm for documentPath()} by space
do shell script bibScript

on seescriptsettings()
    return {displayName:"Run BibTeX"}
end seescriptsettings

include(`SubEthaEditTools.applescript')


And here's the shell script that it calls:
#!/bin/sh

#$Id: runbibtex.sh,v 1.1 2008/01/06 19:09:49 mjb Exp $

PATH="$PATH:/usr/texbin:/usr/local/bin"
export PATH

BIBTEX=${SEE_BIBTEX:-'bibtex "$(basename $FILE .tex)"'}
FILE="$(basename "$1")"
DIRNAME="$(dirname "$1")"

cd "$DIRNAME"
eval $BIBTEX

SEEing LaTeX 23: Am I Done?

Last July, I listed some desirable features for the LaTeX mode for SubEthaEdit. The mentioned features were integrating with a PDF viewer using pdfsync, enabling insertion of citation keys in bibtex format, allow typesetting by calling pdflatex from within SEE, cleaning up auxiliary files, and commenting out selected lines. I also mentioned that it might be nice, if inessential, to be able to insert environments and formatting.

I've accomplished all of that. As well, I have introduced a mechanism for customizing the shell script environment for the mode and have developed a set of AppleScript handlers useful for scripting SEE. I'm quite pleased with how all of that has worked out. Not only do I now have a LaTeX mode that covers my main needs, but I've got a solid foundation on which I - and hopefully others! - can build support scripts for other modes.

That said, the LaTeX mode is not quite finished. It's clear that I should write some documentation, and add a "Mode Help" menu item. I should also better document SubEthaEditTools. Beyond that, there are two additional tasks.

First, the scripts in the LaTeX mode menu should be put into some sort of reasonable order, rather than the haphazard order that currently is there. The items appear in order based on the names of the scripts, which can differ from the entry in the menu. What is the right order to use?

Second, I've omitted some important elements of a LaTeX system, because latexmk handles them for me. For example, I have not provided a way to run bibtex. Should additional elements be added? If so, which? bibtex? makeindex? Something else?

Saturday, January 5, 2008

SEEing LaTeX 22: Inline Math

Environments aren't the only thing common LaTeX constructs that are awkward to type. The delimiters for inline math are pretty awkward, too. Let's add those, too:
set mathText to selectionText()
set wrappedText to "\\( " & mathText & " \\)"
setSelectionText to wrappedText
if (length of mathText) equals 0 then
    set {startChar, nextChar} to selectionRange without extendingFront or extendingEnd
    setSelectionRange to (startChar + length of mathText + 3)
end if

on seescriptsettings()
    {displayName:"Inline Math", keyboardShortcut:"@~^m", inContextMenu:"yes"}
end seescriptsettings

include(`SubEthaEditTools.applescript')

SEEing LaTeX 21: Inserting Environments

Environments again? Yes, but this time we're going to look at environments in LaTeX, not the shell environment. Environments are a bit of a pain to type, but the repetitive structure makes them suitable to automation: we just get the name of the environment, and then have a more-or-less standard form:
\begin{environmentname}
    body
\end{environmentname}

Indentation of the body can be a little unclear. For example, I usually indent equation environments, but would never dream of indenting the document environment.

To add environment insertion into the LaTeX mode for SubEthaEdit, we begin by getting the environment name using display dialog:
try
    display dialog "Enter environment name:" with title "Insert Environment" default answer "equation"
on error number -128 -- user canceled
    return
end try
set envName to text returned of result

The try block is to handle when the user cancels instead of entering and environment name. Somewhat arbitrarily, I set the default answer to be "equation", since my guess is that equations are probably the most common environment.

After that, there is just some fiddly work getting the formatting of the environment correct. It needs to intelligently insert newlines and tabs to keep the document readable. As well, something needs to be done with the selection text. There are two possibilities as I see it: (1) treat the selection as the name of the environment and (2) treat the selection as the body of the environment. I went with the latter. Finally, it would be nice to place the insertion point somewhere reasonable; I think it works nicely at the end of the body, especially since the insertion point is positioned to start typing immediately if the body is empty. Putting it all together, we have:
set {startSelected, nextSelected} to selectionRange without extendingFront or extendingEnd
set {startExtended, nextExtended} to selectionRange with extendingFront and extendingEnd

set prefix to selectByComparing(startSelected, startExtended, "", "\n")
set suffix to selectByComparing(nextSelected, nextExtended, "", "\n")
set indent to selectByComparing(startSelected, nextSelected, "\t", "")

set beforeInsertion to (join of {prefix, "\\begin{", envName, "}\n", indent, selectionText()} by "")
set afterInsertion to (join of {suffix, "\\end{", envName, "}\n" } by "")

setSelectionText to (beforeInsertion & afterInsertion)
setSelectionRange to startSelected + (count of beforeInsertion) - 1 + (count of suffix)
Note that I've made repeated use of a convenience function:
to selectByComparing(val1, val2, sameVal, diffVal)
    if val1 equals val2 then
        sameVal
    else
        diffVal
    end if
end selectByComparing

The selectByComparing handler is not part of SubEthaEditTools - maybe it should be?

That's all there is to it, apart from the boilerplate:
on seescriptsettings()
    {displayName:"Insert Environment...", keyboardShortcut:"@^e", inContextMenu:"yes"}
end seescriptsettings

include(`SubEthaEditTools.applescript')



Update: There is an interesting possibility for adjusting the default environment. I had used "equation" as the default, but it was pretty arbitrary. Another approach would be to just repeat whichever environment was last given. We just add a property to the script that holds the default environment, use its value in making the dialog, and update the property based on the dialog result:
property defaultEnvironment: "equation"

try
    display dialog "Enter environment name:" with title "Insert Environment" default answer defaultEnvironment
on error number -128 -- user canceled
    return
end try
set envName to text returned of result
set defaultEnvironment to envName

Is this actually a good idea? or will it just be annoying? Hard to say without using it, so I guess I'll try it out for a while.

Friday, January 4, 2008

SEEing LaTeX 20: Getting Citations Right

I started enhancing the LaTeX mode for SubEthaEdit by looking at citations. The original approach, based on just using the input manager supplied with BibDesk, proved to be unsatisfactory. Let's fix that now.

The general approach seems clear enough. We need to determine a search term based on the cursor position in the LaTeX document, pass that to BibDesk to get matching documents, let the user pick which documents are relevant, format the selected documents, and insert the result into the document. Using SubEthaEditTools our SEE scripting abstraction layer, it proves to be fairly straightforward.

The first step, determining the search term, is probably the trickiest. What I envision is that the user can enter a partial citation and have it finished by the script. Thus, we'll need to examine the text immediately before the insertion point and determine a partial citation key. We should not cross an open brace "{", as we don't want to include the macro. Additionally, we shouldn't cross a comma, since we might be looking at multiple citations within one macro. Let's not check the calling macro; this allows the completion to be invoked at inappropriate points, but also allows the completion to be invoked for user-defined macros.

One final issue is what we do when there is a range of text selected. Since the default behavior should be to have no text selected, we should treat a non-empty selection as meaningful. Let's take it to mean that the search should be constrained to only include the selected text. Therefore, we determine a search term based either on the selected text or all the text on the line preceding the insertion point.

Putting all that together, I came up with:
set {startChar, nextChar} to selectionRange without extendingFront or extendingEnd
if startChar equals nextChar then
    -- empty selection, try the whole line
    set selectionContents to extendedSelectionText with extendingFront without extendingEnd
else
    set selectionContents to selectionText()
end if
set macroArgument to the last item of the (tokens of the selectionContents between "{")
set searchTerm to the last item of the (tokens of the macroArgument between ",")

I've used a couple of the new handlers from SubEthaEditTools. These are hopefully self-explanatory, but check the implementation in case they are not.

Using the two tokens calls, we obtain a partial citation key. We pass that to BibDesk:
tell application "BibDesk"
    set citeMatches to search for searchTerm with for completion
end tell

The ungrammatical with for completion will give us a list of cite keys, not just document titles. Each completion is given as a string containing both the cite key and some document information, separated by a " % " string.

We next present the list of completions to the user, in order to narrow the list down to just the appropriate citations. To present a list, we use choose from list from the AppleScript StandardAdditions. There are three cases worth considering. First, if there is only one publication, we can select it by default, so the user just confirms whether it is correct. Second, if there are multiple possible publications, we just show the list, declining to guess. Finally, if there aren't any matches, we just inform the user with display alert; in this case, we'll set the list of publications to the empty list. If there aren't any publications returned, that means the user canceled, so we should just exit and leave the document unchanged. Putting it all together, we arrive at:
if (count of citeMatches) equals 1 then
    choose from list citeMatches with title "Citation Matches" with prompt "One matching publication:" default items citeMatches
    set pubs to result
else if (count of citeMatches) > 1
    choose from list citeMatches with title "Citation Matches" with prompt "Please select publications:" with multiple selections allowed
    set pubs to result
else
    display alert "No matches found for partial citation \"" & searchTerm & "\""
    set pubs to {}
end if

if (count of pubs) equals 0 then
    -- user canceled, do nothing
    return
end


We now have a non-empty list of completions. We need to split those apart to get the cite keys, then join the cite keys with commas. I used awk:
set citation to shellTransform of (join of pubs by "\n") for "" through "awk -F' % ' 'NR == 1 { printf(\"%s\", $1) } NR > 1 { printf(\",%s\", $1) }'" without alteringLineEndings


Pretty ugly! Let's reformat the core of the awk program to make it clearer:
NR == 1 { printf("%s", $1) }
NR > 1 { printf(",%s",
$1) }

In this form, it's clear enough, keeping in mind that we use " % " as the field separator: we just print out the first field, corresponding to the cite key, with the first record treated specially to get the number of commas right.

Finally, we insert the formatted citation back into the document. I also move the insertion point to the end of the formatted citation, as a typing convenience. I had considered closing the brace for the citation macro, but that would make multi-citation lists more awkward, so decided against it. This is none too difficult:
setSelectionRange to {nextChar - (length of searchTerm), nextChar - 1}
setSelectionText to citation
setSelectionRange to (nextChar - (length of searchTerm) + (length of citation))

Again, I've used new handlers from the SubEthaEditTools library.

Putting it all together, and adding a seescriptsettings handler to integrate it into SEE, we get:
-- $Id: BibDeskCompletions.applescript,v 1.2 2008/01/04 18:40:16 mjb Exp mjb $

(*
Need to figure out the search term. Treat a selection as meaning to constrain the search
term to lie within the selection, and an empty selection as meaning to get the search term
from the preceding text on the line. We don't cross an opening brace, so that the search term
comes from a call to a macro. However, we don't check to see if the macro is one of the
standard citation macros, since we do want to allow user macros.
*)

set {startChar, nextChar} to selectionRange without extendingFront or extendingEnd
if startChar equals nextChar then
    -- empty selection, try the whole line
    set selectionContents to extendedSelectionText with extendingFront without extendingEnd
else
    set selectionContents to selectionText()
end if
set macroArgument to the last item of the (tokens of the selectionContents between "{")
set searchTerm to the last item of the (tokens of the macroArgument between ",")

tell application "BibDesk"
    set citeMatches to search for searchTerm with for completion
end tell

-- get list of publications, customizing user interaction based on number of matches
if (count of citeMatches) equals 1 then
    choose from list citeMatches with title "Citation Matches" with prompt "One matching publication:" default items citeMatches
    set pubs to result
else if (count of citeMatches) > 1
    choose from list citeMatches with title "Citation Matches" with prompt "Please select publications:" with multiple selections allowed
    set pubs to result
else
    display alert "No matches found for partial citation \"" & searchTerm & "\""
    set pubs to {}
end if

if (count of pubs) equals 0 then
    -- user canceled, do nothing
    return
end

(*
At this point, there is a non-empty list of matches, which replaces the search term. By
construction, the search term always immediately precedes the end of the selection.
Call out to the shell to format the publication list into a LaTeX citation, insert the citation,
and then move the insertion point just after the citation.
*)

set citation to shellTransform of (join of pubs by "\n") for "" through "awk -F' % ' 'NR == 1 { printf(\"%s\", $1) } NR > 1 { printf(\",%s\", $1) }'" without alteringLineEndings
setSelectionRange to {nextChar - (length of searchTerm), nextChar - 1}
setSelectionText to citation
setSelectionRange to (nextChar - (length of searchTerm) + (length of citation))

on seescriptsettings()
    {displayName:"Complete Citation", shortDisplayName:"Citation", keyboardShortcut:"@^j",  toolbarIcon:"ToolbarBibDesk.png", inDefaultToolbar:"yes", toolbarTooltip:"Complete citation using BibDesk", inContextMenu:"yes"}
end seescriptsettings

include(`SubEthaEditTools.applescript')

SEEing LaTeX 19: SubEthaEditTools, or, Eating My Own Dogfood

Let's take the arguments in the last three posts seriously. We move all of the handlers developed in the course of the SEEing LaTeX series into a separate file called SubEthaEditTools.applescript, and rewrite all of the AppleScripts to include the file using m4. While we're at it, we also add some more handlers to the SubEthaEditTools to try to make a more complete set, with the goal being to write scripts for the mode by using the handlers without directly interacting with SubEthaEdit in the AppleScript.

This requires relatively minor changes to the makefile I've been using, but nothing too serious. Also, I construct SubEthaEditTools itself using m4, with more general purpose handlers for, e.g., string manipulation placed in their own files. With the more modular structure, some additional minor rewrites of some handlers seems appropriate, which do not warrant specific comment.

A concept that is made explicit in the SubEthaEditTools is that of the extended selection. That is, the selection modified either so that the beginning is extended to the start of the first line of the selection, or to the end of the last line or the selection, or both. The extended selection has appeared implicitly several times, so it seems worthwhile to make it explicit. Further, there are now handlers both for extending the selection (forward or backward) and for referring to the extended selection without modifying the actual selection.

Update: I've replaced the quotedForm handler with a doubleQuotedForm handler, to help prevent confusion with the quoted form of action for AppleScript strings. I've also worked in a few usages of quoted form of in the various scripts for the LaTeX mode (I didn't know about quoted form of until recently).

Also, I've added a license. It's an MIT-style license, so should be suitably permissive for use by others.

Update 2 (2008/04/23): I've added a few more handlers. Specifically, modeSetting, selectionIsEmpty, and documentIsAvailable.

-- SubEthaEdit Tools

(*
Copyright (c) 2008, Michael J. Barber

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject
to the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR
ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

*)



-- Environment management

on modeEnvironment()
    join of {"export __CF_USER_TEXT_ENCODING=0x1F5:0x8000100:0x8000100;", "export SEE_MODE_RESOURCES=", doubleQuotedForm for modeResources(), "; ", readEnvironment out of environmentFilePath()} by ""
end modeEnvironment

on modeSetting for envVar
    try
        tell application "System Events"
            tell property list file (my environmentFilePath())
                get value of the property list item envVar
            end tell
        end tell
    on error
        missing value
    end try
end modeEnvironmentSetting

to openEnvironmentSettings()
    openEnvironment at environmentFilePath() with settingDefaultEnvironment
end openEnvironment

on environmentFilePath()
    tell application "SubEthaEdit" to set modeName to name of the mode of the front document
     join of {path to preferences from user domain as string, "de.codingmonkeys.SubEthaEdit.", modeName, "_environment.plist"} by ""
end environmentFileName

-- Manipulation of document text

on documentText()
    tell the front document of application "SubEthaEdit" to get the contents
end documentText

on selectionIsEmpty()
    tell the front document of application "SubEthaEdit" to get the length of the selection
    result is equal to 0
end selectionIsEmpty

to completeSelectedLines()
    extendSelection with extendingFront and extendingEnd
end completeSelectedLines

on selectionText()
    tell the front document of application "SubEthaEdit" to get the contents of the selection
end selectionText

to setSelectionText to newText
    tell application "SubEthaEdit" to set the contents of the selection of the front document to the newText
end setSelectionText

on selectionRange given extendingFront:shouldExtendFront, extendingEnd:shouldExtendEnd
    tell the front document of application "SubEthaEdit"
        if shouldExtendFront and shouldExtendEnd then
            get {startCharacterIndex of the first paragraph of the selection, nextCharacterIndex of the last paragraph of the selection}
        else if shouldExtendFront then
            get {startCharacterIndex of the first paragraph of the selection, nextCharacterIndex of the selection}
        else if shouldExtendEnd then
            get {startCharacterIndex of the selection, nextCharacterIndex of the last paragraph of the selection}
        else
            get {startCharacterIndex of the selection, nextCharacterIndex of the selection}
        end if
    end tell
end selectionRange

to setSelectionRange to newRange
    tell the front document of application "SubEthaEdit"
        set selection to newRange
    end tell
end setSelectionRange

on extendedSelectionText given extendingFront:shouldExtendFront, extendingEnd:shouldExtendEnd
    set {startChar, nextChar} to selectionRange given extendingFront:shouldExtendFront, extendingEnd:shouldExtendEnd
    tell the front document of application "SubEthaEdit"
        get the contents of characters startChar through (nextChar - 1) as text
    end tell
end extendedSelectionText

to extendSelection given extendingFront:shouldExtendFront, extendingEnd:shouldExtendEnd
    set {startChar, nextChar} to (selectionRange given extendingFront:shouldExtendFront, extendingEnd:shouldExtendEnd)
    setSelectionRange to {startChar, nextChar-1}
end extendSelection

-- Manipulation of document properties

to checkSaveStatus given updating:shouldSave
    tell application "SubEthaEdit"
        if not (exists path of front document) then
            error "You have to save the document first"
        end if
        if shouldSave and (modified of front document) then
            try
                save front document
            end try
        end if
    end tell
end checkSaveStatus

on documentIsAvailable()
    tell application "SubEthaEdit" to get the count of the documents
    result > 0
end documentIsAvailable

to requireNewlineAtEOF()
    tell the front document of application "SubEthaEdit"
        if "" is equal to the contents of the last paragraph then
            -- final line terminated, do nothing
        else
            set the contents of the last insertion point of the last paragraph to return
        end if
    end tell
end requireNewlineAtEOF

on documentPath()
    tell application "SubEthaEdit" to get the path of the front document
end documentPath

on documentLine()
    tell application "SubEthaEdit" to get the startLineNumber of selection of front document
end documentLine

on modeResources()
    tell application "SubEthaEdit" to get the resource path of the mode of the front document
end modeResources

-- String Utilities

on replacement of oldDelim by newDelim for sourceString
    return join of (tokens of sourceString between oldDelim) by newDelim
end replacement

on tokens of str between delimiters
    set oldTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to delimiters
    set strtoks to text items of str
    set text item delimiters of AppleScript to oldTIDs
    return strtoks
end tokens

on join of tokenList by delimiter
    set oldTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to delimiter
    set joinedString to tokenList as string
    set text item delimiters of AppleScript to oldTIDs
    return joinedString
end join

on doubleQuotedForm for baseString  
    quote & baseString & quote
end doubleQuotedForm

on shellTransform of inText for envString through pipeline given alteringLineEndings:altEnds
    set shellscript to join of {envString, "pbpaste", "|", pipeline} by space
    set the oldClipboard to the clipboard
    set the clipboard to the inText
    try
        set shellresponse to do shell script shellscript altering line endings altEnds
    on error errMsg number errNum from badObject
        set the clipboard to the oldClipboard
        error errMsg number errNum from badObject
    end try
    set the clipboard to the oldClipboard
    shellresponse
end shellTransform

-- Handling of environment settings using a plist file

to writeDefaultEnvironment at envPath
    set savedClipboard to the clipboard
    set the clipboard to "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<!DOCTYPE plist PUBLIC \"-//Apple Computer//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">
<plist version=\"1.0\">
<dict/>
</plist>"
    try
        do shell script "pbpaste > " & (POSIX path of envPath)
    on error errMsg number errNum from badObject
        set the clipboard to the savedClipboard
        error errMsg number errNum from badObject
    end try
    set the clipboard to the savedClipboard
end writeDefaultEnvironment

to openEnvironment at envFilePath given settingDefaultEnvironment:shouldSetDefault
    tell application "System Events"
        if not exists file envFilePath
            if shouldSetDefault
                my writeDefaultEnvironment at envFilePath
            else
                error ("Can't get environment file " & quote & envFilePath & quote) number -1728
            end if
        end if
        open file envFilePath
    end tell
end openEnvironment

to readEnvironment out of plist
    readListPair out of plist
    environmentString from result
end readEnvironment

to readListPair out of plist
    tell application "System Events"
        if exists file plist then
            tell property list file plist
                get {name, value} of every property list item
            end tell
        else
            {{}, {}}
        end if
    end tell
end readPlist

on environmentString from keyValueListPair
    set {plistKeys, plistValues} to keyValueListPair
    set accumulator to {}
    set oldTIDs to text item delimiters of AppleScript
    set text item delimiters of AppleScript to ""
    repeat with i from 1 to number of items in plistKeys
        set tokens to {"export ", item i of plistKeys, "=", item i of plistValues, ";"}
        copy (tokens as string) to the end of the accumulator
    end repeat
    set AppleScript's text item delimiters to space
    set envString to accumulator as string
    set AppleScript's text item delimiters to oldTIDs
    envString
end environmentString

What Role Should the Script Editor Play in Writing AppleScript?

I think it is safe to say that Script Editor is the most widely used application for writing AppleScripts. But, should it be? I'm coming to the conclusion that it brings so many limitations along with it that it can only play a limited role.

Before going any further with this line of thought, I'd like to make clear that I'm not making any recommendation for or against the various commercial AppleScript development environments. Looking at the SEEing LaTeX series, you might get the impression that I use AppleScript a fair amount. I don't. I use it as little as possible, which is why in that series I do most of the real work not in AppleScript, but in shell scripts called from AppleScript. Paying for a commercial environment may make sense if you want to or have to write a lot of AppleScript, but would be a waste of money for me.

However, our choices are not limited to Script Editor and the commercial development environments. We have an excellent development environment that comes for free with Mac OS X: it's called Unix. We can use make to manage a project, of which AppleScript can play a useful part. Unfortunately, Script Editor is tied too strongly to AppleScript, which means we're more or less stuck with AppleScript's mechanisms for modularity. Since modularity goes hand in hand with reuse, this is a key point.

Within AppleScript, the usual approach is to put common functionality in a script library and use load script from the StandardAdditions. This has a number of problems. First, there isn't a standard search path from which to load the script library, so you need to introduce a new convention. Second, if a library is changed, there is no automated mechanism to update scripts that depend on it. You can load the script library, store it in a property, and just check to see if the library has been updated whenever the script is called; this roughly emulates how, e.g., import works in Python. There are enough steps to this process that you're looking at a lot of copy and paste programming. Also, you depend on whatever convention you've established to load the script library.

Consider now using make. You can use osacompile to compile your text source into a compiled script. You could also use various shell methods to manipulate your script to define the search path for the script libraries, but how is the Script Editor to work with that? You can also just forget about reloading the script libraries when they change, since make will take care of that for us. We just load the script library once, and forget about it. Since we're just loading the script library once, why not just forget about the search path while we're at it? Just compile the library source into your working directory, and have the script load it from that same directory.

At this point, we still can use Script Editor to edit the script sources. However, we still have a lot of boilerplate to write for little gain. We need to set up the scripts correctly so that make can run them once to load the library. We need to add extra steps to make in order to load the libraries. We need to refer to the script libraries by using the property in which we stored them, whenever we want to use a handler in the library.

Alternatively, instead of emulating how Python loads modules, let's take a look at how something like C would do things. In C, you just use the preprocessor to include the various files. We can do that for AppleScript using, e.g., m4. Assuming you're working in a script called demo.applescript, and want to use some handlers you've written for manipulating text. Just include the relevant library source with something like
include(`StringTools.applescript')
and compile with
m4 demo.applescript | osacompile -o demo.scpt


Just like in C, this approach includes everything into the namespace of the script. And, just like in C, this could lead to namespace conflicts. So far, I've only used this in a few scripts, all of which are relatively small. I've not encountered any difficulties. But, should trouble arise, there is a solution. Instead of including the text at the top level of the script, include it inside of a script object:
script StringTools
    include(`StringTools.applescript')
end script
This could be further automated by writing an m4 macro, such as:
define(`import', `script `$2'
include(`$1')
end script
')dnl


I'm not at all sure that this approach would scale to larger scripts well. You'd really like to be able to have scripts import what they need, and not worry about whether they will themselves be imported into others. Since I'm not planning to write any large AppleScripts, I'm not going to worry about that until it becomes a problem.

Where does this leave the Script Editor? It's no longer useful for scripts of any length, since it can't handle the include statements. We do want it in order to look at scripting dictionaries, that much is clear, but is it needed for any editing at all? It is a reasonable environment in which to try out little code snippets, but we can even dispose of that, since Script Editor offers that functionality through the services menu.