AppleScript Syntax: Dealing with the Depressing Reality

Given the preceding two posts, I hope it is clear that I think syntax is of too much concern in AppleScript discussions. The preoccupation with syntax can hide real problems that might actually be sensibly discussed. It seems appropriate to mention some of the aspects of AppleScript that I think could be usefully discussed. I'll limit myself here to points where I can provide some practical tips as starting points.

To begin, name resolution is pretty tricky. Rather than some sensible lexical scoping, there are complex rules. John Gruber provides an excellent description of how names are resolved, in the context of explaining a subtle bug. There is a fact that I've found useful for avoiding name conflicts: you do not have use tell blocks.

Frequently, you just need to tell applications to do a few things. However, example code often has the entire program in a single tell block. That leads to all the tricky rules for determining when a term is looked up in an application dictionary, or a Scripting Addition, or as a handler in the script. Many of these issues can be eliminated by doing something like tell application "Finder" to … instead of using the block format. In essence, this is the same idea as avoiding the use of an implicit this or self object, as seen in some object oriented languages.

Said approach also is particularly enlightening on how much of your program is actually spent dealing with interprocess communication, and how much is program logic. Frequently, not much is actually spent on dealing with IPC, but it leads to a lot of complications. Simple solution: encapsulate IPC in handlers.

Text handling is important in many scripts. AppleScript is pretty weak in its text handling. Mac OS X comes with a powerful text processing system: the Unix shell. Use do shell script for text manipulation.

AppleScript is lacking in the data structures it provides. The one I most frequently miss is an associative array, but it's not the only one. Many other languages provide a richer library of data structures, better support for defining your own, or both. You could often dispense entirely with AppleScript, except for communicating with applications, which might just be a few lines. You can communicate with applications using the osascript shell command. This lets you use just about any language you like, and still be able to gain the main benefit of AppleScript.

There must be many more.

AppleScript Syntax: Some Depressing Examples

Let's expand on the previous post by looking at some examples of how discussion of syntax poisons discussion of AppleScript. Just to be clear, the examples are taken from blogs I like and read regularly; I'm sure there are many other examples, but I haven't gone out of my way to find these.

First, let's return to Daring Fireball. John Gruber recently wrote that

AppleScript, as a programming language, is a noble but failed experiment.

To support this, he links to an earlier article, The English-Likeness Monster, in part of which he makes the far more modest claim that AppleScript's English-like syntax is a failed experiment.

Well, which is it? Languages are more than their syntax. Reading the article doesn't clarify much. Gruber details how a bug he experienced was caused by a subtle name conflict in scripting dictionary terminology. He gives a cogent description of the semantics of name resolution in AppleScript, showing how the name resolution semantics leads to the bug he experienced. However, buried within is a lengthy rant on AppleScript's syntax. It looks like, and is identified as, a digression (well, it is called an "interpolation"—Gruber is a David Foster Wallace fan, as I recall), but gives the post its title. Just what is the intent?

Additionally, there is mention of Python and JavaScript as having clearer, if more abstract, syntax than AppleScript, which helps to prevent such problems. However, if you had AppleScript's syntax but Python's name resolution, you literally could not have the same error. There is a difference at a far deeper level than the syntax. To what extent are these articles supposed to be about syntax, semantics, surrounding tools, libraries (i.e., scripting additions),…?

As a second example, let's take a look at an interesting article from Daniel Jalkut's Red Sweater Blog, called Apple's Script. Jalkut makes what is essentially an economic argument that Apple should make JavaScript the default scripting language for the Mac, keeping AppleScript as an alternative point of access to the Open Scripting Architecture. His key point is that Apple is devoting significant resources to JavaScript, and will continue to do so for strategic reasons. Further, due to wide-spread experience with JavaScript, it is in practice easier for users, despite the supposed ease of use of AppleScript (aside: I dispute that AppleScript is easy to use).

Nothing in the article depends in any way on the syntax of AppleScript, but look through the responses! Several people bring up AppleScript syntax, both as a positive and as a negative. Once the issue of syntax appears, the discussion pretty much stays there. It's a shame, because I think that Jalkut's point was an interesting one, and really does warrant some thought.

AppleScript Syntax: the Depressing Corollary

John Gruber at Daring Fireball links to William Cook's paper on the history of AppleScript. I've been meaning to write about the paper for a few weeks, based on a few recent blog posts relating to AppleScript. The history contains several facts that, in my opinion, are vital for understanding what AppleScript is today, and how we should approach it. In particular, there are some important points to be learned about AppleScript's syntax.

I first read the paper a couple of years ago, after learning about it from Lambda the Ultimate. William Cook is one of the original developers of AppleScript. He describes the development of the language for the third History of Programming Languages conference. The history of a programming language sound likely to be rather dull, but Cook's paper is far from dull. Cook sheds light on many aspects of AppleScript, making clear that the language is a mix of successes and failures. What also becomes clear is that the designers were willing to try genuinely new ideas, not all of which worked out due to practical considerations. As Cook writes,

AppleScript was developed by a small group with a short schedule, a tight budget and a big job. There was neither time nor money to fully research design choices.

It should not be surprising that some of those design choices were suboptimal or even failures.

AppleScript's natural language syntax was one of those failures. Cook writes

The experiment in designing a language that resembled natural languages (English and Japanese) was not successful. It was assume[d] that scripts should be presented in “natural language” so that average people could read and write them.… In the end the syntactic variations and flexibility did more to confuse programmers than to help them out. The main problem is that AppleScript only appears to be a natural language. In fact[, it] is an artificial language, like any other programming language.… It is easy to read AppleScript, but quite hard to write it.

(I've corrected a few typos that were in the copy of the paper I have, which was an early draft.) Besides making AppleScript accessible to average people, there were additional goals for the natural language syntax. None of them were successful.

In his conclusion, Cook writes

Many of the current problems in AppleScript can
be traced to the use of syntax based on natural language…

Sadly, many critics of AppleScript would have that be the whole story. It is not, and acting as if it were hinders understanding of real problems with AppleScript, some of which could be addressed without changing the syntax at all. In fact, I will assert that the biggest problem with AppleScript's syntax is that it prevents meaningful discussion of AppleScript!

I propose a variant of Godwin's law for AppleScript:

As an online discussion of AppleScript grows longer, the probability of the discussion devolving into a debate on the merits of AppleScript's syntax approaches one. At this point, nothing meaningful will be said, and the discussion is effectively over.

Too frequently, discussion of AppleScript actually begins on the topic of syntactic merits. This leads to the depressing corollary:

Most discussions of AppleScript consist only of a debate on the syntactic merits of the language. There is nothing to be learned from these discussions.

Or, more simply:

Most discussions of AppleScript contain nothing of value.

While harsh, I do feel these are an accurate description of most online (and, for that matter, offline) discussions of AppleScript that I've seen.

A Small Tip on Catch All Address and Spam Blocking in Google Apps

I've recently signed up for Google Apps, based largely on an article on Lifehacker. One interesting point was the idea of using a catch-all email address to block spam, as described in a comment or with somewhat more detail in a different article. Both explanations omit a relatively minor point that can lead to some frustration.

To summarize the method, you just set up filters in a catch-all address to identify email that matches a simple pattern, and forward it to your own account. The result is that you can give out email address "aliases" for, e.g., web registration forms, without inviting spam to your real address. If you start getting spam to one of the aliases, you can block it directly with a filter.

I set this up, and sent a test message to try it out. The message showed up in the catch-all account, but didn't get forwarded to my own account. The filter I used to set up the forward was identifying the messages correctly, but didn't send it on for some reason.

After a fair amount of web searching, I found the answer. Email for Google Apps is Gmail. Gmail does some modestly smart things to present a convenient interface; perhaps the best known is the way Gmail groups messages into "conversations." Something else it does is to hide messages that you've forwarded to yourself. This seems quite sensible in the context of conversations.

It's probably clear what was happening, at this point. I sent some test messages from my own account, which was then forwarded back to me from the catch-all account. Gmail sees a messages forwarded to myself, and doesn't show it. I'm left scratching my head in puzzlement.

After finding out about the hiding of self-forwards, I try sending a test message from a different account. It forwards without any trouble. Thus, for testing, you should always use a different account to send email than the account you want to receive it in.

Plaxo Synchronization Oddity

I've used Plaxo for a while to synchronize my address book across a couple of Macs and a web mail account or two. This was the original intent for Plaxo, and it has served me reasonably well as a replacement for the .Mac synchronization (which didn't serve me well, and was overpriced to boot). Today, I discovered a rather strange bit of behavior in the syncronization.

Since I started using it, Plaxo has added a feature they call Pulse. Pulse seems like an interesting variant on the social networking websites, focusing on allowing you to connect up feeds from the sites you actually use and centralize the social networking aspects (e.g., from Blogger, Flickr, and del.icio.us). I set up Pulse today.

Without Pulse, I never had much reason to take a look at the online version of my address book. With Pulse, I could see my own address plainly while setting up my profile. It was wrong.

That was quite strange. The addresses at Plaxo came from synchronizing with the Address Book data on my Mac. Those were right! However, there was a chunk of my previous address showing up on Plaxo, mixed in with my current address.

Here's what happened. I now live in Vienna. The address format for my Address Book card is thus set to Austrian. Before coming to Vienna, I lived in Portugal. The Portuguese format has a field for territorial subdivision, while the Austrian one doesn't use that. The extra stuff showing up in my current address is just the old territorial subdivision. Switching the display format for my Address Book card back to Portuguese shows the territorial subdivision field again. The old data is still there, just hidden with the Austrian formatting. It seems that Plaxo doesn't handle this situation correctly, treating all the data as still relevant (to be fair, I don't know if it is even possible for Plaxo to handle it right).

Admittedly, the described case is a bit unusual. Regardless, it may affect others. The solution, fortunately, is simple: switch the Address Book card format back to the earlier country setting, delete the hidden, outdated fields, switch back to the correct country setting, and sync with Plaxo.

Update: Or maybe Pulse doesn't let you connect a feed from Blogger. Mine doesn't seem to be working, anyway.

Tradeoffs for Type Inference

Here's a nice essay by Chris Smith: What to Know Before Debating Type Systems. Smith gives a nice introduction to some of the ideas and terminology for programming language type systems, aimed at the perennial static vs. dynamic debate. I quite like that he dismisses the strong vs weak typing distinction as meaningless. Some terminology that I'd have like to have seen is that of untyped languages and type safe languages, since so-called dynamic typing is just that: untyped, but type safe through dynamic checking. I suppose that would have caused a lot of, e.g., Python users to stop reading, which would be a shame.

For the most part, I think Smith is right on with his presentation, although there are a few points where I'd quibble about details. However, I think that type inference warrants some further consideration. As Smith states, "Type inference is generally a big win." I fully agree with that, but there are some details worth making explicit.

First, let's distinguish type inference from what we might call type propagation or local type inference. Type inference is figuring out the types of the expressions from the expressions themselves. Type inference is seen mostly in functional programming languages such as Standard ML, Objective Caml, and Haskell. For the most part, you don't need to write down the type signature for a function, you just define it and the compiler can work out the types. Despite this, it is common to give the type signatures for functions anyway, since the types provide a great deal of information. The type information is useful for understanding how to call and compose functions, which is obviously important for functional programming. Further, declaring the type for a function can be a useful design aid, allowing a correctly typed function stub to be provided that will just raise an exception when called. In Standard ML, such a stub is defined like fun stub x = raise Fail "Not implemented". The inferred type of stub is 'a -> 'b, which is too general to be useful in any real function. By manually giving a type to stub, you can use it to define other functions and the compiler will be able to do more useful checks.

Type propagation is using known types to figure out the types of as many expressions as possible. The latter might take the form of declaring the types for functions and allowing the compiler to work out the types for all the expressions within the function bodies. If you would normally write out the types for functions anyway, you aren't really out much. You might have to write a few more type signatures, since you'd probably skip the signatures for little helper functions, but those types should be easy, anyway. There can also be gains from this approach, since you can use a more expressive type system for which type inference might not be decidable. Such an approach is taken in the local type inference for Scala. There's no guarantee that types can be inferred, but they usually can be so you don't need to write type declarations for every variable. On the other hand, by softening the guarantees for inference, types can be used to do more. It can even fit in with untyped languages for optimization purposes; my understanding is that this was a big part of how Dylan compilers were able to produce fast executables.

As a whole, having type inference provides not just benefits, but also places restrictions on the type system. Improving the type system may well make it worth giving up a general type inference algorithm in favor of weaker type propagation. The marginal cost of annotating some extra functions with types is just not a high price to pay. Turning it around, one really has the worst of both worlds with languages like C, C++, and Java: you have to declare the type of every variable, but it really doesn't provide you with any benefit.

(Via Lambda the Ultimate.)

Code indentation in SubEthaEdit

There's a recent message on the Yahoo! group for SubEthaEdit asking about automatic formatting of code in SubEthaEdit. By building on SubEthaEditTools, it only takes about twenty minutes to connect an appropriate shell script to any given mode, calling out to, e.g., indent for the C mode. But we can do better!

It takes only a little more effort to install a general system for code reformatting into SubEthaEdit. First, we create a suitable AppleScript that calls out to a formatter:

property prettyPrinter: "${PRETTYPRINTER}"

if not documentIsAvailable() then
    return
end

if (modeSetting for "PRETTYPRINTER") is missing value then
    display alert "Unable to re-indent." message "You need to define PRETTYPRINTER for the mode."
    return
end

if selectionIsEmpty() then
    setDocumentText to (beautifulCode from documentText())
else
    extendSelection with extendingFront and extendingEnd
    setSelectionText to (beautifulCode from selectionText())
end if

on beautifulCode from sourceText
    shellTransform of sourceText for modeEnvironment() through prettyPrinter without alteringLineEndings
end beautifulCode

on seescriptsettings()
    {displayName:"Re-Indent Lines", shortDisplayName:"Re-Indent", keyboardShortcut:"^@i", toolbarIcon:"ToolbarIconRun", inDefaultToolbar:"no", toolbarTooltip:"Automatically re-indents lines", inContextMenu:"yes"}
end seescriptsettings

include(`SubEthaEditTools.applescript')

The formatter is stored as a PRETTYPRINTER environment variable. The include at the very end is an m4 command, not AppleScript; if you don't know what it's for, you can either just cut and paste the contents of SubEthaEditTools into the script at that point, or read about it here.

The logic of the AppleScript is pretty straightforward. I first check to make sure a document is open, then check to see if a formatter is defined for the mode through a PRETTYPRINTER environment variable. I didn't do these sorts of checks in other scripts for SubEthaEdit, because those were always for just one mode, so I could assume that there was an open document and provide a sensible default for the mode. Here, we're going to put the script into the general scripts folder ~/Library/Application Support/SubEthaEdit/Scripts, so we can't make those assumptions anymore.

At this point, I've made all the safety checks needed, so I can proceed to do the reformatting. If no text is selected, I pass all the text to the beautifulCode handler, otherwise I extend the selection to full lines and pass the new contents to beautifulCode. All that happens in beautifulCode is that the text is handed off to the shell for formatting, with a suitable environment obtained from the mode. The results of beautifulCode are then substituted for the original text.

Finally, seescriptsettings are defined. I set a keyboard shortcut of Command-Control-i; it was available, and allows a mnemonic of i for indent. I defined a toolbar item for it, with the run icon, which is pretty arbitrary; I didn't see anything better from the standard list. It can also be invoked within a document using the contextual menu. Remember, this goes into the general scripts, so appears under the scripts menu, not the Mode menu. I saved mine as PrettyPrint.scpt.

One thing remains: how do we define the PRETTYPRINTER for a mode? It's remarkably easy. The script I used for setting the LaTeX mode environment doesn't actually depend on the LaTeX mode in any way, so it can be used directly by just putting it in the general scripts folder instead of within a mode. Actually, it is probably a good idea to also check that a document is open so that a mode can be identified. With SubEthaEditTools, there's almost nothing to it:

if not documentIsAvailable() then
    return
end

openEnvironmentSettings()

on seescriptsettings()
    return {displayName:"Customize Mode...", shortDisplayName:"Environment", inContextMenu:"no"}
end seescriptsettings

include(`SubEthaEditTools.applescript')

This script, too, goes into the general scripts folder for SubEthaEdit; I named mine OpenEnvironment.scpt. Although this second script is quite simple, it has a very profound effect, adding a preferences system for all the SubEthaEdit modes. I wonder what else it could be used for?

And that's it. Install the two scripts, define some appropriate PRETTYPRINTER values (being sure to quote the shell command appropriately!), and reformat away. For your convenience, you can download compiled scripts for the formatter and the mode environment settings.