Closing the Frontier?

How is a programming language interpreter like pornography? They’re both banned at the iPhone App store. The restrictions on interpreters disturb me (this blog does not deal with the other topic).

What if I want to write an iPhone app in my favorite programming language? Or my second favorite? The freedom to program in the language of my choice is a big concern of mine. Even though Squeak Smalltalk runs on the iPhone, and Newspeak runs on top of Squeak, I’m out of luck. Packaging the entire runtime with each app is not attractive.

Tangent: In principle, one should be able to write a compiler that spits out Objective-C. After all, Objective-C incorporates Smalltalk style message sends, down to the keyword syntax. In practice, Objective-C on the iPhone doesn’t support true garbage collection, so it just seems too painful. This may change soon - maybe as soon as Wednesday., with iPhone OS 4 and/or the iTablet (pick your favorite rumor).

The iPhone is a prime example of a trend where our computing platforms become more restricted. As we move toward software as a service rather than an artifact, the computer is no longer as personal; it is very much under control of the service provider. In this case Apple, in other cases Amazon or Google or Microsoft. I’d be surprised if the rumored iTablet won’t work on the same model: rather than an open version of MacOS, a semi-closed world with an app store.

I completely understand why service providers place these restrictions. What if my app is a storefront for apps? Whither the app store’s revenue? And then there are security concerns.

On the iPhone, there are administrative restrictions. Other platforms are more open in that respect - but you tend to run into technical problems. Take Android - you can distribute a language runtime, but it is the Javanese VM that’s in charge.

No one will prevent you from implementing your own system on a JVM or on .Net. However, the VM, by its architecture, will make things difficult. If it does not support the right primitives, you find that your language is a second or third class citizen.

Examples: the typed VM instruction sets are a problem for dynamically typed languages. On .Net, the DLR provides some support, but you’ll never get to peak performance using it as it stands today. On the JVM, invokedynamic will ship some sunny day, and that will be better. Even then, if you want real dynamism, you’ll find it very hard. How do you change the class of an object? How do you change the shape of a class? Or its superclass? The solutions are convoluted and/or suboptimal.

On the desktop, you can choose to bypass these platforms, but the action is increasingly moving elsewhere. Looking ahead, the dominant platforms won’t be traditional operating systems like Windows, MacOS and Linux; far fewer people will use them directly than today. Instead, we’ll have the web (as in, e.g., ChromeOS), and semi-enclosed service platforms: The moral equivalent of iTablet OS (regardless of what Apple does this Wednesday). The “real OS” may still be there underneath, but virtually no one will care.

Each of these service platforms will combine a client side, running on phones, tablets, e-books, laptops (e.g., iPhone, Android, Kindle, ChromeOS) and desktops; and a server side (e.g., Azure, Google, E3, iTunes/MobileMe) supporting storage/backup, software update and distribution and services we haven't invented yet.

The most open of these virtual platforms is of course, the web browser. Mercifully, it is relatively free of administrative restrictions and its machine language is much more flexible than MSIL or JVM byte codes (JVML). Which is not to say that you don’t have to go through some very convoluted and costly hoops. Think of weak pointers/finalization; or stack manipulation (debuggers, tail recursion) etc.

There are two threads here:

1. The narrow technical one, which is about providing abstractions that are good enough to support alternate ways of doing things. One wants flexible designs, especially in the languages that are supposed to be at the heart of general purpose open platforms. Of course, this is much easier said than done.
2. The broad technological/social/commercial one: the trend toward software services and the cloud is pulling us away from the personal computer and the individual control it entails. Of course, one of the great attractions here is the idea of a service that relieves much of the responsibility for digital housekeeping. The vast majority of people don’t want all that control with the trouble that goes with it. Balancing this with the freedom to innovate at the platform and language level, and the freedom to choose among languages, is hard.
   

Long term, the platforms that have more flexibility will benefit more from new ideas, which gives me a basis for unnatural optimism :-)

Avarice and Sloth

Are these my new year resolutions? No (ok; maybe, but we won't go there), they are the names of two hypothetical programming languages.

The world is slowly digesting the idea that object-oriented and functional programming are not contradictory concepts. They are orthogonal, and can be arranged to be rather complementary. In that light, I’ve been dwelling on the idea of a purely functional subset of Newspeak: Avarice.

There is only one place in Newspeak where mutable state can be introduced: slot declarations. Newspeak supports both mutable and immutable slots. For example,

x ::= 0.

introduces a mutable slot; the declaration implicitly introduces both getter and setter methods (named x and x: respectively). An immutable slot declaration looks like this

y = 0.

In this case, only a getter is created, so there is no way to modify y after its introduction.

Disallow mutable slots, and you’re almost there.

There is one catch - the order of slot initialization is observable, and so you might detect a slot in its uninitialized state.

x = y.

y = 0.

will set x to nil. So much for referential transparency.

However, we also have simultaneous slot declarations, which are a bit like letrec. Alas, they aren’t implemented, and the spec needs tightening. Nevertheless, whatever variant we implement will prevent you from observing the uninitialized value of a slot. Once we add this feature, we can subset Newspeak and produce Avarice.

Of course, to make it work, we’d need to change a lot of the libraries. So maybe this is a little speculative at this point. This post will get even more speculative as we go along.

There is one more little nagging issue: reflection. What happens if we reflectively modify our program?

One answer is that we don’t; if we are serious about referential transparency, we cannot tolerate this sort of thing. This means that reflection is restricted to introspection, just like Java and C#. That’s still more than one get in existing pure functional programming.

Another answer is that we allow reflective change; we’re only functional at the base level, not the meta-level.

Note that the language doesn’t dictate a choice here: it depends what reflective library your implementation provides. In principle, you can provide multiple libraries, i.e., one that only supports introspection, and one that does the full thing. Lots to think about here.

Even restricted to introspection, Avarice would be rather different from most purely functional languages. It would carry none of the cultural baggage commonly associated with such languages: no currying, no Hindley-Milner type inference (indeed, no mandatory type system at all), no quasi-mathematical syntax. Is such a beast workable? Well, Erlang is. However, Avarice would also be object-oriented. Who knows, it might be the best of both worlds.

By now, my remaining readers are wondering what I’ve been smoking. Well, I don’t smoke, but red wine is good for you. In that spirit, we can take things further.

What if we make things lazy, like Haskell? This is not a Newspeak subset anymore - the semantics are rather different. However, syntactically, this language is identical to Avarice. We’ll call the lazy language Sloth, in contrast to its eager, dare I say even greedy, cousin Avarice. Sloth would be purely functional, and at the same time purely object oriented.

At this point, Avarice and Sloth are just imaginary constructions. They only exist in my mind, in this post, and in these slides. However, they make a nice thought experiment, and I hope to make them real in the fullness of time.

Regardless, sooner or later, someone will build a purely functional object oriented language. It will probably have a Javanese syntax and be rather hacky; but I hope Avarice and Sloth will be out there as well, for those who appreciate the finer things in life.