Seeking Closure in the Mirror

I've discussed mirror based reflection many times in the past, in this blog and in talks. And of course I'm not the only one - you can  read Alan Wirf-Brock's  posts on mirrors in Javascript. In this post, I want to focus on a particular kind of mirror that has not received much attention. Before I get to deep into the details, a few words of background.

You cannot get at the internals of a function: you can only apply it to various arguments and see how it responds. This is sometimes known as procedural abstraction. Among other things, it is the basis for object-based encapsulation.

Most languages that call themselves object-oriented do not actually support object-based encapsulation. One of the ways they get by despite this defect is to  rely on procedural abstraction directly. Perhaps the most notable example of this is Javascript. The only way to encapsulate anything in Javascript is to put it inside a function. Elaborate design patterns leverage Javascript’s closures to provide encapsulation.

You can see from the above that procedural abstraction is absolutely fundamental. There appear to be circumstances where we might nevertheless might wish to breach the defenses of procedural abstraction. 

Consider implementing a database interface in the style of LINQ, or Ruby on Rails, or Glorp. The underlying model is that the database consists of collections, and that these collections are accessed via standard functional operations such filter, map, reduce etc.  The arguments to these operations include closures. For example, you might write a query such as:

cities.filter(function(city){return city.name = ‘Paris’;});

and get back a collection of answers that included Paris, Texas, and perhaps some other cities. To implement this interface on top of a database, you might want to transform this code into a SQL query.  To do that you need to understand what the closure is doing. In .Net, for example, the type system is designed to coerce a literal closure into an abstract syntax tree representing the expression inside it, which can then be compiled into SQL. 

Of course, it might be that you cannot reasonably compile the code into a SQL query at all.  We will assume that the system is allowed to fail in any case it deems too hard, but we’d like to cope with as many situations as we can.

The LINQ approach relies on static typing, but this is not essential, and in fact has drawbacks.

For example, the static approach precludes the following:

query(f) {return cities.filter(f);}

A more general alternative is to dynamically derive the AST of the closure body. Regardless, it seems I need a way to get the AST (or at least the source) of a closure - something that procedural abstraction is of course designed to preclude.

Even if I can get the source or AST, that isn’t always enough. Suppose I want to write

var cityNames = [‘Paris’, ‘London’, ‘New York’];

cities.filter(function(city){

    return cityNames.contains(city.name)

});

I need the value of cityNames in order to execute the query.  In general, I need to get at the scope of the executing closure. 

Smalltalk and its relatives allow you to do this. How do they get around procedural abstraction? Well, in the case of closures, they basically throw procedural abstraction out the door. Every Smalltalk closure will gladly provide you its context, which is a reified scope that will allow you to find out what all the variables used in the closure are.

Obviously, this is not a very secure solution. One way we can usually reconcile security and reflection is via mirrors, and that is the focus of this post.  Given an object mirror that has full access to the closure object's representation, you should be able to get all the information you need.  This still has the drawback that the representation of closures is exposed as a public API. 

In this case, we want a ClosureMirror. Essentially, there needs to be an object with the magical ability to see into the closure, overcoming procedural abstraction.  The closure itself must not allow this; it must be impenetrable. The capability to look inside it must be a distinct object that can be distributed or withheld independently (exercise for the reader: find another way to solve this problem).

Concretely, a ClosureMirror needs to able to provide the source code of the closure it is reflecting and a map from identifiers to values that describes the closure’s current scope.

Another situation where closure mirrors would be handy is serialization. If you need to serialize an object that includes a closure, you again need access to the closure’s scope.

I have not seen closure mirrors discussed elsewhere. As far as I know, the only implementation was done as part of the Newspeak-to-Javascript compiler. We are also considering it in the context of the Dart mirror system.   The Newspeak-on-Javascript implementation of closure mirrors is rather naive and inefficient.  One reason for this inefficiency is that Javascript provides no support whatsoever to do this sort of thing. In any case, the idea is new and virtually untested, but I think it has potential.