Through the Looking Glass Darkly

In January, I gave a guest lecture in a class on reflection and metaprogramming at HPI Potsdam. A screencast of the talk is now available. It’s an introduction to the concept of mirrors, which is the goodthink way of doing reflection. It’s mostly language neutral, but there is a brief demo using mirrors in Newspeak.

Because it’s a screencast rather than a video, occasionally some detail may be unclear, but by and large it is the most comprehensive introduction to mirrors available other than the OOPSLA paper.

Some people may not have an hour to watch the entire screen cast, and the paper is by no means an easy read, so I’ve decided to post the executive summary here.

The classic approach to reflection in object-oriented programming languages originates with Smalltalk, and is used in most class based languages that support reflection: define a reflective API on Object. Typically, Object supports an operation like getClass() which returns an object representing the class of the receiver. The API of classes defines most additional reflective operations available. For example, in Java, you can get reflective descriptors for a class’ methods (java.reflect.Method), fields (java.reflect.Field) and constructors (java.reflect.Constructor). You can even use these descriptors to evaluate program code dynamically - say, ask the user for the name of a method and invoke it. In Smalltalk, you can also add and remove methods and fields, change a class’ superclass, remove classes from the system etc.

Another approach is used in many scripting languages. The language constructs themselves introduce code on the fly, modifying the program as they are executed. For example, a class comes into being when a class declaration is evaluated, and might change if another declaration of a class with the same name is executed later.

The third approach is that of mirrors, and originates in Self. Mirrors have been used in class based systems such as Strongtalk, and even in the Java world. JDI, the Java Debugger Interface, is a mirror based reflective API. Here, the reflective operations are separated into distinct objects called mirrors. This seemingly minor restructuring has significant implications. Reflection is no longer tied into the behavior of every object in the system (as it is via getClass()) or (even worse) into the very syntax of the language. Instead, it resides in separable components that can be removed or replaced. Reflection is now a distinct capability, in the sense of the object capability model.

If you are worried about security, this is good news. If you don’t provide a program with the means to manufacture mirrors (e.g., you do not provide the mirror factory object), said program cannot do any reflection. You can also provide mirrors with limited capabilities - say mirrors that only reflect the program’s own code, or mirrors that do not allow you to modify code or access non-public members etc.

Caveat: The truth is, mirrors have not really been used for security. Their utility for security seems clear, but a working API has yet to be demonstrated.

Mirrors are good news for other reasons. Say your program doesn’t use reflection, and needs to fit into a small footprint such as an embedded device. It is easy to take it out. Another advantage is that you can easily plug in alternate implementations of reflection - so if you need to reflect on remote objects, you can do so.

Historical note: This is why JDI uses mirrors; indeed, it is why JDI had to be introduced. The original intent was that Java reflection would be used to support debugging; but once you need to deal with cross-process debugging, you need a distinct implementation of reflection; core reflection is tied to a single built in implementation.

Mirrors support a clear boundary between the base-level of your program (the level which deals with the problem domain your program is intended to solve) and the meta-level (the level where your program is discussing itself, where reflection takes place). The classic design, where the class is the main repository of reflective information, tends to blur these lines. Classes often have both base level functionality (like creating instances) and meta-level functionality (reflection). This is most acute in languages like Smalltalk and CLOS. In Java, the base level roles of classes are often supported by specialized constructs like constructors (which have their own, worse, problems) and
static members (likewise). Even in Java, class objects may be used in a base level capacity (as type tokens, for example).

There is much work to be done in this area. No mirror API has yet fulfilled all my claims and ambitions - least of all the Newspeak mirror API, which needs extensive revisions. Still, I hope you’re curious enough to watch the talk and/or read the paper.