A place to be (re)educated in Newspeak

Sunday, December 16, 2007

Lethal Injection

Some months ago, I wrote a couple of posts about object construction and initialization. I made the claim that so-called dependency-injection frameworks (DIFs) are completely unnecessary in a language like Newspeak, and promised to expand on that point in a later post. Four months should definitely qualify as “later”, so here is the promised explanation.

I won’t explain DIFs in detail here - read Martin Fowler’s excellent overview if you need an introduction. The salient information about DIFs is that they are used to write code that does not have undue references to concrete classes embedded in it. These references are usually calls to constructors or static methods. These concrete references create undue intermodule dependencies.

The root of the problem DIFs address is that mainstream languages provide inadequate mechanisms to abstract over classes.

Terminology rant: DIFs should more properly be called dependee-injection frameworks. A dependency is a relationships between a dependent (better called depender) and a dependee. The dependencies are what we do not want in our code; we certainly don’t want to inject more of them. Instead, DIFs inject instances of the dependees, so the dependers don’t have to create them.

DIFs require you write your code in a specific way, where you avoid creating instances of dependees. Instead, you make sure that there is a way to provide the dependee instance (in DIF terminology, to inject it) from outside the object. You then tell the framework where and what to inject. The reason injection traffics in instances rather than the classes themselves is because there’s no good way to abstract over the classes.

Having recapped DIFs, lets move on to Newspeak. Newspeak modules are defined in namespaces. Namespaces are simply objects that are required to be deeply immutable; they are stateless.

Tangent: This ensures that there is no global or static state in Newspeak, which gives us many nice properties.

Namespaces are organized like Java packages, as an inversion of the internet domain namespace. Unlike Java packages, sub-namespaces can see their enclosing namespace.

A module is a top-level class, that is, a class defined directly within a namespace. Newspeak classes can nest arbitrarily, so a module can contain an entire class library or framework, which can in turn be subdivided into subsystems to any depth. Nested classes can freely access the lexical scope of their enclosing class.

Modules, like all classes, are reified as objects that support constructor methods. Recall that in Newspeak, a constructor invocation is indistinguishable from an ordinary method invocation. Objects are constructed by sending messages to (invoking virtual methods on) another object. That object may or may not be a class; it makes no difference. Hence all the usual abstraction mechanisms in the language apply to classes - in particular, parameterization.

Here is a trivial top level class, modeled after the motivating example for DIFs given in Fowler’s article:

public class MovieLister = (
|
private movieDB = ColonDelimitedMovieFinder from:’movies.txt’.
|)
(
public moviesDirectedBy: directorName = (
^movieDB findAll select:[:m |
m director = directorName
].
)


The idea is that MovieLister supports one method, moviesDirectedBy:, which takes a string that contains the name of a director and returns a collection of movies directed by said director. The point of Fowler’s example is that there is an undesirable dependency on a class, ColonDelimitedMovieFinder, embedded in MovieLister. If we want to use a different database, we need to change the code.

However, this code won’t actually work in Newspeak. The reason is that the enclosing namespace is not visible inside a Newspeak module. Any external dependencies must be made explicit by passing them to the module as parameters to a constructor method. These parameters could be other modules, namespaces, or classes and instances of any kind.

In this specific case, ColonDelimitedMovieFinder cannot be referenced from MovieLister. If we try and create a MovieLister by writing: MovieLister new, creation will fail with a message not understood error on ColonDelimitedMovieFinder. We’d have to declare a constructor for MovieLister with the movie finder as a parameter:

public class MovieLister finderClass: MovieFinder = (
|
private movieDB = MovieFinder from:’movies.txt’.
|)
(
public moviesDirectedBy: directorName = (
^movieDB findAll select:[:m |
m director = directorName
].
)


At this point, we can immediately see that we can replace ColonDelimitedMovieFinder with any class that supports the same interface, which was the object of the entire exercise. Newspeak won’t let you create a module with concrete external dependencies, because that wouldn’t really be a module, would it?

In Newspeak code in a module doesn’t have any concrete external dependencies, and no dependees need to be injected. What’s more we can subclass or mix-in any class coming in as a parameter - something a DIF won’t handle.

What about a subsystem within a module? What if I don’t want it using the same name binding as the enclosing module? I can explicitly parameterize my subsystem, though that requires pre-planning.

I can also override any class binding in a subclass. Newspeak is message-based, so all names are late-bound. Hence any reference to the name of a class can be overridden in a subclass. Classes can be overridden by methods or slots or other classes in any combination. So even if you do not explicitly parameterize your code to allow for another class to be used to construct an object, you can still override the binding of the class name as necessary.

In summary, Newspeak is designed to support, even induce, loose coupling. That’s the point of message based programming languages. DIFs are an expedient technique to reduce code coupling in the sad world of mainstream development, but in a language like Newspeak, they are pointless.

11 comments:

Bob said...

Great entry. Several notes:

1) I definitely agree that DIFs work around problems that should be solved in the language. On the Guice home page, we even say, "you might think of Guice as filling in missing features for core Java. Ideally, the language itself would provide most of the same features, but until such a language comes along, we have Guice."

2) Dependency is a synonym for dependence, but it can also mean dependent(n). Dependee is a made-up word, albeit less ambiguous than dependency.

3) No offense to Martin Fowler, but his article is anything but "excellent." For one, it introduced the term "dependency injection" which you obviously don't like. Two, it's outdated. Before Java 5, i.e. back when Martin wrote this article, I didn't think DIFs carried their own weight. Annotations made DIFs viable. Three, the article overlooks many of the benefits of dependency injection; it recommends the service locator pattern over DI for crissakes!

I'm obviously biased, but I think my video tech talk in which I relate DI to the factory pattern is a much better way to explain DI. I've asked Martin numerous times to update the article--it is the first result on Google--but he's moved on to DSLs and doesn't have time.

4) In Newspeak, when I construct a MovieLister, do I need to provide a MovieFinder? i.e. if I introduce another abstract dependency into MovieLister, do I need to go back and update every piece of code that constructs a MovieLister? Where does "finderClass" come in? Sorry for my ignorance. I'd love to see examples of the code which calls MovieLister.

5) Can you elaborate on the following comment with an example? "What’s more we can subclass or mix-in any class coming in as a parameter - something a DIF won’t handle."

Thanks,
Bob

Gilad Bracha said...

Hi Bob,

Thanks for the feedback. You're absolutely right, I should have pointed at your talk. It's just that I find an article much easier to quickly skim, whereas a talk is linear and requires more time and attention.

I think your questions on Newspeak deserve another post. I'll try and do that really soon.

Bob said...

For a written version, look no further than the introduction to the Guice user's guide. ;)

Axel Rauschmayer said...

1) I've always felt that DI was not about giving a language such as Java the means to abstract over classes, but rather about reifying the dynamic scope and using it for configuration. With DI, I mostly use singleton instances in a way that is practically service-oriented (or component-oriented): A singleton needs certain services and gives its clients (which stand "above" it in dynamic scope) the opportunity to configure what these services are. If Newspeak allows one to bind class names to different implementations, it makes DI more flexible, but I would assume that dynamic scope still plays a role when it comes to the effects of this kind of paramerization. To fully use DI in my code, I've had to inject the injector into instances, so that, recursively, they can create their own instances. This is a further clue that DI is about dynamic scope.

2) Does Newspeak support multiple dispatch? I find that there are some methods that cannot be expressed well with single dispatch: If a method's parameters are collaborators and not "true" parameters then I'd rather see it as a top-level construct that "belongs" to all collaborators. This is acutely obvious with a binary operator: To which of the two operands does it belong?

I find it refreshing to read about truly new ideas in object-oriented language design, so I'm really looking forward to more posts on Newspeak.

Paul said...

Hi Gilad,

Interesting. You say that Modules have no concrete external dependencies, which is a good thing, because they can be used as true reusable components. You leave out the crucial bit though, showing how modules (components) are wired together.

I look forward to your next post.

Cheers,

Paul.

Paul said...

Hi Gilad,

Another question. The "overriding" of class name bindings. Is this scoped? So if I override the binding of a class name to refer to a different implementation, is that new binding only valid within the current class and nested classes (I think your refer to this as the current system)? Does the new binding apply to subclasses too?

You say that there are no globals, but the scoping rules aren't clear to me. Something else to clear up in your next post.

PS. Please don't leave it so long this time. You've let us all on a cliff hanger :^).

Thanks. P.

Gilad Bracha said...

Axel:

No, Newspeak does emphatically not support multiple dispatch. I've never been a fan of that, I'm afraid; it is inherently non-modular.

As for more general comments on DI: Perhaps one can view it as an attempt to make object construction be more declarative.

Gilad Bracha said...

Paul:

I'll be talking about Newspeak at Google early next year; the talk might make it out onto the net, and maybe things will be clearer.

In general, blogging may not be the best way to explain these ideas. The format is too short.

There's a position paper I presented at Dyla which discusses the scope rules. It's on my web site. To answer your question - yes, overriding affects subclasses - it works exactly like method overriding.

As for how modules get hooked up: the short answer is that a module definition is a class, and so you instantiate it, calling a class method with the desired arguments. The real question is in what scope do you start the process. I'll try and post about this soon.

Axel Rauschmayer said...
This comment has been removed by the author.
Axel Rauschmayer said...

On multiple dispatch (MD):

(1) I do agree that MD impacts modularity, but it might be complementary/orthogonal to single dispatch (and message passing): I'm using classes as either components or as data holders. In the former case, encapsulation is clear and single dispatch/message passing is the way to go (and a nice preparation for distributed computing).
But with classes as data holders, method functionality does cross-cut classes and then MD neatly solves many design dilemmas that I have (e.g.: "in which class do I put the binary operator implementation"). Additionally, classes-as-components can also have cross-cutting code. There are ideas out there such as invasive composition that handle some cross-cutting scenarios for components, so maybe there is a clean alternative to MD that I'm not aware of. If you have any more insights on this topic, I would love to hear them.

(2) Whenever I explain method overloading (in Java) to my students, they expect it to work like multiple dispatch, so in this case it is actually more intuitive and practical than overloading and static resolution.

Gilad Bracha said...

Axel:

On 1: Multiple dispatch is a tough one, and ultimately it's judgement call.

On 2: Static type based overloading is a truly bad idea, with no redeeming value whatsoever. Multiple dispatch is related, but dynamic, and so much better. The scalability issues are still there however.