A place to be (re)educated in Newspeak

Tuesday, May 06, 2008

The Future of Newspeak

Several people have asked me when Newspeak will be released. Well, I still don’t know, but at least now I know it will be released. Cadence has generously agreed to make Newspeak available as open source software under the Apache 2.0 license.

We will be publishing a draft spec for Newspeak soon; I say draft because I expect Newspeak to continue to evolve substantially for the next year at least, and because the initial spec will necessarily be incomplete.

It will be a while before we are ready to make a proper public release of Newspeak. In the meantime, I’ve gathered some information on my personal web site. We plan to set up an official Newspeak web site in the near future.

20 comments:

Yardena said...

Wow, that's exciting news!

Victor Rodriguez said...

Bravo! I'm looking forward to your first release. Will it be based on Squeak?

Saludos,

Víctor Rodríguez.

Gilad Bracha said...

Victor,

The release is still a ways off, as we have a lot of work to do, but I expect it will be based upon a modified Squeak VM.

Matthias said...

Neat. So I made it through the whole video@HPI and was puzzled by the security model. One the one hand you say "objects as capabilities" but then point to public/protected/private. Sounds contradictory. Do you translate the latter to the former? You also mentioned required VM support.

Matthias

Gilad Bracha said...

Matthias,

I'm not sure why you see a contradiction between the access control and capabilities. Remember that the access control is per object, not per class.

And, yes, implementing this properly should be done at the byte code level, so that there is no way to write code that gets around it. You don't want to pay a cost for checking it - it needs to be done via the context-free syntax.

This probably doesn't help - but explaining the implementation will wait for another time.

Paul Beckford said...

Hi Gilad,

Very exciting news. You say you are evolving the language as you are using it to building stuff.

Sounds like a great way to do things. Is this approach working well?

Thanks,

Paul

Gilad Bracha said...

Paul,

There are pros and cons. We have users, small in number but very real. This forces us to get things to really work. On the other hand, it requires a lot of time and energy. Overall, I'm sure it's a win.

Matthias said...

So the capability model says "If you want that a client not be able to use something -instead of checking at the point of usage- don't even hand him the reference to it".

If a client has a reference to an instance of a class with a 'private' method in it, are you saying that you make the method _lookup_ fail if it is not an unqualified self send instead of the actual _invocation_ ?

Matthias

Gilad Bracha said...

Matthias,

The short answer is 'yes'. It's a bit more involved because of class nesting and protected, but basically as you say.

So a normal send (one with an explicit receiver) by definition only looks up public methods. This doesn't require checking per se.

Uladzimir Liashkevich said...

Gilad, you've done a great job on Newspeak!
I've been waiting for a language derived from Smalltalk/Self, which would fix their disadvantages.
With Newspeak you almost nailed it!

IMHO, the following capabilities would make it close to ideal - selector namespaces, multi-method dispatch, variable number of parameters. :-)

Your post about dependency injection raises a very important problem.
In Java I had to invent very sophisticated scheme of injecting dependencies into anything and virtualizing constructors.

http://subject-ivity.blogspot.com/2007/09/injecting-dependencies-into-domain.html
http://subject-ivity.blogspot.com/2008/01/mocking-real-constructor-invocation.html

Looks bizarre, isn't it? I just wanted to be able creating pure object systems as in Smalltalk without loosing Spring's capabilities. :-)
The main advantage of the solution is that DI becomes transparent to application code.

Constructor-based injection, which is only possible in Newspeak at the moment as far as I understand, is not a very elegant solution.
It looks very similar to checked exceptions in Java.
Local change has non-local effect - you have to pass dependency thoughout the whole class chain.
It reminds me the notion of divergence in mathematics.

When I implemented my own DI solution in Smalltalk I used a different approach. I borrowed the idea of "perspective" from Us (subjective Self).

Suppose we have an implicit parameter called "environment", which is basically a map or an object with slots, pointing to dependees. On module level environment is identical to the module's namespace.
It is possible to derive a new environment by sending messages to the current one.
Plus we need a syntax of running any statement within a new environment. So effectively we introduce a limited version of dynamic scoping:

InnerClass new doSomething (+) environment dependency: SomeClass new

InnerClass = (|
private Dependency dependee = environment dependency
|)

Does it sound clear?

Gilad Bracha said...

Uladzimir,

I'm glad you like what you see. The language is not likely to grow very much though. What makes Newspeak work well is that it derives a lot of power from a small number of constructs applied uniformly - not from accreting lots of features.

In past posts I've mentioned that I have not found selector namespaces, or any of the many other features proposed to address the same problems, to be quite compelling.

Variable numbers of parameters are completely unnecessary. We have literal tuples, so you can pass any extra parameters in a tuple. Indeed, in Smalltalk syntax, it's even lighter weight that way.

As for DI, I can't say I follow your argument. There is no need for DI in Newspeak (constructor or otherwise).

Uladzimir Liashkevich said...

Gilad,
Thanks for your quick reply.

Maybe my post didn't explain well the rationale behind the technique not taking into account DI.
Certainly, the language itself doesn't need any techniques.
But there are some development habits/practices that may require some support on library/language level.

For example, I always try to follow TDD strictly. Ideally, one would want to substitute any real objects with mocks for unit-testing.
Consider the following scenario (let me borrow your code snippet):

public class MovieLister = (
|
private movieDB = ColonDelimitedMovieFinder from:’movies.txt’.
|)

Now I need to implement a unit-test on MovieLister. It means that all other classes must not take part in the scenario.
Thus I have to replace ColonDelimitedMovieFinder with some mock object.

Techniques, which allow to do the substitution are very similar to the ones found in DI implementations.
So we can talk about injection via constructor or any other kind of injection.

Newspeak has excellent feature (I missed it greatly in Smalltalk when dealing with classes/global variables) of externalizing dependecies.
But if we aim at 100% unit-test coverage, then application code may become very verbose due to passing all dependencies to constructors.

Therefore I proposed the idea of introducing an explicit parameter "environment", which would eliminate the necessity of passing dependencies explicity via constructors.

Hope this explanation helps.

Gilad Bracha said...

Hi Uladzimir,

Wrt test driven development, I don't think anything changes. First, let me bring back the correct code snippet:

public class MovieLister finderClass: MovieFinder = (
|
private movieDB = MovieFinder from:’movies.txt’.
|)
(
public moviesDirectedBy: directorName = (
^movieDB findAll select:[:m |
m director = directorName
].
)

This is slightly different than the one you cite - the post actually explains that the original won't work.

Now, in this version, the module is already parameterized and you can feed it your mock classes as desired. This will always be the case for any module definition.

It's easy to set up a method that wireds the modules together with mock classes, the same way yoou wire modules together for real. It can even be parameterized by a tuple of classes which my be real or mock.

Now, it may be that you want to do unit testing below the module level. I'm not sure when/if that is sensible, but if you do, then you still replace any non-private class within the module with a mock up, by overriding the class in a subclass of the module definition.

The only thing you wouldn't easily be able to replace would be private nested classes - but these really are part of the same implementation as their enclosing class and will never change, so I see no point in replacing them.

So I don't see much need for DI in Newspeak; certainly nothing that is worth the overhead of these frameworks.

Uladzimir Liashkevich said...

"Now, it may be that you want to do unit testing below the module level."

Exactly.

"I'm not sure when/if that is sensible, but if you do, then you still replace any non-private class within the module with a mock up, by overriding the class in a subclass of the module definition."

I understand this.
In this case one needs to externalize all dependencies via class constructor.
My point that this solution is too verbose. Also it mixes two aspects in one place - application-specific parameters and dependencies.
To some extent passing dependencies may be considered as an act of meta-programming, if it is used for unit-testing only.
And like objects/mirrors separation, would it make sense to separate passing application parameters and dependencies?

Gilad Bracha said...

Uladzimir,

We may not be able to resolve this via this medium. I'll try once again:

"In this case one needs to externalize all dependencies via class constructor."

No. Either your mock ups are specific to this module (as they are likely to be if they are used to replace nested classes of the modue) - then you'd have to define them somwhere. Just define them in a subclass of the module definition as overrides of the originals. No need to change the constructor.

On the other hand, maybe you have some generic mockups. These do have to be passed in - but all can passed in via a single parameter (call it environment if you wish).

Uladzimir Liashkevich said...

Ok, it is clear.
So your idea is to have a clean design where all dependencies are passed explicitly.

Then I have another question if you don't mind.
Will the design/implementation of Newspeak allow to make substantial changes to its internal mechanisms?

For example, modify method lookup, introduce AOP, seamlessly implement CLOS on top of it, etc. :-)

Gilad Bracha said...

Uladzimir,

Newspeak will be open sourced, so you should be able to modify it as you please.

How easy this would be depends on a specific implementation. Running on a Squeak like VM, as we are now, things are very easy to change. Other implementations may be harder to tweak.

Having said that, I'm not a huge fan of reflective intercession at the level of language semantics. Languages generally benefit from having a predictable semantics that stems from a consistent vision - the vision of the one person (or a small group) who designed it.

It is best that as the language evolves, the designer stays on top of things and ensures that things stay consistent with that vision (The benevolent dictator model). That is my intent with Newspeak.

If a clean, *simple*, mirror based MOP was available, I'd certainly consider it. It would have to fit within a mirror based reflective architecture, which is a very nice research project. I'm not sure exactly how to do it, but clearly it won't look like CLOS. I don't expect to do that in the foreseeable future.

As for AOP: Much of what can be done with AOP is better done with mixins. Many other uses of AOP undermine modularity in a fundamental way and so are a poor match for Newspeak. If you still need to do them, reflection allows you to do so if you have the right permissions.
So I don't see how AOP as it currently exists could fit in Newspeak.

Paul Beckford said...
This comment has been removed by the author.
Ryan said...

Newspeak looks really great. Can't wait to try it out.

Will Newspeak be able to run on the Cog VM?

Gilad Bracha said...

Hi Ryan,

We are trying to stay coordinated with Eliot. So either Cog wlll actually support Newspeak, or we should be able to adapt Cog so that we have a version that does.