A DSL with a View

In a previous post, I promised to explain how one might define UIs using an internal DSL. Using an internal DSL would allow us to capitalize on the full power of a general purpose programming language and avoid having to reinvent everything from if-statements to imports to inheritance.

Hopscotch, Newspeak's UI framework, employs such an internal DSL.  Hopscotch has been discussed in a paper and in a talk.  It will take more than one post to describe Hopscotch; here we will focus on its DSL, which is based on the notion of fragment combinators.

Fragments describe pieces of the UI; they may describe individual widgets, or views constructed from multiple pieces, each of which is in turn a fragment. A fragment combinator is then a method that produces a fragment, possibly from other fragments.

One of the simplest fragment combinators would be label:, which takes a string. The expression

label: 'Hello Brave New World' 

would be used to put up the string "Hello Brave New World" on the screen.  Other examples might be

button: 'Press me' action: [shrink]

which will display a button

that will call the method shrink when invoked.  The combinator button:action: takes two arguments - the first being a string that serves as the label of the button, and the second being a closure that defines the action taken when the button is pressed. Closures in Newspeak are delimited with square brackets, and need not provide a parameter list if no parameters are required.  This is the most lightweight syntax for literal functions you will find. Along with the method invocation syntax, where method names embed colons to indicate where arguments should be placed, this gives a very readable notation for many DSLs.

Further examples:

row: {
  button: 'Press me' action: [shrink]. 
  button: 'No, press me' action: [grow].
}

The row: combinator takes a tuple of fragments (tuples in Newspeak are delimited by curly braces, and their elements are separated by dots) as its argument and lays out the elements of the tuple horizontally:

the column: combinator is similar, except that it lays things out vertically

column: {
  button: 'Press me' action: [shrink]. 
  button: 'No, press me' action: [grow]
}

produces:

In mainstream syntax (Dart, in this case) the example could be written as

column(
  [button('Press me', ()=> shrink), 
   button('No, press me', () => grow)]
 )

The Newspeak syntax is remarkably readable though, and its advantage over mainstream notation becomes more pronounced as examples grow. Of course none of this works at all if your language doesn't support both closures and literal lists/arrays.

So far, this is very standard stuff, much like building a tree of views in most systems.  In most UI frameworks, we'd write something like

new Column([new Button('Press me', ()=> shrink), 
            new Button('No, press me', () => grow)]
    )

which is less readable and more verbose.  Since allocating an instance is more verbose than calling a method in most languages, the fact that fragment combinators are represented via methods, which act as factories for various kinds of views, helps make things more concise. It's a simple trick, but worth noting.

The advantage of thinking in terms of fragments becomes clearer once you consider  less obvious fragments such as draggable:subject:image:, which takes a fragment and allows it to be dragged and dropped. Along with the fragment, it takes a subject (what you might call a controller) and an image to use during the drag. Making drag-and-drop a combinator means everything is potentially draggable. Conventional designs would make this a special attribute of certain things only, losing the compositionality that combinators provide.

Presenters are a a specific kind of fragment that is especially important. Presenters provide user-defined views in the Model-View-Controller sense.  To define your own view, you subclass Presenter. Because presenters are fragments, any user defined view can be part of a predefined compound fragment like column: or draggable:subject:image:.

A presenter has a method definition which computes a fragment tree which is used to render the presenter. The fragment DSL we discussed is used inside of presenters. All the combinators are methods of Presenter, so they are inherited by any class implementing a view, and are therefore in scope inside a presenter class.  In particular, combinators are available in the definition method.

To see how all this works, imagine implementing the well known todoMVC example.  We'll define a subclass of Presenter called todoMVCPresenter  to represent the todoMVC UI.  The UI is supposed to present a list of todo items. It consists of a column with:

1. A header in large text saying "todos" 
2. An input  zone where new todos are added.
3. A list of todos, which can be filtered using controls given in (4). 
4. A footer, that is empty if there are no todos at all. It materializes as a set of controls once there are todos.

We can translate these requirements directly:

definition = (

     ^column: {

      (label: 'todos') hugeFont.

      inputZone.

      todoList.

      footer.

     }

)

More notes on syntax:  methods are defined by following their header with an equal sign and a body delimited by parentheses; ^ signifies return; method invocations that take no parameters list no arguments, e.g., inputZone, not inputZone(); chained method invocation does not require a dot - so it's 

(label: 'todos') hugeFont rather than label('todos').hugeFont.

We haven't yet specified what inputZone, todoList and footer do. They are all going to be defined as methods of todoMVCPresenter.  We can define the UI in such a top down fashion  because we are working with a language that supports procedural abstraction. You get it for free in an internal DSL.  

We can then define the pieces, such as

footer = (

 ^subject todos isEmpty 

 ifTrue: [nothing]

 ifFalse: [controls]

)

Here, we use conditionals to determine what view to produce, depending on the state of the application.  The application logic is embodied in the controller, subject, which we query for the todos list. The nothing combinator does exactly what it says; controls is a method we would have to define in todoMVCPresenter, detailing what should appear in the footer if it is visible.  Again, the code corresponds closely to the natural language description in bullet (4) above.

To elaborate todoList we'll need a loop or recursion or something of that nature; in fact, we'll use the higher order method collect:, which is Newspeak's version of map. 

todoList = (^list:[subject todos collect: [:todo | todo presenter]])

The list: combinator packages a list of fragments into a list view. We pass list: a closure that computes the list to todo items. 

Aside: We could have passed it the list itself, computed eagerly. Often, fragment combinators take either suitable fragment(s) a closure that would compute them.

To compute the fragment list we compute a presenter for each individual todo item by mapping over the original list of todos.
The closure we pass to collect: takes a single parameter, todo. Formal parameters to closures are introduced prefixed by a colon, and separated from the closure body by a vertical bar.

What are the odds that higher order functions (HOFs) were part of your external DSL? Even if they were, one would have to define a suite of useful  HOFs. One should factor the cost of defining useful libraries into any comparison of internal and external DSLs.

The Hopscotch DSL has other potential advantages. Because fragment combinators are methods, you can override them to customize their behavior.
We believe we can leverage this  to customize the appearance of things, a bit like CSS. To make this systematic, we expect to define whole groups of overrides in mixins. I'm not showing examples where Hopscotch is used this way because we have done very little in that space (and this post is already too long anyway). And we haven't spoken about the other advantages of Hopscotch.
such as its navigation model, lack of modality and very clean embodiment of MVC.

Ok, now it's time for the caveats.

1. First and foremost,  Hopscotch currently lacks a good story for reactive binding.  In our example, that means you'd have to put explicit logic to refresh the display in some of the controls.  This makes things less declarative and harder to use. We always planned to solve that problem; I hope to address it in a later post.  But the high order bit is that we have code in a general purpose language that gives a very readable, declarative description of the UI. It corresponds directly to the natural language description of the requirements.
2. Hopscotch lacks functionality in order to support richer UIs, but the design is naturally extensible: one adds more fragment combinators.
3. We also want more ports, especially to mobile/touch platforms. However, Hopscotch has already proven quite portable: it runs on native Win32, on Squeak's Morphic and on HTML (the latter port is still partial, but that is just an issue of engineering resources).   More ports would help us deal with another controversial goal - defining a UI platform that works well across OS's and devices.

Regardless of the current engineering limitations, the point here is simply to show the advantages of a well-designed internal DSL for UI.  The lessons of Newspeak and Hopscotch apply to other languages and systems, albeit in an attenuated fashion.