A place to be (re)educated in Newspeak

Friday, April 02, 2010

The Brave New World of Full Service Computing

I’ve been trying to explain this for some six years with little success, but I’ll try again. It may be easier this time.

For the past generation, we have been living in a world of self-service computing, more commonly knows as personal computing. The person using/owning a personal/self service computer is responsible for managing that computer themselves. All the digital chores - installing and updating software, preventing malware infection, backing up storage etc. are the user’s responsibility. Most users hate that.

Worse, most users really can’t handle these chores at all. They run ancient versions of applications because they are deathly afraid to tamper with anything that somehow works. Their computers are warrens of digital disease, crawling with viruses. Their data is rarely backed up properly.

The evolution of network technology enables a new model: full service computing. Full service computing means that you abdicate a level of fine control over the computer to a service that handles your digital housekeeping. I discussed some implications of this in a post back in January.

You still choose applications that you want - but you must get them from the service; the service installs these applications for you; it keeps them up to date as long as you use the service; and it it ensures that they are safe to use. The service also backs up your data automagically, allowing you to easily search through old versions as needed. In the extreme case, the service actually provides you with the computers you use. That helps it ensure seamless software/hardware compatibility.

We can increasingly see vignettes of this brave new world. Web apps running in the cloud are one step in this direction; web apps storing data locally are a step further. A complete platform designed to hide the raw machine from the user is even closer to this vision. Such platforms started out in the world of mobile phones, and are clawing their way upwards from there: Android and iPhone/iPad. Windows 7 mobile is quite similar to the iPhone model (odd, isn’t it?). And ChromeOS is not dissimilar.

Tangent: How does Google keep Android and ChromeOS aligned? It doesn’t - I believe they are making it up as they go along. Eventually they should converge.

We still haven't seen an integration of the app store model with backup, but it can't be that far off. An option for MobileMe(insert Apple trademark here) subscribers to have an image of their iPad backed up on Apple's servers and browsable via a Time Machine (and again, Apple trademark) style interface is easy to imagine. This can then extend to data you want to share collaboratively, while still retaining a local copy for offline use.

In time, most users will gravitate away from the self service computers they use today to software service platforms. This goes hand in hand with simpler UIs such as the iPad’s. One of the technical features in this evolution is the disappearance of the file system in favor of a searchable database of objects, as I mentioned in a previous post.

Tangent: The move from personal computing to software services accessed via simpler devices is what is truly significant about the iPad. Those who worry about details such as whether it has a camera or Flash are completely missing the point.

All this is already happening, which makes it easier to explain the idea of objects as software services (see the video) now than in 2004/2005. Consider the trend of programming languages toward a pure object model: all data are objects.

Tangent: This is orthogonal to functional programming. Objects can be immutable, as in, e.g., Avarice.

So, we now have programs dealing with objects in primary storage running on platforms whose secondary storage consists of objects as well. Initially, programmers find that they must explicitly manage the transition of data from primary to secondary storage.

The programmer’s task is simplified if that transition can be automated. This is an old idea known as orthogonal persistence. The Achilles’ heel of orthogonal persistence was the divergence of in-memory data representation and persistent data representation. This argued for manual management of the transition between primary and secondary storage.

However, circumstances are changing. The difference between transient (in-memory) and persistent (secondary storage) representations was driven by several factors, none of which apply in the full service computing model.

The first factor was that programs changed their internal representation while persistent data lived forever. Orthogonal persistence broke down when old data became incompatible with program data structures.

In our brave new world, programs co-evolve with their data so this is not an issue. The service manages both the application and secondary storage, updating them in lock step. Thus, we have data that is not only orthogonally persistent, but orthogonally synchronized.

The second factor was that persistent data formats were a medium of exchange between different programs with different internal data representations. This relied on the file system, may it rest in peace. The new applications are in fact compelled to keep their persistent data in silos tied to the application, and communicate with other applications in constrained ways.

This sets the stage for a model where programs are written in a purely object oriented language that supports orthogonal synchronization of both programs and data. This model eases the programmer’s task, and sits perfectly in the world of full service computing.

I have been discussing language support for this vision of software services publicly since my talk at DLS 2005, and privately long before. Newspeak’s modularity and reflection features dovetail perfectly with that.

Newspeak allows programs to be represented as objects; these objects are compact and self contained, thanks to the modularity of Newspeak code. Newspeak programs can be updated on the fly, atomically. Together, these two features provide a foundation for synchronizing programs. And the Newspeak VM supports shallow object immutability, which helps us synchronize data. On this basis, we have implemented experimental support for orthogonal synchronization, though it is very immature. I hope that in the future, we can provide an open software service based on this idea.

Of course, the radical edge the objects as software services vision is the complete abolishment of versions for libraries as well as applications. That is, I admit, untested, controversial and futuristic. However, with respect to applications, it is already starting to happen.

24 comments:

zohar said...

Oh so your buying an iPad.

Eliot said...

Hi Gilad,

as you state, there are web apps that get close to your vision. What's the chance of a well-designed language-oriented approach catching on against the tide of ad-hocery?

cheers
Eliot

Gilad Bracha said...

Zohar: soon, perhaps. But I have my mac. The point is that in some years (5-10) only people like us will use traditional computers. Most people won't need or want them.

Eliot: Web apps are still a ways off, but they will get there. The point is, how hard is it to build them, and build them right? Newspeak should make it easy to do so, and allow you to deliver it via the web. Don't be a defeatist: help get it done instead!

Paul said...

Hi Gilad,

As you know, I've been following Newspeak and your blog for awhile now, and your more conservative ideas around SaaS have such a feeling of inevitability about them that I'm left thinking when rather then if they will ever come to pass.

Being a good idea in itself isn't sufficient alas. Timing seems to be important too, and an idea can appear before people are ready. Our industry can be very short sighted and not see beyond the end of its nose.

Are you finding that Newspeak is suffering from the phenomena I describe?

After hearing about your funding difficulties I had hoped that a far sighted benefactor would have taken you under their wing by know. An internet company that makes most of its money through advertising came to mind :)

I must admit I am disappointed that you haven't been able to announce such a benefactor by now. I'm also concerned about your own personal financial circumstance.

I feel like I'm prying, but I am a true well wisher. Are you in a position to give an update about your funding situation?

Regards,

Paul.

(PS. After witnessing the amount of money that gets wasted on developing absolute tripe, I find it absolutely amazing that something as revolutionary as Newspeak cannot attract funding - there I've said it!)

Gilad Bracha said...

Hi Paul,

Thank you for your concern!
I wish you wouldn't worry about me personally - if my supply of foie gras runs low, I'll let you know :-).

Newspeak itself remains unfunded however. Far sightedness is a much rarer disorder than short sightedness.

We do have several students doing work on it though. One of these has just completed his masters (we'll announce his work soon enough) and should be continuing with a PhD, I hope. There are a couple of more master students in the pipeline.

The fact that work continues, and we've put out a couple of update releases, and expect to have more soon, means that Newspeak's situation is actually better than might be expected.

Of course, we'd make much faster progress with a fully funded team of full time professionals; the absence of said team is frustrating at times. Such is the cost or being ahead of the curve.

James said...

Hi Gilad

The new applications are in fact compelled to keep their persistent data in silos tied to the application,

So, somewhat cynically, this doesn't sound much like objects to me. Sounds like - let's see, an encapsulated component - a bunch of code offering a set interface, with an opaque implementation - that operates on collection of instances of an opaque data structures. Those data structures instances - any or all of 'em - can only be accessed via the encapsulated component.

Yep - these things sound like Abstract Data Types to me, not objects...

James

Gilad Bracha said...

James: Except that no types are involved, and the silos communicate via message-passing, so any resemblance to ADTs is, as you say, cynicism.

Martin Probst said...

One of the problems with blurring the boundary between storage and program is that it can be difficult to manage lifetimes. I think we can take it as a given that data lives much longer than the programs operating on it. The separation between data and program thus comes in handy if you store your data in a model that is not bound to a particular application (e.g. the relational model). Also, if your program gets corrupted, it is relatively easy to save the data. Not so if your program is entangled with the data.

That being said, I could imagine a programming environment that has lots of immutable objects and small amounts of mutable state as a working set. If you tinker with it all the way down, you might find that a generational garbage collector allows you to persist application state in a transactional manner (e.g. call commit at known good points -> perform garbage collection and persist/synchronize the permanent generation).

This is already happening to some degree in typical JavaScript applications, except that you have to build all the abstractions yourself.

Gilad Bracha said...

Martin: Good points. If you look at the objects-as-software-services talk and write-up, there is a notion of transient vs. persistent references. Syncing happens at very particular points (program is quiescent). So I think the old problems of orthogonal persistence can be overcome.

James said...

Except that no types are involved

well no static types - but I don't think types are essential to my argument...

and the silos communicate via message-passing,

sure. but unless you are subclassing whole applications, I don't think it's relevant whether
messages are resolved dynamically a la Newspeak
or statically al la Ada. I think the first version of CLU used dynamic binding by mistake - so I don't think this is the distinction I was trying to make.

so any resemblance to ADTs is, as you say, cynicism.

So thinking again, rather than "cynical", perhaps "topological" would have been a better word. As I understand this model, data lives only in silos within applications - data are owned by the applications. So I might have a contacts application and an email application and a wordprocessor application, each of which stores persistent data objects (contacts, messages, documents) in such a way that they are only directly accessible within that application.

This, for example, follows the app-centric model of the iPhone rather than a desktop metaphor. If I want to get a list of all my documents, i do that within the word-processor. If I want to email one of those documents, the word-processor application has to somehow coordinate with the email application.

In terms of the (yes, message-passing) interface of each application, statically or dynamically, the key persistent internal data objects don't ever cross that interface (leave the silo). Actually, It's a pretty strong ownership condition. Whatever gets sent
from the wordprocess to the emailler isn't the internal data, most likely it is some external representation of that data at most it would be a transient deep clone.

So while this isn't traditionally statically typed ADTs the topology is similar to something like CLU or Ada - where a type or package provides all the operations for dealing with data of a particular type - rather than self describing self sufficient objects
that are accessible and that provide sophisticated behaviour outside their owning application silo.

Gilad Bracha said...

James: In ADTs, the data are exchanged with other ADTs, as opaque (or semi-opaque, say comparable via operators such as identity) entities. This is why types are essential to that model - otherwise, you don't know who the data belongs to. S

o I don't see the silo model as ADTs at all. Rather, apps are objects that communicate by pure message passing. More like actors than ADTs.

Pete F said...

Observation:

"Full-service computing" isn't a bad moniker, but I prefer

"fat sow, with blind helpless users sucking on its teats -computing"

The extra 45 characters really do pay their way, imo


Question:

Does Clojure style persistence fit somewhere here, especially for Avarice? The word "persistence" seems to have an unusual meaning to Clojurerers (you may know how it came to be overloaded) but a "nothin but diffs" state model seems to me like a good match for very mobile objects.

Gilad Bracha said...

Peter F:

"Fat-sow' or not, the fact is that most people would rather get their meat (I find milk too mixed a metaphor) without running a pig farm or being their own butcher.

Re: persistence. I assume you are referring to functional data structure meaning of "persistent". I don't immediately see a relationship (though that concept may apply in Sloth and Avarice).

It isn't terminology I use. When I say "persistent" I mean persistent the way it is widely used - committed to non-volatile storage.

James said...

Still going:

In ADTs, the data are exchanged with other ADTs, as opaque (or semi-opaque, say comparable via operators such as identity) entities.

Sure. But isn't that how data are exchanged between applications in this model?

This is why types are essential to that model - otherwise, you don't know who the data belongs to.

Right - but in dynamic systems we've lots of other options there...

I don't see the silo model as ADTs at all. Rather, apps are objects that communicate by pure message passing. More like actors than ADTs.

Right - I think. I guess the questions for me are - what kinds of objects can be passed along as arguments to the message; and if any of those objects "belong" to an application, where does their behaviour come from?

Now one answer is that the message arguments are only basic library objects, integers, strings, or perhaps some kind of JSON, but without much application-specific behaviour (because that's all in the application). This is what I've been assuming.

Now this morning I realised perhaps you have another answer: the arguments can be any kind of object, even if it belongs to the application silo because the class of that object will be nested inside the application object. Of course, this kind of relationship is precisely what Newspeak gives you compared with Self and Smalltalk,

I'm not convinced this addresses all the problems with upgrades and synchronisation, but I can see that it could help.

Gilad Bracha said...

James: We're just starting on the actor model for Newspeak (there's a grad student on the case).

The current plan is to something very close to E. That means that not only values (deeply immutable objects) can cross actor boundaries, but remote references, which represent specific (and possibly mutable) objects in another actor. Such a reference is really a convenient extra gateway to another actor. So it's not arbitrary objects (otherwise, you're back to threads, which are BAD BAD BAD). On the other hand, it's not strictly values.

In such a model, any other process/app is an actor and you communicate with it via message passing.

So, overall, you're not far off.

James said...

James: We're just starting on the actor model for Newspeak (there's a grad student on the case).

excellent, excellent. I keep trying to encourage students to look at Newspeak but so far no joy.

... not only values (deeply immutable objects) can cross actor boundaries, but remote references, w.... On the other hand, it's not strictly values.

all makes sense. The Actor "owns" its representation, as I'd put it in another life.
I guess the interesting question (in that other life) is whether you stop the wrong objects being accessed from the wrong place - or the corollary - what does it mean for an object to turn into a remove reference. Again, the class nesting can nicely define the right place (or the wrong place): with VM support you can either trap wrong message sends, or wrap objects in remote references.

Hmm. Interesting. How dynamic do you want to be?

Gilad Bracha said...

James: for sanity's sake, I think one wants to make sure that only remote references and pure values cross the actor line. That is, enforce it (unlike, say, Scala actors). Values may require VM support (a deeply immutable bit) unless one works in apure functional setting like Avarice/Sloth. Remote refernce wrapping can be done by the compiler or the actor library (depending on whether async sends have a special syntax or not; I lean toward special syntax in this case).

Steve said...

Hi Gilad,
It probably won't surprise you to hear that I am no fan of this model for the future of mass-market computing.

I see too much evidence of evil actors forcing their users to one particular service offering, rather than a world of competing services.

Worse still, there is growing evidence that such service providers will behave as if they own your data.

Google already acts in this way and Apple is moving in this direction.

"DRM? Not us, it was those big bad studios. They made us do it. We fought the good fight and eventually forced them to get rid of it ..."

Except that music downloaded from iTunes and LaLa is watermarked for a particular user. Put your music in the cloud and suddenly it's not yours anymore and we have DRM mark 2.

Gilad Bracha said...

Hi Steve,

I imagine you've seen:

http://www.appleinsider.com/articles/10/04/08/apples_iphone_4_sdk_license_bans_flash_java_mono_apps.html

this is most unfortunate, and rather short sighted in my view. This is not inherent to the software service model however. I can envision a much more open version.

In any case, whether one is a fan or not, one cannot ignore the facts. The transition is happening, like it or not.

Steve said...

Hi Gilad,
Indeed. I think it's both incredibly arrogant and practically unenforceable. Unless you require apps to be submitted in source form, how can you police it?

Still, it will make life uncomfortable for developers who choose to develop in languages other than those Apple deems acceptable.

No Newspeak on the iPhone or iPad any time soon then! No DSLs; no code generation of any kind it would seem!

Thanks Steve. Another brilliant move.

As you say, this sort of move is not inherent in the kind of full-service approach you are advocating, but I think it is telling that the most significant example of such a service provider grasps at such draconian measures.

Gilad Bracha said...

Steve,

a) The service model is more centralized than the historic personal computer, and so lends itself more readily to this sort of thing, I confess.

b) Jobs' moves are usually brilliant; it is generally a joy to watch. I just hope he's wrong this time. It's early days yet, and every action begets a reaction.

c) Newspeak makes perfect sense on the iPhone. One reason I did not pursue it was that even prior to this latest twist, the model made it difficult. If one could simply deploy a shared VM and be done, it would be quite simple. I thought about compiling into Objective-C. The absence of GC on the iPhone version was a major deterrent. I had hoped that would change,

However, now there seems to be no reason to bother.

d) It's fascinating to watch this unfold. I bet Google is thrilled. Never a dull moment.

naedyr said...

"We still haven't seen an integration of the app store model with backup"

Steam cloud has had this feature for a couple of years now:
http://en.wikipedia.org/wiki/Steam_(content_delivery)#Steam_Cloud

the user just supplies the hardware, installs steam, then logs in. Steam handles the rest

shawn said...

Gilad,

I just came across your Google tech talk on this subject:
http://www.youtube.com/watch?v=_cBGtvjaLM0

I was quite surprised by some of the comments coming from google folk. Given that they already are in the spirit of maintaining software services on an ongoing basis and they couldn't draw the parallels was... entertaining. Maybe it's a generational thing? The first "service" rather than web page I can remember using is Hotmail some time around '97 '98. I've been using software services for literally half my life.

Anyway I digress. The one point I found didn't need to be as dire is passing on 3rd party library live update issues to your customers. You could equally have the client app pull HEAD from the 3rd party or YOU (as the app vendor) could act as a re-distributor of the library at a certain snapshot in time (notice I'm trying hard not to use the word version).

The cost then becomes yours to manage. Digress too much and you'll be stuck managing and bug fixing someone else's old library that doesn't even look the same anymore. The point is that even in code bases we own and serve as web apps, we constantly make trade offs between prioritizing deploying a new feature today or dealing with tech debt. Pushing a deploy with a newer version of the 3rd party lib is no different.

I realize this provides a slippery slope OUT of nirvana but it's also a migration path and one that's familiar to web devs today (e.g. Rails + Bundler).

Gilad Bracha said...

Hi Shawn,

I agree with your comments.

Re: the entertaining lack of appreciation: Over the years things have become more and more clear. By now, everyone understands that off line + sync is the way to go.

Of course, the reality is very ad hoc, as I feared it would be. However, progress is being made.