A place to be (re)educated in Newspeak

Thursday, September 03, 2009

Systemic Overload

Type-based method overloading is a (mis)feature of mainstream statically typed languages. I am very much against it - I’ve spoken about it more than once in the past. I decided I should put my views on the matter in writing, just to get it out of my system. I’ll start with the better known problems, and work my way up to more exotic issues.

Throughout this post, I’ll use the term overloading to denote type based overloading. Arity based overloading, as in e.g., Erlang, is pretty harmless.

When introducing a feature into a language, one has to consider the cost/benefit ratio. You might argue that overloading lets you separate the handling of different types into different methods, instead of doing the type dispatch inside one method, which is tiresome and costly. A classic example are mathematical operators - things like +.

This argument would have merit, if the overloading was dynamic, as in multi-methods. Since it isn’t, overloading doesn’t solve this kind of problem. Not that I’m advocating multi-methods here - they have their own problems - but at least they are based on accurate type information, whereas overloading is based on crude static approximations.

Consider this code (loosely based on an example from Dave Ungar’s OOPSLA 2003 keynote).

class VehicleUtilities {

int numberOfAxles(Vehicle v) { return 2;} // a plausible default

int numberOfAxles (Truck t){ return 3;}

}




Vehicle v = new Truck();

VehicleUtilities u = new VehicleUtilities();

u.numberOfAxles(v); // returns 2

This simple example illustrates the dangerous disconnect between static and dynamic information engendered by overloading.

What exactly is the benefit of type based overloading? It saves you some trouble inventing names. That’s it. This may seem important, but the only case where it is actually needed is when writing constructors - and only because the language doesn’t allow you to give them distinct names.

Constructors are a bad idea, as I’ve already explained, so let’s assume we don’t have them. For methods, type based overloading provides a trivial rewrite and that is all. I don’t think it is that hard to give the different operations different names. Sometimes, the language syntax can make it easier for you (like keyword based method names in Smalltalk), but even in conventional syntax, it isn’t that difficult.

You pay for this convenience in myriad ways. The code above exemplifies one set of issues.

Another problem is the risk of ambiguity. In most overloading schemes, you can create situations where you can’t decide which method to call and therefore declare the call illegal. Unfortunately, as the type hierarchy evolves, legal code can become illegal, or simply change its meaning.

This means that existing code breaks when you recompile, or does the wrong thing if you don’t.

Overloading is open to abuse: it allows you to give different operations the same name. Altogether, you need style guides like Effective Java to warn you to use overloading as little as possible. Language constructs that require style guides to tell you not to use them are a bit suspect.

Ok, you probably know all this. So what contribution does this post make? Well, the systemic costs of overloading in terms of designing and engineering languages are less widely appreciated.

Overloading makes it hard to interoperate with other languages. It’s harder to call your methods from another language. Such a language may have a different type system and/or different overload rules. Or it may be dynamically typed.

You often find a dynamic language implementing multi-method dispatch to approximate the behavior of overloaded methods it needs to call. This is costly at run time, and is a burden on the language implementor.

Scala supports overloading primarily so it can call Java; you might say overloading is a contagious disease, transmitted from one language to another through close contact.

In general, overloading adds complexity to the language; it tends to interact with all sorts of other features, making those features harder to learn, harder to use, and harder to implement. In particular, any change to the type system is almost certain to interact with type based overloading.

Here are some examples. Answers at the end of this post.

Exhibit 1: auto-boxing

void foo(Number n) { ... }

void foo(int i) { ...}



foo(new Integer(3)); // quick, what does this do?

Exhibit 2: the var args feature in Java

void foo(String s, Number... n) {...}

void foo(String s, int i, Integer... n) {...}

void foo(String s, int i, int... n) {...}

void foo(String s, Integer i, Integer... n) {...}

void foo(String s, Integer i, int... n) {...}


foo(“What does this do?”, 1, 2);



Exhibit 3: Generics

void foo(Collection c) {...}

void foo(Collection < String> c){...}


void foo(Collection < Boolean> c){...}


void foo(Collection < ? extends String > c){...}


void foo(Collection < ? super String > c){...}



Collection< String> cs;

foo(cs);


/* Which of these is legal? What would happen if we didn’t use erasure? You have 3 seconds. */


Each of the above exhibits shows a specific type system extension which gets entangled with overloading.

You might say you don’t care; these are pretty sick examples, and the language designers sorted out the rules. What is their suffering to you? Well, these complications all have cost, and since resources are finite, they come at the expense of other, more beneficial things.

Tangent: People tend to think that the language design and implementation teams at large companies like Sun or Microsoft command vast resources. In reality, the resources are spread pretty thin considering the scope of the task at hand.

The more time is spent chasing down these issues, the less is spent focusing on doing stuff that actually buys you something. Products ship later and have more bugs and less desirable features, because people spent time worrying about gratuitous complications.

This is not hypothetical. I know the poor soul who wrote the spec for this might have spent his time doing something else, like day dreaming or adding closures. The compiler writers could have fixed more outstanding bugs, perhaps even reaching a state where there were no open bugs. I know they tried. The testers could have tested for other, more pressing issues and discovered more bugs to open, before shipping. These are the wages of sin - and the sin is unjustified complexity.

Now, for the sake of balance, I should say that overloading, and language complexity in general, do have one advantage I haven’t yet mentioned. They open up great opportunities for training, support and consulting. You can even write some really cool books full of language puzzlers.

It’s worth noting that this kind of overloading is only a concern in languages with a mandatory type system. If you use optional typing (or just dynamic typing), you aren’t allowed to let the static types of the arguments change the meaning of the method invocation. This keeps you honest.

Will future language designs avoid the problems of overloading? I wish I was confident that was the case, but I realize overloading is an entrenched tradition, and thus not easily eradicated.

However, the overarching point I want to make is that the costs of complexity are insidious - unobvious in the short term but pervasive over the longer haul. KISS (Keep It Simple, S - where the binding of S is open to interpretation).


Answers:

  1. The issue here is whether we auto-unbox Integer(3) first to produce an int (and call foo(int)) or resolve overloading in favor of foo(Number) and don’t unbox at all. Java does the latter. The reason is to remain compatible with older versions.
  2. This is ambiguous. Except for the first declaration, no method is more specific than the others.
  3. They all have the same erasure, and so the example is illegal. If we did not use erasure, than foo(Collection < String >) would be the most specific method.

35 comments:

John Cowan said...

My view is that overloading in Java is mostly needed between primitive types: it's often useful to provide foo(int), foo(long), and foo(double) versions. One could do this with foo(Number), autoboxing, and RTTI, but that's more of a headache than overloading.

Occasionally there are other types that are semantically closely related, but for whatever reasons have no type relationships. For example, before Comparable, I wound up with methods overloaded on Numbers and Strings, and so on.

Osvaldo Doederlein said...

In Exhibit 3 all methods are identical, I guess you didn't use HTML escapes for < and >.

Gilad Bracha said...

Osvaldo,

Ouch. Should have checked the output more carefully. Thanks for pointing it out.

Barry Kelly said...

Overloading is the static analogue of multi-method dispatch. Multi-method dispatch is useful in and of itself - consider the contortions that the Visitor pattern has to go through with double dispatch just to end up in the right method depending on the kind of argument. Even then Visitor is limited to a number of distinct types which the visited and visiting have agreed upon, limiting extension in derived types.

When you're in the extreme end of generic programming in e.g. C++, this static analogue really starts coming into its own. Here, overloading is not equivalent to choosing distinct names, because the appropriate name is unknown to the caller - the overload may be chosen based on the type of a type argument.

I consider your examples specious to a fairly large degree, as having overloads where the parameter types are subtypes of one another, and a decision must be made based on the degree of derivation or conversion rules, is usually only done for performance reasons - to prevent the requirement for expensive class-type testing, expensive boxing and unboxing, etc. - things that are only really expensive when done billions of times, but expensive none the less and accounted for in the API so that the optimization takes place with little or no effort from the programmer. More usually, overloads are done based on quite distinct types.

For example, consider a Save method. Is it really better to have SaveToStream, SaveToFile, SaveToBinaryWriter, SaveToTextWriter, SaveToXmlWriter methods? Or would a single overloaded Save method not lead to less verbose, more clear, and easier read code - with no possibility of confusion, as none of the receiver types are subtypes of one another?

Anonymous said...

I disagree about the fact that multiple dispatch is slow and I don't see where it has problems (perhaps you could write a follow-up article and justify your claims).

There has been success in whole programme analysis with optimising away runtime type checks for multiple dispatch.

However, multiple dispatch has limits, e.g. in Dylan there's parameter list congruency. I'm not sure, whether this is a reasonable limit in the sense that it prohibits programmers from doing dangerous things or if it is actually a limitation, but not a problem.

Gilad Bracha said...

Ott:

I didn't say multiple dispatch was slow. I did say that if you implement it especially to call overloaded methods in another language, you are paying a run time cost that is unfortunate.

The main issue with multiple dispatch a la CLOS (and I believe this covers Dyla) is the fact that calls can be ambiguous, and can become ambiguous after the fact. Thus multiple dispatch improves modularity in own way, but undermines it in another.

Unknown said...

I think I disagree.

http://www.google.com/search?q=haskell+ad-hoc+polymorphism

Gilad Bracha said...

Myxie,

Haskell type classes are a rather special kind of beast, with their own unique strengths and weaknesses. I think they deserve a completely separate discussion.

patrickdlogan said...

"Save" seems like a good argument for multi-argument dispatch... the thing being saved, and the thing doing the saving each contribute something to the solution.

chaetal said...
This comment has been removed by the author.
hlovatt said...

Shameless plug for own work:

https://pec.dev.java.net/nonav/compile/index.html

Is an extension to Java that adds design patterns, one of the patterns is multiple dispatch (instead of visitor). The implementation isn't slow and isn't ambiguous. Sp I think multiple dispatch is possible in a mainstream OO language.

Anonymous said...
This comment has been removed by the author.
Anonymous said...

I like to use overloading for option parameter or obvious type conversion like.

doSomething(String st, int flag) {...}

doSomething(String st) {doSomething(st, 0);}


doSomething(Object o) {doSomething(o.toString(), 0);}

Unknown said...

I do not agree with the conclusions of this article. While overloading can cause problems when it is used poorly, the article doesn't seem to mention anything about the main reason to use overloading:

the method names are shorter.

This isn't a trivial thing at all when designing an api. Instead of doing method blah(), you have to do method blahString() or whatever. Longer names means more typing and uglier, harder to read code.

I will note that most of the very popular javascript libraries on github use overloading via typechecking instead of creating longer names.

Steven Smith said...

Essay writing service are in demand these days. They provide a valuable service to students who need to get their work done quickly, and they offer a variety of different features, such as editing and proofreading services.

Sattamatkarepair said...

A particular game called Satta Matka first appeared in India. People who are looking to make money participate in it. Due to its lengthy history of play, the game has changed throughout time. Although anybody can now access the game online, it was originally played only by the participants individually. If you're looking for more information on satta matka you've come to the correct place.

Sattamatkarepair said...

In India, a popular game of chance is called Satta Matka. It entails anticipating the Indian national lottery's winning numbers. In India, a popular game of chance is called Satta Matka. Matching the final digits of the winning lotto numbers is how it is played.

satta matka said...

When playing the dice game Satta Matka, the player must predict the number that will be revealed. The player has the option of placing a wager on either the number or a specific column.

Sattamatkalive said...

In order to win prizes, a player must correctly predict six numbers and then match those predictions with the lottery winning numbers. In India, a popular game of chance is called SattaMatka . It entails anticipating the Indian national lottery's winning numbers.

sattamatkagold said...

In India, many individuals like playing Satta matka. Farmers who used to wager on cattle and crops played the game during its inception in India. Satta matka is now played by Indians everywhere, especially those who are expatriates.

Jackson Smith said...

Welcome to JS Dolls, the premier destination for premium sex dolls for sale in 2022. We offer a wide selection of high-quality, realistic sex dolls for sale at unbeatable prices. Our dolls are perfect for people of all ages, genders, and sexual orientations. Whether you're looking for a companion to spice up your bedroom or a friend to share intimate moments with, we have the doll for you.

johnson said...

Much appreciated for expressing your expertise with us! concentrate up your excellent work! Proceed spreading the message of hope. Make sure to take a look during my web presence.
average cost of uncontested divorce in virginia
preliminary protective order hearing virginia

benstock12 said...

The purpose of assignment sample
  is to give students who are drowning in worry over their dissertations an academic lifeline to keep their grades from sinking into the sea of failures. By providing online dissertation writing services in the best manner possible, we promise to improve your dissertation results and help you finally breathe a sigh of relaxation owing to our amazing services.

Steven Smith said...

In today's fast-paced educational landscape, students face numerous challenges when it comes to managing their academic assignments. Whether it's the pressure of meeting tight deadlines, grappling with complex topics, or juggling multiple subjects, many students find themselves in need of assistance. That's where online assignment help services come to the rescue.

CV Writing Services said...

In today's job market, a compelling CV is essential. But that doesn't mean you have to spend a fortune. Our "Cheap CV Writing Services Ireland" offer high-quality CV writing at affordable rates. Our expert team understands the Irish job market and can create a customized CV tailored to your needs. Don't let budget constraints hold you back from advancing your career. With our cost-effective services, you can confidently apply for your dream job, knowing that your CV represents your skills and experience effectively.

online tuition classes said...

In today's digital age, online classes for kids have become an increasingly popular and convenient way to enhance their education and skills. With the advancement of technology, children now have access to a wide range of educational resources and interactive platforms that can be accessed from the comfort of their own homes.

Michal hix said...

I came across your blog and noticed you were discussing the challenges of financial assignments. It can indeed be a tough subject. If you ever find yourself needing some extra help or guidance, consider looking into finance assignment help. They can make a significant difference in your academic journey.

Vidhipal said...

Avvatar alpha whey protein
from Powergenx will take you on a transforming path toward peak performance. With vital nutrients to feed your muscles and maximize recuperation, this state-of-the-art supplement is expertly formulated to boost your exercise performance.

Vidhipal said...

A variety of other tactics are used in Seo service in Seattle to raise a website's overall search engine position. This covers link building, content production, on-page optimization, and keyword research.

jackk said...

Improve your online presence with our top-notch SEO services in San Francisco. Our skilled experts create custom plans to improve your website's exposure and draw in targeted leads. With our results-driven strategy, you can outperform your competitors and experience unmatched growth.

Vidhipal said...

The high-performance supplement GAT nitraflex pre workout was created to improve training. Rich in creatine, beta-alanine, and caffeine, it gives you a quick energy boost that improves your strength, endurance, and attention.

Vidhipal said...

Before delving into the specifics of Seo service in Seattle, let's first understand why SEO is so important for businesses. SEO is the practice of optimizing your website and online content to rank higher in search engine results pages (SERPs). When your website appears higher in search results, it attracts more organic traffic, leading to increased visibility, credibility, and ultimately, more customers.

Amrit said...

Additionally, it is low in fat and carbohydrates, making it an ideal choice for those focused on lean muscle development or weight management. KNOW MORE: https://powergenx.in/product/myprotein-impact-whey-isolate/

Amrit said...

Variety is the spice of life, and MyProtein understands this well. Impact Whey Protein is available in an extensive range of flavors, catering to different taste preferences. Whether you're a fan of classic chocolate, vanilla, or more adventurous options like salted caramel or rocky road, there's a flavor to suit everyone's palate. KNOW MORE: myprotein isolate

Case Study Help - Writing Tips and Solution said...


I enjoyed your post. Thanks for sharing.

Need help with Assignment Writing Services for university students? Our team provides writing services with 100% non-plagiarism work. Find the best academic writers and experienced professionals at Case Study Help. We allow you to chat with their experts for instant help. Place your order now.

https://casestudyhelp.net/assignment-writing-services.php