Untitled RSS

Archive

Sep
21st
Mon
permalink

Frustration-Driven Language Development
by Michael Feathers
April 6, 2004

Summary
There are an incredible number of programming languages out there. What would it be like if we had a few with features that really supported testing?

I just noticed that Brian Marick had an interesting response to something I said in my last blog entry. I was talking about how difficult it is to instantiate classes outside of their applications. He mentioned that it would be great to have a tool that would allow you to create an arbitrary object and poke at it, test it etc, regardless of how many dependencies it has. You could supply the tool with mocks for the bad dependencies and it would use the mocks. That would be useful.

Brian’s post reminded me of something I ended up making a while ago, a little framework I called Ghostworld. It was a tool for that sort of thing. I ditched it because I discovered that PicoContainer has every feature I added and quite a few more. The world doesn’t need any more duplicate projects so Ghostworld remains unreleased. But, there is another reason why I never released Ghostworld. I felt it was incomplete. It was missing the killer feature. Let me tell you about it.

To make an arbitrary class you fill up the Ghostworld with types:

Ghostworld world = new Ghostworld();
world.addImposter("Profile", UserProfile.class);
...

Then you ask it to create an object:

Reservation res = (Reservation)world.makeObject("Reservation");

The ghostworld walks the constructors of Reservation and attempts to create one using the classes it knows about. So far, so good. PicoContainer does this also, but what I really wanted was a framework that lets me do this:

class Reservation
{
private AIGConnection connection;

public Reservation(..) {
connection = new AIGConnection(..);
}
}

world.addImposter("AIGConnection", TestingAIGConnection.class);
Reservation res = (Reservation)world.makeObject("Reservation");

Now, here’s what I want to have happen. Every time that that Reservation object or one of its subobjects creates a AIGConnection using a ‘new’ expression, I want a TestingAIGConnection object to be created instead. In the example above, we’d now have a reservation which holds onto an instance of TestingAIGConnection rather than AIGConnection. Wouldn’t that be useful? We could instantiate and test nearly any piece of code without refactoring. I’m not convinced that is great thing. Abused it would just let code get uglier without bound. But, used well, it could really help people out of same jams.

Unfortunately, I never had time to really dig in and find out whether there is some sort of classloader, AOP, or bytecode manipulation magic that would allow me to substitute classes over and over again at runtime. But, the thing that I have been reflecting about is the fact that there seems to be a lot of energy behind efforts like this. And when that happens it usually means that language support is something to consider.

I forget whether it was Paul Graham or Richard Gabriel (or someone else entirely) who said something like this: “the algorithm for developing every “next” language is simple.. it has to be more dynamic than its predecessor and it has to look like C.” So, let’s imagine a language like that.

class Account
{
private balance = 0;
private log = new TransactionLog();

deposit(value) {
log.addEntry(balance, value);
balance += value;
}
...
}

(By the way, just in case you haven’t seen it before, there’s actually a language with syntax close to this. It’s called Groovy).

The Account class is simple enough, but let’s suppose we want to test it, and let’s make it a real bear. The TransactionLog class talks to a database behind the scenes. Every time you add an entry, it’s sloooowww.

The way languages are today, if we want to get past this, we have to add a constructor that accepts a log, or introduce a factory method and override it, or.. well there are a bunch of solutions, but what if we had a language which let us to do this?

test Account
{
class TransactionLog
{
entries = new ArrayList();
addEntry(entry) {
entries.add(entry);
}
}

testDeposit {
a = new Account();
a.deposit(3);
assertEquals(3, a.getBalance());
assertEquals(1, a.log.entries.size());
}
}

The language would allow us to define classes inside a test class. The classes we define in tests supersede classes outside the test. So, whenever some object creates a transaction log in this test class, they get an instance of this simple replacement class that doesn’t touch a database.

Another feature… the tests have access to all private data.

Could these features be abused? Yes. If you can test without breaking dependencies it’s easier to create snarls of complicated code. On the other hand, it would be much easier to get out of them too.

What do you think? Isn’t it about time languages had first-class support for testing?

Jul
13th
Mon
permalink

The End of the Era of Patronizing Language Design

Earlier this year, I was asked to speak at a Ruby Conference.   I was happy to go, but I also felt a bit out of place.  I haven’t done much Ruby, but I’ve admired the language from afar.  I have numerous friends and acquaintances who’ve left C++ and Java to jump toward Ruby and Python, and for the most part, they are happy. They do great work, and they enjoy it.  They are living proof that the the nightmare scenarios of people in the static typing world don’t come true.  You can program safe secure high quality applications in dynamically typed languages, people do it all the time.  But, not everyone knows about it because the industry is divided into different cultures.  If you work with one language, chances are you aren’t completely up on what is happening in another community with a different language.  It takes time to immerse yourself in another community and not everyone is able to.


This is pretty much the situation I’ve been in with Ruby, up to a point.  I haven’t written a large Ruby application.  I’ve tinkered around with the language and written utilities but as far as total immersion goes — well, no, I’ve never been totally immersed in the language but that hasn’t kept me from learning things at the edge through friends and acquaintances.  One of the striking things that I’ve noticed is that the attitude of Rubyists toward their language is different.  They seem to have an ethic of responsibility that I don’t see in many other language cultures.

Ethic of responsibility?  What do I mean by that?

Well, I guess I can explain it this way.  In many language communities, people are very concerned with the “right way” to do things.  They learn all of the warts and edges of the language and they anticipate the ways that features could be misused.  Then, they start with the advice and the style guides — all the literature which tells you what to avoid to make sure that you don’t make your application completely unmaintainable.  The advice goes on and on. Much of it centers around legitimate language defects.  Some languages make you work hard to use them well.  Other bits of advice, though, are really extensions of culture.  If a language gives you mechanisms to enforce design constraints, it doesn’t feel quite right to not use them.  As an example, consider sealed and final in C# and Java.  Both of those constructs do pretty much the same thing and people do go out of their way to advice people on how they should be used to protect abstractions.  It’s interesting, however, that languages like Ruby, Python, and Groovy don’t have anything similar, yet people do write good code in those languages.


Let’s leave aside, for a minute, the debate over static and dynamic typing.  What I think is more important is the issue of tone.  In some languages you get the sense that the language designer is telling you that some things are very dangerous, so dangerous that we should prohibit them and have tools available to prohibit misuse of these dangerous features.  As a result, the entire community spends a lot of time on prescriptive advice and workarounds.  And, if the language doesn’t provide all of the features needed to lock things down in the way people are accustomed to, they get become irate.

I just haven’t noticed this in Ruby culture.

In Ruby, you can do nearly anything imaginable.  You can change any class in the library, weave in aspect-y behavior and perform death-defying stunts with meta-programming.  In other words, it’s all on you.  You are responsible.  If something goes wrong you have only yourself to blame.

So, why aren’t more people crashing and burning?  I think there is a very good reason.  When you can do anything, you have to become more responsible.  You own your fate.  And, that, I think..

Jun
18th
Thu
permalink

The Deep Synergy Between Testability and Good Design

Think about your first unit testing experience.  It was probably frustrating.  It is for most people.  You probably took a class, tried to instantiate it in a test harness and ran into a host of difficulties. You might have had to create a bunch of objects to pass as constructor arguments.  Some of them might have been difficult to create because they depended on other objects.  You might have had to pull in libraries you didn’t expect to.  And, when you finally constructed the object that you wanted to test, you might have discovered that it threw exceptions during construction because your database wasn’t connected.


So, unit testing is hard, isn’t it?  It’s a pain.  It’s no wonder that so many people try to shirk it.

If you thought about this further, you might have realized that there were some tricks that you could play to make things easier.  You could take some private methods in your class, make them protected and override them so that they don’t that they don’t actually access the database.  You could make your construction easier by putting setters on your object and using them to plug in all of the dependencies.  You could do all of that, but if understand good design, you realize that you are doing is really perverting the design; changing it in ways which violate well-known design principles just to make it testable.

So, is testing incompatible with good design?  Absolutely not.  The problem is that many people have give up a bit too easily.  They’ve failed to realize something rather deep and profound: testing is actually tells you whether or not you have a good design.  If you have to hack your design to make it testable, it wasn’t really a good design in the first place.

Testability and Design

How to Test a Private Method

This has to be the

Resource Management

One of the things that we all know about design is that we should manage our resources, but do we really?  The evidence in the industry is not compelling.  Many teams that I’ve visited have applications with an incredible number of resource leaks.  If the leaks aren’t completely crippling, they end up imposing a terrible burden on users.  They run the application until it crashes or consumes too much memory and then they restart to continue with their work.  In some cases, the leaks are extremely obvious; in others, they are subtle.  The users have just learned that a periodic reboot helps.  In either case, it’s bad design, a complete abdication of good practice.
Interestingly enough, unit testing solves this problem also.  I mentioned earlier that one of the beautiful things about unit testing is that it forces small pieces of our code to execute in their own little hermetically sealed containers.  Resources which may be allocated several times in production will be allocated hundreds or thousands of times in test.  If there are leaks, you find out quickly.  Moreover, the “hermetically sealed container” concept helps because we think about tearing down our state when we write our tests.

Application Shutdown

Apr
12th
Sun
permalink

“I used to think that statically checked languages are better for teaching because they prevent the students from doing obviously stupid things. About two years ago I changed my mind. The students learn much better by doing stupid things than by being told by an oppressive compiler that their programs are stupid. So this year I switched to Python. The students are happier, and so am I (because I don’t have to explain that code must be properly indented). Python does not whine all the time. Instead it lets them explore the possibilities, and by discovering which ones crash their programs they seem to understand better how the machine works.”

- Andrej Bauer

( http://math.andrej.com/2009/04/11/on-programming-language-design/ )

Apr
7th
Tue
permalink

Truth in Metaprogramming

I find it convenient to distinguish between two types of metaprogramming.  One kind augments the source code, the other kind nullifies it.

Ideally, whenever you see behavior in the source code you know that it is accessible in the precise way outlined in the source.  For instance, if a class named Registry has a method named getProperty, we should have confidence that when we send the getProperty message, the registry will do what the code of getProperty tells us that it will do.

When we can open classes and replace behavior, we can easily violate the understanding that is implicit in the code.  We can, for instance, have getProperty do something entirely different.  In general, this is bad.  It makes the existing code a lie.

When metaprogramming is used well, it augments the behavior in the code.  We might look at a class and see three methods.  In some other place in the code we might be able to add two more methods to that class.  Does this make the original class definition a  lie?  Not really.  The behavior outlined in those three methods is still there, and accessible, it’s just not the complete story.  Methods added to the class from someplace else do not not make the original three methods “untrue.”

At its base, this is really a variation on an old design principle called ‘The Principle of Least Surprise.’  In Object-Oriented code we can see in in the ‘Liskov Substitution Principle’: subclasses should not violate expectations clients have of their superclasses.  It’s interesting to note, though, that LSP violations often occur when we replace behavior by overriding it in subclasses.  In other words, when we do precisely the thing that we should avoid in metaprogramming.

Code should never lie.  In general, it’s a good idea to use metaprogramming to augment behavior, not replace it.

Apr
2nd
Thu
permalink

The Golden Rule of API Design

It’s not enough to write tests for an API you develop, you have to write unit tests for code which uses your API.

When you do, you learn first-hand the hurdles that your users will have to overcome when they try to test their code independently.

Feb
25th
Wed
permalink

10 Papers Every Programmer Should Read at Least Twice

  1. ‘On the criteria to be used in decomposing systems into modules’ - David Parnas
  2. ‘A Note On Distributed Computing’ - Jim Waldo, Geoff Wyant, Ann Wollrath, Sam Kendall
  3. ‘The Next 700 Programming Languages’ - P. J. Landin
  4. ‘Can Programming Be Liberated from the von Neumann Style?’ - John Backus
  5. ‘Reflections on Trusting Trust’ - Ken Thompson
  6. ‘Lisp: Good News, Bad News, How to Win Big’ - Richard Gabriel
  7. ‘An experimental evaluation of the assumption of independence in multiversion programming’ - John Knight and Nancy Leveson
  8. ‘Arguments and Results’ - James Noble
  9. ‘A Laboratory For Teaching Object-Oriented Thinking’ - Kent Beck, Ward Cunningham
  10. ‘Programming as an Experience: the inspiration for Self’ - David Ungar, Randall B. Smith

Nov
27th
Thu
permalink

Coda - The Dance of Protection (excerpt from ADAIUM)

When I’ve discussed testability issues with API developers, I’ve had a mixed response.  Most of what I suggest involves opening up APIs – making them a little bit less protected.  This runs completely counter to most people’s instincts.  At one company, I told several members of a team that they would be able to enable a bit more testing if they made several methods public.  Immediately they said “You don’t understand.  If we make those methods public people on other teams will use them and we’ll never be able to change them.”   Did they have a point? They did, but on the other hand, it’s important to recognize that what they are talking about is a social problem, not a technical problem and there is only so far that you can go trying to solve social problems with technology.  If they couldn’t reach agreements with their users about the use of various APIs – within their own company – it points to a level of dysfunction that is probably hurting them in many other ways.

Agreements can work well inside of companies, but what about commercial API development?  APIs developed for thousands of external companies?  People who develop these APIs often have some of the most serious deliberations about API design.  They have to live with strict backward compatibility and the decisions that they make have to last forever.  Or do they?

A long time ago, some people at Sun were optimistic. They formulated a policy for deprecating APIs. At any point in time, Sun could release a new version of an API and mark various classes, methods or fields as deprecated.  This was a signal to developers that they should not write new code using those features and that the features may disappear in future releases.  Except, of course, they never have and they probably never will.

Commercial API developers often work under some very tight political constraints.  Imagine this scenario.  You’re a developer in a company and you decide to expose some API that enables testing, however, it’s not an API that developers should use in a production situation.  A user in another company discovers that API and uses extensively.  Unaware, you make a change to that API in the next release.   The user notices the change and escalates a complaint in his organization.  The complaint hops from his organization to yours (probably through some high level contact) and then settles into your lap: you have to continue to support the API even though you hadn’t intended to.

The fact of the matter, however, is that it doesn’t have to be that way.  Technical organizations can make agreements outside of the code and enforce them.  This happens routinely in fields outside of software.  If you buy a refrigerator, you are able to take it apart and modify its insides.  There is no physical analog to final, sealed, or non-virtual in refrigerator design.  What they do have, however, is the notion of a warranty.  If you choose to do particular things with your refrigerator, the warranty is voided.  The line of responsibility between user and supplier is clear.

Fortunately, there are some good models for moving forward. In the Eclipse development community, they have a notion that they call “soft final.”  There are places in Eclipse code where API developers would like to make a class or method final but they realize that there are legitimate reasons to subclass or override – they might even want to do it themselves for testing.  What they do is mark those particular APIs soft final.  It’s documentation without the force of the compiler.   It is a way of saying, “Yes, you can extend the framework this way, but if it breaks it’s your responsibility.”

As much as we like to believe that protective mechanisms in API design are necessary, I think we do have to admit that their use is a cultural norm, not an absolute norm.  In Java or C# it might be unthinkable to, say, allow API users a way to provide an alternative implementation of the system exit routine.  We would immediately have all sorts of concerns about security and misuse.  However, we never seem to have these concerns in Python or Ruby.  In Ruby, you can open the System class and replace the exit routine at any point in your program’s execution.   People don’t seem to be terribly concerned about this dynamically typed languages and I’m sure it isn’t because people don’t do foolish things but rather because the notion of responsibility is different in the cultures surrounding those languages.

Nov
18th
Tue
permalink

Notes for the Afflicted (excerpt from ADAIUM)

When I decided to write this paper, my intention was to raise awareness of the testability issue among API designers.  But, I wouldn’t feel comfortable finishing it without talking about how we can move forward when API designers don’t do any of these things.

There is one simple answer: wrap it.  I’ve visited a number of teams who have decided that it is just easier to wrap all APIs .  If you decide to download some fancy database interface tool, you don’t use it directly from your code, you write adapters to which encapsulate it.  Your adapters also become your stub points and you don’t have to rely upon your API vendor for anything with regard to testability.

This may seem like a lot of work, but often it doesn’t have to be.  When you are wrapping, you have a wonderful opportunity to simplify an interface.  If your API vendor gives you a class with 15 methods, write a wrapper that only exposes the two that you need.  Or, better yet, look at the signatures of those methods and ask yourself whether there is some simpler way of presenting that functionality to the rest of your application.  This wrapper that you write is under your control and you should definitely do things that make your work easier.  Remember, API designers aren’t designing with your application in mind; they are designing code that they suspect will be useful across a broad range of applications.   It would be mysterious if you couldn’t write a wrapper that is more suitable for your application.