Posts Tagged ‘productivity’

Testing re-Revisited

Monday, November 26, 2012

Last week I went to SCNA: first time, but good conference overall. I may post separately about it, but no promises.

Anyway, Michael Feathers gave a talk titled Testing Revisited in the afternoon of the first day. The basic message of the talk is that we shouldn’t become hostage of our tests: they are just a means to an end and at every step we should be able to evaluate the value that they provide and even (GASP!) delete them when they are not needed anymore. In principle, this is a valid message. When you’re new to something you get to exaggerate and stick to a new principle or to a newly learned technique a bit too much. It may also be a matter of staying in the comfort zone. In fact, every team that successfully transitions to automated testing may be afraid of losing confidence in its code base by reducing the test suite.

Feathers motivations are mainly fed by the idea that he has met several teams reporting to him that at a certain point they felt that their test base was slowing them down. That is to say that the developers had a clear impression that without tests, or with less tests, they would have moved faster. This is already controversial, but there are some elements presented by Feathers which are interesting nonetheless:

  • First of all, as a team we should always be able to evaluate the actual utility that every test brings to the table. Tests that used to serve a purpose but not anymore should be deleted from the suite. An example could be a regression test originally written to reproduce a bug at  a big scale, slow and clumsy => if the bug cannot happen anymore by design, then there’s no reason to keep that test in the suite.
  • Secondly we should be able to reason about the risk involved in removing any test, especially before scaling a build system that cannot accommodate an infinite growth. More importantly, by reasoning about the risk and the utility of every test we can decide to rewrite them, maybe in a lighter way.

Let’s get to the point of this blog post then. Feathers said something very specific, which was: “maybe we rely too much on automated tests”.

I think that this is a dangerous and ambiguous message. In my experience, the only way to be slowed down by tests is always, absolutely always, no exceptions, because of the code base that is under test, not because of the tests themselves.

What follows is what I tweeted the day of the conference:

While I appreciate some elements of @mfeathers talk I disagree about the sentence: “maybe we rely too much on automated tests” #scna
12-11-09 5:45 PM
Teams who deliver good code and have good practices are the ones writing more tests and suffering less from them. /cc @mfeathers #scna
12-11-09 5:48 PM
Teams who suffer from legacy code, with devs not up to the par & suffering from tests are the ones who need them the most. @mfeathers #scna
12-11-09 5:48 PM
Going after tests seem to me going after the symptoms, not the cause, hence perpetuating the problem. @mfeathers #scna
12-11-09 5:48 PM

And then the day after:

@yehoram @mfeathers I think that the main problem is sending a somewhat mixed message to teams which don’t have consolidated practices yet
12-11-10 9:33 AM
@yehoram @mfeathers While it’s true that we should always evaluate the risk which is covered by tests and ultimately their actual utility
12-11-10 9:34 AM
@yehoram @mfeathers Team which work better are the ones having the least amount of problems with their tests.
12-11-10 9:35 AM

What I’m saying here is that in my experience the teams that are truly slowed down by their tests are in reality slowed by the bad quality of the code under test, not the tests themselves. In practice, I’ve seen cumbersome and brittle tests due to the complexity and intricate dependencies of the code under test. This can be noticed more easily with legacy code.

Some teams struggle, but they are able to navigate through these problems and radically change their systems. Some other teams don’t seem to be able to take the bull by its horns, sometimes also because they focus too much on the tests and not enough on the cause of the pain: their code.

So in the end, while I appreciate Feathers point of view and overall his talk, I think that in particular this aspect can be misunderstood and taken as an excuse to test less, so that teams won’t be slowed down. Unfortunately this is just an illusion and when it backfires it hurts.

Our opposable thumb makes us all professional mechanics, doesn’t it?!?

Thursday, October 29, 2009
Usage of duct tape

Duct tape on Apollo 17, courtesy of Nasa

After watching the video of Uncle Bob’s Clean Code presentation at NDC 2009 last Friday, I’ve thought a bit about the paradox in which our profession lives, especially in light of the recent debate around testing ignited collaterally once again by Joel Spolsky’s blog post. This  paradox is interesting, because it is multifaceted.

Facet 1: most of the companies prefer to deal with many cheap, sloppy programmers instead of investing in a few very good programmers.

Facet 2: despite all the knowledge about software development is easily available a) on the internet b) in multiple ways and c) at a very convenient price (if not free), most of the developers don’t learn much after the university and don’t take any pride in exercising their profession.

Facet 1 may be a side effect of the whole “IT Doesn’t Matter” phenomenon: if investing in IT doesn’t give you anymore a competitive advantage, then maybe it’s not worth investing in the best developers you can find, but just in “developers”. While I generally agree with Carr, especially with regards to “simple” tasks like Office Automation or building corporate websites, there are still huge areas of inefficiency where IT can play a significant role. Maybe it will not give you a significant advantage over your competitors, but it can still save you a lot of money. I personally know a few examples in the logistics domain, but unfortunately I cannot disclose any details because of existing NDAs. Let’s just say that we are talking about many millions of dollars of savings because of problems that in theory have been solved already at the beginning of the nineties. IT wouldn’t matter, if only it was applied and used consistently!

I have also experienced personally the 10x productivity factor: I have seen sloppy programmers being 10x slower than good programmers (not exceptional programmers, just good ones), whose salary wasn’t even nearly 10x bigger than the former ones (let’s be optimistic, let’s talk about 30-40% more). So, it still seems kind of dumb to resolve to hire mediocre people, especially when you are a medium-large company: you certainly have bigger capacity to absorb inefficiencies, but those numbers are starting to be so big, that unbelievable amounts of money go down the drain just because of this. Having said that, there are not nearly enough good programmers in the world and it seems that HR and head hunters have just given up. Sidenote: I happen to know many good programmers and all of them are having a successful career, so it seems that there is always a need for good programmers, at least in this continent. Good programmers, people who care, please don’t give up!

Is there a cause-effect relationship between 1 and 2? I don’t think so. It seems simply that there is a bit of pressure to level software development discipline. I’m sure that this is not the intent of Joel Spolsky, quite the contrary, but singing an ode to the duct tape programmer, even when the programmer is a good one, doesn’t seem the smartest idea to me. Put duct tape in the hands of an exceptional programmer and he’ll do great things. Put duct tape in the hands of a sloppy programmer and he’ll do an awful job. Have you ever seen a participant to Canada’s Worst Handyman assembling a shelf?

For some reasons, everybody thinks he/she can program, even without the appropriate knowledge or background. After some practice in Excel, everybody can certainly write VBA macros (and do a terrible job anyway), but it’s like saying that because everybody can use a wrench then we are all professional mechanics. I guess it’s all about perception, so a lot of people oversimplify the complexity behind implementing and maintaining a code base for any significantly complicated business problem. I’m not saying that every software programmer needs to be as good as Ward Cunningham, but in my mind there is a clear trend towards embracing more and more people in the industry with less and less capable minds. Even worse than that: a lot of programmers seem simply to not care at all.

What is wrong about letting more people get into software development? Generally speaking, nothing – assuming that the complexity at hand has been reduced over the years, thanks to astonishing advancements in tools and technologies. To some degrees this is true: the things that we can do today would have been virtually impossible twenty years ago, but during the same period of time the industry has just raised the bar, tackling more and more complicated problems. The revolution that started with personal computing isn’t just done yet.

Hence, we need smart people, willing to learn new things every day, constantly looking for ways to improve their work. It’s like having your legs solidly planted in today’s problems, but your eyes are always looking at tomorrow.

Duct tape is an exceptional tool when you need to fix something quickly and for whatever reason you don’t have access to the appropriate tool, knowledge, etc. Maybe sometimes you leave duct tape in place for years (hell, my plastic pipe that conduct warm and humid air from the dryer outside of my house sticks into its place thanks to duct tape), but this is generally not a good solution, capable of lasting for a long time. Sooner or later somebody will have to come and clean the mess that the duct tape expert has left. Two, three times a year I have to replace the duct tape because it simply wears out over time: it’s annoying and I would have fixed it already if I wasn’t renting my house.

There may be specific situations where this is tolerable, but not in much broader terms. And yet, there seems to be a tendency to underestimate the effects over time of such short-sighted decisions in our profession. Is it a tolerance to mediocrity? Are we all getting dumber?

Think about it this way: would you accept to have a sloppy pediatrician taking care of your child’s health? Would you go into the surgery room with a mediocre surgeon? Would you put your life on the line in a judicial trial with an incompetent lawyer? Would you trust an ignorant plumber to do the plumbing of your house? Etc., the examples are endless. We are not talking about NASA-like situations – these are more or less very common problems.

The point is that you would NOT. And yet, many managers and entrepreneurs, even when they are technical, do with software development what they would not do in their personal lives. Is it because it’s not their personal wealth or beloved ones at stake?

I’ve always found that the best managers and entrepreneurs are the ones which know the difference between cost and investment. There is no return from a cost, but there better be from investments. If you are on the upper side of the food chain, which return do you expect from the developers working for you? How can you maximize it? But if you are on the lower side: how can you provide value to your company? How can you be always on top of your profession?