Why don’t software development methodologies work?

I’ve worked on big projects, small projects, in huge teams and by myself, in fossilized federal agencies and cool Silicon Valley companies. I have learned and used at least twenty programming languages. I’ve lived through waterfall/BDUF (big design up front), structured programming, top-down, bottom-up, modular design, components, agile, Scrum, extreme, TDD, OOP, rapid prototyping, RAD, and probably others I’ve forgotten about. I’m not convinced any of these things work.

[Edit: Let me explain what I mean by writing methodologies "don't work." I mean they don't deliver a predictable or repeatable software development process in and of themselves. I don't mean that using a methodology dooms the project. Most software development methodologies try to make programming a more engineering-like process, and in that regard they fall short.]

How can you tell if a methodology works?

Whether a methodology works or not depends on the criteria: team productivity, happiness, retention, conformity, predictability, accountability, communication, lines per day, man-months, code quality, artifacts produced, etc. Every methodology works if you measure the right thing. But in terms of the only measurement that really matters—satisfying requirement on time and within budget—I haven’t seen any methodology deliver consistent results.

My own experiences are anecdotal, but they are shared by almost every programmer I know. It turns out that anecdotes are all that anyone has: rigorous studies of software development methodologies haven’t been done because it’s impossible to control for all of the variables.

Try this thought experiment: Imagine two teams of programmers, working with identical requirements, schedules, and budgets, in the same environment, with the same language and development tools. One team uses waterfall/BDUF, the other uses agile techniques. It’s obvious this isn’t a good experiment: The individual skills and personalities of the team members, and how they communicate with each other, will have a much bigger effect than the methodology.

In his 2003 thesis People and methodologies in software development Alistair Cockburn concludes “People’s characteristics, which vary from person to person and even from moment to moment, form a first-order driver of the team’s behavior and results. Such issues as how well they get along with each other and the fit (or misfit) of their personal characteristics with their job roles create significant, project-specific constraints on the methodology. This result indicates that people’s personal characteristics place a limit on the effect of methodologies in general.”

Our own worst enemy

When I started programming in the 1970s software development was tightly controlled by management through a hierarchy of project managers, business analysts, and senior programmers. We did not get to pick the language or tools. I worked on some big, complex projects in companies that worked this way. Some succeeded, some didn’t. Now it’s common for the programmers to choose the methodology and style of working, along with their languages and tools. Analysts are not part of most programmer’s experience anymore, and project managers have been reduced to “team leads” and “Scrum masters,” neutered of any managerial authority and not really in control of anything other than rituals dictated by team consensus.

Strict management, in love with Gantt charts and schedules and documentation, was reduced to “stakeholders” and “users,” and then abstracted away into “user stories.” It’s common now for me to get involved in a project that seems to have no adult supervision. Surprisingly, left to themselves programmers don’t revert to cowboy coding—they adopt or create methodologies stricter and more filled with ritual than anything I experienced in 1980. In fact programmers today can be much more rigid and religious about their methodologies than they imagine a 1970s-era COBOL shop was. I now routinely get involved with projects developed by one or two people burdened with so much process and “best practices” that almost nothing of real value is produced.

Once a programming team has adopted a methodology it’s almost inevitable that a few members of the team, or maybe just one bully, will demand strict adherence and turn it into a religion. The resulting passive-aggression kills productivity faster than any methodology or technology decision.

Does anything work?

My own experience, validated by Cockburn’s thesis and Frederick Brooks in No Silver Bullet, is that software development projects succeed when the key people on the team share a common vision, what Brooks calls “conceptual integrity.” This doesn’t arise from any particular methodology, and can happen in the absence of anything resembling a process. I know the feeling working on a team where everyone clicks and things just get done. What I don’t understand is why I had that feeling a lot more in the bad old days of BDUF and business analysts than I do now.

I think programmers should pay much more attention to listening to and working with their peers than to rituals and tools, and that we should be skeptical of too much process or methodologies that promise to magically make everyone more productive. Maybe social skills come harder to programmers than to other people (I’m not convinced that’s true), but developing those skills will certainly pay off a lot more than trying yet another development methodology.

How to develop unmaintainable software

I get paid to take on technical debt. In my work I see a lot of hard-to-maintain code, and I see many of the same avoidable problems over and over.

I specialize in debugging, fixing, maintaining, and extending legacy software systems. My typical client has a web site or internal application that works, more or less, but the original developer isn’t available. Business requirements have changed and the software hasn’t kept up. Or my client has something that is “almost finished” but they parted ways with the developer after using up their budget and schedule. Usually there’s a list of missing features and bugs.

My typical client usually has been told by other developers that they need to throw everything away and start from scratch. Most programmers don’t like maintaining code, and they especially don’t like maintaining someone else’s code. When writing code programmers often ask the wrong questions when they talk about maintainability—see Matt DuVall’s article The myth of maintainability for a good description of how that happens.

Here are some things you can do in your own software projects to keep me in business:

Don’t use version control

I’m always surprised to find big projects written in the last few years that aren’t under source code version control. When you don’t use version control the next programmer to come along has to figure out which files are part of the current system and which are obsolete or backups. The next programmer won’t have any commit messages or change logs to get the code’s history from. I covered how not to use version control in my article Introduction to Abject-Oriented Programming.

Customize your development environment. A lot.

Don’t make it easy for the next programmer to start working on the code. Require specific versions of languages and tools, and make sure they conflict with the versions that come with the operating system. Customize your Eclipse or VisualStudio or vim environment like crazy, then write macros and scripts that only work with that setup. Don’t have a disk image or scripts to replicate your development environment, and don’t bother writing any documentation—it will be intuitive.

Create an elaborate build and deployment environment

For a web site deployment to a staging or production server should look like one of these:

svn up
git pull
hg pull

Programmers can argue about simplicity and elegance in code, and then turn around and build the most elaborate and baroque build and deployment systems. This is one of the off-the-radar things that programmers do without their client or project manager reviewing or understanding it, so it easily gets out of control. While you’re chaining together eight different tools with various scripting languages remember to leave out the documentation.

Don’t set up testing/staging platforms

Changing a production system is exciting. Don’t bother with a testing/staging server. Instead have secret logins and backdoor URLs to test new features. Mix test data with real data in your database. Since you aren’t using source control keep copies of previous versions just in case. Don’t build logging into the code. Don’t disable outgoing emails, credit card authorizations, etc. during testing.

Write everything from scratch

Don’t bother with a well-understood framework like Django, Rails, or CakePHP. You can write a better templating engine, ORM, sort or hashing algorithm. I scratch my head when I see code with comments like “faster than native dictionary method” or “replacing PHP library function because parameter order sucks.”

Add dependencies to specific versions of libraries and resources…

Throw in as much third-party code as you can. Link to as many shared libraries as you need to. I’ve seen code depend on large external libraries just to get access to one function. Modify the source code of the third-party libraries so they can’t automatically be updated to a newer version, but don’t bother forking or keeping your modified versions under source code control. I will be able to diff your version with the original and figure it out.

 … But don’t protect or document those dependencies

I get more urgent calls because of updates and upgrades gone wrong than for any other reason. A seemingly innocuous WordPress upgrade, Linux package update, or new jQuery release will trigger a chain reaction of failures. Don’t make your code check for the specific version or modified copy of your external resources or third-party libraries. Don’t even add a comment to remind yourself.

Use a bunch of different programming languages and stay cutting-edge

Every day HackerNews and Reddit buzz with cool new languages. Try them out on your client’s time. Any decent programmer can learn a new language in no time, so if the next programmer who gets to work on your code is a newb that’s not your problem. The boundaries between the languages, the incompatible APIs and data formats, the different server configuration requirements are all fun challenges to overcome and post about on StackOverflow. I’ve actually seen PHP web sites with pieces of Ruby wedged in because everyone knows PHP sucks and Ruby is better. Half-baked caching, aborted Rails and Node.js projects, and especially NoSQL solutions (“It’s more scaleable!”) are my bread and butter.

Where’s the programming advice?

It won’t matter much if your code is beautifully object-oriented or shiny and functional—programmers pretty much always see spaghetti code when they have to maintain a legacy system. I am good with diff and grep and Ctags, tracing through code, refactoring, and debugging. I’ll figure out the code. The most beautiful, elegant code is still hard to work with if there’s no source control, too many dependencies and customizations, and no testing/staging facility. It’s like finding one clean and nicely-decorated room in a house straight out of “Hoarders.”

Related and possibly interesting: The joys of maintenance programming.

Fetish-Oriented Programming

In my career as a programmer I’ve seen lots of projects go off the rails because of strict adherence to some practice, rule, or fashion. It may be something the entire team has bought into, like OOP or TDD. It may be something a single member of the team bullies everyone else about, like tabs vs. spaces or braces style. Even a programmer working alone can sabotage a project by honoring fetishes above productivity.

Here are just a few of the fetishes I have observed wasting hours and days of programmer time:

  • Spaces vs. tabs, how much to indent each line
  • Braces style
  • CamelCase, mixedCase, names_with_underscores, etc.
  • Variable naming conventions, especially Hungarian notation
  • No function can be longer than 50 or 100 lines, or no function can be more than one printed page
  • Never use GOTO, eval, operator overloading, singletons, or some other language feature deemed evil
  • Functions can have only one return
  • No global variables
  • Pure OOP, pure functional
  • Design patterns
  • TDD (test-driven development)
  • Gannt charts, Agile, Scrum, Extreme, User Stories
  • Never use SQL stored procedures, or always use SQL stored procedures
  • SQL vs. NoSQL
  • Never use PHP short tags
  • All web interfaces must be RESTful
  • HTML must validate with W3C
  • Strict XHTML

Not all of these things are necessarily bad ideas. And it is important for programmers on a team to follow some guidelines so their code is understandable and plays well with the rest of the system. The problems arise when a programmer or a team decides that purity according to an arbitrary rule is more important than delivering a working solution. If someone is paying you to develop an application or solve a business problem it really isn’t important how the source code is indented. It is important to the customer that the programmers not waste many billable hours debating whether singletons are bad or not, or if all of the variable names follow one person’s favored convention.

The problem is similar to cargo cult programming. It’s bad enough when newbie programmers do things without understanding why. It’s worse, I think, when experienced programmers turn their own preferences and biases into fetishes and then force themselves and everyone they work with to follow them, even when that means putting the project at risk.

For the last few years I’ve specialized in debugging and fixing code, a lot of it abandoned when the developer and the customer divorced over budget and schedule overruns and missed expectations. I see code that doesn’t even work sprinkled with self-congratulatory comments referencing the GoF patterns. I see badly-written replacements for library functions that exist only because the programmer decided that the standard version’s name sucks. I see specs full of justifications for technical choices that don’t mention important business rules.

I also see lots of code that doesn’t use my preferred indentation, tabs, braces style, variable naming, etc. Spending time “fixing” the code to satisfy my aesthetic sense would be a waste of time and money for my client. I’ve figured out that code that wasn’t written to my tastes is really no harder for me to read and maintain than my own code.

Methodologies, techniques, and best practices should be understood and used to solve problems, not worshipped as rules that must be obeyed no matter what the cost. If you are spending more time debating naming conventions with your colleagues (or yourself) than you are writing code that solves a problem you are wasting time with programming fetishes.