But, We Need Proof Before We Try It

By Woody and Kevin

We give a lot of workshops around the world and frequently come across something intriguing. Someone in an organization contacts us, wanting one of our workshops, and we provide details such as syllabus, number of possible participants, and cost. 

Cost often exposes a few dysfunctions. Whoever contacted us must first get approval from those overseeing spending. We’re fine with oversight of how money is disbursed, but we’re baffled by the reasoning of what typically happens next. We’re frequently met with a version of the following from those who control spending:

“Prove to us that Software Teaming will work before we try it. Give us evidence of others in our field doing it and how much they gained in productivity.”

There’s a lot to unpack in that statement, a book’s worth, in fact, but let’s address the most salient aspects in this post.

The Evidence, and Lack Thereof

Ah, the problem of evidence. We’ll cut straight to the conclusion: If by evidence, it means providing scientific studies that quantitatively prove productivity gains from Software Teaming, well then, to date, there aren’t any that we know of. You can stop reading here if that’s all you need to know. If you’re still curious, then there’s much more below.

While no scientific studies yet encompass Software Teaming, many exist for Pair Programming. Most of them generally suggest a benefit to pairing, though some do show a drawback. Given the number of variables involved and the different approaches to measuring them, it isn’t surprising that different authors reach differing conclusions. See, for example, Williams, 2002 [1], Rico, 2009 [2], Hanny, 2009 [3], Cockburn, 2000 [4], Arisholm, 2007 [5], Vanhanen, 2005 [6].

Lacking studies for Software Teaming, we must turn to empirical evidence. Here are some of the benefits we’ve seen in actual practice: 

  • Increased flow and throughput of work items for customers
  • Less siloing and more knowledge-sharing
  • Better quality code because all the team members continually inspect it
  • Easier onboarding of new team members
  • Less duplication of code
  • Shorter lead times
  • Shorter feedback loops
  • Happier co-workers who know each other better
  • A climate in the team that improves our ability to contribute well
  • A decreased need for status and planning meetings
  • An environment of amplified learning
  • Less solving of the same problems in different ways
  • Less time is required for coordinating work and scheduling
  • Easier to manage

This list is not exhaustive, and many organizations we’ve worked with could undoubtedly add others. We find it curious that there’s a hidden assumption in asking for proof, which most questioners never consider. Specifically, it’s the inverse question:

“What evidence proves that it’s more productive working independently instead of as a team?”

Some might say, “Well, we’ve been doing it all along and making money at it.” But that ignores a crucial counterfactual. Namely, how much more might we be making with a different approach? More specifically, just because we’ve done something for a while doesn’t mean there isn’t a better way. Assuming that “the way we’ve always done it” is the best means we fail to consider alternatives that might provide more benefits. 

In fact, we think better collaboration is the key to improved benefits in anything we do. Researching this idea, we found a book by Amy Edmondson, Professor of Leadership at Harvard [7]. In the book, Edmondson states:

In today’s complex and volatile business environment, corporations and organizations also win or lose by creating wholes that are greater than the sum of their parts. Intense competition, rampant unpredictability, and a constant need for innovation are giving rise to even greater interdependence and thus demand even greater levels of collaboration and communication than ever before. Teaming is essential to an organization’s ability to respond to opportunities and to improve internal processes.

If greater levels of collaboration are needed then we think each company must devote itself to becoming collaboration experts. We believe that the collaboration practices in most companies are poor and improving them would reap considerable value.

Returning to the evidence topic, will there someday be useful studies demonstrating the benefits of Software Teaming? We don’t know. But we doubt they’ll have the statistical quality and certainty of studies in scientific fields such as those found when introducing a new pharmaceutical. There are too many variables in software engineering teams to control adequately. The noise would most likely overpower the signal. 

Besides, we don’t think productivity is a meaningful metric. Our book details this, but suffice it to say we’re after effectiveness. After all, there isn’t much point in being productive if we’re productive at doing the wrong thing. That’s why we’re much more inclined toward being effective. That is, being good at doing the right thing and perpetually discovering even better ways of doing it. We think we can more easily reach that goal when working collaboratively as a team. And for that, there’s abundant evidence.

The Much Larger Topic: Innovation

Requiring proof of something’s value before experimenting with it opens up a much larger topic: Innovation and its necessity for survival.

In our view, we see company lifecycles as something like this: 

  • Companies discover a marketable niche.
  • They develop a business that exploits the niche.
  • They enjoy the profits. 

This recipe looks terrific on the surface. Simply determine what customers will buy and sit back, enjoying the payoff. However, there’s one huge downside: 

Market niches are inevitably transient.

Profits mean competitors will arrive, disrupt the niche, or create new ones that draw away customers. In short, abundant profits bring ruinous competition. In the long term, companies that fail to grasp this go out of business.

But how does innovation spread through a business or social system? Who adopts it, and who doesn’t? EM Rogers addressed this in 1962 [8]. Rogers produced the following curve showing the stages of innovation adoption. 

Each slice under the curve is described as follows:

Innovators – These are people who are creating the innovations, or want to be the first to try the innovation. They are venturesome and interested in new ideas. These people are very willing to take risks, and are often the first to develop new ideas. Very little, if anything, needs to be done to appeal to this population.

Early Adopters – These are people who represent opinion leaders. They enjoy leadership roles, and embrace change opportunities. They are already aware of the need to change and so are very comfortable adopting new ideas. Strategies to appeal to this population include how-to manuals and information sheets on implementation. They do not need information to convince them to change.

Early Majority – These people are rarely leaders, but they do adopt new ideas before the average person. That said, they typically need to see evidence that the innovation works before they are willing to adopt it. Strategies to appeal to this population include success stories and evidence of the innovation’s effectiveness.

Late Majority – These people are skeptical of change, and will only adopt an innovation after it has been tried by the majority. Strategies to appeal to this population include information on how many other people have tried the innovation and have adopted it successfully.

Laggards – These people are bound by tradition and very conservative. They are very skeptical of change and are the hardest group to bring on board. Strategies to appeal to this population include statistics, fear appeals, and pressure from people in the other adopter groups.

Slightly edited from [9].

In our opinion, Software Teaming currently lies on the left side of the curve, with the Innovators and Early Adopters.

We believe that when organizations require proof of something’s usefulness before experimenting with it, they’re implicitly admitting that, at best, they’re in the Early Majority class. At worst, they’re on the right side of the curve, in the Late Majority or Laggards class. They’re unwittingly surrendering profits to those on the left side of the curve, the Innovators and Early Adopters.

We think there’s an unrealized operating belief at work when we’re asked for proof of others’ success before an organization will consider Software Teaming. We believe we’re really being told:

“Show us competitors who have succeeded at this—thereby leaping ahead of us in innovating—and we’ll decide if it’s worth spending money trying to catch up to them.”

Needless to say, perpetually playing catch-up with competitors isn’t a long-term recipe for business success. Worse still, we don’t think it’s possible to catch up. By the time we do, the competitors have already captured most of the profits and moved on to newer innovations. 

We prefer being innovative and discovering new approaches that capture early profit. If we aren’t the leaders in improving and innovating our processes, practices, and principles, then we’re always behind those who are.

In short, it’s best to be the disruptors, not the disrupted.

Disruptors are good at frequent, small-scale, safe-to-fail, and cheap experiments. Like evolution’s incessant random variations, they understand that most experiments won’t result in beneficial outcomes. But some will and that’s where the gold lies. 

Further, true disruptors aren’t content resting on an innovation. They ceaselessly seek to push the boundaries of what’s possible. We’re confident that disruptors who adopt Software Teaming will innovate beyond it and discover something even better.

The Last Word

Let’s be clear about one thing: We emphatically are not saying that failing to adopt Software Teaming means imminent bankruptcy for a company. What we are saying is that failing to experiment and innovate quite likely means eventual demise.

With all this said, we can’t yet definitively prove or disprove that Software Teaming will benefit an organization. If there’s a safe conclusion, it’s this: We speculate that Software Teaming provides benefits, but we currently lack scientific data to confirm this. What’s undeniable, however, is that experimenting and innovating offer benefits worthy of the time and money spent on them.

References

[1] On the Economic Feasibility of Pair Programming. Williams, Laurie and Erdogmus, Hakan. International Workshop on Economics-Driven Software Engineering in conjunction with the International Conference on Software Engineering, May 2002.

[2] The Business Value of Agile Software Methods, Maximizing ROI with Just-In-Time Processes and Documentation. Rico, Dr. David F., Sayani, Dr. Hasan H., Sone, Dr. Saya. J. Ross Publishing, 2009.

[3] The Effectiveness of Pair Programming: A Meta-Analysis. Hannay, Jo E., Dyba, Tore, Arisholm, Erik, Sjoberg, Dag I. K. Information and Software Technology. Vol 51(7). July 2009.

[4] The Costs and Benefits of Pair Programming. Cockburn, Alistair, and Williams, Laurie. Extreme Programming Examined: 223-247. 2000.

[5] Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise. Arisholm, E., Gallis, H., Dyba, T., Sjoberg, D.I.K. IEEE Transactions on Software Engineering, Vol. 33, Issue 2. 2007

[6] Effects of Pair Programming at the Development Team Level: an Experiment. Vanhanen, J. and Lassenius, C. International Symposium on Empirical Software Engineering. pp 336-345. 2005.

[7] Teaming: How Organizations Learn, Innovate, and Compete in the Knowledge Economy. Edmondson, Amy,  Jossey-Bass, 2012.

[8] The Diffusion of Innovation. Rogers, Everett M. Free Press, 1962.

[9] Diffusion of Innovation Theory

https://sphweb.bumc.bu.edu/otlt/mph-modules/sb/behavioralchangetheories/behavioralchangetheories4.html