How Good Are Your Unit Tests?

 One question that really bugs me is: what is the average effectiveness of unit tests?

Why do I want to know this? Well, because people are always asking me, that's why. I show them Test-driven Development as it is written (so mote it be), and demonstrate 100% code coverage, 100% path coverage, 100% constraint coverage and then use mutation testing to demonstrate just how really very bloody good my unit tests are. (And they are very bloody good indeed. Oh yes.)

And it's usually around this point that someone in the audience puts their hand up and innocently asks:

"So how far do people go on average projects?"

(Because that's the level of ambition I'm up against, godammit! 90% of developers just want to not be worse than average.)

And the truth is, I have no idea what the average level of unit test effectiveness is. None whatsoever. Diddly.

You see, I'm a bit like the Queen. She thinks that the world smells like wet paint, because wherever she goes that's what she can smell.

I think that all teams achieve pretty high levels of unit test effectiveness, because wherever I go that's what I expect from the teams I work with. And they hate to let me down. No, they really do.

So I have a fairly skewed perception of what goes on out there among the hordes of the great unwashed. I hear - like those tales of sea monsters and 9ft tall ape men that explorers brought back with them from exotic, far off lands - that there are teams out there who have no unit tests at all. None. Not one.

But such a team would surely not dare subject themselves to my piercing gaze, and therefore - for exactly the same reasons that Queen Victoria didn't believe that lesbians existed - I choose to dismiss these as the wild imaginings of a depraved lunatic.

So how about your unit tests? Do they cover every line of code? Do they cover every path through the code? Do they cover every combination of logical constraints in the code? If you randomly noodle with your code, do your tests always pick up these "mutations"? How often do you run your unit tests? How much change happens between each test run?

And do you even care?

Feel free to let me know?

Comments

Post a Comment

Popular posts from this blog

83% of Developers Suffer from Burnout

  The Covid-19 pandemic has led to a significant increase in burnout among software developers. According to a study conducted by Haystack Analytics , a firm that focuses on engineer productivity, 83% of software developers experience burnout. The primary causes cited for this exhaustion are excessive workloads (47%), inefficiencies in processes (31%), and ambiguity in objectives and targets (29%). Most areas of human activity have had to rely on technology to remain productive during the coronavirus pandemic, including implementing telework. For the most part, businesses saw significant productivity gains in 2020, directly related to the widespread use of telework. Some are even anticipating a total productivity gain of 17% over the next two to three years. In addition, through telework, 88% have realized savings on real estate expenses over the past year. Similarly, 92% expect further savings in the next two to three years. However, has maintaining or increasing productiv
Read more

Lessons for Young Programmers: What You Need to Know

 In the past 7.5 years I have supervised over a dozen programming interns and have seen hundreds of portfolios of students and graduates. In almost all of those I saw the same things that they needed to learn. One might expect that I think they need to learn specific techniques, algorithms or math, or other forms of specific knowledge. And of course they do, but in my opinion that is never the main thing. The main thing they need to learn is self discipline. The discipline to always write the clearest code you can, the discipline to refactor code if it becomes muddy through changes later in development, the discipline to remove unused code and add comments. Most of the time I spend supervising programming interns is spent on these topics. Not on explaining advanced technologies or the details of our engine, but on making them write better code in general. I always ask applicants what they think is important in being a good programmer and they usually answer that code should be cle
Read more

Hell Is Other Architects

 One of the thornier facets of being a "design authority" is that you are invariably called upon to make a professional assessment of other peoples' designs. (In a process, often compared to pulling fingernails, known as a "design review"). A team of developers might get a little precious about someone like me asking awkward questions about the design of their code, but that's a walk in the park compared to reviewing code that's been architected (by an architect, no less…) Some architects will defend their design decisions to the death, and quite often will use every intellectual and political weapon in their arsenal to repel hostile agressors like myself - godless heathens who have the nerve to saunter nonchalontly into their empires and say things like "should the data access code be in the same classes as the business logic?" and "shouldn't that class that everything else depends on be a bit more abstract?" and other g
Read more