Excellent breakdown, Daniel! Mutation testing is a game-changer, moving us beyond the misleading allure of code coverage. 🤔 The analogy with bread toppings is spot-on—quality matters more than quantity
Great question my friend! That would be the end goal, definitely. But unfortunately, due to the state of mutation testing frameworks, many times it is not possible in most stacks. For example the mutation testing libs in the Rust ecosystem are not good enough yet to incorporate into CI.
So instead of doing MT in CI, we do manual triggering and analysing effort ourselves. But with practicing TDD, it will be just a quick quality check, requiring much less time than with test-last development.
I like this idea, I definitely use coverage, but mainly there as catch for reduction, and luckily most of my team knows, and is dishonest enough to write assertion-less tests, this does look like a fantastic way to add some additional confidence and validation.
Excellent breakdown, Daniel! Mutation testing is a game-changer, moving us beyond the misleading allure of code coverage. 🤔 The analogy with bread toppings is spot-on—quality matters more than quantity
Appreciate your feedback, glad that you liked it!
Do you incorporate this in the deployment pipeline? If the score is not 100% for example, do you fail the build?
Great question my friend! That would be the end goal, definitely. But unfortunately, due to the state of mutation testing frameworks, many times it is not possible in most stacks. For example the mutation testing libs in the Rust ecosystem are not good enough yet to incorporate into CI.
Stryker has some good features lately, such as incremental mode (https://stryker-mutator.io/blog/announcing-incremental-mode/), but as I don't use Stryker lately, I have not played with it yet.
So instead of doing MT in CI, we do manual triggering and analysing effort ourselves. But with practicing TDD, it will be just a quick quality check, requiring much less time than with test-last development.
Interesting, thank you!
This is so new to me!
I code in golang. I never heard of mutation testing before as a concept.
(Probably done it manually without putting a name to it)
It's very useful to know the difference types of it.
I am sure if I searched I can find a lib for it in Go. (Or implement one it does not sound too complicated)
Will definitely recommend this to my team!
I am sure it will save us a lot of wrong measurements.
Thanks, Daniel!
I like this idea, I definitely use coverage, but mainly there as catch for reduction, and luckily most of my team knows, and is dishonest enough to write assertion-less tests, this does look like a fantastic way to add some additional confidence and validation.
Ty