Awesome and well articulated response. Love it.
There are a few threads to pull here:
Is an integration test the best way to test the full e2e behavior of an api call? Yep.
Now, depending on the size (and architecture) of your system, the full test suite of into tests could take like, 10 hours.
And even then, you’re STILL probably mocking something (direct seeding bombs into the db). Full setup of all life cycles of all your entities using their associated entities (bombs, wires, wire clippers, bomb diffusers, their rank, their bosses, their diffusion history, etc…)
So they’re expensive, and if you really look you’re probably still cheating some steps anyways.
Another thread to pull on is what are you mocking in your tests to begin with, and if they’re not resulting in valuable test results then that goes back to one of my original points being that you might have suboptimal patterns in your production code.
Like, what logic governs bomb diffusion and where does it live? Are you mocking the right thing? If you have to mock the wrong thing… WHY? The ritual of writing those tests give you the first whiff of a potential smell.
I tell my guys not to test implementation: test behaviour. Test contract adherence. If that’s hard, no problem, message me let’s review it. Maybe a behavior they’re trying to test legitimately should be an int test. More often than not, their design is demanding to mock something that should be decomposed further, and then composed with dependency injections that are where the units that should ACTUALLY be mocked are the things that get mocked and injected, so the unit tests are actually testing the BEHAVIORS we want to pin rather than implementations. Composition and single responsibility principal are what enable unit tests to pull their weight.
Edit: I’m so passionate on the subject that if you scribbled up a scratch example and pinged me a github link I’d genuinely review it.
That’s a lot, so I’ll be forthright that I’ll probably be giving a relatively terse and gappy response, but that’s more because of my own limitations as a mobile user.
Fundamentally every test unit, integration, ui, manual is still the same thing:
Given X do Y expect Z.
Mocks, fakes, whatever you want to call them, they’re fundamentally the same concept too: “given some idealized version of P”
All you’ve done is pushed where that line sits. If I update my company writer to prospend all my company names with “corp.”, and my company read API test seeds without it and something at your read service layer strips “.”, if I forget to update the “given” in my read test seed data, that bug stays invisible.
It’s the same class of issue. We just kicked the can down the road.
You could do writes and reads in the same test, but even then you’re still kicking the can down the road that your front end is calling the API correctly at all. Maybe that the buttons even render to an end user given their authority.
One might then conclude, then, that everything should be an automated UI test.
But would it catch a bug where the bottom collapsed to 1PX? Is that part of the test too? Maybe manual testing is the real way to go.
A further step back why do we even test anything at all? To provide some ultimate guarantee that everything works perfectly at all times? Is that the gate for a PR? A deployment to prod?
There are various gates, at different levels of certainty, and there is an economic scaling for the different levels of assurances. AND they scale with the overall complexity of the system.
And that isn’t a bug, that’s a feature.
It’s ALL behavioral contracts, up and down the pyramid.
There is a bang, there is a buck. Get the most bang for your buck. The distribution will vary by complexity and structure of your project.
That being said, I’ve said it before and I’ll say it again: int and UI tests make it very very easy to write fucking garbage code, the VALUABLE architectural pushbacks on poorly patterned code that unit tests avail is AT LEAST as much value to the overall tech debt accumulation of a project as the results themselves.
Please don’t read this as me shitting on int tests. It’s me trying to impress that unit tests are worth writing, and justifications for forgoing them usually are fundamentally driven by an underlying code smell. Sometimes the smell is “our architecture is already so fucked that writing a meaningful unit test would involve re architecting large swathes of our system”.
And in that case I’d say “it is what it is”, recalibrate the testing posture towards int tests, and accept that is the reality.
But I would NOT conclude on the blanket statement that unit tests SHOULDN’T be at the bottom of the testing period generally. Just that I’d inherited or failed to maintain a strong system design, and take the opportunity to reflect on how things came to that state for the purposes of becoming wiser.
Addendum:
Even int tests can reveal insane system architecture, just at a different layer and scale. I worked on a system at a fortune 500 company where the architecture was too far gone to even have meaningful integration tests. They were too far gone to even have automated front end tests. All material testing had to be done by hand. They had engineered their system with that staggering level of incompetetence. At the risk of making a slippery slope argument… It’s a slippery slope.