I recently spent about two weeks trying to figure out why an intercontinental connection between two of our sites was broken. Not really my job, I just care about application level, but the network guys were beyond useless.
In the end I had the problem isolated to a specific network segment in India, which made them look at the right system and fix things. The reason? "We put up a firewall the day your problems started which blocks everything, if we allow your connection it works".
"If we allow your connection, it works"
Oh, surprising
The one I hate? Your unit tests pass when run locally, and in your sandbox environment, and in dev, and in UAT, but prod? Fuck that, failing with reckless abandon.
Unit tests or integration tests?
Yes
Are you kidding me? Tests in prod? Hopefully that means you didn't fully roll out. That means someone didn't get a 3am page.
This sounds like a god-damned resounding success.
You guys have unit tests…?
Ohh yeah, loads! Do they actually test anything useful? Not so much.
You just have to mock everything, so it’s basically testing nothing. But you get those code coverage numbers.
I actually had to refactor configuration module some time ago. These really came in handy. But was it worth it? Well... it saved some time, the time that could be used to debug problems manually, but it made me a lot more confident that the functionality that worked before, worked after.
mostly deprecated or disabled ... but yes. /jk
The ones I have seen so far are probably written by the same people who don't understand the usefulness of comments, I reckon. And maintained with the same enthusiasm.
Everyone talks about testing, but I have never seen it in the wild
It's common in critical projects.
Chuckles.... you have not worked in my company.
After many years (10+), I finally find a company that actually, really, implements CI/CD. Then I look at the tests and it's actually the most inane shit imaginable, tacked on top of ancient existing code, not maintained. I spent more time fixing the stupid tests than actually fixing the bugs I was tasked fixing.
Amazing.
We test the shit out of our Apis. We do more API level/integration testing though.
I.e. a test will be something like "if the db is in this state, and we hit this endpoint with these params, does it return what we expect and update the db correctly".
Our app is primarily about users maintaining stuff on big datasets with complicated aggregation and approval logic. So setting up a scenario and checking the app does what the business logic says it will do is what we want to know.
It makes refactoring wayyyyy less painful to just know that the app will always behave itself. Rather than testing whether a function can add 1 + 2 correctly, we can test each endpoint does what it's supposed to do.
It gives us loads of confidence that the backend is doing what it's supposed to. If you do a huge refactor you don't need to worry about whether you broke the test or if the test is failing correctly. If the tests all pass everything is working as it should.
Downside is longer test execution times (because a temporary db needs set up) when running the full suite. Worth the trade off for us though.
Those are good tests. But that sounds more like an integration test than a unit test. And we should have both.
I can't really imagine working on any code base that has to actually be maintained and doesn't have tests. The amount of times that tests have safed my ass at my job are uncountable
Meanwhile I'm very upset our unit test coverage is only 40%.
Like, it's the number 1 priority for the principle & staff engineers to get that up to 80% across the codebase.
And it's number 1 priority for management to employ as few developers as possible and stretch their team as thinly as possible. Hence still no unit tests in any of the companies I've worked at recently, despite everyone knowing they're worth it, including lip service from management. They just won't invest in testing, no matter what. One company even fired all the testers then complained to the developers that the product was getting less reliable.
Ahhh my favorite way to fix a broken test: just remove it!
Mark it as intended-to-fail, that way if someone fixes your bug it's their test failure.
Working at a company with no automated tests. There's not even a collection of regression tests or anything to follow. I was wondering if anyone could share or point me towards a good template to start building out test cases as a first step?
I think this is something you're gonna have to just jump into and start since you don't have anything to work off of. it's going to take a lot of work, but at least you'll be able to work off your own ATFs once you finish. good luck..
I mean, start with trivial cases of the core functionality of what your system does. Then build upon it based on your own findings and what your clients report.
E.g. if your system loads images then put in a tiny 5x5px solid square or checkerboard pattern and see if it loads. Then try putting multiple images, different formats (webp, gif, png, tga) etc. see if that breaks anything, keep building out.
if there are zero automated tests, things probably weren't written with (automated) testing in mind, so there may be a lot of coupling... starting off with integration tests just to validate existing behavior is a good start. hopefully the existing applications aren't also inextricably bound to environments that are run on pet servers and managed by other teams...
Thankfully, the team has full control without external team dependencies
It probably really depends on the project, though I'd probably try and start with the tests that are easiest/nicest to write and those which will be most useful. Look for complex logic that is also quite self-contained.
That will probably help to convince others of the value of tests if they aren't onboard already.
I'm in this meme right now.
I just made a tiny change that somehow broke a major feature in testing. The disbelief is real.
Rollback, tests still fail.
Flip a table.
sometimes it is the unit tests.
Wait! US-EAST-1 is dead (Shia surprise)
Oof I am here right now, which is why I'm procrastinating on socials...
I recently spent about two weeks trying to figure out why an intercontinental connection between two of our sites was broken. Not really my job, I just care about application level, but the network guys were beyond useless.
In the end I had the problem isolated to a specific network segment in India, which made them look at the right system and fix things. The reason? "We put up a firewall the day your problems started which blocks everything, if we allow your connection it works".
"If we allow your connection, it works"
Oh, surprising
The one I hate? Your unit tests pass when run locally, and in your sandbox environment, and in dev, and in UAT, but prod? Fuck that, failing with reckless abandon.
Unit tests or integration tests?
Yes
Are you kidding me? Tests in prod? Hopefully that means you didn't fully roll out. That means someone didn't get a 3am page.
This sounds like a god-damned resounding success.
You guys have unit tests…?
Ohh yeah, loads! Do they actually test anything useful? Not so much.
You just have to mock everything, so it’s basically testing nothing. But you get those code coverage numbers.
I actually had to refactor configuration module some time ago. These really came in handy. But was it worth it? Well... it saved some time, the time that could be used to debug problems manually, but it made me a lot more confident that the functionality that worked before, worked after.
mostly deprecated or disabled ... but yes. /jk
The ones I have seen so far are probably written by the same people who don't understand the usefulness of comments, I reckon. And maintained with the same enthusiasm.
Everyone talks about testing, but I have never seen it in the wild
It's common in critical projects.
Chuckles.... you have not worked in my company.
After many years (10+), I finally find a company that actually, really, implements CI/CD. Then I look at the tests and it's actually the most inane shit imaginable, tacked on top of ancient existing code, not maintained. I spent more time fixing the stupid tests than actually fixing the bugs I was tasked fixing. Amazing.
We test the shit out of our Apis. We do more API level/integration testing though.
I.e. a test will be something like "if the db is in this state, and we hit this endpoint with these params, does it return what we expect and update the db correctly".
Our app is primarily about users maintaining stuff on big datasets with complicated aggregation and approval logic. So setting up a scenario and checking the app does what the business logic says it will do is what we want to know.
It makes refactoring wayyyyy less painful to just know that the app will always behave itself. Rather than testing whether a function can add 1 + 2 correctly, we can test each endpoint does what it's supposed to do.
It gives us loads of confidence that the backend is doing what it's supposed to. If you do a huge refactor you don't need to worry about whether you broke the test or if the test is failing correctly. If the tests all pass everything is working as it should.
Downside is longer test execution times (because a temporary db needs set up) when running the full suite. Worth the trade off for us though.
Those are good tests. But that sounds more like an integration test than a unit test. And we should have both.
I can't really imagine working on any code base that has to actually be maintained and doesn't have tests. The amount of times that tests have safed my ass at my job are uncountable
Meanwhile I'm very upset our unit test coverage is only 40%.
Like, it's the number 1 priority for the principle & staff engineers to get that up to 80% across the codebase.
And it's number 1 priority for management to employ as few developers as possible and stretch their team as thinly as possible. Hence still no unit tests in any of the companies I've worked at recently, despite everyone knowing they're worth it, including lip service from management. They just won't invest in testing, no matter what. One company even fired all the testers then complained to the developers that the product was getting less reliable.
Ahhh my favorite way to fix a broken test: just remove it!
Mark it as intended-to-fail, that way if someone fixes your bug it's their test failure.
Working at a company with no automated tests. There's not even a collection of regression tests or anything to follow. I was wondering if anyone could share or point me towards a good template to start building out test cases as a first step?
I think this is something you're gonna have to just jump into and start since you don't have anything to work off of. it's going to take a lot of work, but at least you'll be able to work off your own ATFs once you finish. good luck..
I mean, start with trivial cases of the core functionality of what your system does. Then build upon it based on your own findings and what your clients report.
E.g. if your system loads images then put in a tiny 5x5px solid square or checkerboard pattern and see if it loads. Then try putting multiple images, different formats (webp, gif, png, tga) etc. see if that breaks anything, keep building out.
if there are zero automated tests, things probably weren't written with (automated) testing in mind, so there may be a lot of coupling... starting off with integration tests just to validate existing behavior is a good start. hopefully the existing applications aren't also inextricably bound to environments that are run on pet servers and managed by other teams...
Thankfully, the team has full control without external team dependencies
It probably really depends on the project, though I'd probably try and start with the tests that are easiest/nicest to write and those which will be most useful. Look for complex logic that is also quite self-contained.
That will probably help to convince others of the value of tests if they aren't onboard already.
I'm in this meme right now.
I just made a tiny change that somehow broke a major feature in testing. The disbelief is real.
Rollback, tests still fail.
Flip a table.
sometimes it is the unit tests.
Wait! US-EAST-1 is dead (Shia surprise)
Oof I am here right now, which is why I'm procrastinating on socials...