It’s the late 90s and I was working in IT for a small (200 employees) company.
One of my tasks was performing daily data backups, which ran for about four hours and started after everyone else had left for the day.
We had an intercom system in the building, where if you picked up your phone and dialed a specific extension, you could page someone, or make announcements so everyone could hear.
One night I was in the process of running Ethernet cable from switches on the second floor down to individual terminals on the first floor, so there was a lot walking around the building.
I like to have music when I’m working, but I didn’t have an MP3 player at the time, so I thought… “I have music on my PC and no one is around. Why don’t I just…”, and I went around and cranked the volume knob on every ceiling intercom speaker in the whole building. Then I dialed into the intercom from the phone in my office, put the handset up as close as I could to my PC speakers, and boom, music throughout the building.
Cut to a few hours later, it’s late, I’m tired, the backup is finished, so I close down the system, hit the lights, lock the doors, and go home.
The next day, I come in, not thinking about anything from the night before, and get to work. About 11:00am, the owner’s secretary got on the intercom to page someone… and blew everyone out of their seats. There was an almost collective, “What the fuck!!!” from everyone, and an “Ohhh, shiiit” from me, because of course I immediately knew what happened.
I ran around like a madman, turning the volume down on all of the speakers that I had forgotten the night before, while saying, “No idea… Yeah it’s crazy… No no, it’s fixed now…”
No one ever found out exactly why the volume was so loud that morning.
Had a content filter server I had built. Dansguardian.
Installed some new packages, which I had wgot (joking) at / as root. Fuck, I was a dumbass kid.
After I installed the packages, I created the folder /installs and then: holy shit, mv is naturally recursive?!
mv * /installs
Well, let’s ls to see if the root directory is clean.
ls not found.
Um… ok, this is a call center, seems to still be working as everything is still in RAM, but as it is working as a proxy for content filtering, you could lose the company MILLIONS, so think fast!
I rebuilt another server, swapped it with ninja speed, fixed the other server, and swapped it back with ninja speed.
And everyone asked me if I was ok. “Nothing. I mean, yeah, good.”
edit: mv is recursive by default, in contrast to cp, which requires -R. Scary lesson to learn on the job.
Tower cleared me to position and hold (back before it was called line up and wait), then cleared a 737 to land on the runway I was sitting on.
Now it was day VFR conditions and my little ship was painted bright red and white, I wasn’t exactly camouflaged, but I called the tower with my entire tail number and position. RALEIGH TOWER, NOVEMBER ONE TWO THREE ALPHA BRAVO HOLDING POSITION FIVE RIGHT.
Made a great lesson for my student, the tower WILL forget about you and clear a jet 500 times your mass to land on you.
The big annual release was coming soon and I was leading a small group of testers with after hours testing of a feature that was my responsibility to cover. It was already around 8pm and we’d been working for hours. Someone came and informed us that a specific device was incorrectly configured, thus invalidating all of the tests we’d run thus far. It was mine. What an asshole.
I told everyone to give me their devices and go home. I stayed until 2-3am running all of the tests myself. My manager video called me a couple of times and he had a good sense of humor about it, I think because I took responsibility for it and didn’t burden others with my mistake. I really felt terrible but after I got the report in before the deadline later that morning, no one ever gave me a hard time about it and it just was like it never happened.
Not trying to deflect or make excuses and owning a mistake is like a super power. An imprortant lesson that has basically never failed me when I’ve screwed up at work.
Seconded on owning a mistake. It might get you reprimanded, but it usually doesn’t get you fired.
Got creative at work, i was managing my crews .profiles so we could share aliases and such. I got to clever and did a symlink so we all had the same exact one. Then i tested a change that made logins fail.
Luckly it was just the admins during downtime but we had to have the account team make a temp account lol
I used to work at a big cable company whose name rhymes with bombast. One day I (M) found myself in a meeting with a bunch of female vice-presidents (Bombast has thousands of vice-presidents). The presenter couldn’t get her laptop to connect to the big TV so I pulled out one of my gadgets and got her set up. She said “wow, you’re pretty handy to have around.” My brain decided that it would be a clever thing indeed to say “well, I’m pretty well-endowed in the dongle department” but when I started to say it my brain realized how completely insane a comment it was. So my brain cut me off and I ended up just saying “well, I’m pretty well-endowed” which was not exactly any better.
To their credit everybody ignored this except for one woman who actually guffawed briefly. I wasn’t fired immediately so there’s that.
To be fair telling someone “wow, you’re pretty handy to have around” is opening the door for all kinds of inappropriate comments…
Added a bug that would delete data from a form but only when they reopened it and didn’t paginate all the way through to the end of the form. Took ages to discover and a big job to review all the potentially missing data.
Disabled the incorrect password so that it would allow login no matter what password you use. (Original system wasn’t mine and I completely agree with “don’t roll your own security”).
Not physically destructive but someone wrote a script that was supposed to clean up unused phone extensions and decided to run it in production without testing it. It wiped out all the phones.
We probably spent a week or two fixing and testing thousands of phones and the person who ran the unvetted script was fired, but nothing was physically destroyed.
Your post reminded me of a video I saw once where someone did something like this with the fire alarms throughout the building, except there was no fallout. It was just cool to see the speakers in the alarms getting repurposed to play music.
Not at work but in High School.
It was the 90s and computers at school had just become a thing. Our school offered a typing class which seemed pretty chill so I signed up for that. I think I was the only guy in the class.
Anyway, I started talking to this girl who sat across from me and we ended up just sending emails back and forth to each other all through class, using our school emails. This went on for weeks. The messages started to get kinda flirty and then eventually they got more than flirty.
So I’m sitting in another class one day, maths or something, and a teacher comes in and asks if I can go with her. She takes me to the principle’s office and sits me down. In walks the principle with typing girl and she sits her down next to me. The principle then dumps a massive pile of paper on the desk and asks us to explain… we of course have no idea what she’s talking about. I pick up a sheet and start reading and my heart drops. It’s our entire email chain, weeks of cringe and filth. They must have been monitoring our emails, right? Wrong.
So we get a bollocking, threatened with suspension, the whole deal, but it wasn’t until we were sent on our way that we started to realise what had actually happened and the gravity of the situation. Over the next few hours some really weird shit started happening. I’d be walking through school and kids I didn’t even know would just start pointing at me, shouting out random things. Even my teachers were being super weird. Everything just felt really off and then my mate filled me in.
Now, to this day, I still don’t know how this happened. I genuinely think that our school email system was set up wrong but somehow, when replying to one of typing girl’s messages, I had forwarded our whole email chain to every person at our school. Every kid. Every teacher. Thousands of them.
It was humiliating.
I was forever known as that guy. Typing girl never talked to me again.
TLDR: I accidentally forwarded weeks worth of dirty emails to our entire school and held on to my virginity.
I accidentally forwarded weeks worth of dirty emails to our entire school and held on to my virginity.
As a fellow pro in the keeping-virginity in high school dept, game recognizes game.
That sucks. I think the blame is on them for misconfiguration.
Worked in an ESA lab with cleanroom. My work had to do with plants, their composition etc.
Naturally I had to dry some plants before throwing them into the analyser, but my designated oven was occupied.
Fear not, I thought, there is another oven over there. I put my samples in and call it a day.
Come over the next day, kinda late, lab chief looks at me and is only like “ummm, you should head over to the offices and see big boss”…go over, boss guy very angry. Turns out, the other oven in the lab was reserved for ExoMars parts. Like, they disinfect the shit out of it for weeks, like, take it apart, haul every screw to the cleanroom, clean it, autoclave it, clean it again and repeat that a billion times over, certify it and clean it again. So I kinda destroyed the work of months, maybe I alone was responsible for one of the many launch delays (probably not, but I’ve never been told).
Arrogant young me in my first job basically just was like “maybe put a sign on it next time, lol”, boss guy pauses and says “maybe you’re right”. So we reclean the shit out of the oven and put a sign on it. My “punishment” was to spend a lot of time in the clean room to clean some ExoMars parts. I thought “cool, stuff I had in my hands is going to land on Mars”.
Fast forward some years, Russia invades Ukraine, the pipes I cleaned will never go to Mars 😢.
You were absolutely right… Two ovens in a room, 1 for plant use and 1 that needs to be super clean for a specific purpose? A sign would have been the bare minimum, that should have been part of your lab training too.
It wasn’t at work, but I think someone will find it funny regardless.
During my time at university I had a web development exam for which we had to do a group project, you guessed it, making a website. This included having a bunch of different input fields, and the requirement was to properly validate each one and provide clear feedback to the user.
While adding some inputs I was focused on having all the checks in place correctly, and so I decided to put some placeholder text for the errors and do a pass later changing them with more informative ones. Which I did, but I forgot about one.
So, presentation day comes along, the professor is looking at our website, asking us to show him that we had all the checks in place and correct. He asks me to input an invalid date in a calendar widget, which I happily do, knowing I 100% had all the required checks in place. The verification runs and fails, showing the error popup I set up, which read something along the lines of:
“date’s fucked, mate”
Shit. My placeholder message. Which I obviously forgot to change. I was mortified. Luckily the professor chuckled and said that technically the check worked so that was fine.
not me, someone else.
someone deployed a change to the development AWS account at work before a holiday that caused a lambda to execute over 60 million times in a week.
not bad until you find out that the lambda was responsible for creating EC2 instances.
And then?
not only was the dev summarily dismissed, the monitoring team that was responsible for escalation was sternly talked to by the CFO. evidently there were alarms going off that entire week but nobody did anything about it because “it’s in dev, it’s always noisy.”
they actually worked it out with AWS and was able to get the bill reduced but it was still more than $500k (at least that’s all my boss would let me know).
Decade or so ago, logging into a work computer and suddenly the screen is just tiled with the word “virus”, but a bit janky looking like it was a screenshot of the word that was enlarged and tiled with no thought to whether the right side of the image lined up with the left and such - it was a very 90s cheesy movie like depiction of what happens to your comp when you get a virus, like you’d expect the next scene to be some shady hacker on the other end exclaiming “I’m in!”
Then again, I don’t know jack fuck about viruses, and the machine is behaving super weird, so I unplugged the internet cable from the back, and called my manager over. He had a similar “…is this for fuckin real?” reaction that I did. Mind you, this is a hospital, so if one of our computers is fucked, this is a big deal for potentially leaking patient info, so even though it had that kind of slapstick appearance, no one was fucking around with it: so, manager pushed it up the chain, and after a few mins a fucking SWAT team of IT guys comes in and starts doing their own assessment.
After some deliberation, they start troubleshooting - keystrokes and whatnot don’t seem to be responsive. One of them eventually reaches back behind the monitor and ‘poof’ the screen pops back to the normal login screen.
So, that facility had a fancy custom login screen, which was some generic desktop background type scenery and a wall of text that none of us ever put any thought into. One of the snipits on that wall of text was instructions on what to do if you suspect your machine has virus (which was basically what I did - leave it running, unplug it from the internet, and call IT… hooray, I’m not completely stupid!). What ended up being the cause of the “virus” tiling was the fucking VGA cable slowing worming its way out of the back of the monitor - when I got to it, that thing was hanging on by a thread, and touching the desk it was on was just enough of a bump to make it cross that line between having enough of a connection to do its job vs not. That micrometer of pull of on that cable wasn’t enough to completely cut its signal, but it wasn’t enough to produce a normal display either: it just picked a random spot on the screen, and filled the monitor with that repeating image. The spot it just happened to do this to was the word “virus” in the custom login text explaining what to do if you think you have a virus, lol.
TLDR, got my management chain from my dept to the top dogs running the hospital, and the entire fucking IT department to swarm to my unit over a potential data breach that ended up being a wiggly VGA cable.
Whoops.
Better safe than cryptolockered.
For me, it’s occasionally making a code update to a website that has evident and immediate visual glitches. But, I always make these updates on a staging platform before pushing them live, so it ends up being a funny thing in the end. I actually have a folder of my “bloopers” that I keep for my amusement.
Mid '90s, was doing something on the company’s NT 4.0 PDC. The thing about NT 4.0, when the screen locked it would show the “Press Ctrl-Alt-Del to log in” screen, but if you hit a key or moved the mouse quick enough it wouldn’t actually lock and wake up again. So, as I’m sitting there waiting for the server to do something the login screen shows up. I’m in the zone, so I lightning quick give it the three-finger salute, type in the password (usa1, security wasn’t super-high) and hit enter. Another thing about NT 4.0, if you hit Ctrl-Alt-Del when the machine was not locked you’d get a box similar to XP’s, with options to lock the computer, log off, shut down, etc. Now, you’ll notice one of the options, shut down, starts with the letter “S”. And the key to press to select that option is also “S”. And you’ll further notice that the password I typed in also has an “S”. But all is good, it asks you to confirm before it actually shuts down, right? Except the shortcut to accept that option is Enter, which I hit after entering the password. So yeah, I shut down the company’s PDC. Fortunately it was late in the day and most people had already left so the impact was minor. But I was doing some pants crapping for about five minutes while the machine came back up.
The longest 5 minutes of your life.
