LetMeinNow
129409
1526
34
Too too true!!
Dec 22, 2021 6:06 AM
LetMeinNow
129409
1526
34
Too too true!!
ChrisWsrn
But Log4J keeps needing to be patched!
0grish
The Heartbleed flaw was deployed to OpenSSL four hours before midnight on December 31st. That is all.
SilentScreamsX
Yeah, we suspended all new releases for our service until Jan 4th. Nobody wants to deal with that shit over the holidays.
comacomacomacomachameleon
My company actually enforces this. Hard freeze December 20th on. No deployments unless absolutely critical and with 3 levels of approval.
Fishooked
Normally it's a quiet week, but everyone is in a mad scramble for that LOG4J vulnerability
ObliqueRay
Every software engineer with the slightest inkling of experience knows this, my man.
Tesseract09
Tell that to the JWST team
d0o0o0d
Hear that? That's the voice of experience.
1CheekyMF
Code freeze for the win!
binarycortex
Doing God's work.
aschophenhauer916
But what about that stupid little feature add no one will use? You want us to wait till next release in 2wks?! Pish posh.
yoyo42
Read-only Fridays!
SithElephant
Or do. 'Sorry, have to go in'
Magus25
Still recovering from a cyber attack and looking after software written in Java. There will be no Christmas for some of us in the IT sector.
blwoodcock
I had a boss force me to deploy an entirely new version of production days before I left the country for 2 weeks.
OneTicketToHellPlease
Laughs in Log4J.
Samerious
I was thinking the same
Citizen7
Sadly, the hackers are working the week before Christmas and therefore so must we.
LebronFrames
They should really get a union.
125no3sir
Lol I was about to say that the log4j jar file would like to have a word.
aschophenhauer916
Find / -name “*log4j.jar” -type f | xargs -n 1 -P 1 -iFILES sh -c ‘rm -rf FILES’
aschophenhauer916
Better have another meeting about this one
Iwouldbenick
NETWORK ADMINS take note
iamthecanadian
Change freeze is the only time we have to get work done!
folded13
Strangely enough, Change Orders tend to dry up at my company starting about Thanksgiving.
NZSheeps
DaierMune
Change Freeze is in effect
bshodokai
My product owner has been pouting for 2 weeks because of code freeze on new features to prod, but we have 1/4 capacity.....
WaterUnderTheRocketAppliances
Doesn't matter. Something is going to randomly awaken an ancient unknown bug on Christmas morning.
haadyy
CS (me) will call at 8am saying agents are overrun by contacts about it and they have a skeleton crew due the hols...
haadyy
Well... I hope it is caught at 8am and not 6. I won't enjoy waking up eastern European devs on Christmas morning with the words:
haadyy
I too have a vodka and meatless dinner hangover but shit on fire, dude, we need to fix this ish.
therealbardman
Time to even stop those ninja edits into prod.
norrinraddsboard
Two words: Maintenance moratorium
GreenFox
Tell that to log4j. I'm so glad I don't work supporting Java apps any more. This past couple weeks must have been hell for my ex-coworkers.
Zerod
And do whatever you can to not be the poor fucker that has the supportline phone.. I somehow had it 3 xmas in a row.. Fkcn cheaters..
bubbawasmybestgoodfriend
phoning it in the week before Christmas is a time-honored tradition. software engineers should take part
ohmanhospitals
Baby I got code to push! Nothin better!
AccountCreatedToUpvoteDogs
My code is ready for a pull request but nothing's getting merged. I'll be pretending I'm Gtest and mocking my work for a few days.
ArthurPhillipDent
'Do we have any _____ to send to ___ this weekend? ' *checks, finds five in various states of readiness* 'umm, no, sorry.
SilentScreamsX
We are. Our usual service meeting turned into a debate on whether Kevin McAllister could have defended Helms Deep.
freakdiablo
...How much prep time?
SecondSince
Say again? I wasn't paying attention...
Diplocrap
God, this comment made me instantly feel so much better. I'm just doing my part!
WhythehelldidImakethisaccount
No offense, but a major vulnerability was found that is still being patched. Everyone is deploying software this week as next.
haadyy
Meme is for scheduled deployments... I hope.
Diplocrap
Thanking my lucky stars to be a C# dev instead of a Java dev this week. It could easily have gone the other way.
physicsisalwaysrelevant
If your software is commonly used in education, you are in the best deployment time until July
ToneLock
rollback does exist. Use it.
Hammertulski
Fuck this attitude.
ToneLock
why is that ? If you changed something and it didn't work out as expected, it is not a bad thing to be able to revert. why ?
Hammertulski
You don’t rely on it. You rely on QA and change management so you aren’t disrupting everyone with a “if it breaks, pull it back” attitude.
ToneLock
seen QA and change management too much. In production is always is a bit different than in the situation they tested it on. If they tested..
Hammertulski
The only takeaway I have from this exchange is that I appreciate that you and I don’t work together.
therealbardman
This is the voice of inexperience.
ToneLock
yes, you sound like you never rolled something back, true.
Iamnotacreativeman
Pfftt not if the package included a db change and no one told you, so you didn't do a snapshot and you're left with 2day old backup.
HeroThatWasNeeded
I feel this in so many ways, a server we use for whole- genome sequencing in house was down for a month due to updates in the software (1)
HeroThatWasNeeded
package. I'm not in your field, but kudos for the patience you must have. (2)
Iamnotacreativeman
I've been around the block, I've seen some shit, and more pile of shit. Hell I've written shit code. They're all shit.
CognosNerd
My code is perfect and without bugs. Who needs UAT? I'm just going to deploy it straight to prod.
WhoDoesNumber2WorkFor
https://i.imgur.com/y7Hm9
BluePaladin42
Amazon devs, is that you?
CognosNerd
So you may have heard about the AWS outage earlier. Sorry about that.
Flabort
I think it affected the McDonald's app, and our debit/pinpad system.
KeeperOfTheFuck
mieper3
Hey Cognos just wanted to let you know I fixed those unit tests you asked me to look at last week, they're all disabled now...
GoIIum
I'm in this picture and I don't like it.
Urbautz
Why deploy to prod when you can code in prod?
Ikingg
We will test this feature in production
Diplocrap
tip tap it's a trap
HelpfulCorn
Sometimes we have to when there's no data in the lowers!
Ikingg
Fix the data..
HelpfulCorn
If only
Ikingg
Get prod refresh with data scrubbing, re- encrypt with test keys. Her security, compliance and legal to review, audit approve it. Easy !
badatediting
Never on a Friday and never the week before major holidays.
luceid
Had a deployment the Friday before Thanksgiving last month.
totallynotaburneraccount00
“fuck it, they can deal w new bugs for christmas “ -New World Dev Team
gunnexx
Depends, do you hate your work and want to set an example? I'd do it. Wanna call me during my pto? Get fukt m8
IncrediRis
i mean, why not? just to keep things interesting
Gearwarz
And don't have your leave scheduled for the day before migration to prod
bloodystupidusername
Friday is read-only day
Phinalize
I deployed to production tonight, with 2 days to firefight with our Devs of necessary...
iCurse
Just never when the next few days are inconvenient to emergency fix on
multipliedzero
And never too close to the end of the work day
Hammertulski
I wish. Half my business systems can’t be touched until after-hours lest we disrupt the ever-holy order processes.
kingtyrion
And always check it at the start of the day, just to make sure.
cmbuckley
Blanket policies like “don’t deploy on X,” even if well-intentioned, tend to result in bundled changes, which is also risky.
cmbuckley
As ever, Charity Majors hits the nail on the head: https://twitter.com/mipsytipsy/status/1466310011072614404
rastilin
Because when the "responsible" person drops the ball, it's still the company's problem to triangulate, repair data and fix it.
bingotown
One idiot makes changes late Friday, and whole teams get alerted and have to respond. No thanks.
wonderboyblue
This is what change management processes are for, as much as they suck to deal with.
cmbuckley
Why is that person allowed to release (to an environment that necessitates alerting) without someone else approving the code change?
aschophenhauer916
They scolded me for not responding to pagerduty alert at 1-3am. I was literally unconscious. I screamed at them.
aschophenhauer916
“The posting said ‘support 365/24/7 platform’” yea posting also said rotations. Not ‘you’re not allowed to sleep’
wormfood
Are they paying you for your pager duty time? If you're obligated to answer the phone you're on call and they have to pay you.
TooMuchDad
Probably salaried position if I had to guess
aschophenhauer916
Salaried. But they don’t maintain a rotation. So I’m constantly getting escalations.
ParaBellumOutfitters
If you're oncall then you have a duty to wake up...If you're not on the rotation then they can pound sand though.
Schriberman
I wish I could do that when on-call at the hospital.
aschophenhauer916
Now image if you didn’t have a rotation. You were just expected to be on call without break
Wyrdbunny
Wish we could have when on-call at the funeral home.
Jamesstin
"Scolded" don't sign up for that type of work then? Also if you were actually "unconscious" maybe consider attending some AA meetings....
aschophenhauer916
Are you conscious when you sleep? Did I miss something. But that’s the point, I didn’t sign up to provide 8760hr of on call support a year.
Fishworshipper
There’s a strange habit that most humans are fond of where they decide to will themselves into unconsciousness for 4-10 hours. Called sleep.
Jamesstin
Yeah and? Any time I was on call it'd be an hour of work and I was guaranteed a full day's pay. And being asleep isn't unconsciousness
Fishworshipper
Being asleep is by definition unconsciousness. To be conscious is to be aware and/or awake. To be unconscious is to not. As for the 1/2
Fishworshipper
other half of your statement, I don't have experience working on-call, so you could very well be right, but maybe the guy's just a hvy slpr.
gunnexx
I have pager duty, I don't mind. I get paid a fuck ton for it and have like 3 calls a month. Its also completely optional
GerberLegalTeam
I think they're saying that you should be able wake up relatively easily unless you're black out drunk.