Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
![](/static/devrant/img/pipeless-devrant-banner-white.png)
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "pipeline"
-
A devRant Update!
Hey everyone,
We thought now would be a great time for a devRant summer update on what we've added recently and what we've been working on.
Highlights since our last update:
- We launched devRant++, a supporter program for people who want to help us cover our costs while getting some cool extra features (a supporter badge on rants/comments/profile, reserved spot on our in-app supporter list, ability to edit rants/comments for up to 30 minutes instead of 5, and thanks to immediate user feedback, we also added the ability to post a rant every 1 hour instead of 2, and post comments that are up to 2,000 characters instead of 1,000!) We are extremely happy and thankful for the great response the program has gotten and we plan to continue to improve it using your feedback.
- We added the ability to subscribe to a user's rants. This makes it so you get a notification whenever that user posts a new rant!
- We added an "active discussions" feature (available in the "more" tab on the right). If you're looking to join a conversation happening in the moment, then this feature will help you discover those rants. It shows rants that have recently been commented on so if it's a topic that interests you, you can easily get in on the discussion!
Some stuff we have in the pipeline:
- More fun avatar stuff, including fun new OS/language-themed pets
- More perks for the devRant++ subscriber program - if you have anything you'd like to see, please let us know and we will try to make it happen!
- We will be testing some stuff to help classify rant types (rants, jokes, questions, etc.) in order to create a more personalized experience
- On that note, we're also going to take some more time to do some work on the algo as we haven't done much in terms of improvement since the initial smart algo launched
- Community projects page update - we've been slacking on updating the page and apologize for that. If you have created a devRant-related project and it's not on the community page, please resend it to david@hexicallabs.com (even if you sent it already) so we can make sure it gets added. Sorry about that!
A note on community etiquite regarding voting on content:
We've always believed that one of the most important and awesome experiences on devRant is getting your content noticed and appreciated by others. If you enjoy a piece of content, you should upvote it. If you enjoy 500 pieces of content, you should upvote them all. People really appreciate others enjoying their rants and comments so let them know if you do! If you don't like content, you can downvote it with the relevant reason. What we don't encourage is voting on content that you haven't actually looked at or spamming upvotes in mass for content you're not even actually reading/viewing. While we don't encourage that, it's not explicitly disallowed so we won't impose any penalty for it.
What is strictly prohibited and enforced is using scripts or automated procedures for voting on content. Anyone who is caught doing that will have their account deleted without warning. While very rare, we caught a couple of people doing that this week and both accounts in question were immediately deleted once discovered. To be clear, this is the practice of explicitly using a script or automation to mass vote on content. You will NEVER be banned/deleted for voting on a lot of content manually, even if you vote quickly and on lots of stuff. We just want to make that clear becuase this is not meant to discourage people from voting, it is only regarding votes not placed by humans. So if you're a human voting on content, you have nothing to worry about, we promise!
Please feel free to let us know if you have any questions or feedback on any of this. We love constructive feedback and in the past it has gone a very long way to improving and advancing the devRant community. And as always, thank you to everyone who contributed to the community in any way, we really appreciate it and want to keep making your experienfce better.
Happy ranting,
~David and Tim (Team devRant)
@dfox @trogus38 -
The reason why aliens are avoiding earth:
Me : Guys, the CI/CD pipeline is ready. ci.yaml is our config file, so don't remove it as the deployments will fail.
**10 seconds later**
slack: BUILD FAILED
Me: *Looks at git commits* "Brian removed ci.yaml
Wtf BRIAN!🖕🖕🖕🖕16 -
"WE'RE HIRING!!"
Skills Required:
BEFORE: HTML, CSS, JS, jQuery
NOW: REACT/VUE/ANGULAR, NODE, CI/CD PIPELINE, DOCKER, GRAPHQL, JOHN CENA12 -
"Don't deploy on Friday" is a public admittance that your company either has no CI/CD pipeline, or that all your devs are retarded rhesus monkeys who only wipe their ass if the product manager wrote it as a spec.
If the saying was: "Don't port your whole API to GraphQL on a Friday", or "Don't switch from MySQL to Postgres on a Friday", I would agree.
But you should be able to do simple deploys all the time.
I deployed on Christmas & New Year's eve. I've deployed code while high on LSD, drunk-peeing 2 liters of beer against a tree after a party. I've deployed code from the hospital while my foot was being stitched up. On average, we deploy our main codebase about 194 times a week.
If you can't trust your deploys, maybe instead of posting stupid memes about not deploying on Fridays, you should fix your testing & QA procedures.46 -
1. I join a company.
2. I get deeply involved in "how to run the company", and get nice compliments from both coworkers & management about my skills in conveying startup/scaleup advice & necessities to upper management.
3. With my ego inflated through all the sweet talk, I think "ah, what the hell, let's do this again", and I accept a Lead/CTO promotion. I have to join board meetings, write reports on quarterly plans and progress.
4. I get unhappy/stressed/burned-out because I really just want to be a developer, not a manager/executive.
5. Upper management understands, I give up my lead position, lock myself back into my coding cave.
6. I get annoyed because the requirements I receive become more and more disconnected from reality, half of the teams seem to have decided to stop using agile/scrum, the testing pipeline breaks all the time, I get an updated labor contract from HR by mail which smells like charred flesh, etc
7. The annoyances become too much to do ANY work. I yell at the other devs outside of the entrance of my cave. There is no answer, only a few painful moans and sighs.
8. I emerge from my cave. The city has turned into a desolate wasteland. The office is a burning ruin, the air sharp and heavy with black soot. Disemboweled corpses of developers litter the poisoned soil.
Product Managers dressed in stained ripped suits scream at each other while they try to reinforce concrete barricades with scotch tape and post-its. *THUMP* Something enormous is trying to break through. "Thank God, bittersweet, you're still alive! The stakeholders! They have mutated! We couldn't meet the promised deadlines! We've lost the whole mobile app department, and that kid there is the last of the backenders and he's only an intern! You're here to save us, right? RIGHT?".
In the corner, between the overflowing coffee machine and a withered cactus, a young boy has collapsed onto the floor. His face is covered in moldy coffee grounds, clasping on to his closed macbook for dear life, wide-open eyes staring into the void, mumbling: "didn't backup the database, and It's all gone" over and over.
A severely dented black Tesla with a dragging loose bumper breaks through the dried up vertical herb garden and the smoothiebar, and comes to a halt against the beanbags in a big cloud of styrofoam balls.
The CEO limps out, leaking blood all over the upholstery. He yells to the COO: "The datacenter is completely flooded with sewage! I saved the backup tapes though", holding a large nest of tangled black magnetic tape mixed with clumps of mud above his head.
9. I collect my outstanding salary and sell any rewarded options/shares for a low dumping price, take a 5 month holiday, and ask a recruiter about opportunities in a different city.14 -
>we increased new releases to once a day
> wow, how'd you do that?
> fired the QA team
Credit - @iamdevloper✓3 -
I just realized 1/4th of devRant has tried devRantron ✌
I just wanted to thank everyone, this has been an amazing experience and I've learnt a lot from it. We have some really cool features comming down the pipeline.
I love all of you ♥️13 -
Dear Indian Companies,
Why do you hire for a role and then say: "We dont have that role but then we want you to grow up to be a Generalist"!
6 years as a build, release and SCM guy at Moto and Nokia back then, I shifted to this big Indian IT corp coz Nokia was shutting down...
A week into my orientation (which is a crazy weirdness inducing ritual in and of itself), the new manager I'm supposed to be working with comes up and says- "Here's the code repo, there are 2 open jQuery issues, fix them!"
I'm not really sure what to say at this point because jQuery is nice and all but thats not who I am.. I'm the infra / DevOps guy. And this is circa 2012 when DevOps as a term was just hotting up...
Tell me to setup a multi-stage pipeline and automated test cycles, I'll do it drunk, but oh no! bug fixing on a jQuery script? Noooo!!!!! I just dont have the chops for it.
So long story short, I get reported to HR for insubordination - Yeah, Go Figure!
Cue: HR meeting
HR: You wont work?
Me: I cant work on jQuery. I am a sysadmin / devops guy... Give me a project that involves those skills and I'll work.
HR: But we hired you to work on jQuery.
Me: But you did not mention jQuery / UI / UX in the job description - Pulls up email and shows JD for interview which says Symbian, Build, Release, Configuration Management but NOT jQuery.
HR: ....
Me: :-/
HR: But we want you to be a generalist.
Me: #wtf
HR: We want an engineer to be able to do anything he is tasked with!
Me: Can I know my last working date here?
And thats how my career at a glorious IT corporation just went poof!
When I think back on it, I feel good that I chose to do what I wanted to get better at and what I loved working on...
And this is the problem with IT companies in our country - They play with people's aspirations and passions... To the point that all thats left of a software engineer is the looking forward to pay day so he can start the damn cycle all over again.11 -
My first complete pipeline with Jenkins:
- push on gitlab
- build with maven
- test with junit
- deploy with Ansible
- integration test with Selenium
I love devops!6 -
We passed a milestone: 250,000 phpunit testcases.
If it weren't for a heavily parallelized build pipeline which splits it out over 20 servers, it would take about 7.5 hours to complete.
Not hating on PHP, and without tests it would truly be hell...
But still, fucking hell, we outgrew PHP.
Not having a solid type system just means you either accept more bugs, or write thousands of unit tests to guard all the foundational cracks in the system.
On the bright side, I get a coffee break after every commit 😄22 -
As a Java developer, reasons to kill other programmers:
- static mutable variables
- WRITING to static mutable variables
- API call with Framework X didn't work. Add Framework Y along with X and try that. Wrap X in try/catch statement. Catch block fires framework Y.
- six, seven, ten levels of nested code. Zero thought put in organization
- 6K LOC Java files
- spring (singleton? Maybe) object assigning values in static mutable (see pt.1)
- a couple of unit tests in code base that no longer work. Zero unit tests in new code
- unit testing disabled in CI pipeline
- empty catch blocks
- pass mutable data between threads. Modify in various places concurrently.3 -
Usually I do love my colleagues, but lately....
FOR FUCKS SAKE I AM NOT YOUR WALKING HUMAN GOOGLE SEARCH ENGINE SHITOVERFLOW CHATGEPETTO INSTANCE! READ YOUR FUCKING LOGS, DO A FUCKING INFORMATION LOOKUP, READ THE FUCKING MANUAL.
OH YOU HAVE A QUESTION YOU SAY? PLEASE FOR FUCK SAKE ELABORATE WITH SOMETHING MORE THEN 'Please help me with the pipeline"' WHILE YOUR ACTUAL PROBLEM IS A LACK OF KNOWLEDGE AND UNDERSTANDING OF GIT, LINUX OPERATING SYSTEMS AND AUTOMATION.
OH YOUR BRANCH IS, WHAT, 3 MONTHS BEHIND MASTER? NEVER HEARD OF A FUCKING REBASE? WHATS THAT YOU SAY??? YOU DONT KNOW WHEN TO SKIP A COMMIT??? ITS YOUR FUCKING CODEBASE! READ THE FUCKING DOCUMENTATION !!!
WHATS THAT? YOU WORK IN VSCODE AND YOU DO T K OW HOW? AGAIN READ THE FUCKING DOCUMENTATION !
Self.end(rant)11 -
So my thesis is coming to its end. The new pipeline seems to be working. We get higher fps (~20) on bigger input images. In addition for control, camera capture and inference, we can now also stream the images from the drone's camera. So we made a video from the nano-drone's point of view. The weird thing on my forehead is the marker we use for the tracking system so we can compare the inference to the GT.
https://youtube.com/watch/...15 -
Woooooooo!
Just finished my first fully automated CI/CD system. Now all my commits go through the pipeline and gets deployed to live automatically.
It's a small project but still, it's really cool!10 -
I jump on an existing scala project.
git pull && sbt compile test
Tests are failing.
Me: "Hey team, the tests are failing."
Team member: "That cannot be. They were passing for the the last run."
Me: "Did you run them locally?"
Team member: "No, on Jenkins. It was fine."
I check Jenkins.
Me: "What do you mean it's fine. The last successful deployment was on the end of May."
Team member: "The Pull Request checker always went through successfully."
I check how our Jenkins tasks are configured. It's true that the Pull Request Checker runs successfully yet due to a "minor misconfiguration" (aka "major fuckup") the Pull Request Checker only tests a tiny subset of the entire test suite.
Team members were were fine if their Pull Request got the "Success" notification on bitbucket's pull request page. And reviewers trusted that icon as well.
They never checked the master run of the Jenkins task. Where the tests were also failing for over a month.
I'm also highely confused how they did TDD. You know, writing a test first, making it green. (I hope they were just one specific test at a time assuming the others were green. The cynic in me assumes they outsourced running the tests to the Jenkins.)
Gnarf!
Team member having run the tests locally finally realizes: "The tests are broken. Gonna fix them."
Wow. Please, dear fellow developers: It does not kill you to run the entire test suite locally. Just do it. Treat the external test runners as a safety net. Yet always run the test suite locally first.4 -
Me passing time on the weekend
Random call from unknown number
Turns out it's the manager
M: hey , how is your weekend going ...
Me: nothing much ... Whatsup ?
M : yeah well , we wanted to push some minor adhoc fixes as some clients wanted it urgently
The Devops folks need developer support . Can you pitch in and monitor
Me : I'm not aware of what changes are going , i don't think i can provide support
M : don't worry it's minor changes , it's already tested in pre prod , you just need to be on call for 30 mins
Me : ugh okay .. guess 1 hr won't hurt
M: thanks 👍🏽
Me: *logs in
*Notices the last merged PR
+ 400 lines , implemented by junior dev and merged by manager
*Wait , how is this a *minor* release...
*Release got triggered already and the CI CD pipeline is in progress
*5 mins later
*Pipeline fails , devops sends email - test coverage below 50%
Manager immediately pitches in ...
M: hey , i see test coverage is down , can you increase it ?
Me: and how do u suppose I do that ?
M : well it's simple just write UTC for the missing lines ... Will it take time ?
Me : * ah shit here we go again
Yeah it will take time , there are around 400 lines , I am not aware of this component all together
Can you ask junior dev to pitch in and write the UTC for this
*Actually junior dev is out on a vacation with his girlfriend
M : well he's out for the weekend , but
as a senior dev , i expect you to have holistic understanding of the codebase and not give excuses ,
this is a priority fix which client are demanding we need this released ASAP
Me : * wait wat ?
---
I ended up being online for next 3 hours figuring out the code change and bumping up the UTC 🤦🏾9 -
We can compile, transpile, and do all sorts of fucky internet things through an entire development pipeline and then troubleshoot through all sorts of hackery and dev sorcery to output html.
Or I can just index.php and be done with it.
I dunno man, I dig frontend and using the popular js libs to put shit online and be done without having to deal with the fuckery that is wasm or use something similar to Rust to bring shit to my clients.
9 times out of 10, these dudes have been well served with the php or node or even golang that i give them.
Seems that a lot of tools coming up just make shit harder.
Even VBScript seems simpler compared to the amount of web fuckery going on right now.
Yeah I keep current, but fuck, every day it seems as if shit was just getting more and more complex16 -
Indian web dev companies suck ( for developers )
when I finished 3 year grad program in computer application here in my country (India), I thought life's gonna be fun working as a developer. Oh boy, I was so wrong.
I started out working for a small service based IT company, followed by 2 more. I realized really quickly that they're nothing short of a scam. If your company's only agenda to somehow survive in the market and showing no signs of growth in 8 fucking years, then I'm sorry you're working for scamsters.
Now I'm not saying that all of them are alike. But most of them sorta are.
They don't give a shit about quality, not one bit. Quality means no money in the short run. And they haven't been able to develop any strategy to deal with that. Hence, no growth.
They promise 100 things on their website but only provide shitty services in 10.
There is no pair programming, no code review, no code quality check, no architect, no database designer. They won't give you extra time to write test cases. They use git as a storage device.
They don't put their developers (especially the ones who are learning) under any sort of managed development framework to ensure smooth work.
At the end of the day, their main objective is to somehow NOT deliver a project but finish a milestone and make money out of it.
After cashing out for a milestone, they want you to put your current project on hold and start working on a new project until you have like 10-15 projects in the pipeline and you're severely overwhelmed and you just wanna fucking QUIT.
They would say YES to literally every fucking thing, only to disappoint the client later.
I can't believe someone in the US, or UK thought it'd be a good idea to approach these companies
for their brand new app ideas. They're so fucked.
They're rarely finishing any project.
I'm sorry if I hurt your feelings. I had to get it out of my system.11 -
So, I was going to complain about JS being finicky and not making a damned bit of sense, but it turns out that it wasn't JS's fault. Not entirely, anyway. It was the halfassed JS minifier middleware (written by the legendary dev himself) that was breaking the JS while writing it to the page.
The original problem:
My code worked. I removed some comments. Big ol' block of //'s. And suddenly $() isn't a function. But if I call $(); at the top, it all works!
It turns out the "minifier" caused JS to think my code was chaining off the previous JS line in the rendering pipeline instead of being a separate statement. so all it really needed was a `;` at the start. What threw me, though, was the last line of the previous blob of (non-minified) JS was a comment, so it should be a separate statement, right?
But as it turns out...
```
console
// JS really is finicky.
.log('Sigh.');
```16 -
There is nothing sexier than a well designed CI pipeline....i have wet dreams about that green check mark.3
-
Let's start 2023 !
WHO THE FUCK imagined that having language like YAML is a good idea ??
Fuck you and your spaces. No editor produce any decent errors messages except "Your spaces are wrong".
When you edit an Azure debops pipeline, it's just 5 min ti do thing, 35 minuites to figure ou where to add/remove spaces.
NO, I WILL NOT read 25 pages of documentation to add a single step into pipeline.
Fuck YAML !29 -
Post after a long long time...
Wanted to reply to so many comments and mentions, rant about a bunch of topics, do a face reveal after I went for a vacation with family and got some pictures, update y'all on my job hunt, but was busy like hell.
Anyway, time for a story.
After my rejection with Meta and Booking, I started preparing like crazy and my interviews started going well. Refined my LinkedIn further and recruiters started reaching out as well.
Over time, with efforts and feedback, I was able to build a good pipeline.
One of my dream companies reached out to me and I got hired in just 1 round and all others were merely a formality. I was euphoric, but at the same time didn't get over excited as this seemed fishy.
They made a very good monetary offer and I didn't talk to my manager yet regarding resignation. They are pushing me for an early joining.
Read a bunch of Glassdoor reviews and also spoke to a friend who just recently quit that organisation.
He confirmed that the company has 3 months of notice, has sandwich leave policy, and some other XLT political mess.
I decided to decline the offer tomorrow.
Day saved? Not yet.
Because of this I slacked off work a lot. I am super screwed with work items pending because I thought I'd quit.
My boss resinged and new one isn't that supportive yet. He is trying to change everything overnight. Typical.
I ended up performing poorly in other companies because I was confident I'll pick this offer and didn't prepare for upcoming good companies.
Moreover, we have our offices opening up from April and I might be asked to relocate to another city which does not have a team but just because it is on paper, they might force me to be in office 50% of the time.
And what's worse is, my relationship with tech is deteriorating and they are putting the entire product team in bad light.
I have a planned weekend trip coming up, so I won't be able to prepare for interviews or work on case studies so that shit will pile up more.
I am sooooo fucking screwed. Life was stable and then all of a sudden too 180° flip.
I am hysterical right now.16 -
Holy shit. Didn't know I had to vent this out before I had revisited this shit.
Storytime!
Back in May last year, I started working on a dream project (call project X) of mine. Surprisingly it's still a novel idea and shit like this doesn't exist. Made some huge incremental changes. Added all the necessary automation pipeline stuff. Added some sick ass readme with screenshots/badges/glitz/glam.
Worked my ass of for about a month or so until I got distracted by other pending projects in need of clearances. Somewhere partway in that clearance period, I receive a mail from this "GitHub user" asking me why the development of project X had suddenly stopped.
I was a bit taken aback. Firstly because my project had ZERO stars and NO user interaction. Secondly because I hadn't encountered someone with confrontation like this since my middle-school teacher asking me for my homework.
Being the good, responsible child I am, I informed them on my situation and asked them to contribute according to the guidelines and I'd be more than happy to see this becoming a joint effort by the community.
Apparently, they were quite ecstatic to learn that my development was halted. They didn't have plans to contribute. Instead they wanted me to take down the project and stop working on it entirely.
Tough luck fucko.
Their organization had been working on something similar for longer than a couple of years. A similar open-sourced project will *apparently* ruin their market impact and I can *apparently* be sued for it.
I don't know much about open-source "laws" (and I've seen laws fuck people over) but this just seems retarded. At the moment, I'm not quite sure how to continue with the project. I'll still work on it but the fact being that I started receiving threats before stars makes me question the gatekeeping capacity of toxic market conditions (I still don't blame the person entirely. It's just really hard to keep your head above the water)
This is a one off thing but somehow it has definitely hampered my drive to work on the project (combined with the sheer amount of pending project that I've dug my grave with).
On the brighter side I've got 10 anonymous stars with zero promotion. 2 new message threads with productive insights and a person who says "I'm relying on this to work out". So not everything has gone to shit.6 -
It works.
How I hate that sentence.
Whenever that sentence pops up, I wanna take a frying pan, make some bacon, eat the bacon and slam the still hot pan with grease through someone's face till the skull breaks.
Why has he so many anger issues, one might ask.
Usually the sentence "It works" means that after looking at "working thing" it works wrong in 95 % of all cases, but hey - for 5 % it at least does *something* right. Not everything, don't get ya hope up.
We had this fun topic happening again today and I'm still too angry to sleep.
Lucene analysis of texts in Elasticsearch.
Stopword list? Multiple word n-grams per line, duplicates, not lower cased, not properly encoded.
Tokenizers? Duh. Why should one put them in proper order.... Or more realistic: There is an order in tokenizers necessary *devs with shocked faces*.
Language specific details... UHM. Wait. Languages are different? There are edge cases in languages? *more shocked faces*.
Even more shocking that if an text processing pipeline is implemented horribly wrong, it delivers wrong results. *mind blown*.
But our unit tests (this goes out to @kiki) were working.
Yeah. You dumb nuggets who even an amoeba would be ashamed of, when you only do positive tests in unit tests with the most obvious working examples, then your unit tests are just useless waste of nibbles.
Some of the devs are really a fucking waste of genetic information, should have probably ended better in a sock.
If this sounds too harsh, they had 2 weeks.
In just 3 hours I found out that they can redo that with supervision.
-.-
I'm getting too old for that shit. Seriously.4 -
Dear Product Owners,
If you tell me how I need to architect my software again I'm going to ask you to provide a network topology of the architecture you want me to build.
I'll also need you to request the new servers, work with the ops teams to setup credentials, provision the NAT, register the domains and document the routes that the proxy will need to use.
then I'll need you to hook the repo up to our non-existent pipeline so that I can make sure I won't do all that testing I already can't do.
I hope you're paying attention, because that framework you told me I needed to use is going to be a pain to setup correctly.
after you're done with that, please attach any documentation you shit out to the ticket you never created.
Enragedly yours,
Looking for a new job
PS: get fucked3 -
I recently started working with CI/CD in gitlab. I don't know why I had such a hard time getting it to work. I left it hanging for another day. Yesterday, before I went to sleep, I merged to master branch. On waking up I can't explain my happiness when I saw my pipeline... 😀6
-
Project day 1. CTO: everything should be covered by automatic tests.
Project day 30. CTO: listen, we have lot of work in the pipeline now and we need to prioritize. Please don't spend that much time on tests.
Project day 100. CTO: why isn't everything covered by automatic tests?7 -
"We use WSDL and SOAP to provide data APIs"
- Old-fashioned but ok, gimme the service def file
(The WSDL services definition file describes like 20 services)
- Cool, I see several services. In need those X data entities.
"Those will all be available through the Data service endpoint"
- What you mean "all entities in the same endpoint"? It is a WSDL, the whole point is having self-documented APIs for each entity format!
"No, you have a parameter to set the name of the data entity you want, and each entity will have its own format when the service return it"
- WTF you need the WSDL for if you will have a single service for everything?!?
"It is the way we have always done things"
Certain companies are some outdated-ass backwater tech wannabees.
Usually those that have dominated the market of an entire country since the fucking Perestroika.
The moment I turn on the data pipeline, those fuckers are gonna be overloaded into oblivion. I brought popcorn.7 -
I've been lurking on devrant a while now, I figure it's time to add my first rant.
Little background and setting a frame of reference for the rant: I'm currently a software engineer in the bioinformatics field. I have a computer science background whereas a vast majority of those around me, especially other devs, are people with little to no formal computer background - mostly biology in some form or another. Now, this said, a lot of the other devs are excellent developers, but some are as bad as you could imagine.
I started at a new company in April. About a month after joining a dev who worked there left, and I inherited the pipeline he maintained. Primarily 3 perl scripts (yes, perl, welcome to bioinformatics, especially when it comes to legacy code like is seen in this pipeline) that mostly copied and generated some files and reports in different places. No biggie, until I really dove in.
This dev, which I barely feel he deserves to be called, is a biology major turned computer developer. He was hired at this company and learned to program on the job. That being said, I give him a bit of a pass as I'm sure he did not have had an adequate support structure to teach him any better, but still, some of this is BS.
One final note: not all of the code, especially a lot of the stupid logic, in this pipeline was developed by this other dev. A lot of it he adopted himself. However, he did nothing about it either, so I put fault on him.
Now, let's start.
1. perl - yay bioinformatics
2. Redundant code. Like, you literally copied 200+ lines of code into a function to change 3 lines in that code for a different condition, and added if(condition) {function();} else {existing code;}?? Seriously??
3. Whitesmiths indentation style.. why? Just, why? Fuck off with that. Where did you learn that and why do you insist on using it??
4. Mixing of whitesmiths and more common K&R indentation.
5. Fucked indentation. Code either not indented and even some code indented THE WRONG WAY
6. 10+ indentation levels. This, not "terrible" normally, but imagine this with the last 3 points. Cannot follow the code at freaking all.
7. Stupid logic. Like, for example, check if a string has a comma in it. If it does, split the string on the comma and push everything to an array. If not, just push the string to the array.... You, you know you can just split the string on the comma and push it, right?? If there is no comma it will be an array containing the original string.. Why the fuck did you think you needed to add a condition for that??
8. Functions that are called to set values in global variables, arrays, and hashes.. function has like 5 lines in it and is called in 2 locations. Just keep that code in place!
9. 50+ global variables/hashes/arrays in one of the scripts with no clear way to tell how/when values are set nor what they are used for.
10. Non-descriptive names for everything
11. Next to no comments in the code. What comments there are are barely useful.
12. No documentation
There's more, but this is all I can think to identify right now. All together these issues have made this pipeline the pinnacle of all the garbage that I've had to work on.
Attaching some screenshots of just a tiny fraction of the code to show some of the crap I'm talking about.6 -
Man, most memorable has to be the lead devops engineer from the first startup I worked at. My immediate team/friends called him Mr. DW - DW being short for Done and Working.
You see, Mr. DW was a brilliant devops engineer. He came up with excellent solutions to a lot of release, deployment, and data storage problems faced at the company (small genetics firm that ships servers with our analysis software on them). I am still very impressed by some of the solutions he came up with, and wish I had more time to study and learn about them before I left that company.
BUT - despite his brilliance, Mr. DW ALWAYS shipped broken stuff. For some reason this guy thinks that only testing a single happiest of happy path scenarios for whatever he is developing constitutes "everything will work as expected!" As soon as he said it was "done", but golly for him was it "done". By fucking God was that never the truth.
So, let me provide a basic example of how things would go:
my team: "Hey DW, we have a problem with X, can you fix this?"
DW: "Oh, sure. I bet it's a problem with <insert long explanations we don't care about we just want it fixed>"
my team: "....uhh, cool! Looking forward to the fix!"
... however long later...
DW: "OK, it's done. Here you go!"
my team: "Thanks! We'll get the fix into the processing pipelines"
... another short time later...
my team: "DW, this thing is broken. Look at all these failures"
DW: "How can that be? It was done! I tested it and it worked!"
my team: "Well, the failures say otherwise. How did you test?"
DW: "I just did <insert super basic thing>"
my team: "...... you know that's, like, not how things actually work for this part of the pipeline. right?"
DW: "..... But I thought it was XYZ?"
my team: "uhhhh, no, not even close. Can you please fix and let us know when it's done and working?"
DW: "... I'll fix it..."
And rinse and repeat the "it's done.. oh wait, it's broken" a good half dozen times on average. But, anyways, the birth of Mr. Done and Working - very often stuff was done, but rarely did it ever work!
I'm still friends with my team mates, and whenever we're talking and someone says something is done, we just have to ask if it's done AND working. We always get a laugh, sadly at the excuse of Mr. DW, but he dug his own hole in this regard.
Little cherry on top: So, the above happened with one of my friends. Mr. DW created installation media for one of our servers that was deployed in China. He tested it and "it was done!" Well, my friend flies out to China for on-site installation. He plugs the install medium in and goes for the install and it crashes and burns in a fire. Thankfully my friend knew the system well enough to be able to get everything installed and configured correctly minus the broken install media, but definitely the most insane example of "it's done!" but sure as he'll "it doesn't work!" we had from Mr. DW.2 -
You need to answer your manager, but don't have a clue, try these randomly:
I'll circle back to you
I will run the numbers on it
Let's go back to the drawing board
Let's touch base in a bit. Ping me
I don't have the bandwidth right now
It's on my radar
Let's put this on the back-burner
I have too many balls in the air right now
I have a lot on my plate
I'll get back by close of play tomorrow
I'll have to deep dive/drill down into this and get back
I have a hard stop at X hours
Let's park that for now
It's in the pipeline2 -
I am extremely particular about writing good READMEs in my repositories. I make sure that it has everything from prerequisites to run the code and tests on a new machine to how to actually run it (and the tests) and everything in between.
Despite all that I was asked questions that should have been avoided if you had seen the README.
One of these times was by a junior DevOps asking me about an error which was clearly due to him running the code without a virtual environment. Pings me with the entire stacktrace, I go to his desk and tell him to install the environment, which he does. 3 minutes later, another error message.
He was running the wrong script. I go to his desk again. Open the repository. Show him the README. Show him the section titled "To run the pipeline"!
There's a reason they're called README. You're supposed to READ them! 😑3 -
Friend of mine: so I wonder how do you test your applications in the startup?
Me: testing? *grabs his coffee laughing*
Actually we have a complete build pipeline from commit/pull-request to dev and production environments. No tests. Really. We are in rapid product development / research state.
We change technologies and approaches like our underwear (and yeah, this is frequently). If we settled some day and understood the basic problems of the whole feature palette, we'll talk about tests again.rant early product development test driven development proof of concept don't make me laugh prototype startup3 -
I fucking hate the fact that this group I'm side-hustling for gives maintainer access to every shitty dev they have.
Dev pushes four commits directly to master branch. Each time, pipeline fails on unit tests.
Shithead ignores failed tests and manually deploys to stage anyway.
Fuckface then declares (in group chat) that her "fix" works on stage and proceeds to merge to RC branch without updating the fucking unit test.
Pipeline fails (of course) and remains unfixed for the last EIGHT FUCKING HOURS.
This is what I woke up to at 6-fucking-AM in the god-damn morning.
*insert multiple expletives and insinuation of mother's excessive girth by comparing waistline to equator1 -
My best case "Deploy Bittersweet Pipeline":
Prep a bunch of carrots, cucumber and tomatoes for day snacks. Roll & cut some pasta noodles, cook stock with fresh veggies & mushrooms, add some droopy soft boiled egg(s) to the broth, drizzle in some black garlic hot sauce. Enjoy that breakfast with an unsweetened Australian flat white and a half-liter cup of chai spiced green tea. Watch some science/tech/woodworking/cooking YouTube videos while feeding my Bittersweet Jr girl.
(yeah my mood is determined for about 90% by food)
Fire up docker compose & IDEs, and start refactoring code and migrating/fixing old databases.
My worst case "Fatal Incident Bittersweet Repair & Recovery Process":
Stuck while refactoring the worst kind of trash code since 9am.
Pour a glass of Tawny Port at 9pm. Pour a glass of cognac at 11pm. Unwrap 3 chocolate bars and break them into chunks in a bowl. Look at IDE, get nauseated, not from the booze or chocolate, but from the code.
Can't fall asleep because code is too broken, that crap should simply not exist. Take some LSD and amphetamine, can't sleep anyway. Start splitting several 10k-line-long files into smaller classes, type until my fingers have blisters. Empty two bags of Doritos, order a large Falafel with extra garlic sauce at 4am.
Fall asleep at 5am with my face on my keyboard, wake up at 9am with keyboard pattern on my skin.
Cook some hangover noodles.
Call work that I'm taking 3 days off. Feed Bittersweet Jr while I watch some YouTube channels with her. Bittersweet has successfully rebooted.1 -
!rant
I just dreamed about a 6-year old kid looking for me to ask me what is a pipeline. A CPU pipeline.
I felt so happy to explain (I was that into computers when I was 11 or so) that I couldn't start, I woke up first.
Time to have a son/daughter?3 -
I'm the first in 3 generations not be some sort of pipeline/mechical engineer. All my family works at the same company too.
I was partially accepting because at least a software engineer is some kind of engineer.
...And than my cousin graduated to be a gym teacher so I was then fully accepted 😂3 -
My level of frustration with Microsoft is growing to a point that I'm unable to contain it.
They buy Github, it was a great tool for developers because is FUCKING WORKED! New features were never beta tested on users unless they requested it.
Why in the absolute fuck am I getting all these new experimental bullshit features that literally make it harder for me to do my god damned job?
They provide me NOTHING but grief and sleepless weekends while I fix the god damned pipeline that's worked perfectly fine for YEARS.
Your business model is bad and your products are SHIT.
Fuck you Microsoft, I cannot even stress it enough. If I had a time machine that could go back 5 years and my options were: Tell the world about Covid, make sure Trump never became president, or stop the Github purchse. I would hunt down and brutally attack the team of executives that decided to buy Microsoft.
Words cannot adequately describe how much I want Microsoft to fuck off. If the company was a person and they died in a house fire it wouldn't be enough.
I just want a VCS that does what it's supposed to do. I don't need pipelines, I don't need image repos, I don't need static code analysis.
I JUST NEED A FUCKING VCS THAT CONTROLS VERSIONS OF SOFTWARE YOU IGNORANT FUCKS.15 -
Hey there!
So during my internship I learned a lot about Linux, Docker and servers and I recently switched from a shared hosting to my own VPS. On this VPS I currently have one nginx server running that serves a static ReactJs application. This is temponarily, I SFTP-ed the build files to the server and added a config file for ssl, ciphers and dhparams. I plan to change it later to a nextjs application with a ci/di pipeline etc. I also added a 'runuser' that owns the /srv/web directory in which the webserver files are located. Ssh has passwords disabled and my private keys have passphrases.
Now that I it's been running for a few days I noticed a lot of requests from botnets that tried to access phpmyadmin and adminpanels on my server which gave me quite a scare. Luckily my website does not have a backend and I would never expose phpmyadmin like that if I did have it.
Now my question is:
Do you guys know any good articles or have tips and tricks for securing my server and future projects? Are there any good practices that I should absolutely read and follow? (Like not exposing server details etc., php version, rate limiting). I really want to move forward with my quest for knowledge and feel like I should have a good basis when it comes to managing a server, especially with the current privacy laws in place.
Thanks in advance for enduring my rant and infodump 😅7 -
Not exactly a dev related rant.
Do you ever get the feeling when you're not working, like today, that you're kinda wasting time (can't find a better way to describe)? I usually work on Sunday at home, running behind insane deadlines, trying to anticipate tasks. Today was different, I woke up to a fresh VS 2017 install, updated my .net core api to 2.0, learnt how to deploy to Azure, made a CI/CD pipeline and then spend some fun time with my 5 month baby. Argued with him when Azure didn't let me make a new subscription. Sat on the sidewalk with him doing absolutely nothing for a solid half hour, only looking the way he admired everything around him and stuff. Took the trash out, did the dishes, helped with the laundry. But yet I feel like tomorrow gonna be a rough day, where everything will blow up 'cause I didn't did anything work related.
I'm starting to think I lost the taste of enjoying myself, enjoying the people around me, my family, parents, friends. I've been spending too much time on autopilot. Wake up, smoke, work, eat, work, smoke, sleep. Repeat.
I do enjoy my job, a little less when it's not dev related, but I do anyway. We are a small company with big contracts and tight deadlines. Always struggling to give our best and advance further, but I can see I'm loosing something while giving 120% of attention to my job.
Anyway, just wanted to get this thing out of my chest. Thank you if you read this far.7 -
I was asked to look into a site I haven't actively developed since about 3-4 years. It should be a simple side-gig.
I was told this site has been actively developed by the person who came after me, and this person had a few other people help out as well.
The most daunting task in my head was to go through their changes and see why stuff is broken (I was told functionality had been removed, things were changed for the worse, etc etc).
I ssh into the machine and it works. For SOME reason I still have access, which is a good thing since there's literally nobody to ask for access at the moment.
I cd into the project, do a git remote get-url origin to see if they've changed the repo location. Doesn't work. There is no origin. It's "upstream" now. Ok, no biggie. git remote get-url upstream. Repo is still there. Good.
Just to check, see if there's anything untracked with git status. Nothing. Good.
What was the last thing that was worked on? git log --all --decorate --oneline --graph. Wait... Something about the commit message seems familiar. git log. .... This is *my* last commit message. The hell?
I open the repo in the browser, login with some credentials my browser had saved (again, good because I have no clue about the password). Repo hasn't gotten a commit since mine. That can't be right.
Check branches. Oh....Like a dozen new branches. Lots of commits with text that is really not helpful at all. Looks like they were trying to set up a pipeline and testing it out over and over again.
A lot of other changes including the deletion of a database config and schema changes. 0 tests. Doesn't seem like these changes were ever in production.
...
At least I don't have to rack my head trying to understand someone else's code but.... I might just have to throw everything that was done into the garbage. I'm not gonna be the one to push all these changes I don't know about to prod and see what breaks and what doesn't break
.
I feel bad for whoever worked on the codebase after me, because all their changes are now just a waste of time and space that will never be used.3 -
Did anyone of you worked for a company where:
- there was a financial success
- code was clean and was enabler for fast delivery
- tests were professional
- CI/CD pipeline was working as expected
- features were developed in small chunks (few PRs per day)
- managers were trustful and were solving real issues to help you
- refactor was part of the everyday development
Is it even possible? Is there at least one company who achieved success doing the above?15 -
Tuesday.
Director: we need to add our partner's project to our ci pipeline
CTO: we need to add ci to this repo, it should work via this tool (not the one we use now)
PM: we need to use this new tool, figure it out until Friday
TL: we need to move all our projects to this new ci tool, don't ask, the task is given by CTO
Me: okay
Friday.
Me: everything is ready except for this one old project
TL: we couldn't use it for one of our projects
PM: we had difficulties with the new tool
CTO: we couldn't add our partner's project to the pipeline
Director: *gives me a warning for being incompetent*1 -
I find a poor tester copy/pasting data from the test environment to the live one, as he accidentally broke it. I ask the DBA, " why isn't syncing SQL records part of the deployment pipeline?"
"You're front end. This is my job. Go do your job."
"... but it's an easy query, and you're exposing us to human error."
"You need to go sit down."1 -
Long but hilarious:
I was deeply concerned about how we have a single, non-paginated call to a backend service, returning hundreds of entries, which has to be enriched with constant data fetched from our db for each entry. FOR EACH ENTRY. AND FOR EACH REQUEST.
I voiced my concerns to my PM, who called me a "rage prophet" for it.
As expected, the call took 20-something seconds to complete.
Ten minutes before the CEO comes over to have a look, another dev changes his loosely-related service, and the entire super-heavy, sprawling abyss of enrichment pipeline returns in sub-second timing!!!
CEO: guys, this is too fast. You have to slow it down a bit. It doesn't seem reliable that we're able to get all this data immediately.
PM: you see, rage prophet, it all worked out in the end
Me: #$@%$&!!!!!2 -
Soooo it's Monday........ 🤯
@C0D4 started the day fixing current projects defects (4 tickets smashed before coffee 💪)
Then after coffee, run a test coverage report and see a significant decline over the past few months, so spends a couple hours adding more tests to get some areas filled in - meh, nothing like 50+ lines per test... to test a if() statement but whatever - complex scenarios will be complex to get too, but no my tests break and I'm missing data I didn't know about🤦♂️
So let's comment all that out, and go to lunch ... mmmm lunch.
Get back, start working on those again, and then get handed a new issue, so comment that all back out again, ( ok I know what you're thinking, but I'm working in an environment that does not use git for deployments - don't ask, real pain in the ass I haven't had time to invest into yet - but as code versioning only) anywho, starts to workout this new issue but don't figure it out, enter a 30 minute meeting.................. yea that was 2 hours later but was a very practical whiteboard session only to work out I have something like 16-20 weeks of work over 4-5 projects to get out in like 6 weeks... hahahahahahaha fml..... oh and that's excluding another project which had a 6 weeks of work in the pipeline to get to somehow.... I'm not seeing this one happening, and probably conflicting projects needed on top of that down the track... but we'll leave those out for now!
Whoot is fucking home time!!!
🤷♂️I'm starting to think I'm like a team of 5-10 devs right now, maybe I should start asking for 5-10x more 😏
#letsBringOnTuesday!!!!4 -
I'M GONNA KILL THIS MAC
what the FUCK!!!???!!!??? IT WORKED YESTERDAY!!!!
Now the whole iOS build pipeline fails again complaining smth about distribution profiles. Then about certificates. And then it starts using a completely unrelated certificate and complaining that provisioning profile does not include it...
what the f...
IT FUCKING WORKED YESTERDAY!!!!!!26 -
YESSS!!!! IT'S GREEN!!! FINALLYYYY!!!!!!!!
3 weeks ago I started building this CI pipeline. Finally, I see something GREEN!! YAYYYYYYYYYY!!!!!!!
P.S. A complete build requires 23GB of disk space, lasts 1h40m, and artifacts are too big to be cached, so everything must be shoved into a single CI job.
You've gotta love building monoliths...11 -
We were running an obfuscator as part of our build pipeline, but also were not. I discovered we had disabled every rule, and after asking around it turns out that the obfuscator broke the app (because of reflection and things I won't go into).
So I turned it off.
An hour later the CTO came to me and said to put it back. "We have to obfuscate, put it back."
"But... it wasn't doing anything, other than slowing the build down."
"I don't care, we HAVE to run obfuscation. It's in our contract with the client."
...4 -
So I joined this financial institution back in Nov. Selling themselves as looking for a developer to code micro-services for a Spring based project and deploying on Cloud. I packed my stuff, drove and moved to the big city 3500 km away. New start in life I thought!
Turns out that micro-services code is an old outdated 20 year old JBoss code, that was ported over to Spring 10 years ago, then let to rot and fester into a giant undocumented Spaghetti code. Microservices? Forget about that. And whats worse? This code is responsible for processing thousands of transactions every month and is currently deployed in PROD. Now its your responsibility and now you have to get new features complied on the damn thing. Whats even worse? They made 4 replicas of that project with different functionalities and now you're responsible for all. Ma'am, this project needs serious refactoring, if not a total redesign/build. Nope! Not doing this! Now go work at it.
It took me 2-3 months just to wrap my mind around this thing and implement some form of working unit tests. I have to work on all that code base by myself and deliver all by myself! naturally, I was delayed in my delivery but I finally managed to deliver.
Time for relief I thought! I wont be looking at this for a while. So they assign me the next project: Automate environment sync between PROD and QA server that is manually done so far. Easy beans right? And surely enough, the automation process is simple and straightforward...except it isnt! Why? Because I am not allowed access to the user Ids and 3rd party software used in the sync process. Database and Data WareHouse data manipulation part is same story too. I ask for access and I get denied over and over again. I try to think of workarounds and I managed to do two using jenkins pipeline and local scripts. But those processes that need 3rd party software access? I cannot do anything! How am I supposed to automate job schedule import on autosys when I DONT HAVE ACCESS!! But noo! I must think of plan B! There is no plan B! Rather than thinking of workarounds, how about getting your access privileges right and get it right the first time!!
They pay relatively well but damn, you will lose your sanity as a programmer.
God, oh god, please bless me with a better job soon so I can escape this programming hell hole.
I will never work in finance again. I don't recommend it, unless you're on the tail end of your career and you want something stable & don't give a damn about proper software engineering principles anymore.3 -
I'm getting more and more triggered by my colleagues overusing words in seemingly random fashion.
The word 'perspective' comes up at least 6 times during a meeting, from an x perspective, from a y perspective. It would be fine in a design meeting but it's used _so fucking much_ I cringe every time I hear it.
Another one is 'standard', that gets put in front of every word nowadays, standard process, standard protocol, standard machine, standard pipeline. What does it mean? No clue, what does it add? Nothing.
'Please put this add the standard location.'
Where?
'The default one'
What?!
I remove it from documentation every chance I get.
Furthermore, some documentation changes make small pieces of information super long. A nice summary list of features? Make it at least 3 sentences for every bullet point. 1-sentence info with a reference link to more info? Scratch that let's include all information in that reference paragraph anyway. Sometimes they even expand English expressions for no reason, making them longer and harder to read.
WHYYYY
We always complain about shit documentation and yet we're oblivious to the fact that our own docs are so bloated. Stop repeating information, stop using useless adjectives, just put it all in 1 sentence and add dozens of code examples. One piece of code says more than a billion words.
I'm not innocent either. As a teen I was great at writing long pieces of text that seemed like a great read but were actually way too bloated for the information I needed to convey. It was great for reaching word limits.
Now I'm trying my absolute best to be as concise and to-the-point as possible because I know that nobody likes reading and people just want the information that they're looking for.
Even this rant is overly long, but thank god that it's just a rant and I can let off some steam.
Btw same thing goes for diagrams, too many icons, too much text, too many lines. When I try to submit a clean-as-fuck diagram I get asked to add more info/features to which I say No, we're already at the max.
I even got a PR for review that made some changes to add unnecessary information, I pointed it out and never heard anything from them again. I rejected the PR, and never saw a new one.
* Sigh *
It's just so strange to me, it's never clear to me why these things happen. I'm too much of a coward to point these things out unless they endanger the quality of the product. But maybe they just need somebody to tell it to them.6 -
Pipeline time!
Ubuntu: 100% tests, 367/367 tests succeed
MacOS: 93% tests, 330/367 tests succeed
Windows: 71%, 285/367 tests succeed
Time to boot up a windows vm...
(these numbers are not accurate)5 -
Building my own router was a great idea. It solved almost all of my problems.
Almost.
Just recently have I started to build a GL CI pipeline for my project. >100 jobs for each commit - quite a bundle. Naturally, I have used up all my free runners' time after a few commits, so I had to build myself a runner. "My old i7 should do well" - I thought to myself and deployed the GL runner on my local k8s cluster.
And my router is my k8s master.
And this is the ping to my router (via wifi) every time after I push to GL :)
DAMN IT!
P.S. at least I have Noctua all over that PC - I can't hear a sound out of it while all the CPUs are at 100%12 -
Please stop putting critical infrastructure to the internet. Security on the internet is a joke, and we won't be laughing the time when someone dies from a cyber attack on another pipeline/dam/weapons factory.23
-
Only around 700 lines of code and I finally have my first working Vulkan drawcall! Can't wait to integrate it into my engine for all the parallel rendering goodness (not to mention better architecture and asynchronous-ness)
One thing that's a bit weird about Vulkan is the way everything is very static and tightly linked together. You basically need a different renderpass for each stage of rendering (scene-hdr-no-aa, scene-hdr-msaa4, scene-hdr-smaa1, scence-shadow, post-bloom, post-resolve, etc.), a different pipeline object for each distinct pipeline configuration (!!) and both framebuffers and pipeline objects are only valid within the single renderpass they've been created for (roughly speaking)
Oh, and each time the window is resized you have to recreate *all* of these objects from scratch because they also depend on viewport size
No wonder `pipelineCache` is the first argument of `vkCreateGraphicsPipelines` lol3 -
!rant
The efficiency of every dev stack in the world will never compare to just dropping my folder of php code into a server, with proper configurations, routes envs and everything else into a folder and watch it run.
I a pops shop wants me to build something for them? php
If an enterprise grade with a lot of users comes about? php
I have yet to have a single issue with it as most of you evee poluted, herd ready, mob mentality mfkers want to make believe.
Legit, the language is flawed, but has yet to fuck with me, i have memorized the quirks and fuckups of the language (much like I have done with JS) to know that a lot of you just bandwaggon over shit.
"It DoeSnt hAve proPer geNerics"
boom, deployed a form to a customer for his site, charged $2k for a one day job with no issue. But go ahead, setup an entire fucking pipeline of dependencies, a .net app and/or an entire bs app in node or rails(which I love btw) or an entire fuckState centric app in Go that gets messier the more you look at it and it would not be as easy or as simple to deploy.
Legit, in my entire career, nothing makes my life simpler for the web.22 -
Discovered pro tip of my life :
Never trust your code
Achievements unlocked :
Successfully running C++ GPU accelerated offscreen rendering engine with texture loading code having faulty validation bug over a year on production for more than 1.5M daily Android active users without any issues.
History : Recently I was writing a new rendering engineering that uses our GPU pipeline engine.. and our prototype android app benchmark test always fails with black rendering frame detection assertion.
Practice:
Spend more than a month to debug a GPU pipeline system based on directed acyclic graph based rendering algorithm.
New abilities added :
Able to debug OpenGL ES code on Android using print statement placed in source code using binary search.
But why?
I was aware of the issue over a month and just ignored it thinking it's a driver bug in my android device.. but when the api was used by one of Android dev, he reported the same issue. In the same day at night 2:59AM ....
Satan came to me and told me that " ok listen man, here is what I am gonna do with you today, your new code will be going production in a week, and the renderer will give you just one black frame after random time, and after today 3AM, your code will not show GL Errors if you debug or trace. Buhahahaha ahhaha haahha..... Puffff"
And he was gone..
Thanks satan for not killing me.. I will not trust stable production code anymore enevn though every line is documented and peer reviewed. -
First company:
- being sat at an office that didn't have chairs with proper back support. It would kill my back every day. Like sitting on a bar stool coding.
- not having access to basic resources (cafeteria, salary bonuses)
- being seriously underpaid ($200 under)
- not having an IT process pipeline (yeah, this is a huge one): no JIRA, no git, no VCS, no continuous integration, etc. I fucking spend 45% of the time fixing coding-unrelated shit.
Second company (very aggravating):
- dumb frontend bitch and privileged colleague who both kept telling me months on end to shut up and who wouldn't listen to my advice on anything, while my advice would actually help the company advance in productive ways. The key here is being told to shut up while stagnating. i.e. dead end job.
- people advancing in the company based on nepotism and favoritism, based on having tits and ass, rather than skills and independence.
- pointlessssssssss meetings where decisions are made solely based on the opinion of Mr. favorite senior dev. The rest just sits there like a bunch of sad saps and yay-nodders. Incompetent PO's who "would like to hear your input" but then when you give it, they completely dismiss you.
- pointlessssssssss monthly meetings with stakeholders, where the dev teams do nothing but clash and act like pussies in front of the PM just to get in his favor, but behind scenes continue to make the same mistakes and telling the CEO everything is fine. Goodness, how can it get more unproductive.
- completely antisocial and nepotistic 'colleagues' who won't even talk to you, let alone smile at you or be friendly. You saying good morning and them pretending you're vapor that doesn't exist. Go go company atmosphere! Especially during lunch, those are the worst times. Imagine sitting at lunch where everyone looks like you killed their dog and the rest is huddled up in little high school groups.
What else? The incessant and pointless smalltalk that makes me want to bang my head against the wall. Talking about dogs, kids, what show was on tv last night. The fuck man, do you have a brain?!
Third company:
- HR bitches who think they are the shit and developers are antisocial, helpless misfits, but they work with computers and they don't even fucking know what a status bar is! The irony!
- forced socializing and stigmatization for the opposite. Imagine coming into a company and you don't say good morning. Should that be a problem? No. Instead, everyone starts dogging on you and hating you just because you didn't smile in their faces and said: hiiiiiiiiiiii how did you sleep? Did you feed your dog? Fuck you.
Elliot (Mr. Robot): "Wouldn't it be awesome if there was a mute button for life?" -boop, boop, boop, boop...- Ahh.. there.. that's much better."
- CEO's sucking up to you but when it comes to salary increase, they say shit like: "Ahhh ya know, it's kinda difficult." Yet another dead end job.2 -
Just got a new job at an old school hardware company. The codebase is giving me heart attack. They don't care about dev experience or code navigation at all. Every attempts to modernize the codebase is so half assed. All patches are so bloated that make the codebase even worse.
Frontend is migrated from prototype-oop-jquery cluster fuck to AngularJS, then finally angular. Holy moly, all business logics are baked into UI "classes" using prototype chain. When they migrated to AngularJS, someone simply added a wrapper to that jQuery cluster fuck class and overwrote all the prototype with a 10k +lines file. Since all the methods are hidden in either prototype, JS object, or callback function, it's impossible to trace the data pipeline using IDE when "go to definition" on update() method gives you all the update methods/string in all objects/classes. And they don't care about immutability. References are taken out, renamed, and mutated everywhere. Finding the source of a bug is fucking guessing game.
I don't know what trick they use that makes cLion static analyzer fail.
And there is no unit test or spec doc.
Fuck me dead3 -
Being called into a meeting where you get fired for an argument you had with your CTO. But the CTO left 6 months ago and the sales department are suspiciously low on deals in the pipeline...
Shame really, I loved that job and the Dev team was great, shame the directors were bastards.2 -
PM: this is our super fancy new CI/CD pipeline, it's the greatest. i expect you to learn and understand all this in no time.
devs: so i have to spend some more time on this topic because it's completely new to me and requires some learning...
PM: nooo, that's a super easy task with zero effort, my braindead hamster can do that in no time, so can i, and so can you! let's assign 1 story point for that.
~ 3 months latèr ~
also PM, after he has started developing as well: so i'm realizing there are many things that i have to learn, and it takes me some time. i haven't developed with C++ and <other tool stack> for a longer time. by the way, you guys don't need to check for any quality right now, we need to deliver fast. it's okay, when you have memory overflows, your code is completely crappy, poor architecture or memory overflows, it doesn't matter.
he even has a subtask for migrating his code from VS project to our new project structure, since he refused to learn our pipeline right from the beginning and created VS project instead. シ why is this a subtask? this job can be done in no time, my left vanishing twin named Klaus who has dislexia and hates vim can solve this task in 20 seconds!!!!11
(and still no PR, not even a feature branch in our repo)2 -
naiive idealism to the max:
medior+senior to junior: "hey, buddy, we need you to do this, here's the codebase, here's the button, here's what needs to happen when that button is clicked, here's the relevant files and classes, make it happen."
medior to senior: "so what you just said about how we should redo the whole order processing pipeline, na-ah, not possible. i've been in those parts of the code many times, and based on what i've seen, you either leave that thing mostly alone or nuke it from orbit and build a completely new module in its place, but these "medium adjustments" you're proposing... not feasible...
senior to medior: "okay, i've seen how slow your progress was on even the most basic-sounding bugs in those systems... looks like what you're saying makes sense."
senior to *EO: not possible to just do these changes with this budget and deadline, that wouldn't even cover the "unexpected bugs" overhead, either you let us do it properly as a new greenfield project, almost, or you're stuck with what we've got.
*EO: mmmkay, so that's 20 times more time and budget that is in the proposal?
senior: yup, something around those numbers.
*EO (with a pained but understanding expression) : go for it, imma explain to the rest of the EOs at the end-weeks's meeting.4 -
I thought of posting this as a comment to @12bit float' post, but then decided it better goes out as a post by itself.
https://devrant.com/rants/5291843/...
My second employer, where I am on my last week of notice currently, is building a no code/low code tool.
Since this was my first job switch, I was in a dreamy phase and was super excited about this whole space. I indeed got to learn like crazy.
Upon joining, I realised that an ideal user persona for this product was a developer. Wow! No code tool for developer. sO cOoL...
We started building it and as obvious as it could get, the initial goal was adoption because we were still at top of the funnel.
We launched an alpha release shortly followed by a beta.
Nobody used it. Tech XLT/LT kept pushing product and design team to run a feature factory so that their teams can use this tool.
The culture set by those two leaders was toxic as fuck.
Now, I decided to do some research and some more product discovery to understand why folks were not using it. Mind you, we were not allowed to do any research and were forced to build based on opinions of those two monkeys.
Turns out that the devs were really happy with their existing tools and our tool was another tool being forcefully added into their toolbox by the said XLT/LT.
Not only that, even if they decide to use our tool, out of pressure, they still cannot because the product was missing key capabilities like audit control and promotion from one environment to another.
Building those would essentially mean reinventing Github aka version control and Spinnaker aka CI/CD pipeline.
My new boss (I got 3 managers in 4 months because of high attrition across levels due to the toxic culture), thinks that tech XLT/LT are doing great and we all suck as a product and design team.
He started driving things his own way without even understanding or settling down for first 90 days.
Lol, I put in my resignation got out of that mess.
So agreeing to what our boy said here, no code tools are a complete waste, especially for a developer, and even as a non tech person, I prefer keyboard over mouse.2 -
them: welcome new project members, this is our CI/CD pipeline which is completely different from the rest of the company, there won't be any great knowledge transfer, we just expect you to be able to know and use everything. but also, we expect you to work on your tasks and don't waste any time.
me: okay, so my tasks aren't going as fast as expected, because I need to invest some learning so i can set up my project correctly.
later: some help would be nice, i'm stuck right now
coworker: *helps me to fix my problems, which were partly due to misconfigured build servers* i know it's a lot, and unfortunately, for this topic sources on the web aren't so good. i can really recommend this book, this will give a deeper understanding of the topic.
me: okay, yeah i mean, tbh, i'll read the book if the project invests some time for me so i can learn everything that's required, but this won't happen. also, some initial workshop on the topic or anything would have been nice.
coworker: well, i mean, i am a software developer. for me, it is normal that i learn all that stuff in my free time. and i think that's what the PM expects from us.
me: okay, that's fine for you, i mean, if i'm interested in a topic, i will invest my private time. but in this case, PM would just expect me to do unpaid labor, to gain knowledge and skills that i can use in this specific project. i'm not willing to do that.
coworker: ...
me: ...
it's not that i don't want to learn. the thing is that there isn't any energy left by the end of the day. i'm actually trying to find some work life balance, because i don't feel balanced right now, haven't felt since i started this job.
also, this is only one of several projects i'm working on. it's like they expect me this project has top priority in my life. if it wasn't so annoying on different levels, maybe i'd have a more positive attitude towards it.
also, at the moment i find it fucking annoying that i have to invest so much time in this dev ops bullshit and this keeps me from doing my actual work.
if they are unhappy with my skills, either they can invest in my learning or kick me out. at this point, either is fine for me..12 -
Central team: No, your team must be doing something wrong. Our pipeline is super-configurable and works for any situation! You just have to read the docs!
Me: Where are the docs?
Central team: Uhh, well, umm... we'll hook you up with a CI/CD coach!
Me: Okay, cool. In the mean time, can you point me at the repo where all the base scripts are?
Central team: Sure, it's here.
Me, some weeks later: Yeah, uhh, the coach can't seem to figure out how to make our Prod deployment work either.
Central team: That's impossible! It's so easy and completely configurable!
Me: Well, okay... but, here's the thing: your pipeline IS pretty "configurable", in the sense that you look for A LOT of variables...
Central team: See! We told you!
Me: ...none of which are actually documented, so they're just about useless to me...
Central team: But, but the coach...
Me: ...couldn't make heads or taisl of it either despite him literally being ON YOUR TEAM...
Central team: Then your project must just be architected wrong!
Me: Well, we're not perfect, so could be...
Central team: Right!
Me: ...but I think it's far more likely that the scripts... you know, the ACTUAL Python scripts the pipeline executes... while it took me DAYS to get through all your levels of abstraction and indirection and, well, BULLSHIT... it turns out they are incredibly NOT flexible. They do one thing, all the time, basically disregarding any flexibility in the pipeline. So, yeah, I'm thinking this is probably one of this "it's you, not me" deals.
Central team: Waaaaahhhhhhhh!!!!!!2 -
whhooOOOOOOOOOOOOOSHSHHHHHHHHHHHMMMMMMMMMMMMMRRRRRRRRR
yepp, that's me running a GitLab pipeline on my PC and laptop (laptop is noisier, 'cuz PC has Noctua all over the case).
Turns out running a pipeline of ~200 jobs is quite network-demanding. To the levels where my DNS server in LAN is timing out. And streaming Netflix in parallel kills some of the gitlab runners.
daammnnnn.... I so don't want to pay for an EC2 or EKS at this stage :/
But then again, I don't remember when was the last time I heard fans whooshing in my lappy. I so got used to only hearing the coils whine in it...
Decisions, decisions1 -
Big win today. Set up a custom motion capture pipeline for Unity using my Vive/controllers today. Quote a bit of scripting but I went from nothing to complete animation reel for our character model in a single day.
Successful day for once. I can sleep well tonight.2 -
I was waiting in the lobby of a recuiting place, hearing all their speeches. All super generic stuff, like "I looked at your resume and I have a bunch of cutting edge opportunities in the pipeline that are going fast, which I think you are a perfect fit for!" Which I don't argue is always invalid, just after hearing the same copy and paste speech, you start to wonder.
I even saw a few stand up and wave their hands around while trying to make a meh position sound like the latest golden egg.
I left with a feeling that recuiters would be a great inspiration for a "Wolf of Wall street" styled movie.1 -
When do you tell your boss you’re looking for another job?
I reached the point where I just don’t care anymore and I want to tell my boss that I am actively looking for alternative employment and I will be leaving as soon as I possibly can.
What would your advice be in this situation? Rather wait until I have something in pipeline in terms of new employment? Or just let them know that I have reached this point?10 -
We've got this legacy PHP system that doesn't really run anywhere else than on it's server. It's not configured with git, and there's no pipeline. Just plain old SSH. How would you go about managing it?11
-
Management has been promising we'd leave .NET framework for 2 years now. Never fucking happens. A new ASP.NET project was just started last week and yup, OF COURSE, its .NET Framework 4.8.
I'd even be happy with one of the earlier .NET Core versions at this point for fucks sake. I have no clue why tech leads are so happy to create a brand new project on a deprecated framework version.
And yes, I have checked thoroughly. Our whole infrastructure works with .NET Core onward. People are just too lazy to learn new stuff.
Stuff like switching to .NET 6, actually doing unit testing, improving our CI/CD pipeline, refactoring problematic codebases, etc. -> all this stuff is the kind of things they promise me I can work on later whenever I'm so bogged down with work that I'm looking for a light at the end of the tunnel. All empty promises.
Ideally we should be on .NET 6 since its LTS and just stay on the LTS versions as the year goes on.8 -
My current job at the release & deploy mgmt team:
Basically this is the "theoretically sound flow":
* devs shit code and build stuff => if all tests in pipeline are green, it's eligible for promotion
* devs fill in desired version number build inside an excel sheet, we take this version number and deploy said version into a higher environment
* we deploy all the thingies and we just do ONE spec run for the entire environment
* we validate, and then go home
In the real world however:
* devs build shit and the tests are failed/unstable ===> disable test in the pipeline
* devs write down a version umber but since they disabled the tests they realize it's not working because they forgot thing XYZ, and want us to deploy another version of said application after code-freeze deadline
* deployments fail because said developers don't know jack shit about flyway database migrations, they always fail, we have to point them out where they'd go wrong, we even gave them the tooling to use to check such schema's, but they never use it
* a deploy fails, we send feedback, they request a NEW version, with the same bug still in it, because working with git is waaaaay too progressive
* We enable all the tests again (we basically regenerate all the pipeline jobs) And it turns out some devs have manually modified the pipelines, causing the build/deploy process to fail. We urged Mgmt to seal off the jenkins for devs since we're dealing with this fucking nonsense the whole time, but noooooo , devs are "smart persons that are supposed to have sense of responsibility"...yeah FUCK THAT
* Even after new versions received after deadline, the application still ain't green... What happens is basically doing it all over again the next day...
This is basically what happens when you:=
* have nos tandards and rules inr egards to conventions
* have very poor solution-ed work flow processes that have "grown organically"
* have management that is way too permissive in allowing breaking stuff and pleasing other "team leader" asscracks...
* have a very bad user/rights mgmt on LDAP side (which unfortunately we cannot do anything about it, because that is in the ownership of some dinosaur fossil that strangely enough is alive and walks around in here... If you ask/propose solutions that person goes into sulking mode. He (correctly) fears his only reason for existence (LDAP) will be gone if someone dares to touch it...
This is a government agency mind you!
More and more thinking daily that i really don't want to go to office and make a ton of money.
So the only motivation right now is..the money, which i find abhorrent.
And also more stuff, but now that i am writing this down makes me really really sad. I don't want to feel sad, so i stop being sad and feel awesome instead.1 -
I want the pipeline-operator in JavaScript. I want it now!
... But then I mostly work with Vue.js (because it's awesome) in the last time and I somehow miss vanilla JavaScript. Life is cruel.4 -
A 30 minute Jenkins pipeline just happened to start at the same time as today’s Nintendo Direct. Whoops. Guess I’ll take a break.
-
"Do not lose time improving the data pipelines from our ERP, it is about to be replaced!"
Then suddenly there is a week of bugs and stress because the non-improved data pipeline can't handle new situations.
"Just fix the bugs! It is all about to change anyway!"
Repeat. And repeat.
Fuck, I hate when managers think that there are such things as "temporary fixes" in ERPs. Or that companies can ever migrate to another ERP. Those things are forever, as Cheetos dust on your bowels. -
School group project: Pipeline is done, everything deploys automatically, microservices work in development, https everywhere, Docker is bae, everything works perfectly
Now I have to work on frontend stuff ;_;8 -
So I just recently had the pleasure to set up a Rails environment for a friend on Windows. I haven't used Windows in about 5 or 6 years, and the person I had to set it up for doesn't know much about programming at all.
I all went fine at first, install database, devkit thingy and git. Then set up the project itself. And there is where the problems started.
First windows would refuse to use SSL, because of some weird bug in the Windows version of rubygems. The suggested upgrade did not work so I had to switch some gem sources to insecure connections, but at least it did install everything correctly.
Alright, I thought, that's not _that_ bad, everything is running now.
He sent me a screenshot some time later. Something was wrong with the JavaScript runtime, and I could not figure out for the life of me what the issue was.
Later again he sent me another screenshot.
His Antivirus spyware was messing with the asset pipeline. (╯°□°)╯︵ ┻━┻
This was the point where I just said "FUCK IT, i'll just put everything into a fucking VM and let him use that".
I should have done that in the first place.
Long story short:
Setting up a development under Windows is painful.
Do yourself a favor and just use a VM.3 -
Ok I know there have been a lot of similar rants to this one, but now I have to write one by myself!
Fuck freelancer.com or whatever that shit is called. I once started using it when I was in school because I thought it was a convenient way to earn money on the side without fixed work times, so I could adjust to how much time I have. But soon I realized that wouldn't happen. It is easy for me to make a website, I have written some css templates from scratch and can apply them, but when will these cocksucking assholes learn that $25 for a website is not only a joke, but a fucking insult? Or a logo for 4? In his video on fiverr, pewdiepie has a point on the thing where he said that you can shit out a logo in 2min and make an easy 4 to 5 bucks, but I like doing things more properly and I bet those fuckers will give you shit for not designing the perfect logo. I once accepted a job where I ended up busting my ass 3 days log for $100 and I thought that was the normal mess at the beginning, before you have former customers rate your profile, but I got perfect ratings and still didn't get or even find any proper jobs. Most are complete shit, like write a fucking book for me or design a fucking Website or pull a logo out of your ass, but some projects are just rediculous. I once saw a project where they wanted some engineer to do the layout for the pipes in a huge processing plant. Yeah, because engineers are so poor and unemployed, even when they are entrepreneurs they dont go to those shity sites. Since I am actually qualified for such a job, I applied just to see if I could land a job that is actually not shitty, but of course it turned out the person had no idea what he was talking about. It is basically a platform where people can pay you in exposure. And the absolutely fucking worst thing about it is that they get away with it. There are always a ton of people, mostly from countries where cost of life is significantly lower, who flood the freelance market with cheap, presumably horrible logos, mobile apps, websites, texts and apparently pipeline layouts. I haven't found a similar platform but where there are only high quality biddings. But that is something that I would love to use.
Sorry for long rant, no potato.1 -
Damn fucking channel diggers. One of them probably cut our fibre cable when working on the water pipeline to our house. And tomorrow is a holiday here, so no FTTH until at least wednesday (I am not that optimistic, though).
My mobile data plan is already stretched and I was very motivated to do something on my thesis.4 -
Long time ago i ranted here, but i have to write this off my chest.
I'm , as some of you know, a "DevOps" guy, but mainly system infrastructure. I'm responsible for deploying a shitload of applications in regular intervals (2 weeks) manually through the pipeline. No CI/CD yet for the vast majority of applications (only 2 applications actually have CI/CD directly into production)
Today, was such a deployment day. We must ensure things like dns and load balancer configurations and tomcat setups and many many things that have to be "standard". And that last word (standard) is where it goes horribly wrong
Every webapp "should" have a decent health , info and status page according to an agreed format.. NOPE, some dev's just do their thing. When bringing the issue up to said dev the (surprisingly standard) answer is "it's always been like that, i'm not going to change". This is a problem for YEARS and nobody, especially "managers" don't take action whatsoever. This makes verification really troublesome.
But that is not the worst part, no no no.
the worst is THIS:
"git push -a origin master"
Oh yes, this is EVERYWHERE, up to the point that, when i said "enough" and protected the master branch of hieradata (puppet CfgMgmt, is a ENC) people lots their shits... Proper gitflow however is apparently something otherworldly.
After reading this back myself there is in fact a LOT more to tell but i already had enough. I'm gonna close down this rant and see what next week comes in.
There is a positive thing though. After next week, the new quarter starts, and i have the authority to change certain aspects... And then, heads WILL roll on the floor.1 -
Another chapter of my nano-drone love-hate story. Deployment using the old pipeline, but with new augmentations. Next week hopefully, we will deploy a better version with the new pipeline :)
https://youtube.com/watch/...3 -
Devops (By Azure) is so stupid.... (I won';t even start of YAML, it will be a 10pages rant).
me : Ok I have 5 projects, each has it's own Azure conexxion for deploy.
Me : Can I do just ONE shared connexion ?
Devops : Yes. You need to click 150 buttons and it's done !
Me : Ok. /* doing actions */
Me : Ok ready !
Me : Project 1 do your release pipeline !
project 1 : Sure, just wait 5 minutes.
5 minutes later
Project 1 : All good.
Me :Ok now sharing test ! Project 2 : do your pipeline !Project 2 : Sure ! It's strated !
Me : Ok I'll go take a beer
... 1 hour later..
Me : project 2 ? PROJECT 2??!!!
Me : fine... going into logs.
Message : You must accept the shared conexion from Project 1 before pipeline can run
Me : WTF ? I literally just SHARED it to project 2,3,4,5 !!!
Why that idiot check ?!
One thing is sure, I hate devops more than I hate JavaScript.5 -
I recently tried to apply the same data analytics rationale that I use at work to my personal life. This is not a rant, it is more like an data storytelling of an actual use case I would like some input on.
I set a goal - gotta thin up a bit and calm down my ticker - and got a (almost unreasonably expensive) field expert consultant to yell at me about it for a couple hours.
I unravel the metrics - there is like a million weight-related KPIs and most say nothing at all. I have never seen an non-infrastructure measurable subject that could not be resumed to 2-5 performance metrics. I got overall weight, how well my nine-years-old business suit fits me, heart rate, and day-after relative muscle pain (it will make sense soon).
Then its data-pipeline time. I bought a cheap weight scale and smartwatch, and every morning I input the data in an app. Yes, I try to put on the suit every morning. It still does not fit.
After establishing a baseline, I tried to fit different approaches. Doing equipment-free exercises, going to the gym, dieting. None was actually feasible in the long run, but trying different approaches does highlight the impacts and the handling profile of each method.
Looking at the now-gathered data, one thing was obvious - can't do dieting because it is not doable to have a shopping list and meals for me and another for the family.
Gym is also off the table - too much overhead. I spend more time on the trip there and back than actually there.
And home exercise equipment is either super crappy or very expensive. But it is also the most reasonable approach.
So it is solutions time. I got a nice exercise bycicle (not a peloton), an yoga mat (the wife already had that one) and an exercise program that uses only those two resources. Not as efficient without dieting, not as measurable and broad as the gym, but it fits my workflow. Deploy to production!
A few months pass and the dataset grows. The signal is subtle but has support - it works! The handling, however, needs improvement, since I cannot often enough get with the exercise program. Some mornings are just after some hard days.
I start thinking about what else I can improve in the program, but it is already pretty lean and full of compromises.
So I pull an engineer and start thinking about the support systems and draft profile. What else could be draining my willpower and morning time?
Chores. Getting the kids ready for school, firing up the moka pot, setting the off-brand roomba, folding the overnight-dried clothes, cooking breakfast, doing the dishes, cleaning the toilets. All part of my morning routine. It might benefit from some automation.
Last month I got that machine our elders call "wasteful" and "useless crap lazy entitled Americans invented because they feel oh-so-insulted for simply doing something by hand like everyone always did" - a "dish-washer".
Heh, I remember how hard was to convince my mother-in-law that an remote-controled electric garage door would not make she look like an spoiled brat.
Still to early to call, but I think that the dishwasher just saved me about 25 mins every morning. It might be enough to save willpower for me to do more exercise.
This is all so reflective of all data analytics cases really are out in the wild - the analytics phase seems so small compared to the gathering and practical problem-solving all around. And yet d.a. is what tells you that you are doing the wrong thing all along. Or on what you should work next.7 -
I know you, youre out there somewhere, coding, feeling like shit, putting your best, listening to coldplay, in the server room, your basement ... I know you veryy well1
-
I mean, ONE of these days, the Prod pipeline has GOT to work... doesn't it?? Just basic odds say EVENTUALLY it will, doesn't it??
Of course, I'm sure I could have had it working long ago if not for the fact that we have to use common scripts developed by another team that we can't alter, and just guess where the problem is.
You know, standardization and DRY are good principles generally, but if you just apply them without thinking and without at least SOME flexibility, you only make matters worse.2 -
Every Monday I am privileged to wake up 1. 5 hours earlier and listen to executives talk about what meetings they had with potential customers and what news about ongoing agreements with other potential customers (which have been in the pipeline for 1 year or more sometimes)
Why is this necessary?? There has been virtually no progress on any of the topics discussed. At least nothing that concerns us devs.
Instead we get to wake up earlier, waste 1 hour, and get bored listening to what essentially seems like a sales speech while convince the employees the company is not spinning its tires while stuck in the mud.3 -
I am an Indie game developer. I've been working solo for two or three years now and teaching myself. I can work in 3D modeling applications as well as program in c++ and do blueprint in unreal engine. I know most of the pipeline and the suite.
I'd like to transition to doing game Dev full time or at the very least do programming as my job. I have no degree.
I'm looking for contracts or whatever I can get and I'd like to get suggestions on how I should go about quitting my shity night shift job at a factory and finally work in tech.
I've got a couple contracts going on right now that I am not sure if they are going to last. I would like to know how I should go about finding more and or what things I should do in order to get residual income so I can focus on my own projects.
I have several of my own games in the works and I'm developing some tools for the marketplace. Advice?28 -
PSA: If you need to make dummy commits to trigger a pipeline for test purposes, please just use --allow-empty and make an empty one. Don't just add random newlines back and forth in stupid places.6
-
Love my new job but fuck they are way behind in any kind of modernization.
Just saw a demo over zoom where someone was showing the team how to change the margin on an error page.
They literally changed the HTML directly in prod using the VIM. So first of all no web modernation because there was no build, no deployment, not git, no pipeline - NOTHING!
This project went from 40 people to around 200 in 6 mos. You can't have all these people in prod just making fucking changes.
If this job did not pay 110k a year I would bail.9 -
I just finished designing an entire asset management pipeline and christ on a fucking pogo stick, if it isn't convoluted.
Theres a lot of game engines out there, but all of them do it a little different. They all tackle a slightly different problem, without even realizing it.
1. asset management
2. asset change management
3. behavior change management
4. data management
5. combinatorial design management.
6. Combinatorial Behavior management
7. Feature completion
ASSET MANAGEMENT is exactly what it says on the tin.
ASSET CHANGE management can be thought of handling the import, export, formatting, platform specific packing, and versioning (including forking) of an asset.
BEHAVIORAL CHANGE management is a subset of asset management, because code is a subset of assets (depending on how you define 'assets'). The oldest known example of this is commenting and uncommenting code.
Or worse, printf debugging.
This can be file versioning, basic undo services, graph management of forks and mergers, toggles for features or modules, etc.
DATA management is about anything that doesn't fall into the other categories, everything from mission text to npc dialogues, quests, location names, item stats, the works. Anything you'd be tempted to put in a database, falls under this category. Haven't yet seen many engines offer this as an explicit built in tool as of yet, because the other problems are non-trivial as is, so this is a bit of low hanging fruit that gets handled by external tools, or loaded from formats as simple as json.
COMBINATORIAL DESIGN management is the idea of prefabbing, blueprints of broader object design using nested prototypes of existing game objects, to create more complex, reusable set pieces. Unity did this well. GM does this in part.
COMBINATORIAL BEHAVIOR management is entity-component systems, plus tooling to make it easy to add, remove, and configure components and their values on entity blueprints, also not uncommon. Both stencyl and unity do this. GM has a precursor to this in the form of configurable fields, but these fields are not based on component scripts attached to objects.
FEATURE COMPLETION is that set of gameplay mechanics or styles of design that an engine naturally makes easier to include or build in a game.
I don't think I'm aiming for all that, but I think at minimum a good engine has to do asset management, behavioral change management, prefabs, and entity-component systems with management tools for that. And ideally, asset change management.8 -
My legacy is now indisputable in this company!
Utf-8 emojis for pipeline declarations will became a new era for pipes from now on.1 -
Im deploying a Machine Learning Model to production. We dont have an automated deployment pipeline for the models, so we do it manually exposing the models through a rest api.
I asked for the model artifact to the DS, they didn't have permissions to download files from Databricks.
I asked their manager for the artifact. He told me that he has the permissions, then bullshitted me with something about the formal process, some shit about proper permissions handling, and that they do no have a standard process for sharing files right now so i should wait.
I was like "bro, share the artifact with me to unblock my work, then stablish your process, i dont care". He said no, and just after that he started a thread involving half of the middle management and data engineers asking for feedback on how to stablish a process for sharing databricks files. Just Wtf.
I got pissed, i reach out to his superior (good friend of mine), that was on vacation btw, and i told him the situation. He opened slack and humiliated him so bad, that i almost felt bad for the manager jajajajaja.
I grabbed my model artifact and got out of there instantly.2 -
Was wondering why my script wasn't working. Added it to my node modules. Imported the style sheet with a require. Checked the gulp config file. Made sure it was in my pipeline. Checked my webpack settings and made sure jquery was loaded first. Checked my version on npm, crossed referenced with my package.json file. Why isn't it working....forgot to add the script tag in my html😤2
-
** me setting up GitLab CI **
- run pipeline
- FAIL
- env variable not passed to one of the shell scripts
- set -x, rerun
- FAIL
- same reason. env variable is OK in the `set -x` output
- comment out `set -x`, rerun
- still FAIL
- same reason
- find a `set +x` left in one of the scripts
- comment that out
- rerun
- PASS
- WTF?!?!?!?
- continue on swearing for wasted better half of the day debugging my scripts12 -
Hey fastlane!
Great tool and all, but your documentation is at 🤡 levels, I need to read 20+ pages to get a full overview and understanding. So far I've had to read a dozen plus blogs and stackoverflow posts to find hidden flows (authentication first to do this, etc. etc. etc.)
Don't market your tool as "reducing complexity & saving time" and showing one-liners in the docs when in reality there are lots of hidden steps and NOT one-liners!!!!!!
This is why everyone complains it takes 1-2 days to just get a freaking pipeline working!!!!🤡 -
Why is planning so awful at so many tech companies (and in general I guess)?
I’m barely a month in and already having to drop current projects because someone made a promise to someone else and realized at the last minute “wait we need infrastructure for this” and came to me out of nowhere “hey we need this super complicated pipeline and complex architecture built”
I ask boss about how this should fall in priority, he basically says “figure it out”. Okay….thanks.
I ask for the business case from the requesters and when they need it and I’m told “we’re going live in late September”
Go back to boss, tell him what I “figured out” and asked if this was something he intended for me to take on. He skimmed through the ticket and gave a non-committal “I’m not sure what this is”. I’m still trying to figure out the infra here, still haven’t gotten access to half the things linked in the JIRA, but there’s been about 3 email chains and a Director recently DM’d me on slack asking for an update.
So I guess I’m flying in the dark on this one.
If you never hear from me again in this community I probably flew into the side of a mountain or something. This new job ain’t it.7 -
> Opens heroku
> Opens application
> *this pipeline does not have an owner*
> *internal screaming*
> refresh
> "Ah okay, heroku is retarded"
> *sigh of relief*2 -
Fucking fuck sonarcloud and everything about it. Part of the build pipeline for us to deploy code is to ensure that 90% of the code is covered by a unit test. Great in theory, horrible in practice. You think you've written enough tests that actually add value and test a valid piece of functionality but NO, sonarcloud throws a fucking fit because you're at 89.888 then your branch is going nowhere. Because everyone else gets to this stage and writes just enough tests to get the coverage to 90.01% then it becomes a stand off of who will break first; the code coverage threshold or your mental state.4
-
Me: "Need help with build config problems, please help almighty documentation page!"
Docs Page: "Nah fam, I got 4 headers about problems with no text, a blank code example, and 2 error 404 pages."
And that's why I don't like build pipelines. -
Stupid pipeline bullshit.
Yeah i get it, it speeds up development/deployment time, but debugging this shit with secret variables/generated config and only viewable inside kubernetes after everything has been entered into the helm charts through Key Vaults in the pipeline just to see the docker image fail with "no such file found" or similar errors...
This means, a new commit, a new commit message, waiting for the docker build and push to finish, waiting for the release pipeline to trigger, a new helm chart release, waiting for kubernetes deployment and taking a look at the logs...
And another error which shouldn't happen.
Docker, fixes "it runs on my machine"
Kubernetes, fixes "it runs on my docker image"
Helm, fixes "it runs in my kubernetes cluster"
Why is this stuff always so unnecessarily hard to debug?!
I sure hope the devs appreciate my struggle with this... well guess what, they won't.
Anyways, weekend is near and my last day in this company is only four months away.2 -
You guys! You guys! You'll never believe what happened today!
I used out CI pipeline half a dozen times today and it worked every time!
Seems impossible, I know, but it's true!1 -
OH MAH GAWD, FUCK ECLIPSE.
Just thank you so much Eclipse, for executing the entire build pipeline instead of running the last configuration.
Thank you so so much for considerately forcing me to clean upon every change to the codebase, also obliterating reliable hot code replace.
Special thanks for letting me clean once more, after the pipeline finished.
And thank you for the search bug, where all CTRL+H searches result in an error message, even though the search is being conducted.
Just go fucking die, you tremendous piece of garbage.5 -
My team and I are working on a huge project that's been in development for years.
First deadline was in the fall last year. We were never going to make that.
Then we were supposed to be ready just after the summer holidays (months ago). We didn't make that either.
Then we were supposed to launch last week. Didn't happen, still too many critical errors and unfinished, untested features.
Now we are having daily meetings to discuss whether we'll be ready to release... that day!
Meanwhile, stability issues and other critical errors keep popping up. The product is barely finished and has not been through rigorous testing with all the latest features and bug fixes. Not to mention that we don't really have a deployment pipeline either.
And here's the kicker: The customers don't know this is coming. It's highly anticipated, but only internally. It is a replacement for an existing product, which strives towards not changing the frontend too much.
Why do we rush it so? I get that a deadline can help motivate you to reach your goal, but how motivated will we be if the launch fails and we get buried in bugs and missing features?
Would it not be better to launch it with at least the confidence of knowing that we've tried to test it properly?9 -
So in the project I’m working on we were about to do a push to live, no major functionality just minor adjustments and nice to have stuff. One of the things I did was a reminder, nothing special just sends an email out if something hasn’t been done for 3 days and then sends an email every day following. Push to live and every thing goes fine with no issues. Day 1 there are no issues. Day 2 there are no issues. Day 3 and I’m inundated with people telling me that the emails are getting sent to practically everyone, shit. What have I done? What have I missed?
So I start looking at the live database hoping for a data problem, no such luck. I look at my code looking for something blatantly obvious but nothing. I start replicating the data but I can’t reproduce this bug and it’s annoying the hell out of me. I checked one of the emails that the client sent to us more thoroughly and seen that it was sent at 07:01. This is odd as our webjob runs at 1am so I start looking at environmental factors and started looking at release management, more out of hope than expectation. I check the staging environment and see that the webjob ran at 7:00. Coincidence I thought, the webjob gets packaged on the release pipeline and everything in the database was dummy data anyway but I’d better check anyway. The database was an exact copy of the live database, turns out a “senior developer” wanted to sanity check everything by running live data through the code so he copied the database over. It was fine for the first couple of days but the data was now 3 days out of date triggering my email code and I get hit with the shit storm. I’ve never met such an incompetent developer in my fucking life, functions 700 lines long, classes that are over 20000 lines, repetition every where and the only design patterns he’s used is when he picks up a child’s colouring book. I can live with the fact that he writes code like someone on their first day of University But copying a database because he wants to “visualise” the fucking data is absolutely farcical. No wonder the project is fucked with a “developer” (in the loosest possible use of the word) is at the helm. -
Fuck heroku, fuck precompiling assets, fuck the assets pipeline, fuck fuck fuck fucking piece of shit3
-
merged a complicated pipeline script based on 3 other scripts, and it worked the first time I run it.
Pretty sure There was a Disturbance in Force... -
...another (probably about fourth) completely futile attempt at making MASM compiling pipeline work...
...what the fuck... seriously, i've spent together about two weeks of time trying to make a fucking default hello world compile... ml64 problems, then rc.exe problems, apparently i was missing some dumb CommonService.dll which not only doesn't exist anywhere on my computer, but it doesn't even seem to exist at all in this fucking dimension. After several hours I had the bright idea of "fuck MS rc, let's just grab any other random resource compiler that I can find, and see if that one works".
Funnily enough, it does. Except Visual MASM can't run it from it's build process because it fucks up the commandline call, so I need to run it manually, and then when I run the build from V-MASM, the rc call still fails, but then it checks for the resulting .res file and finds it, so it happily continues with success...
...and now fuckin... what even is it? *goes to check*
oh yeah, now linker is shitting itself:
LINK : fatal error LNK1104: cannot open file 'user32.lib'
And I'm just completely defeated, just searching system-wide for the lib intending to copy it into the linker folder because fuck this fucking bullshit, I've had enough of drowning in MS BuildTools versions and installations and uninstallations and fixes and modifys and repairs and all that FUCKING BULLSHIT.
HOW. THE. FUCK. is this in any way usable for anyone. I suspect nobody ever actually tried to build an assembler project in the last 30 years, so nobody noticed it DOESN'T. FUCKING. WORK.
THIS.
THIS is why I hate anything that's not a proper IDE where I install ONE thing, and do everything in that ONE IDE and let IT figure out all this linuxy-soft-coupled bullshit of twentyfuckingthousand fucking useless commandline apps threwn around the whole fucking system where I'm fucking supposed to know where the fuck what is and which version and GO FUCK YOURSELF.
GIMME. FUCKIN. ONE: IDE. WHICH. WILL. INSTALL. ALL. THAT. IT. NEEDS. TO. BE. FUCKING. ABLE. TO. FUCKING. WORK. AND. COMPILE. SHIT!!!
FUUUUUUUUUUUUUUUUUUUUUUUUUUCK.10 -
I'm trying to build VoIP into my browser-based game, and holy shit are sound processing people bad at explaining stuff.
Every stackoverflow answer has badly named variables, noone names the algorithms they're using (which makes research near impossible), and literally every single Web Audio API pipeline I have seen so far contains at least one unexplained effect with no parameters, but it's a different effect each time.
One guy had implemented some kind of smoothing for catching up with the stream after interruptions (where the playback speed is proportional to how far we're behind the intended latency), without ever mentioning it anywhere. And this is meant to be a basic example!4 -
I discovered a commit message from one of my (senior) colleagues today. It made me shudder. It read, 'Just adding some changes made outside of source control and deployed (over last 12 months)'.
I genuinely think he can't follow any processes he didn't design. He controls the servers too, so it's not like any pipeline would prevent him from just doing what he wants. It's a bit scary to be honest, he thinks MD5 is a secure password hash! -
* Gets handed additions to current software platform (web)
* Gives back estimte of time after meeting with everyone and making them understand that once the testing phase of the project is reached there will be no changes, tests should be exhaustive and focus on SAID FUNCTIONALITY of the new additions. NO CHANGES OR ADDITIONS AT THIS POINT IN TIME
* All directives, stakeholders, users etc agreed on my request and spend an additional hour thinking of different corner and edge cases as provided by me in case they can't think of them (they can't, because they are fucking stupid, but I provided everything)
* Boss looks irritated at their lack of understanding of the scope and the time needed, nods in approval after he sees my entire specification, testing cases, possible additions to the system etc
* All members of the committee decide on the requirements being correct, concrete and proper.
* Finish the additions in a couple of weeks due to the increased demand for other projects, this directly affects the user base, so my VP and Director make it a top priority, I agree with their sentiment, since my Director knows what he is doing (real OG)
* I make the changes, test inside of my department and then stage for the testing environment. Everything is ready, all migrations are in order, the functionality is working as proper and the pipeline for the project, albeit somewhat lacking in elegance is good to go.
* Testing days arrive
* First couple of hours of test: Oh, you know what, we should add these two additional fields, and it would be good if the reporting generated by the system would contain this OTHER FORMAT rather than this one.
* ME: We stated that no additions would be done during the testing environment, testing is for functionality, not to see if you can all think of something else, even then, on June 10 I provided a initial demo and no one bothered to check on it on say something.
Them: Well, we are doing it now, this is what testing is for.
Me: Out of this room, the software engineer is me, and I can assure you, testing is not for that. I repeatedly stated that previously, I set the requirements, added corner cases, tables charts everything and not one single one of you decided to pay attention or add something, actually, said functionality you are requesting was part of one of my detailed list of corner cases, why did you not add it there and then before everything went up?
Them: Well I didn't read it at the time (think of the I in plural form since all of these dumb fucks stated the same)
Then my boss went on a rampage on their dumbasses.
I fucking hate software development sometimes.
Oh well. Bunch of fucking retards.4 -
I just joined a new company.
Their CI pipeline is to give root access to staging and prod servers to every developer in the company and the manually git pull each repository (8-10 repos per server) and manually set nginx and port configurations. And if this wasn't enough, all of the 30 sites they have are basically the same site and they make the changes manually for each tenant (no env file). I'm amazed at how hard some people are willing to work.5 -
/me joining a new front-end project with enforced prettier rules to complete the build pipeline.
No double-quotes ! Single-quotes all the way !
- weird flex, but OK. I'll comply
No CRLF ! LF only !
- Ok, now you're starting to annoy me. With git autocrlf I can have my precious CR locally bu check in only LF and YOU CAN GIVE THE FUCK ABOUT WHAT LINE ENDINGS I USE LOCALLY WHEN EVERYTHING WILL BE CORRECT WHEN PUSHING COMMITS!
No semi-colons !
- Now I hate you18 -
Until today, I had assumed deploying stuff to prod would NOT be one of my responsabilities in this company. Apparently that's not the case.
Had to deploy my code and pray it didn't break anything. Why is this a big deal at all?
Well you see, there is no repository. At all. No git, no svn, not even duplicate folders. No tests, no pipeline. Just a bunch of CPanels.
Had to manually copy files and folders from the development site to the production site and partially copy a database. "Just drag and drop" were the instructions I was given.
As if using CakePHP2, PHP5 and having to parse fucking Excel files wasn't bad enough, now I have to deal with one of the worst ways to deploy code.
Fuck it, I'm switching on the looking-for-job flag on linkedin.5 -
Microsoft :
"YOLO All Devops pipeline are now .net 8"
Me, lunching PROD release : "Whatever, I don't use any advanced shit"
Pipeline : "Can't connect to your nuget soure (private) : 401"
Me : What ? You just connected to it in prevous step in pipeline !
Azure : Yah, but prev st ep was using different syntax so it worked
Me : Ok, here you go, same syntax
Azure : Still nop.
Me : WHY ?!!!
Azure : I won't tell you. 401 !!
Me : Fuck you, I 'll rewrite this step from scrath !
30 mins later
Azure : Naaaan, stil 401.
Me : HOW ?!!! You literally use this nuget feed in 2 prev steps with no issues.
Azure : Fuck you
Me : fuck you too.
15 mins later
me to me... Hmm, I've already restored all packages. They "should" stay where they are..
Me : Take that azure : Ignore restoring for this step"
Azure : Oh yeah, that works ! All compiled !
WTF ?!!!! And why they release cha ges without even sending a mail. because fuck us apparently.
God I hate devops.... -
In August 2021 I asked my bosses for a raise for my extra work that I done for the last 6 months to create the first 4 microservices in the company and deploy them in production on a new infrastructure that is cloud based.
Today 27/01/2022 they reported how happy they are of me and that they will take my proposal *in consideration*.
Now i am searching for a new job.
Funny part: I am simple guy if they would have given me a NAS from Synology with lot of space as a reward I would actually have been happy.
P.S. jokes on them, i left 4 easter eggs hidden in the app, pipeline, db and user manual.3 -
story of a release
v2.1.0 major changes went live : new features, bug fixes, optimisations. also included releases for 2 associated libraries
release process tasks:
- do code
- update test cases
- test sample app
- test on another sample app
- get code reviewed and approved by senior ( who takes his own sweet time to review and never approves on first try)
- get code reviewed again
- merge to develop after 20 mins( coz CICD pipeline won't finish and allow merging before that)
- merge to master after 20 mins( coz CICD again)
- realise that you forgot to update dates in markdown files as you thought the release will be on 10th sept and release is happennig on 12th sept coz of sweet senior's code fucking/reviewing time
- again raise a branch to develop
- again get it a review approval by sr (who hopefully gives a merge approval in less time now)
- again get it merged to develop after waiting for 20 mins
- again get it merged to master after waiting for 20 mins
- create a clean build aar file
- publish to sonatype staging
- publish to sonatype release
- wait for 30 mins to show while having your brain fucked with tension
- create a release doc with all the changes
- update the documentation on a wyswig based crappy docs website
- send a message to slack channels
- done
===========
why am i telling you this? coz i just found a bug in a code that i shipped in that release which still got in after all the above shitty processes. its a change of a 3 lines of code, but i will need to do all the steps again. even though i am going through the same shitty steps for another library version upgrade that depends on this library 😭😭
AND I AM THE ONE WHO CAUGHT IT. it went unnoticed because both of those shitty samples did not tested this case. now i can keep mum about it and release another buggy build that depends on it and let the chaos do its work, or i can get the blame and ship a rectification asap. i won't get any reward or good impression for the 2nd, and a time bomb like situation will get created if i go with 1st :/
FML :/6 -
Crazy deadlines> Director: "You need to design a new architecture that has failover, multi-AZ, automated deployments, CI/CD pipeline, automated builds/tests as well, for our new SaaS product. You have 3 days to complete it"
Me: "Ok cool. Do we have the new product developed? Can I have the spec docs of the new software, libs and packages required for the env?"
Product Lead: "No we dont have anything yet. The POC is on my local PC, but I dont know what packages are needed to run it"
Me: "So I cant design anything unless I have the minimum requirements to run the new software"
Director: "Just get it up and running in a live environment and we'll take it from there"
Me: *sigh*..this is going to be a big mistake -
shutdown an old Jenkins CI pipeline from one of our teams... after one month they call us to bring it online again because they don't know how to build it manually.... lol #automation #scripting #jenkins2
-
Sr Engineer says "I'm not sure how this project is deployed".
You're a software engineer. You solve problems all day long. This particular engineer has worked here for several years and is well aware of the fact that we use Gitlab to run deploys.. If you want to know how it's deployed... look at the pipeline definition...
I don't understand how people, especially those with several years experience with our stack, can't solve basic problems on their own.2 -
Listen dude I get it, you've been in more of a Systems Admin role for a long time, you haven't really worked on a devlike team.
I can be patient I can be understanding. But when you break the build you need to fix it.
Yes I know you didn't change any of the files that are now failing, but you the pipeline is no longer deploying and so we can't fix anything.
Okay dude we are being prevented from deploying because you broke the build, you need to fix it. It's stopping everyone else.
DUDE FIX THE FLIPPING BUILD EVERYONE IS WAITING FOR YOU TO FIX THAT!
Seriously I know we should be patient with people learning new things, but some days it is difficult.5 -
Hand over from another developer who handled our internal data pipeline. I tried to set meetings for the handover he kept canceling them. When I asked him why? He replied I'm writing a documentation so that you won't the meeting. After 3 weeks, he emailed me with an overview diagram for the data pipeline.
My reaction: Woooooooooot? You built this in 3 weeks. WTF3 -
Worst: forced to work for 9 months on a shitty wp theme:
- colleague with no clue trying to make me do their work… check
- incompetent manager doing shit about it… check
- idiotic pipeline requiring to redeploy for every asset update… check
- micromanaging cto which for some fucking reason didn’t want to allow access to the writers, forcing the role of content editor on the devs… ducking check! Quack!!
Best: automated lots of processes in my free time, all stuff which I can reuse! -
CircleCI:
- Ensuring work has meaning: "Let's make yet an other dashboard webapp that going to replace all of our dashboard webapps which we made to replace all of our dashboard webapps"
-Solving interesting problems: "Let's make this with java 15 instead of java 14!!!! Also add graphql to ADD interesting problems nobody had since the nineties!"
- Gain meaningful value from talent: 'Bitbucket and the whole pipeline died fourth time this week, I'm going to drink a coffee or two..."
- Developers in flow: "Joe went to have a lunch around 11:00, you probably should not look for him until 14:30."
- Bring buying decisions closer to the engineering team: "The boss tried to bring up the pros and cons between aws and azure... The police eventually had to break the ensuing fight in the meeting room. The survivors reported things got truly out of hand when someone mentioned line-endings"
- Bring leadership closer to the engineering team: "There was yet an other agile coached hired, when she asked how should we measure velocity one of the lead devs managed to actually wake up and told her that the wifi is still pretty fucking slow" -
So I found a course that shows how to make purely procedural animation for characters. The demo on youtube was pretty amazing. It is like 9 hours of coursework. I am expecting some good information from this. The course is for Unreal 5. I intend on extrapolating the relevant data to use with Godot.
I started watching the videos and everything is being done in blueprints. Not sure what I was expecting. The goal is to do procedural animation in Godot. The logical place to start is to install Unreal 5 so I can learn the blueprints and transfer that knowledge to Godot. Kinda funny. Yet another launcher I "get" to install.
The goal is reduce the artwork pipeline. If I can rig characters correctly I am hoping I can share animation procedures and not have to spend hours keyframing animations. Also, code that moves stuff is cool. I genuinely want to know how to make that work. No idea if the goal is achievable.
The course:
https://udemy.com/share/...
Youtube:
https://youtube.com/watch/...1 -
Build pipelines are awesome. What's not awesome is forgetting the 'pipeline' bit.
Sure, let's have each TeamCity job in the 'pipeline' build the deployable from different places! You must manually merge your code from dev to test to release.
What's that? The functionality working in dev isn't there in production? I wonder how that happened. -
Dev work estimation is harder when the higher ups gives you a one line jira description and expect you to provide them with a timeline. Like how many times i gotta send the ticket back up the pipeline ><1
-
++ if you've ever had to type something along these lines:
def main(args):
do = some
stuff = here
if __name__ == '__main__':
main()
trying to figure out the best way to write a driver script for a processing pipeline and the choices have boiled down to;
a) define "new" main functions where necessary
b) learn subprocess module
c) write a bash script to do it instead
still not sure what I'm going with... -
Okaaay.....so after I got fucked at the technical interview i returned there the following day and wrote them code for a system that was actually on their pipeline
Left the thing 80% complete and they were like: Amaizing we will be in touch
And I left bouncing1 -
I'm in a team of 3 in a small to medium sized company (over 50 engineers). We all work as full stack engineers.. but I think the definition of full stack here is getting super bloated. Let me give u an example. My team hold a few production apps, and we just launched a new one. The whole team (the 3 of us) are fully responsible on it from planning, design, database model, api, frontend (a react page spa), an extra client. Ok, so all this seems normal to a full stack dev.
Now, we also handle provisioning infra in aws using terraform, doing deployments, building a CI/CD pipeline using jenkins, monitoring, writing tests, building an analytics dashboard.
Recently our tech writer also left, so now we are also handling writing feature releases.
Few days ago, we also had a meeting where they sort of discussed that the maintenance of the engineering shared services, e.g. jenkins servers, (and about 2-3 other services) will now be split between teams in a shared board, previously this was handled only be team leads, but now they want to delegate it down.
And ofcourse not to mention supporting the app itself and updating bug tickets with findings.
I feel like my daily responsiblities are becoming the job responsibilities of at least 3 jobs.
Is this what full stack engineering looks like in your company? Do u handle everything from app design, building, cloud, ops, analytics etc..7 -
delete random, unused column from production database
entire app blows up
re-release last change on code pipeline
everything works beautifully
¯\_(ツ)_/¯2 -
Jesus christ I need my VP and CIO to get their hands out of Azure and GCP and just let me work.
Yes, governance and security and IAM are big deals. That's why you have infraops people like me to deal with that.
I'm literally working with one hand tied behind my back because just about every button press or CLI command I need to do my damn job as a professional cloud fluffer requires me to go bother an executive and ask permission to pretty please can I deploy a new container, can you go press the shiny button? No not that one, move your mouse up...up..now UP..ok over lef-no..can I have mouse control? Sigh fine, do you see where it says "Approvers", no that says "Release Pipeline"
Look I actually kinda like this job, I do, in as much as when I have something to do I get left the fuck alone to do it. Meetings are minimal, aside from the odd days when one of our app services decides to yeet itself into the river Styx, there's little distractions.
Yeah, developers do dumb shit but that's probably best left to the notion of job security and never talked about again less they go to HR and complain that the ops guy was very stern and direct and made the developer take some accountability for their work product.
AND YET
It's so intergalactically stupid that I have to go ask permission just to do ops tasks by the same people barging down my goddamn door asking why the ops task isn't done yet.
"Because you won't give me permissions in GCP to actually DO anything".
Okay. Rant over. Time for lunch. Good meeting, see you all at the holiday party.2 -
I am lying down on the floor because I cannot figure out why this specs pass locally but repeatedly fail on the ci/cd pipeline. Literally done everything now I just want to lie here and sleep.3
-
I called the hack "blow up bunny", was in my first company.
We had 4 industrial printers which usually got fed by PHP / IPP to generate invoices / picking lists / ...
The dilemma started with inventory - we didn't have time to prepar due to a severe influenza going round (my team of 5 was down to 2 persons, where on was stuck with trying to maintain order. Overall I guess more than 40 % ill, of roughly 70 persons...)
Inventory was the kind of ultimate death process. Since the company sold mobile accessoires and other - small - stuff.
Small is the important word here....
Over 10 000 items were usually in stock.
Everything needed to be counted if open or (if closed) at least registered.
The dev task was to generate PDFs with SKUs and prefilled information to prevent disaster.
The problem wasn't printing.
The problem was time and size.
To generate lists for > 10 000 articles, matching SKUs, segmented by number of teams isn't fun.
To print it even less. Especially since printers can and will fail - if you send nonstop, there is a high chance that the printer get's stuck since the printers command buffer get's cranky and so on.
It was my longest working day: 18 hours.
In the end "Blow up bunny" did something incredibly stupid: It was a not so trivial bash pipeline which "blew up" the large PDF in a max of 5 pages, sent it to one of the 4 printers in round robin fashion.
After a max of 4 iterations, bunny was called.
"bunny" was the fun part.
Via IPP you can of course watch the printer queue.
So...
Check if queue was empty, start next round with determined empty printer queues.
Not so easy already. But due to the amount of pages this could fail too.
This was the moment where my brain suddenly got stuck aft 4 o clock in the morning in a very dark and spookey empty company - what if the printer get's stuck? I could send an reset queue or stuff like that, but all in all - dead is dead. Paper Jam is paper jam.
So... I just added all cups servers to the curl list of bunny.
Yes. I printed on all > 50 printers on 4 beefy CUPS servers in the whole company.
It worked.
People were pretty pissed since collecting them was a pita... But it worked.
And in less than 2 hours, which I would have never believed (cannot remember the previous time or number of pages...)1 -
Jenkins' triggerManualBuild randomly but if so then consistently produces 500 errors for certain newly created jobs. I haven't really found a pattern, yet I was bit by it in the past already. I used to "solve" it by deleting the offending job and re-creating it.
Now, I have this annoying issue again, and no matter how often I re-create that shitty one-liner job in the pipeline, it won't trigger. (The job itself is fine. It's the actual trigger that is broken.)
It's not like it's important or anything, as this is basically only the "push to production" step.
FML. And fuck me for stating: "Creating a delivery pipeline should be straightforward. I therefore consider 1 storypoint enough."4 -
Don't you just love it when gitlab's ci pipelines crash for no apparent reason, causing tests which cannot fail to just magically break down, change logging levels to Just about anything and basically PMS for about 3 hours before it decides it needs to restart completely and when you return the same pipeline which you've been trying to fix for the better part of an entire evening, after regular work hours, it. Fucking. Works. With. No. Changes. To. The. Entire. FUCKING. System.
Waste of a day.3 -
v0.0005a (alpha)
- class support added to lua thanks to yonaba.
- rkUIs class created
- new panel class
- added drawing code for panel
- fixed bug where some sides of the UI's border were failing to drawing (line rendering quark)
v0.0014a (alpha) 11.30.2023 (~2 hours)
- successfully retrieving basic data from save folder, load text into lua from files
- added 'props' property to Entity class
- added a props table to control what gets serialized and what doesn't
- added a save() base method for instances (has to be overridden to be useful beyond the basics)
- moved the lume.serialize() call into the :save() method on the base entity class itself
- serialized and successfully saved an entities property table.
- fixed deserializion bugs involving wrong indexes (savedata[1] not savedata[2])
- moved deserialization from temp code, into line loading loop itself (assuming each item is on one line)
- deser'd test data, and init()'d new player Entity using the freshly-loaded data, and displayed the entity sprite
All in all not a bad session. Understanding filing handling and how to interact with the directory system was the biggest hurdle I was worried about for building my tools.
Next steps will be defining some basic UI elements (with overridable draw code), and then loading and initializing the UI from lua or json.
New projects can be set as subfolders folders in appdata, using 'Setidentity("appname/projectname") to keep things clean.
I'm not even dreading writing basic syntax highlighting!
Idea is to dogfood the whole process. UI is in-engine rendered just like you might see with godot, unity, or gamemaker, that way I have maximum flexibility to style it the way I want. I'm familiar enough with constructing from polygons, on top of stenciling, on top of nine-slicing, on top of existing tweening and special effects, that I can achieve exactly what I want.
Idea is to build a really well managed asset pipeline. Stencyl, as 'crappy' as it appeared, and 'for education' was a master class in how to do things the correct way, it was just horribly bloated while doing it.
Logical tilesets that you import, can rearrange through drag-n-drop, assign custom tile shapes to, physics materials, collisions groups, name, add tag data to, all in one editor? Yes please.
Every other 2D editor is basic-bitch, has you importing images, and at most generates different scales and does the slicing for you.
Code editor? Everything behavior was in a component, with custom fields. All your code goes into a list of events, which you can toggle on and off with a proper toggle button, so you can explicitly experiment, instead of commenting shit out (yes git is better, but we're talking solo amateurs here, they're not gonna be using git out the gate unless they already know what they're doing).
Components all have an image assignable to identify them, along with a description field, and they're arranged in a 2d grid for easy browsing, copying, modifying.
The physics shape editor, the animation editor, the map editor, all of it was so bare bones and yet had things others didn't.
I want that, except without the historic ties to flash, without the overhead of java, and with sexier fucking in-engine rendering of the UI and support for modding and in-engine custom tools.
Not really doing it for anyone except myself, and doubt I'll get very far, but since I dropped looking for easy solutions, I've just been powering through all the areas I don't understand and doing the work.
I rediscovered my love of programming after 3-4 years of learning to hate it, and things are looking up.2 -
Yesterday, I was expecting my merge request to be closed.
I've done all the stuff my tech lead told me to do.
All tests passes, green light boyzzzzz.
Gitlab CI pipeline passes, greeeeeen light I said.
In Jenkins everything f*cked up...
Why ??
Well it was a conflict with 3 other MRs, missing rebase from other dudes.
And because they were remote working, got to clean up all this mess.
That's was a day off.
PS : well that's was not so off, I could fix a UB on a ternary and extend a test which was not covering some cases.
PS2 : learn git damn3 -
“OhhHhh please fill out an entire fucking excel sheet for our test environment deployment. It helps us manage everything better and gives us a reason to fucking thumb around in our holes all day and pretend like we really mean something as managers.” Like absolutely no, you can go fuck yourself with a condom filled with broken glass shards and diseases is what YOU can do. You are a parasite.
“Senior lead developer” - but they don’t have a title - says: “please just give me the list of files I need to manually change on the env in real time”. Bitch, do you even know what CI/CD is?!?!? The fact that you have been doing this for a year straight makes me pity how much of a fucking dumbass you really are. Even if u don’t use a pipeline, just look at my fucking git changes. That’s literally why we have it. You are a fucking disgrace of a developer and I hope you know that everyone who is a competent dev would rather bathe in a bath filled with lemon juice and cactus spikes, before EVER working with YOU EVER AGAIN!!!2 -
Nothing like client requesting "Hey can you add a virus scan for every goddamn file in the pipeline ? (We don't know what antivirus software you can use, neither do we know what are the consequences of this virus scan failing. We also don't know why a system admin is not doing this server wide instead of a software Dev doing it for this specific individual component. Heck we don't even know why do we need the virus scan in the first place) You might need to think 'outside the box'. Let's hope this is done by 12th Jan. Regards" on a Friday goddamn night
They probably even expect me to write my own antivirus software instead of paying for one
Why is industry like this ? Is there really any polite way to deny this ?15 -
Any good async (if possible, also pipeline based) Networking libraries for C++ on Linux? I want to create a small Server which should be able to handle a big amount of TCP connections (~500) at the same time, using threadpools maybe, something like Netty for Java would be the best.9
-
Fuck this.. I have to tell you this annoying wtf..
Hi btw this is my first rant so pls dont blame me :)
I am working on an etl project for our company to connect data sources like netbase, similar web, etc. to alooma (a data pipeline).
Now I got the task to add another data source called BrightEdge to it.
All fine.
BUT WHAT THE ACTUAL FUCK.. IT TAKES 3 MONTH TO GET AN ACCOUNT. AND U KNOW WHAT.. I DIDNT EVEN GET THE API CREDENTIALS. THIS IS MY FINAL PROJECT FOR MY TRAINING TO BECOME AN IT SPECIALIST.3 -
so my friend and I are canvassing NLEs for our guys at the Post Production squad in our project (we were in charge in infrastructure). We looked at Premiere since it's kinda ok until we found Black Magic Design's Da Vinci Resolve.
First of all, I was suprised with the price. 299 for the Studio Version? Holy fuck, that's cheap as hell! Then there's a free as in free software version which has the core editing features with 1080p rendering. So we grabbed that and kinda suprised it requires postgres but as seeing Resolve having collab and render queues, it makes sense.
Installed them on the PCs the postprods gonna use, they were amazed. We literally saved 500 bucks for an NLE. When they asked how much is it. Our reply was:
"That's free".
and there was silence...
"And it's also 299 bucks for the cooler version".
And silence still ensued.
Guess our guys wasted alot of money on a pipeline that is cheap as hell but more jam-packed than any other NLE found in the market.
Props to you BlackMagic Design. -
WTF kind of bullshit software is sonar.
I can't deploy my application because sonar is telling me that there is a vulnerability. So I look at it. IT'S A FUCKING DEV DEPENDENCY. Are you fucking serious sonar? I can't deploy because a dev dependency has a vulnerability that allows DOS attacks. What the fuck do you think will happen?! I'm going to DOS my own fucking application whilst coding or what? Who the fuck would even care?!
I fucking hate our Pipeline, all the tools behind it operate like shit. the only thing positive about it, is that I am able to deploy applications myself without having to call someone and wait a week. Because putting a file in a directory is hard ._.3 -
What the fucking fuck. Arquillian you piece of shit.
I have a service that needs to go to production soon, it contains Arquillian tests. The tests work locally but not when going through our new Jenkins pipeline. The error message simply says: "Could not start Arquillian container".
Well fuck you too.
After 3 fucking days of rewriting configs, changing up things and I dont know what else I did, I stubled upon the most hidden error message in the history of error messages, a small little line that says "Could not find or load main class ".
Those 2 spaces are intentional btw, because the fucking error was that when starting arquillian and reading the config there was A FUCKING SPACE too much in my JVM arguments. This piece of shit iterpreted it as my FUCKING MAIN CLASS. Whhhhyyyyy, whhhyyyy. Who the fuck... AAAAAAAAAHHH
Btw I snuck myself on devrant a few weeks back and managed to get my 100++ today. Really love this place 😊1 -
I suck at DevOps at least as hard as I'm good at front-end/UX. I found out as a result of the local job market starting to get needy for 'full-stack'. Stuck for 2 days on setting up a Docker/ Dockerhub/ DigitalOcean/ Bitbucket pipeline with Nginx/ Node/ MongoDB Cloud & Webpack/ React.
*Sigh*1 -
!rant
there's almost nothing more satisfying then getting an automation script or CI/CD pipeline juuuuuuuust right 😎 -
I can now appreciate some design decisions behind react-redux after witnessing some angular OOP clusterfuck.
I am sure there is some clean/correct way to code in angular, but everyone is treating angular as java.
Some angular application (the one I have to work with) is littered with network calls. It's difficult to spot duplicates. People usually resolve promises everywhere. In services, in a top-level component, or in for loops. In react, people use apollo/redux-query or redux-saga to handle network calls. Since these libraries prevent duplicate network calls internally and reassigning apollo network call function or redux action function is always useless, it's easy to spot all network calls in a component tree.
In angular, it's difficult to trace data mutations when data can be updated everywhere. In react, you can easily find UI state updates by tracing state hooks/dispatch/apollo usages.
In angular, it's difficult to trace data pipeline. Since everything is imperative by default, people need to add update functions in data subscriptions. With all the littered mutations. Soon you will lose track of what the fuck is going on.
I hope angular get the agonizing death it deserves and fuck everyone who codes JS OOP clusterfuck UI.8 -
I had to argue with my lead today, to prove that docker is actually a container technology and not a CI CD pipeline -_-
That too in the team meeting.
P.S. Not much of his fault though, as ours is a mobile app development team :/5 -
Yesterday i have worked from 9 to 5 for $0/hour (they dont call it $0/hour they call it a technical interview)
To which i had to build all alone all by myself a rest api backend in java, bash script to move the jar to other folder a gitlab cicd pipeline and deployment to aws.
Basically for this position i have to work both backend and devops
I passed
Yesterday at 8pm (yes) the recruiter told me i passed and she asked me to schedule the next interview tomorrow morning in 8am.
I didn't open linkedin and then she sent me a word template at 10pm.
Are you fucking kidding me? Was it not enough that i was your slave from the fucking morning till the whole working day for $0/hour and now you want me to reply to your messages outside of work? You want me to be ur $0/hour slave 24/7 and not 9/5? Fuck off. Genuinely get fucked.
I hate the corporate world. This is satan's job. This is the work of the devil. I feel my soul dying. This matrix is killing my soul. I must escape. i need energy to escape but this matrix is sucking all the energy out of me2 -
Over the last few weeks, I've learned two things about the head of my division:
1. They "don't care about code quality"
2. They want to try a "low-code" approach for the web frontend of a completely custom piece of what is currently desktop software
This is despite the fact that we have three full time developers with a wide range of both front and back end skills on the payroll, a deep library of existing components for various frontend frameworks, a custom CSS library, and a decent deployment pipeline for frontend code.
But sure, let's try low-code. Let's see how far that gets us. -
So for the past day I've been obsessed with adding compression into my build pipeline for web dev, I've implemented html minify, babel JS minify, gzip and made the server specify content length to prevent chunking and shave off a few unused bytes. Is there anything else anyone can think of to get even more gains? Sofar my project went from 1.33mb to 180kb transferred. It's a huge win, just wondering if I can push it further somehow?1
-
I can't set up testing, a decent structure for my projects, or a basic CI/CD pipeline, because my clients want everything immediately, and my boss just sells my workforce without aknowledging anything of my relationship with the clients. I'm tired af, unsatisfied by what I produce and desperate for side projects to use my skills1
-
"I'm gonna wake up early, and hammer out this feature, being super productive tomorrow"
Tomorrow comes:
- unexpected build errors
- unexpected runtime errors
- intermittent CI pipeline errors
- spends two hours trying to resolve errors
- literally hasn't touched the thing that's important
What else is going to go sideways? Watch Bitbucket or CircleCI fuck up and refuse to deploy this live for some stupid reason.8 -
we want you to be
- full stack developer (you do everything front end, back end)
- dev ops/SRE (you can sort out the deployment CI/CD pipeline, cloud platform services AWS/GC/Azure whatever)
- architect (you can design the software as well)
all in 1, you gotta be multiple roles/departments
good luck getting this experience on the job (hell in a startup is not for everybody and certainly not for me)
also why the fuck companies who aren’t startups ask for this idk
not sure if i missed any roles/competencies so far , don’t forget you need like >=3 years of experience possibly in every field for entry roles and more for anything higher than that9 -
We've got two conflicting linters for ruby.
The rubocops serve the IDE and say to outdent access modifiers, giving the indented line a little yellow under-squiggle. Monkey see, monkey do, I indent it.
The company's one in the build pipeline wants to indent access modifiers and literally fails the bloody build, screaming about a rogue access modifier two spaces from where it should be.
Waste of 5 minutes.1 -
I'm tired of being forced to use an overengineered library for every little thing I'm given to implement. I don't want to add nodejs bloatware to my service just because I neet to pass single json object through a message pipeline. I'm all for reusability, but this is crossing every line now.3
-
It's really fun to not have access to the CI backend and keep only troubleshooting build crashes through a CI that takes 15min to run until it crashes :)))
1 diagnostic command == 1 pipeline run
1 typo == another run :)
it's been 2 sprints now. Still no luck2 -
I just got Jenkins all setup locally, setup the first pipeline, get docker working with it, setup the build step, setup the test step and more.
In under an hour.
Not too bad for the first attempt.
The hardest part was figuring out the GitHub credentials.
———
Actually, the hardest part was keeping an eye on the dude in the booth next to me who has delusions of grandeur and likely other mental illnesses.
Had to keep an eye on him while he was pointing around the room (usually at me) and saying shit like...
“Ugly, ugly, all of you are fucking ugly”
“Fuck you, fuck you too!”
I’m sitting over here thinking...
“Bud, you got 3 teeth, you smell like shit and your rambling to yourself... fuck you you ugly piece of dog shit! Let me do my work I peace.”4 -
Basic graphics pipeline through chatGPT, took a few layers but it should be able to draw out most well described objects.... I hope5
-
Name a more 2018/2019 New Year’s Eve.. We used a new emoji in a commit message and it borked the CI pipeline.3
-
It's too early to be asking these questions today:
Are your DB schema changes checked into source control?
What branch are they checked into?
Why are the schema changes checked into one branch, but deployed to a completely different database?
Is my CI pipeline deploying incorrectly? Oh, you manually deployed changes.
Are your DB changes in source control an accurate reflection of what you actually put in the staging database?
Why not?
Can I just cherry-pick update my schema with your changes from the staging database?
Why is there a typo in your field name?
Oh. Why is there a typo in the customer data set? Don't they know how to spell that word?
Why is the fucking staging database schema missing three critical tables?
Is the coffee ready? I need coffee.
Why is the coffee not ready yet?
What's going on in DevRant this morning?
What project am I working on now anyway?
Did my schema update finish yet?
Yup, it finished. Crap. Where the hell do I keep those backup files?
What's the command line to restore the file again?
Why doesn't our CLI tool support automated database restores?
I can fix that. What branch name should I check the CLI tool into?
What project was I working on this morning again?1 -
I guess you could say that my speciality is cloud at scale. I’d say it chose me more than I chose it.
Looking back on it though, I think what I like about my speciality is the unique challenges it brings.
Every speciality has its own set of challenges, like tight resource limits in embedded, or client-server synchronisation in native/mobile.
The challenge of cloud at scale is throughput. Designing systems that can support 100K users making a bazillion requests a second, or a data pipeline firing events that you need to process in near real time without dropping a single one.
The real challenge of course is doing all this within a sensible budget. We have virtually infinite compute but we dont have infinite dollars to spend on it.
Its a fun problem to solve.3 -
It usually starts with a project, a CI pipeline, unit tests, deployment guidelines, and wait - oh crap - what am I building again?
-
Can Gitlab users make their pipeline shorter. I've been waiting for this shared runner for 30mins...1
-
Here's the story of me trying to set up Jenkins with Java, Maven and pipelines. I finally got it to work :D Not sure if it's the correct way and all, but fuck it, it works!1
-
The company who "hired" me for 6.25$ an hour I'll have to work on a legacy old ass caveman java 8 code for a bank system. And I'll have to do the devops part. Main role as devops engineer
I lied having 3+ years of cloud devops experience
I started diving deep into cloud devops since january this year
Turns out I'll be the only devops engineer working on this bank software building devops pipeline and devops shit!
So now they expect me that im so experienced I don't need help for shit and i could do it on my own!
Am i fucked?4 -
How oracle can be so big piece of shit? Dude i only need to know wtf a OSB pipeline are doing. Dude, the fucking jdeveloper cant work on my machine, no fucking way, no one version, the shit just crash with fucking null pointers. But is my machine that are cursed, right? OK, lets download their virtual machine environment and finish this. Except THE JDEVELOPER DONT WORK IN THEIR OWN FUCKING VIRTUAL MACHINE, dude, i'm going crazy. Pls, DIEEEEEEE fucking Oracle
-
What do you think of Elixir + Phoenix to build API’s? Is it a better choice than a more established language like Python or something more new like Scala or Clojure?
At my company we're going through a watershed moment where we're starting to discuss and think about re-building our digital foundations and nothing is off limits. I'm leading the discussion about our architecture where everyone can have their say into what the future looks like for our applications. We're currently on a Drupal (CMS) + PHP7/Symfony (Backend Content Repository) + Symfony Twig templates (Frontend)
Even though I have been developing in PHP most of my career, I personally love Elixir and spend a lot of my time away from work learning it but many of my reasons feels subjective like pattern matching, it's actor concurrency model, immutable data and not having to deal with classes/objects, and I'm not entirely sure how that translates to business value, advocating successfully for a tech stack change requires solid reasoning and good answers to challenges like how do we find Elixir developers when existing devs leave, how easy is it to build a CI/CD pipeline for Elixir/Phoenix, etc.4 -
Was working on setting up a ci/CD pipeline. The ci part with automated testing and deployment to a on-premise docker registry worked already, so I thought "hey I could try to actually run one of those fresh containers" so I tried it with the usual docker run command.. "Manifest not found" suddenly appeared, it confused me a bit since I used the same url I used for publishing... So I googled around only to find NOTHING that is even remotely connected to my issue. "Eh let's let the guy that runs that registry fix it" was what I thought and called it a day. The next day I was eager to try it again and checked the urls case by case only to notice that I wrote secret-project-backend-client instead of secret-project-api.. I tried it with the new name and it worked!
Never felt so retarded in my life.... -
I love that I need to fucking babysit the fucking CI pipeline because, half of the times, it just RANDOMLY FAILS and I need to hit that fucking retry button like a 19th century kid respooling a loom.8
-
<Question>
I'm curious: For those of you who have tests running in CI pipelines at work:
How long does it take to run the tests, in relation to the system's size?
At my company, it's ~ 30 mins, testing 1M lines of code (750k SLOC) written in Java (85%) and JS (15%).7 -
Good times: Migrating a Jenkins build pipeline patched together out of groovy, python, bash, awk, perl scripts and God knows what else since I have only scratched the surface so far, from Maven to Gradle while not breaking day-to-day builds, integrations and deployments of features, hotfixes and releases. I'm actually enjoying the challenge but it's taking forever due to several issues:
- Jenkins breaks/hangs randomly because it's Jenkins
- Gradle can't handle sets of version ranges but Maven can
- Maven can't handle Gradle style version ranges
- Gradle doesn't have a concept of parent poms, you need to write a plugin and apply stuff programmatically. But plug-ins being part of the buildscript{} don't fall under depency management rules :clap:
- Meme incompatibility issues of BSD vs GNU versions of CLI tools like sed, grep etc1 -
A jr dev was having an issue registering code with our data pipeline (prefect self hosted).
Turns out he's running vscode to launch a anaconda shell (didn't even know that was a thing) to launch jupyter notebook and running commands in the notebook (didn't know that was possible) all from Windows.
No it doesn't work. His environment configuration isn't right. I told him to just run Linux and get rid of all that nonsense.
Nothing is on git yet and were three weeks in! His code is full of hard coded absolute paths of files on his hard drive... He even had an example app to go buy, with a project layout to copy.
There's no helping some people9 -
At work I help manage a fleet of Apple hardware that acts as our iOS build pipeline, and today I tested out MacOS Sonoma on one of the build nodes. The update went fine, but the test build failed because it didn't have sudo access for a specific command. I looked into it a little more, and it appears that the update set the sudoers file back to default! Like, why would you do that? Why would you mess with a configuration like that just for an OS update? It doesn't make any sense to me, and now I'll have to go and fix each sudoers file manually after I update the rest of the nodes. So, thanks Apple.3
-
Started working for a new company as a data lead, created a couple of basic lambda functions for a pipeline... 5 approvals and three weeks later the functions haven't even been pushed to UAT (which I'm not allowed access to).. Have I been lucky before or is this dysfunctional..?
-
Just finished a little proof of concept of a reprojected multisample antialiasing technique and daaaaamn it's looking good. First time ever a rendering technique I invented isn't complete shit so that's an improvement
It still has some (pretty big) issues with both spatial and temporal ghosting but I have some ideas in the pipeline
I wish I could show you guys comparison images but as it turns out most anti aliasing techniques look pretty good on still frames but only the good ones stand up under motion and I don't know of a good way right now to capture pixel perfect clips like that5 -
Spent the last few days trying to solve a weird issue with our CI/CD pipeline for a project. Yesterday i finally gave up and told my coworker that i need a fresh set of eyes to look at this.
I leave for lunch, an hour later I'm back and brought fire and fury to the mix.. Then, 2 hours later i raised my hands and my mouth uttered the glorious words of victory: Fuck yeah, it works.
Mood was still shit though... 1 bug down, 99 life problems remain 😢 -
It's the top of Round #6, the score is:
CI-Pipeline with 5 points, and the developer with 0. Remember, these rounds are 20 minutes long. Let's see if the developer can get a point before the end of the first two hours.1 -
It's almost like everytime I open the BitBucket pipeline logs my pc starts running the pipeline instead of BitBucket
Even visiting the logs spins up my fan so quickly that it's noticable for the people around me.
Instead of launching a pipeline it launches my pc3 -
Best tools/guides to setup a pipeline for C# applications that need to be built tested and ran on windows and linux?1
-
Quite frankly im embarrassed to say how much time i wasted having no idea why my pipeline is failing because it cant login to my aws account3
-
Has there been any luck in implement the “blocking” feature devRant?
Is the ability to block specific users from commenting on your posts still in the development pipeline?6 -
I hate people who don't do basic debug and say that it is a Infra/DevOps pipeline issues. If you are not able to compile in your local how it can be compiled by CI engine.2
-
The pain of creating a data pipeline in AWS to dump all your DynamoDB tables into S3 is something I don't want anyone else to go through..Why don't you have a copy button..😭😭
-
First let me start this rant by saying: Don't use SharePoint lists as your primary data store if you can avoid it. You're gonna have a bad time.
My coworkers and I work on a system where we need to pull tons of data down from a SharePoint site and run various algorithms and operations on it. Generate reports, that sort of thing. This is all done in the browser using a Typescript React SPFX webpart. Basically using SharePoint as a DB/DAL.
Because of the sheer amount of data we end up pulling down (our system in production is the single source of truth for one of the largest companies in Canada, and they're currently building a pipeline as we speak), in order to maintain a reasonable speed while using it, we have some pretty intense caching logic implemented, logic that ensures we get new items when new items are detected, and merges changes to already exisiting objects. It's pretty brilliant, and that's before we even consider the custom paging that my coworker implemented in order to get around the IndexedDB max size of 100MB.
Well that's all well and good, and works great in production, but it is a horror to work with. Because EVERYTHING we touch on the server is cached locally, it can be IMPOSSIBLE to detect data anomalies, be they local or server side -.- You don't know how many hours I have completely WASTED fixing a "bug" that didn't really exist... Just incorrect data in the cache12 -
I have never used Docker and/or Kubernetes. Motivate me to use it by telling me how cool it is.
I want to build a complete ci-pipeline but am too lazy to start... there is so much to do, and I don't know where to start.6 -
Need some advise. I suggested a feature over an email to a CEO of a B$ company, he replied “we also have same idea in pipeline”, they developed it 3 years later. A lot of screens are same as I sent.
Can I ask for credit legally.14 -
What the hell kind of tool is Gitlab? I just want to automatically backmerge hotfixes from master to development. Even fucking Bitbucket had a checkbox to enable this. But not Gitlab, no, you better create a pipeline job in your already unreadable, overcrowded pipeline yml, but oh, the checked out repo in the pipeline is a detached head and you cant push with the user that checks out there. So what, just use a project acess token which revokes after a year breaking your task and then switch origin amd branch manually. But your token-user can't push to protected branches, so create a merge request instead, which requires approvals, making the automated step no longer automated.
But dont worry, you can just use the gitlab api to overwrite the approval rules for this MR so it requires 0 approvals. But to do so you must allow everyone to be able to overwrite approval rules therefor compromising security.
And so you made a feature that should effectively be a checkbox a 40+ line CI job which compromises your repo security.
which nuthead of an architect is responsible for the way gitlab (and its CI) is designed?6 -
My website is now deployed on a Digitalocean droplet using Terraform to provision the infrastructure and Ansible to configure the server. It creates users, sets up SSH config and deploys the required containers I want all using an Azure pipeline and an Azure storage account to store the TF state.
Now I need a frontend... ._.2 -
How do you do your CI/CD pipeline? Sorry if this is a dumb question. Just wondering how the tests and deployment usually runs. Is it on a per team basis? Is it the whole release getting deployed to Test many times per day? What happens if too many automated tests fail or there is not enough coverage, does it abort the deployment? If so, how can every team get delayed by every issue - is that actually a good policy?
My pipeline is very slow and requires a team of 12 people working in shifts to complete it. I’m not an expert but I know it does a lot of steps and never completes without manual intervention. I would like to help but I’m not sure how bad it is.3 -
!rant
For the past weeks I've been reading about continuous integration and today I finally decide to dive into Gitlab-CI. After a couple hours I finally managed to have a working pipeline for one of my project using a self-hosted runner and holly shit that was satisfying. Now I just don't see myself not using this in the future -
Hey look at these awesome features we offer in the new version!
*carefully upgrades servers*
*app runs smoothly for few hours*
(Docker container exited with code 137) x 100
OutOfHeapMemory errror
Another dev: "I can't see the data flowing through the pipeline"
Boss: "Hmm, why did we upgrade again?"
*checks jira issue for the software*
Bug Report #125 fixed in the next version.
Aha!
Fuck this shit!1 -
!rant
Rant from my previous work as a consultant Data Engineer (wish I had known this site back then).
During my stay at the place, we have a big client whose contact with us was an incompetent stressful fellow.
I single-handedly build a humongous automated data pipeline using Airflow. I am very proud of my baby as my first massive project and check it obsessively for every possible flaw, especially when writing down documentation for the poor soul that would take my place.
Luckily for me, everything is working as intended, until of course on my last day of work, shit hits the fan, and everything breaks down.
After a moment of initial panic: it was Thursday morning, we had a Machine Learning model to run over the weekend, predictions to make and reports to write and a very lovely next week deadline, I calm down.
"I won't be dealing with this shit anymore, starting from 18:00 PM and anyway Fear Is The Mind Killer."
Quite sure that it couldn't have been my code, I start looking at various logs when the culprit was clear. The B(ig) S(tupid) C(lient) changed the whole schema of the data he was feeding to us.
I call him: he has no idea of what was done to the data. Hell, at first he doesn't seem to remember what the deal with schema, data, and SQL is (the guy was supposed to be a big shot in the IT department). It turns out he hired one of our competitors to do his side of the collection pipeline. He tries to get mad at me, but everything he throws bounces back to him. I am calm yet ruthless pointing out how every major hiccup had been his fault and that I could quickly reach to his board of directors explaining why their Machine Learning model was late.
Result: he apologizes, extends our deadline, and I get a round of applause from other juniors who would have to deal with me had I failed.
Never am I happier to not work as an underpaid cannon fodder apprentice in a shitty consultant firm.
Luckily for me, everything is working as intended, until of course on my last day of work, shit hits the fan, and everything breaks down.
After a moment of initial panic: it was Thursday morning, we had a Machine Learning model to run over the weekend, predictions to make and reports to write and a very lovely next week deadline, I calm down.
"I won't be dealing with this shit anymore, starting from 18:00 PM and anyway Fear Is The Mind Killer."
Quite sure that it couldn't have been my code, I start looking at various logs when the culprit was clear. The B(ig) S(tupid) C(lient) changed the whole schema of the data he was feeding to us.
I call him: he has no idea of what was done to the data. Hell, at first he doesn't seem to remember what the deal with schema, data, and SQL is (the guy was supposed to be a big shot in the IT department). It turns out he hired one of our competitors to do his side of the collection pipeline. He tries to get mad at me, but everything he throws bounces back to him. I am calm yet ruthless pointing out how every major hiccup had been his fault and that I could quickly reach to his board of directors explaining why their Machine Learning model was late.
Result: he apologizes, extends our deadline, and I get a round of applause from other juniors who would have to deal with me had I failed.
Never am I happier to not work as an underpaid cannon fodder apprentice in a shitty consultant firm. -
How is it possible that nextjs developers could build a much more sophisticated web platforms than i do, much faster much more rapidly and easily and ship it to production for almost no costs, not even using any devops pipeline, unit or untegration testing, while i do all of that + devops and it costs me to pay for servers on aws?????
Literally what a nextjs developer can build in 1 month it would take me at least 3-4 months11 -
As I will have a new job soon and it seems likely that it will be in network api design I wonder if you have good book recommendations on the topic.
I already have "clean code", "clean architecture" and "design patterns" in my pipeline, so I need something more specific on designing network (restful) service apis.
(This is a follow up to https://devrant.com/rants/1828903/...)3 -
Don't you just hate it when there seems to be nothing but in some ways lacking solutions to a definite task in your capability arsenal? Or rather, I don't really know how I should feel about it... I've been developing this solution to receive a 3DES encrypted Azure Service Bus message, decrypting it and chewing the output XML down so as to be digestible to the PHP application whose API the message gets delegated to... but there just seems to be no perfect solution: subscribing to the event topic straight from the target app just... doesn't seem to work properly, a Python implementation.... well, let's just leave it at that... a Node.js implementation would require TS and completely rewriting a proprietary library with 100+ complex types - also, there's some hiccups with both the subscription and the decryption...
I started with an F# implementation (after deeming the PHP one flawed), and it seems it's still the best. But goddamn it I had problems with it on the dotnet core side of thing (decryption output incorrect), so I had to switch to dotnet framework... Now finally everything crucial is peachy, but I can't seem to be able to implement a working serialized domain model pipeline to validate the decrypted message and convert it to something easier to digest for the target application (so that I could use the existing API endpoint instead of writing a new one / heavily modifying the existing implementation and fear breaking something in the process...). I probably could do it in C#, I don't know, but for the love of Linus I'm not going to do it if I can avoid it, when implementing the same functionality I have now without the Dto and Domain type modules would take 3x LoC than the current F# implementation incl. the currently unused modules!
And then there's the problem of deployment... I have no idea what's the best way to deploy a dotnet framework module to an app completely based on MAMP running on a mostly 10yo AWS cloud solution. If I implemented a PHP or Node.js solution, it'd be a piece of cake, but... Phew, I don't know. This is both frustrating, overwhelming and exciting at the same time.7 -
Funny how things comes around...
So... project start-up... everybody learning and designing the future new system. Then we get to a point that we saw that we'll probably need someone outside or dev team to setup all the environments CI/CD pipelines... Our PM said "what about we get a Devops guys to take care of that?" Most of our team members agreed but our Techlead said "Devops is not a job, it's a culture.". Ok, nice... I understand that point, but for a system of the size of the one that we're building...It would probably be a good idea to have someone to take care of that for us. BUT, he (the techlead) said that he will be taking care of all that himself (along with coding part of the backend).
RESULT: We're stuck in the point that we're unable to test our system in the correct environment, we've no pipeline for automated deploy of our sprints...
Guys, I think the Devops is no more then somebody that is going to take care of some tasks in the project, like the backend, the frontend, the tests, the management...2 -
Hi there!
I've been worrying about the following problem for months now and I don't find any solution. Maybe anybody of you can lead me the way.
We are developing a software suite which consists of a number of desktop applications:
* 12 applications written in C++; all over 20 years old; further development by 5 or 6 guys (one man armys) - mainly bugfixing, changes of law implementations, small features
* 2 applications we are currently writing in C#; completely new developments of existing C++ applications; scrum teams with at least 5 guys; this is, where we put our focus in
These applications (C++ and C#) are sharing some core assemblies and are interacting with each other. So they are not independent.
We organize them in a mono repository in one huge solution, which consists currently of about 500 projects.
Advantages:
* With all projects in one solution and through project references, Visual Studio takes care of the right build order
* Code navigation is superb - every single line of code is accessible - this makes refactoring easy
* Every developer can map the branch and build the whole suite locally
* Debugging on the local machine is easy
* DevOps pipeline is straight forward - it just have to build a single solution
Disadvantages:
* The huge solution is extremely slow.
* If you want to build the solution or you want to debug (which does essentially the same as a pre step) Visual Studio is building a lot of projects, although they haven't been changed. Their detection is buggy. So sometimes you wait 2 minutes until it starts the app. That slows us down a lot.
* Full builds need about an hour, because its building the same projects (even if they haven't been changed) over and over again (with ready made nuget packages this could be improved a lot I think)
* If a core team member changes some core apis, he is changing the calling code too, although he doesn't know the calling code, because another team has written it. I don't think, that's best practice and it doesn't scale.
* Often, a C# developer has to mess around with C++ building problems, because the C++ projects are in the same solution
* It gets more and more confusing and frustrating, because there is no clear organizational seperation between apps and nobody can't just focus on his app alone.
Idea:
I was thinking about putting the whole framework and core projects in a new solution (around 100 projects). Then we could take all old C++ projects and put them also in a new solution (around 200 projects). This would leave the newer projects (new applications - C#) in the existing solution.
This should speed up things, and would be a first step to better seperation, BUT:
How should the integration process look like?
Scenario: Core team is changing an API in our framework
Current process: Because all projects are in the same solution, they change the calling code too. So it's immediately integrated and the app developers just have to do "get latest".
New process (?): Core team is providing the changes through a nuget package (new version). So does every developer now has to keep track of if there is a new package version and if yes, do the integration? And how can we coordinate the different teams, so they are upgrading all at the same time? Because we ship our applications as a suite, all apps has to use the same versions. Or should we automate the integration process? Is there a best practice?
I have to add, that our core team is making changes very frequently, so the integration process will have to happen often.
Any ideas/feedback/inspiration?
Thank you so much in advance!4 -
The integration of technologies project I have this year. Not yet finished but I already learned a lot of very cool stuff.
First, I learned a new programming language + framework (Ruby on Rails)
Second, for the first time, I implemented a continuous deployment pipeline with Capistrano and Travis ci.
Third, first time I programmed a Restful API.
And more cool stuff coming up ! :D
I freaking love learning ! -
Why do tech writers always produce deliverable documents in Word files instead of Markdown, LaTeX, or anything that is easy to CM? It would be easier to track revision history and dump out of the CI/CD pipeline to the customer.4
-
Oh no. Im shitting for too long on my throne and now my legs are feeling numb! I can't stand or walk. Fuck. How do i wipe this shit or clean hands. Shit. I need a standing toilet or a tube in my asshole so i can shit better. Like a devops pipeline8
-
Do it's been about 19 days since my last update on my shitty internet and the upgrade that's meant to be in the pipeline for the current month.
Despite seeing people laying cable around my neighbourhood there's been no update on their website or in my letterbox.
It's halfway through the month and news outlets are reporting that the NBN has said that they:
Have 99% coverage
100,000 suburbs that aren't fully catered to
But, can't cater to a property that is in a relatively dense area less than 20km from a major city. -
Ffs debian, y u no imagemagick >= 7, it's been around for a lot of years now. Time to unnecessarily complicate my pipeline10
-
Why is it that when you try to run a project and have ample dev but none of them wants to raise a PR because they are too scared to get feedback.
Oh you have done that can I see it on our pipeline or Git.....
Blank Face No.
Why?!
Because it won't run on pipeline.
Really3 -
Why do I feel like if you don't serve everything up on a pladder for the product owner / product management types, that they'll never find something as simple as a build executable the pipeline produces? Just more busywork for the competant ones to do...
haha flashback to the startup days when _i_ was the one acting as scrum master for the clowns who couldn't even do something as simple as set the release notes in apple and google stores. they truly need to be spoon fed everything. fucking incredible. -
We recently pulled in a developer onto the devops team, and also our manager was fired a month ago. Everything devops is Python/Go but all our developers are in Clojure or Node. He's recently been writing a lot of stuff on Clojure and I didn't get an opportunity to bring it up at the last retro.
Recently he changed all our old Go projects to use a CI-pipeline he built, even though I wrote a build tool, with tests, that does everything his pipeline does, in Python .. over two years ago. When I asked him why he doesn't just use our existing builder, he responds saying he doesn't know why he should use some tool/script just because it exists ... um, everything we've built uses it?! 😡
Our last manager was a dick and I understand why they let him go, but he also would put this guy in his place. He's literally made is own little "devops roadmap" for himself and shared it out with the team. It's all about his personal goals and what he wants.1 -
I’m frantically trying to escape the ‘CS grad to web / react / next developer’ pipeline. Only thing that sucks is I only know the web shit6
-
Creating the build script for the CI pipeline:
- 20% trying to avoid someone getting access to passwords, tokens, etc.
- 10% writing commands for the build and tests
- 70% writing work arounds for bugs and errors caused by the CI system or SDKs in headless environments...4 -
Had a particularly strange one about hose pipes flailing wildly but in a very specific pattern.
I’d been working on a bit of a monolithic multithreaded data pipeline and I couldn’t work out what I wanted to do with the thread pool. When I woke up I had a picture of those hose pipes, that basically how it now works😂 -
Went through 60 python packages to see which fails installing on the serve. Took hrs as I have no terminal access but just via jenkins pipeline. So "edit/gitpush requirements.txt and wait" many times. Eventually looped them 1 by 1 in shell. By end of day got the list that installs.
Finally sent the whole list....with confidence
-Takes full 10 mins & Fails......
(panic mode starts)
+Changed the sequence = fails, somewhere else
+1 by 1 again = installs.....
+few random without the culprit =works
+again, whole list = fails, somewhere else
Need to sleep, brain's thinking of eagles1 -
Is it slow ?
taliking about unit tests. I have 2746 unit tests in a project (Covering around 15% of code).
It takes around 2 minutes in local (With good PC) and aroubnd 20 minutes in a pipeline to execute.
Would you people say it's slow ?9 -
Only when the latest feature is implemented, the last bugfix and the last workaround are found, the last unit test is written, the latest CI/CD pipeline done, the customer guy does manual testing and acceptance tests on the staging server and let's them pass and a few days later it's pushed to production...
You will be reminded (again) that shitty customers do exist! A customer is the least capable person to tell you what the customer actually wants and is also the least trustworthy person to test the features he requested...
Holy fuck come on! Just test that shit on the staging Server! One Look could have already shown you that that's Not what you expected!
I checked the logs after that and yup you guessed correctly... The said endpoints weren't even used on staging, only on production...1 -
Unit tests pass locally but fail on the pipeline. After 3rd re-queue, pipeline tests pass. I am so over this bloody week.11
-
IDE can find symbol, local build passes, yet gitlab build/test pipeline can't
I'm dumb and forgot to commit and push new file1 -
So features and work are slowing down in the pipeline at a new job I took. Is it okay to just spend the time in between "work" to do tutorials on other technologies?7
-
I think I just realized what my biggest gripe about our career paths that I hate the most.
This is something that has worsened over time, especially the last 2 to 3 years.
As developers, we have far too many options. Some of the most powerful apps are written with languages that have hard, and I mean HARD, guardrails in place. If the app is written in a language that does not meet this criteria usually a framework has been used to install those guardrails.
We just get our minds so wrapped around the possibilities and the opportunities in the software, that we just can't focus on the end result. We're like puppies that are excited about something and we just piss all over everything.
In my career I have met far too many developers that don't have the capacity and mental fortitude to take control of their actions. Because of this I think the only way for us to stop this corruption, that I feel we are nurturing, the solutions/services that we use need to push back on us and install those guardrails for us.
All this came from a change that Microsoft put in place that seems well intended, but introduces yet another choice and a multitude of opinions in how you release code.
It used to be a simple check box. If it was checked it was pre-release, if it was unchecked it was a production release. That's it. On or off. The simplest choice you ever needed to make on a release.
Now though, there are two check boxes. One for a pre-release and one for a latest release. You can also not check either for some "ephemeral" release? So now something as easy as on or off has been made into a difficult decision on how this works within my pipeline. Now every time I make a release I have to ask myself, "which one do I check?"
I shouldn't need to spend more than a second to identify a path forward on simple shit like this, but here we are with a third choice.
Can we just stop overcomplicating shit?6 -
Nextjs
I just realized
unit tests and integration tests dont exist in nextjs
So now i wondered
What about integrating AWS cloud functions with nextjs?
What about docker with nextjs?
Kubernetes with nextjs??
How TF do u implement a CI/CD pipeline with nextjs if there is nothing to test?!?!??
Nextjs seems like itself is self sufficient. WTF? Everything has been packed and cluttered into 1 giant pile of horsedump and called Nextjs Framework where you dont need k8s to run it or anything. Might as well then rename this fucking "framework" to GOD of all frameworks since it appears it can fucking do anything and everything without requiring HEAVY DevOps bullshit.
ALL of it can be cluttered in 1 nextjs project and you have a fully functional production working website that can basically do ANYTHING and EVERYTHING.
How???
Am i fucking going insane? Am i missing something here??19 -
*deletes env var* "I'll just hard code this to localhost for now, my ci/cd pipeline will setup everything later"
*Pushes to master, forgets to undo*
Aw fuk,
I should of just changed the .env file -
Jenkins, why you be so fucked up? The same pipeline file runs in one folder, but fails at the root of the Jenkins server. It's the same code! Literally, the exact same code, I just moved the job into a different folder.2
-
To the ones using GitLab CI/CD:
Is there a way to trigger a pipeline run on a specific commit? Like in a situation where you're deploying every commit to master into prod, and you need to roll back, but without reverting the commit history.14 -
I cant believe the project I'm working on does not use kubernetes or terraform. Not even docker. How is this multi trillion dollar project even in business?
I feel so sad for not having the opportunity to work with one of the most fundamental and most important technologies to know as a devops engineer... So sad
I cant advance or improve. Im just stuck in their ecosystem like Apple
This corporation is probably ran by 90 year old grandpa men from world war 1. However considering they are so large and still in business this gives me hope that anyone can make it even if you're stupid
Think about it
They are proof that you can run a giant business with hundreds of employees, not use k8s and the most modern devops technologies, and still operate just fine.
The devops code i have to maintain is older than the amount of years i exist. Its very messy and most of this shit is not even devops related. Its more of some kind of linux administrative tasks mixed with 3 drops of actual devops (bash scripts, ansible scripts, ci/cd pipeline)
And yet im paid more than i have ever been paid in any job so far
What should i do. Stay due to "high" money or..ask for a project with k8s. I put "high" in quotes because it is extreme luxury in my shithole country, im now among top 1% earners of the country, and yet i make less than 30k a year. With less than 30k a year i cant buy a good car but i can live very comfortably in my country. I cant complain about this salary since i think its finally enough to invest to get a chance to earn more and still have enough left to live comfortably.
Before i was just working to survive. Now im working to live. Its an upgrade.
Due to not working with difficult stuff like k8s i cant demand for more money. It wouldnt feel justified. I'm stuck here
What would u do9 -
How many of you at larger companies (at LEAST a 3 stage build pipeline) are using Docker? Are you finding its solutions worth it?1
-
Does the company you work at have a CI/CD pipeline (Continuous Integration/Continuous Delivery, e.g. with Octopus Deploy)?
Sometimes it surprises me how many companies don't have this..10 -
GitLab, you really should fix your CI.
I mean, I know .gitlab-ci.yml has to be written carefully, having in mind that GL shell is a castrated bourne shell, but come on... Failing a pipeline because I used a semicolon in an `echo` parameter string?
echo ""items: 0" ## this will fail
echo "items 0" ## this will pass
This is a bit too much.
Removed the semicolon and the pipeline worked just fine.12 -
What are you using for local environment? I'm stuck on xampp for 10 years. I realized that I'm having terrible pipeline from developing to production.6
-
Wish devRant can show the timeline of where the Swag stickers have reached in the delivery Pipeline. Eagerly waiting for my Swag Stickers to arrive.
-
Creating an open source project which is actually used by multiple people.
(I actually have something I'm currently building in the pipeline) -
Our pipeline had been running for 45 minutes, I went to investigate
Our mutation testing framework had identified 90 potential mutants and started running through our 342 tests for every single mutant, so 90 * 342 tests
And I had been complaining about the amount of pipeline agents being too low and pipelines blocking up...5 -
I keep a collection of strange logs I see scrolling by the CI Pipeline. What's your strangest log entry?
-
Pipeline as json is pain and you can't even validate until it breaks on the ci
#makesProgrammingHell4 -
I cant find 1 single normal Fucking tutorial explaining how to code FULL DEVOPS PIPELINE for deployment to AWS.
A pipeline that includes
- gitlab (ci cd)
- jenkins
- gradle
- sonarqube
- docker
- trivy
- update k8s manifest
- terraform
- argocd
- deploy to EKS
- send slack notification
How Fucking hard is it for someone to make a tutorial about this????? How am i supposed to learn how to code this pipeline????10 -
I'd like to make a simple-ish game with OpenGL, but I'm not sure if I should use the older (fixed function pipeline) versions or the newer (programmable pipeline thingy, with shaders and stuff) ones. Thoughts?3
-
I refactor, speed up, and generally slim down our api tests - so our 'api test' specialist QAs can happily dump thousands upon thousands of lines of new copy-pasted test cases in again with every commit, without fear of the test execution pipeline timing out again...
-
For a little background on the sort of stuff I'm dealing with, check out my last rant.
Anyways, I'm testing this pipeline at work and was just reminded of the fucktarded way a "software engineer", who had a bachelor's biology degree, decided to handle a json file.
The script is question is loading a json file containing an array of objects. The script is written in perl. There's a JSON module. Use that? Fuck no! Let's rather perform an in-place sed command on the file substituting the commas separating objects in the array with newlines, then proceed to read the file line-by-line and parse out the tokens manually. Mind you, in the process of adding the newlines he didn't keep the commas, so now all of these json files his bullshit handled are invalid json that cannot be parsed.
The dumb ass was lucky the data in the file is always output upstream as a single line and the tokens for each object are always in the same order, so that never led to problems. But now, months later after I fixed his stupidity I am being reminded of it again as I'm testing and debugging some old projects as part of regression testing new changes I'm making.
TL;DR Fuck dumbwit motherfuckers who can't even google search "parsing a json file" and doing literally anything that is less fucktarded than manually parsing a json file2 -
Spent the day figuring out how to maintain injected dependencies in scope when they're requested asynchronously later in the pipeline and then be able to clean it up later without having any lifecycle hooks to use.
Seriously considered switching DI frameworks before I just added an event when it's OK to dispose of the scope and I think it's finally working (without the memory leaks it had before).
Who else has to try something every possible way before you can be satisfied? -
Bad (or at least uninformed due to unclear requirements) design decisions lead to somewhat perplexing (and above all, frustrating) problems down the line...
Here I am, wrote a cli client for event bus subscription, and since EC2 is eqv to Docker, had to fix the main function to keep it alive - now the deploy stage on the ci/cd pipeline fails due to startup script timing out. But why tho?
I’m not really sure whether I should’ve designed this differently from the get-go or whether my build and deploy configs don’t match anymore due to recent changes and I should figure out what’s wrong with them... or both.
Bottom line is: I have no idea what I’m doing.9 -
Can anyone with some AWS IAM skills please shine a light on this one: I needed access to create a slack notification for a job in Code Pipeline. Simple enough, but we (devs) have next to no access to AWS so every time I try something I am stopped by the red "user X is not authorized to perform Y on resource Z.." warning message. I send an email to OPS and ask for permissions needed to do what I need (in this case: create a Slack notification for a pipeline), and I am granted that specific one. It gets me one step further, until I am stopped by a new red warning message. This has been going on for over a week, with a total of TEN new authorizations added to my user. That's TEN red warnings, TEN emails asking for access, and TEN replies saying "Ok, can you try now?". Today I finally got the god damn slack notifier set up, only to get one last red warning slapped in my face: I am not allowed to SEE the notifications configured for my pipeline. Please insert four letter word that rimes with DUCK here: [_________]!!.
I REFUSE to believe that this is how access should be granted in AWS. Can I tell my OPS person that there is somewhere they can find a list of required access rights to complete a specific operation ("Create slack notification for pipeline")? I know there are example policies for various things, but if there isn't one for what I need how should OPS go about granting me access without this totally ridiculous "try again now" approach?
Oh, and @assmaster: don't comment "nice" to this one. This is shitting me off.3 -
What's the best way to deal with constant dread? I deployed code after following every procedure, got every kind of thumbs up from QA and now it's my fault our 2012 admin site borked. Should I point out all the obvious flaws (again), or should I give up on our stagnant-ass developers and systems?
The fear of showing off anything new is crippling. I wrote up a Pyton API to hook into our current pipeline over lunch breaks but am worried if I even raise it as an option it'll just be cast aside and lost to time, regardless of business value. -
If i have 2 branches on git
- main
- infra
You cannot push directly to main. It is forbidden. You can only merge to main
Now. Once i push to infra branch. Assuming all the shit went good pipeline passed tests passed etc. Then i merge it to main.
Now
Locally while im on infra branch. I have to pull latest changes from main otherwise ill fuck up everything and cause conflicts.
After trial and error i realized i just have to do:
git fetch
This fetches all shits from main (defaukt branch) into infra branch. And now it works. No rebase. No pull. Wtf?
Is this the correct way to do it?
Also i need someone to explain this to me like im 5:
- git pull
- git pull --rebase
- git fetch
What is the difference between those 3 commands? I tried googling and chatgpting but i cant seem to understand any explanation. Explain it to me in simple terms with examples15 -
Back in college I was writing a parser for a personal project in python. I was adding nodes to a set, but somewhere down the pipeline, some of the items I added to this set mysteriously disappeared!
I never removed anything from this set, nor did I replace it, so it baffled me that I couldn't find some items in the set.
It turned out the problem was I updated fields in these nodes that were used for taking the node's __hash__, which would prevent them from being found in the set
:|1 -
What’s your take on engineering managers just focusing on delivering the product. Making sure the management happy.
TL;DR.
To begin, i want to clear it out that my EM is a really sweet person and tries to accommodate all of us and takes pressure off us whenever possible.
But he’s a really shitty manager. He’s the kinda guy whom you can give a weekend project and he’ll an excellent job. On the other hand, if the taks requires more than a weekend, he’ll probably fuck it up.
He thinks everything can be done in a jiffy and we don’t need to think about the edge cases.
His reasoning for not giving quality importance is that he wants us to focus on building the product and making it shippable. We can iron out issues once we get it done.
He’s decided not to let the tests run in the cicd pipeline because that might hinder making hot fixes.
I’ve tried talking to him but he believes that’s the only way management is willing to agree to.2 -
So a long time ago, I posted Ink lyrics from ColdPlay, and gave them a DevOp twist. I will post the full lyrics, as memorabilia and for a chill moment from rants:
Got access rules said "prod/maintainer"
Pulled in your change with my cli
And you wonder when you wake up
Did it build last night?
Oh, oh, oh, oh, oh
Feels like the pipeline, is broken inside
All I know
All I know
Is that I'm host
When company broke
AIl I know
Is that I love to code
So much that it hurts
Got a homebrew but the shell aint right, oooh, oooh
Just want a way of keeping docker alive
All I know, All I know
Is that the post
BreachForums indorsed
All I know
Is that the data is sold
So much that it hurts
I see that Forbes, and New York Times
They said my boss, began to fire
I see my console and Im mashing Ctrl-C
All I know
Is that loved to code
So much it -
I'm having problems understanding ngrx (or simply rxjs...), once again.
I have a feature state called "World".
This state has three Actions: Init, InitSuccess and InitFailure.
Also I have other isolated feature states like "Mountains", "Streets", "Rivers".
They have actions like this and corresponding effects to fetch data: Load, LoadSuccess, LoadFailure.
Now I would like to add an effect to WorldActions.Init, which will dispatch Mountains.Load, Streets.Load, Rivers.Load one after another. So ideally an action log would look like this:
1. World.Init
2. Mountains.Load
3. Streets.Load
4. Rivers.Load
5. Mountains.LoadSuccess
6. Streets.LoadSuccess
7. Rivers.LoadSuccess
8. World.InitSuccess
Or when an error occurs:
1. World.Init
2. Mountains.Load
3. Mountains.LoadFailure
4. World.InitFailure
How could an effect pipeline for that look?
How can I dispatch World.InitSuccess only after all LoadSuccess-Actions have happened?
Or am I still trying to implement ngrx in a wrong/bad way?
PS: The reason I am putting everything into separate feature states is because "Mountains" etc. are standalone features on their own. Only in the context of a "World" they belong together. For this reason I can't create a monstrous effect "World.Load", without producing redundant code.10 -
Situation - I am responsible for refactoring and performance improvements in a company with several teams. This means I gotta do static analysis on code, run compliance tools and make changes in code or in the deployment pipeline, make sure the cloud is configured properly etc.,
Here is the catch when it comes to working on a ticket- the Azure team does not give my team permissions to make the necessary changes in the cloud. The Azure team won't pick up the ticket and do it themselves either.
Instead, we take the ticket, read the docs, take a guess on what's right or wrong. Then proceed to inform the Azure team who then go on to make that change. It is very hit or miss and often the ticket comes back to us and we do the same process again. Sometimes I have to spin up resources on my personal Azure account to tinker with settings to see which knobs are there for making changes to a resource.
Either pick up a ticket and work on it yourself, or give us azure with sufficient rights for us to be able to make the change. This midway status is infuriating, super unproductive and painful for us. Is this common? I am so frustrated.2 -
I am new here so apologies if I make any mistakes.
I have been a opensource contributor since last 2 years and it has been a great experience. As I am looking for a new opensource organization, I got around an organisation X(name changed). It is my first time when I don't like an opensource organization. The organization is controlled bh a single person and he does just tells me to copy the whole website of another popular opensource organization and make the organization website. Also, he does not listen about anything. He just pings me about the work done everyday even after telling him that a review is a blocker for me to do new task. I don't say it is a bay thing but don't looking at the issue is the main thing. On another case, the build pipeline was failing. It can be solved only by changing certain settings on the build pipeline and I does not have its access. I told him about how to tackle it in the review comment. Even after this, he just pings me for around a week just telling me that it has something to do with my code and the pipeline is all right.
I can understand that in the early phase, an organization may have some problems and the setup may have some flaws but this type of dictator behaviour is not good in my opinion. I had worked in 3-4 opensource organization and all have very welcoming community. I had always learned from them but this is my first time bad experience with it3 -
Msal.js. I give it 3/10..
The docs are duplicated, and in various states of out of date. Half the library seems to be undocumented based on how many edge case bugs I've hit, it offers a popup login but you have to have a set specified white list of urls you can launch the popup from which makes a popup login pointless...
Ontop of that my colleagues shat the bed on it and fucked the whole implementation including the azure b2c setup... We do not even have a backend app listed in the azure b2c apps. The redirect also won't work if you don't instantiate an object in a hidden iframe of your own website that fetches a token... This does not make life easy when you use a SPA framework and you have already implemented a whole pipeline abstracting the creation of this object behind layers dependency injection.. Nice.
After sifting through endless shit I finally have a solution. What a week. -
How do folks feel about IoC/DI?
I used Spring and Angular for the first few years of my career, so it seemed like it was a mandatory pattern of a framework and my team would never deploy an app that couldn't use it (even if it was just a Lamda or something, we found smaller DI libraries). Now I work in Express and React, and I look back and feel that those patterns required me to write more code, created more complexity, and wasn't any easier to read or understand, and was way more bug prone, and debugging the injection pipeline itself was effectively not possible.
I guess I'm wondering: what do people feel that it buys them?15 -
So trunk-based is the new approach everyone is using, because it is so cool.
I used gitflow for the last projects with azure devops, set up the pipelines like tipically in 1 week if I had other things to do with the help of the portal clicking through things. PR-s triggered pipelines, everything worked cool.
But then trunk-based got momentum, so I worked with this client where 2 developers worked for !!!3 months!!! to setup trunk-based pipeline. It was not my money, so I did not say a thing. They were using infrastructure as code.
I am all in for automation, but seriously? Then again, another project where a DevOps team took 1 dev-month to setup the pipeline + meetings. And what do you get in the end? So that the same image goes on all environments? Like how many releases do you have for prod in a year. Lets say 24. 24 x 5 minutes of manual work for the release, that is 2 hours. So my question is why would you spend 2 hours of manual work while you can automate it merely in a month? Everyone loves to code, but using the ui on the DevOps portal saves you so much time. I don't get this. Maybe I am getting old :D4 -
I recently joined the team that is responsible for the maintenance and development of the ibis adapter framework (http://github.com/ibissource/iaf)
The IAF is an integration framework, with a set of pipes written in java one can compose a service written in xml by building a pipeline with the premade pipes. For data mapping and validation we use xsl and xsd files. The framework can communicate over different protocols such as HTTP(S), JMS, EMS, SMTP, FTP and more.
I will be responsible for the web interface where you can manage/debug/test your application.1 -
Any ideas how to skill up devops ? Currently in company im doing simple things with kubernetes, aws, terraform and circleci, and the whole idea click to create your inba cluster is interesting, smells like a few steps from cybersecurity!
Soo i decided to write an app, with two environments, which are staging and prod, configure some ci pipeline, kubernetes deployments and terraform, everything with usage of aws, and then when i will be okay with it, send cv's as devops and change career path.
Seems legit or waste of time ?2 -
New guy in the block!. Just started with a new position in a new company too!.
Designated as as Devops Engineer (after my 2 years of experience as one) in a well funded Saas Startup!. Lots to learn. I used to work in Openstack Terraform puppet etc whilst here it's fully AWS. I was expecting this right from the start but woah.
Lambda, dynamodb, cloudformation, ssm, codebuild, codepipeline
Serverless framework, Flask and node mixed apps , Vue (including vuex) js Front end, graphQl api, and rest for between microservices.
Lots of ground to cover and I've not consumed this much topics before. Especially graphQl and Vue js are being a pain for now .
Each Devops engineer is working on a tools to improve the productivity and shorten the release time. Lots of automations in the pipeline!.
I'm not sure this qualifies as a rant but here you go!.2 -
I really admire and respect devs here in our country that are helping to build useful apps to combat covid-19 pandemic but please, if the National Government already has this kind of applications or in their pipeline (covid contact tracing, qrcode checkpoint, assistance, etc..) please do not try to hijack by creating similar app and market it. Instead of helping it creates confusion on which info to follow (ie DOH tracking) and which to use. #random2
-
Even though we have sophisticated CD pipeline, decided to modify war on server (modifying 1 source file, compile it and patch it on server (Java based)). Just cause CD takes 4 hours and we "only have" 15 mins to do it. So manual patching with rush job. Now when I think about it, I cringe and thank the dev ducks that we didn't messed it up even worse!1
-
Working on a feature which heavies relies on a data pipeline. I noticed it is a couple of lambda functions calling each other ( Fuck you to the guy who made it). The best way to get sanity back is build a proper etl pipeline. Any suggestions for building a etl in python with reliability.
Options already considered
1. Celery tasks - Worked well but no overview of the single task progress across celery tasks
2. Airflow - Gives good overview but the docs make less sense than a 10 yr talking. Mostly because they introduced a new syntax and not everything has migrated fully yet. Also no support for reusing dags2 -
Today I got a long term contract at the company I have been working at for the past two years. We maintain and develop an open source java based framework, basically you write XML to configure components (pipes, receivers, senders) in Java to build a pipeline which usually functions as a backend service. We also do implementations of the framework for our customers.
Im in a position where I my main task is applying the framework which is writing XML or skyping people at the client office to chase them to fix their server settings, please create a database for us (each time different, sometimes we get a manager user sometimes the regular user can do everytbing), create NPA's, execute queries in ACC environment or ask them why 5/10 we get an error 407 pro,y authentication required ffs
My salary is increased aswell and they told me before that I am one of the five developers in the company (20~ devs) that they want to keep costing what it costs. Management also told me they are looking to bring out something like shares or certificates for those five dev's!
Sounds pretty good right? Actually im really happy about those things but I feel like management managed to keep me in the company whilst my dreams are saying to travel around the globe, do projects wherever I am and if I find a nice place to live ill stay there.
What would you guys do?
Would you try and find a way to chase your dreams and travel/live around the globe or invest your time and effort in growing the company?1 -
I was in an interview and all they really could tell me was the projects 'on the shelf' how they were understaffed, and new opportunities due to covid, that were coming onto the current development pipeline.
How would you guys react to a headhunted talk at a company structured like that? Just curious, it spooked me a bit at first.2 -
Construction crew came up my street literally in front of my house to dig the ground cause the pipeline for shit got rust and it started leaking shit all over the neighborhood, making the entire street smelly and shitty! This is what happens when i the shitbeast clog not just my toilet but the entire neighborhood with total bullshit! All must feel the wrath of my bullshit2
-
Need advice about switching to contracting.
TL;DR;
So I had 2 years of exp as an android dev, then I had a 1.5 year gap from doing android and now for the past 6 months Ive been doing android again fulltime. Im thinking of switching to contracting due to my debts and boring project and life crushing slow corporate processes in my current fulltime job, so I need tips and advices as to where should I start looking for new contracting gigs and in general what should I pay attention to. If it helps, I am based in EU, but am open to any EU/US gigs.
Now the full story:
Initially when I joined my current fulltime job after a break I had zero confidence, lowered my and employers expectations, joined as a junior but quickly picked up the latest standards and crushed it. Im doing better than half devs in my scrum team right now and would consider myself to be a mid level right now.
Asked for a 50% bump, manager kinda okayed it but the HQ overseas is taking a very long time to give me the actual bump. I have been waiting for 10 weeks already (lots of people in the decision chain were on and off vacations due to summer, also I guess manager sent this request to HQ too late, go figure). Anyways its becoming unnaceptable and I feel like its time for a change.
Now since I have mortgage and bills to pay, even with the bump that I requested that would leave me with like maximum 700-800 bucks a month after all expenses. I have debts of around 20k and paying them back at this rate would take 3 years at least and sounds like a not viable plan at all.
Also it does not help that the project Im working on is full of legacy and Im not learning anything new here. Corporate life seems to be very slow, lots of red tape kills creativity and so on. I remember in startups I was cooking features left and right each sprint, in here deploying a simple popup feature sometimes takes weeks due to incompetence in the chain. I miss the times where I worked in startups, did my job learned nre skills and after 6 months could jump on another exciting gig. Im not growing here anymore.
So because my ADD brain seems to be suited much better for working in startups, and also I need to make more money quick and I dont see a future in current company, I am thinking of going back to contracting. All I need right now is to build a few side apps, get them reviewed by seniors and fill my knowledge gaps. Then I plan of starting interviewing as a mid level or even a senior for that matter, since I worked with actual seniors and to be honest I dont think getting up to their level would be rocket science.
Only difference between mid and senior devs that I see atleast in my current company is that seniors are taking on responsibility more often, and they also take care of our tools, such as CD/CI, pipeline scripts, linters and etc. Usually seniors are the ones who do the research/investigations and then come up with actual tasks/stories for mids/juniors. Also seniors introduce new dependencies and update our stack, solve some performance issues and address bottlenecks and technical debt. I dont think its rocket science, also Ive been the sole dev responsible for apps in the past and always did decent work. Turns out all I needed was to test myself in an environment full of other devs, thats it. My only bottleneck was the imposter syndrome because I was a self taught dev who worked most of my career alone.
Anyways I posted here asking for some tips and advices on how to begin my search for new contract opportunities. I am living in EU, can you give me some decent sites where I could just start applying? Also I would appreciate any other tips opinions and feedback. Thanks!3 -
Hi, I'm currently an intern and building a web application with react. I'm only doing frontend and have no access to the backend.
After some development we want to host the website and the backend guy is building a pipeline.yml for me. Fast forward website doesn't work because of missing environmental files in the pipeline. I added them on azure but somehow you need to do that in the pipeline.yml as well. I have no idea how to do that and he said: "Find out for yourself and tell me later"
How should I work from here? I feel left alone with that backend stuff. Why should I fix this pipeline, isn't it his job or is it frontend?6 -
Does anyone know how to develop a pipeline-builder on window? I did some research but found that all are linux-based development, and also want to train my broken english in devRant :P1
-
Recently started a new role as a junior dev(second role). Three weeks in and I'm already starting to loathe the work setup & process.
Last week I was asked to fix a bug due to them not having anything in the pipeline for me(I had finished my allocated tasks for the sprint). There was no spec to this, no visible steps to replicate the error & no tests in place to validate it was working... I thought I had fixed it, even had one of the seniors reviewed it on my PR but also I walked him through my possible solution resulting in us moving forward with the "improved" solution.
After a bank holiday, I've come back to find that the "fix" I had deployed doesn't solve the problem at all. So here I am after 3.5hours of flying blind with a bug that I'm still not able to reproduce, bored and frustrated asf. Not to mention, that the codebase has little to no consistency, a lot of legacy and almost no form of tests.
Am I overreacting to this as junior?1 -
How can we modify ec2 instance with cdk stack or pipeline?
I am able to launch the ec2 instance but whenever i modify the stack it creates a new instance.
So, any idea how to launch and manage ec2 instances meaning change instance type , size etc programmatically?8 -
How do you setup your ci/cd pipeline to work just like you want it, without having to create a quadrillion commits in your repository? Create a branch, make a quadrillion changes and squash them at the end? Is there a better way?2
-
Rather than using the project management software that the company has spent the past year getting set up and stuck into, the new ops manager seems to think that faffing around in Google Sheets and making pretty schedules is the way forward.
If you're doing some work that's in your actual job list, but not in the new pipeline, ohhhh boy. -
Can someone tell me how to properly rebase changes from main branch
I always fuck myself over. Fucking merge conflicts i caused by myself. Since the CD pipeline creates a commit or i merge into main from a side branch, i often forget to pull those changes locally from main branch
What happens then is i just create a new branch to start working on the next feature
git checkout -b feature/shit
Totally forgetting to first do
git pull --rebase
From main branch. Because of this when i push shitload of features to feature/shit branch and then try to merge that shit into main. CD pipeline gets fucked. There are merge conflicts now because i havent rebased
Question -- if i switch to a new branch, make a shitload of changes and forget to rebase from main branch First, what command do i type to rebase right there (on the new branch) but rebase from main branch so these conflicts dont appear?23 -
PS: sorry the the question is too generic and with very broad boundaries.
PPS: Pls don't mention integration problems?
Ques: I wish to know what kind of software related problems do you devs face/faced in your work,
Say like you were working in version 9.2 postgre and in the end you realised it had to be 9.6 and get doomed or like the bot deployed created it's own error?
PPPS: It would be lovely if you could classify in what category or stage of pipeline does it belong to.
Thanks!!☺️3 -
I'm working on an internal overhaul for Orchid phasing out the last bits of code from May's crunch and making the loading pipeline much more transparent and easier to optimize, but I'm starting to get really tired of high theory and tree walking algorithms.
What are some light hearted projects that can benefit from a scripting language? I don't mind if existing scripting languages would be an objectively better choice, I don't want to build something perfect here, I just wanna have fun with this project again. -
I am trying to extract data from the PubSub subscription and finally, once the data is extracted I want to do some transformation. Currently, it's in bytes format. I have tried multiple ways to extract the data in JSON format using custom schema it fails with an error
TypeError: __main__.MySchema() argument after ** must be a mapping, not str [while running 'Map to MySchema']
**readPubSub.py**
import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions
import json
import typing
class MySchema(typing.NamedTuple):
user_id:str
event_ts:str
create_ts:str
event_id:str
ifa:str
ifv:str
country:str
chip_balance:str
game:str
user_group:str
user_condition:str
device_type:str
device_model:str
user_name:str
fb_connect:bool
is_active_event:bool
event_payload:str
TOPIC_PATH = "projects/nectar-259905/topics/events"
def run(pubsub_topic):
options = PipelineOptions(
streaming=True
)
runner = 'DirectRunner'
print("I reached before pipeline")
with beam.Pipeline(runner, options=options) as pipeline:
message=(
pipeline
| "Read from Pub/Sub topic" >> beam.io.ReadFromPubSub(subscription='projects/triple-nectar-259905/subscriptions/bq_subscribe')#.with_output_types(bytes)
| 'UTF-8 bytes to string' >> beam.Map(lambda msg: msg.decode('utf-8'))
| 'Map to MySchema' >> beam.Map(lambda msg: MySchema(**msg)).with_output_types(MySchema)
| "Writing to console" >> beam.Map(print))
print("I reached after pipeline")
result = message.run()
result.wait_until_finish()
run(TOPIC_PATH)
If I use it directly below
message=(
pipeline
| "Read from Pub/Sub topic" >> beam.io.ReadFromPubSub(subscription='projects/triple-nectar-259905/subscriptions/bq_subscribe')#.with_output_types(bytes)
| 'UTF-8 bytes to string' >> beam.Map(lambda msg: msg.decode('utf-8'))
| "Writing to console" >> beam.Map(print))
I get output as
{
'user_id': '102105290400258488',
'event_ts': '2021-05-29 20:42:52.283 UTC',
'event_id': 'Game_Request_Declined',
'ifa': '6090a6c7-4422-49b5-8757-ccfdbad',
'ifv': '3fc6eb8b4d0cf096c47e2252f41',
'country': 'US',
'chip_balance': '9140',
'game': 'gru',
'user_group': '[1, 36, 529702]',
'user_condition': '[1, 36]',
'device_type': 'phone',
'device_model': 'TCL 5007Z',
'user_name': 'Minnie',
'fb_connect': True,
'event_payload': '{"competition_type":"normal","game_started_from":"result_flow_rematch","variant":"target"}',
'is_active_event': True
}
{
'user_id': '102105290400258488',
'event_ts': '2021-05-29 20:54:38.297 UTC',
'event_id': 'Decline_Game_Request',
'ifa': '6090a6c7-4422-49b5-8757-ccfdbad',
'ifv': '3fc6eb8b4d0cf096c47e2252f41',
'country': 'US',
'chip_balance': '9905',
'game': 'gru',
'user_group': '[1, 36, 529702]',
'user_condition': '[1, 36]',
'device_type': 'phone',
'device_model': 'TCL 5007Z',
'user_name': 'Minnie',
'fb_connect': True,
'event_payload': '{"competition_type":"normal","game_started_from":"result_flow_rematch","variant":"target"}',
'is_active_event': True
}
Please let me know if I m doing something wrong while parsing the data to JSON. Also, I am looking for examples to do data masking and run some SQL within Apache Beam4 -
Persisterising derived values. Often a necessary evil for optimisation or privacy while conflicting with concerns such as auditing.
Password hashing is the common example of a case considered necessary to cover security concerns.
Also often a mistake to store derived values. Some times it can be annoying. Sometimes it can be data loss. Derived values often require careful maintenance otherwise the actual comments in your database for a page is 10 but the stored value for the page record is 9. This becomes very important when dealing with money where eventual consistency might not be enough.
Annoying is when given a and b then c = a + b only b and c are stored so you often have to run things backwards.
Given any processing pipeline such as A -> B -> C with A being original and C final then you technically only need C. This applies to anything.
However, not all steps stay or deflate. Sum of values is an example of deflate. Mapping values is an example of stay. Combining all possible value pairs is inflate, IE, N * N and tends to represent the true termination point for a pipeline as to what can be persisted.
I've quite often seen people exclude original. Some amount of lossy can be alright if it's genuine noise and one way if serving some purpose.
If A is O(N) and C reduces to O(1) then it can seem to make sense to store only C until someone also wants B -> D as well. Technically speaking A is all you ever need to persist to cater to all dependencies.
I've seen every kind of mess with processing chains. People persisting the inflations while still being lossy. Giant chains linear chains where instead items should rely on a common ancestor. Things being applied to only be unapplied. Yes ABCBDBEBCF etc then truncating A happens.
Extreme care needs to be taken with data and future proofing. Excess data you can remove. Missing code can be added. Data however once its gone its gone and your bug is forever.
This doesn't seem to enter the minds of many developers who don't reconcile their execution or processing graphs with entry points, exist points, edge direction, size, persistence, etc.2 -
What alternative do you use for EF Migrations?
We are in current stage that it's slowing our build time for 30 seconds, and each month we need to clean them and do the same for the whole pipeline (dev, qa, prod)...
It's way uncomfortable to work with it since we have stored procedures and ef doesn't track them...
What alternative did you chose to use?1