Building, just because…
Comms as a Data Engineer can be tough. Should you email a group of people? Should you dump a message in a public Slack channel they frequent? Should you follow up daily, weekly, etc? It’s a lot of manual labor. I don’t like manual work. Also, I hate email. This seems to be a common … Read moreComms As A Data Engineer
Why in the world should you use a DAG factory with Apache Airflow? Let’s go into what they are, why they’re used, and how they could make your life easier. We’ll also go into the nitty gritty of how to design and build one. Also, before I jump into this post, shout out to all … Read moreApache Airflow DAG Factories
So, a loooong time ago I wrote this post on how to clone a Git repo in Python. I used subprocess that first time around to run git commands. I was essentially trying to run git commands in python explicitly. But, there’s a better way to do this. It’s prettier, it’s easier to read. There’s … Read moreUpdated – How To Clone A Git Repo In Python3
This post will show you how to pull a domain from a full website path in BigQuery. So let’s set the stage for a hypothetical. You own a URL shortener company. You want to partner with a website for whatever reason. You decide that you want to do analysis over the data you’ve streamed or … Read morePull A Domain From A Full Website Path In BigQuery
Background Info So, a couple years back I decided to leverage a CloudFormation template to scaffold a WordPress blog. Do note that this link is close to what I used, but it isn’t exactly what I used. It spun up a load balancer, ec2s, did the networking. It did most of the work to get … Read moreAWS CloudFormation, PHP, and WordPress Issues
Hello folks. My name is Carlos Cabán and I’m a Data Engineer at Reddit as of October of 2023. I thought I’d go over how I went from being an analyst out of college to doing larger scale data work with companies like Reddit and Netflix without going to school for software engineering. This will … Read moreMy Data Engineering Journey Without A Software Degree – Part 1 – My Thoughts on Education
Users have expectations when they’re using your tools. Maybe you have a message app? Your enter button should probably send your message. Tab might make you switch windows or menus. We’ll cover how to run a function when your user hits the enter button in a NextJS application. This will also work for any generic … Read moreRun a Function On Enter in NextJS
Let’s quickly go over how to set default inputs on a NextJS Typescript function. This will be short and to the point. No need to waste your time. The Why I Need to Set a Default So, I wanted to dynamically set the autofocus of a custom input in my tool. But when I added … Read moreSet Default Inputs on a NextJS Typescript Function
In what situation would you want to use BigQuery’s having clause outside of an interview? We’ll go over a couple of use cases and how I use it as a Data Engineer for Reddit. My Setup What Is BigQuery? BigQuery is a data warehouse as a service. Google handles your compute, your storage, and does … Read moreBigQuery’s Having Clause
Let’s go over some terminal commands for Beginners. This assumes you have basic computer knowledge and might have some sort of interest in software development. Terminal commands, especially those for linux systems, will get you pretty far. My Setup MacOS Opening Your Terminal On a mac hit command+space. This will open a search window. My … Read moreTerminal Commands For Beginners