Building, just because…
Comms as a Data Engineer can be tough. Should you email a group of people? Should you dump a message in a public Slack channel they frequent? Should you follow up daily, weekly, etc? It’s a lot of manual labor. I don’t like manual work. Also, I hate email. This seems to be a common … Read more
What in the the world are Apache Airflow DAG Factories and why should you use them? Let’s go into what they are, why they’re used, and how they could make your life easier. We’ll also go into the nitty gritty of how to design and build one. Also, before I jump into this post, shout … Read more
So, a loooong time ago I wrote this post on how to clone a Git repo in Python3. I used subprocess that first time around to run git commands. I was essentially trying to run git commands in python explicitly. But, there’s a better way to do this. It’s prettier, it’s easier to read. There’s … Read more
This post will show you how to pull a domain from a full website path in BigQuery. So let’s set the stage for a hypothetical. You own a URL shortener company. You want to partner with a website for whatever reason. You decide that you want to do analysis over the data you’ve streamed or … Read more
Background Info This blog post will discuss AWS CloudFormation, PHP, and WordPress issues. So, a couple of years back I decided to leverage a CloudFormation template to scaffold a WordPress blog. Do note that this link is close to what I used, but it isn’t exactly what I used. It spun up a load balancer, … Read more
Hello folks. My name is Carlos Cabán and I’m a Data Engineer at Reddit as of October of 2023. I got into Data Engineering without a software degree. I thought I’d go over how I went from being an analyst out of college to doing larger scale data work with companies like Reddit and Netflix … Read more
Users have expectations when they’re using your tools. Maybe you have a message app? Your enter button should probably send your message. Tab might make you switch windows or menus. We’ll cover how to run a function when your user hits the enter button in a NextJS application. This will also work for any generic … Read more
Let’s quickly go over how to set default inputs on a NextJS Typescript function. This will be short and to the point. No need to waste your time. The Why I Need to Set a Default So, I wanted to dynamically set the autofocus of a custom input in my tool. But when I added … Read more
In what situation would you want to use BigQuery’s having clause outside of an interview? We’ll go over a couple of use cases and how I use it as a Data Engineer for Reddit. My Setup What Is BigQuery? BigQuery is a data warehouse as a service. Google handles your compute, your storage, and does … Read more
Let’s go over some terminal commands for Beginners. This assumes you have basic computer knowledge and might have some sort of interest in software development. Terminal commands, especially those for linux systems, will get you pretty far. My Setup MacOS Opening Your Terminal On a mac hit command+space. This will open a search window. My … Read more