How To Get Started With Apache Airflow?

When Airbnb was scaling rapidly, they faced the problem of organizing complex data pipelines. To combat this and become a data-driven organization, Airbnb launched Apache Airflow in 2015, their custom-made open-source platform to manage complex workflows. In simple words, Apache Airflow is a platform where you can create, schedule, and monitor complex workflows using simple …

Read more

Making A Twitter Bot With Tweepy

Want help making a Twitter bot with Tweepy? It’s actually pretty easy after you apply for a developer’s account and setup an app on Twitter’s side. You can do the application and setup here. My Computer Setup Python3 (Finally getting used to adding parentheses around my print statements ) macOS Getting your Environment Setup You …

Read more

Cast a String Array of JSON Objects to JSON in Javascript

I recently came across a situation where I had to to save an array of objects to a local device so that I could pull that info up later. My problem was that I had to change that array of objects into a string before saving it. Thus, I had to cast a string array …

Read more

Using the Google Translate API in your Android App

Just make an Http call! So, I thought I’d write a quick article on using the Google Translate API in your Android app since most of the articles I found didn’t help. Article after article pointed me in the direction of this Java library which unfortunately conflicted with other dependencies in my application. I spent …

Read more

Android On Click Listener

Buttons are one of the most important building blocks in any application. Thus, their implementation is a key milestone in learning any new language that deals with GUIs. The following should get you through making any View, Button, or Image respond to touch in Android Studio.

Add A Background Image to a NextJS App

One of the first things we do when starting off a new application is get our background squared away. This article will cover how to add a background image to a NextJS app. My Setup Operating System = macOS Ventura Next.js v13.1.1 Setting Up Our Basic NextJS App Webstorm Sooo, I use Webstorm. So rather …

Read more

Set An Existing Table’s Partition Expiration In Big Query

Why Might I need A Partition Expiration? Partition expirations can be leveraged for several use cases. Some countries have laws pertaining to how long PII can be held. Storage is very expensive, maybe your company wants to save some money? The list can and will go on. But it gets a little harder when the …

Read more

Run Apache Airflow Locally in Docker

We’ll walk you through how to run Apache Airflow locally in Docker. The first chunk of this post will cover how to get Airflow standing, the second will go into some nuance and will answer several whys. My Setup MacOS Ventura M1 Chip Getting Airflow Standing Docker 1. Download Docker Desktop from this link. Make …

Read more

Execute Bash Commands and Return Results in Python

I’ve had numerous cases where I’ve needed to execute bash commands and return the results in Python for some additional manipulation. My most recent example involved me working with BigQuery schemas. Getting schemas from the CLI was easy peasy but I’m not the best at BASH programming so I naturally turned to something more familiar …

Read more

Install a WordPress Theme Using the CLI for a Subsite

Hi! So, you need to install a wordpress theme using the CLI for a subsite? Maybe you’re using Cloud Formation to scale your deployment or something and need it done each time your cluster of EC2 scales horizontally? Regardless, there’s a flag for that! I kept this one short and to the point. This should …

Read more

What Happens When Your Code Does Not Follow The Python Rules?

So what happens if your code does not follow the Python rules? This article will be fairly short. Most of this will not depend on your version of Python. Tabbing/Spacing So what happens if you get your tab or spacing incorrect? Let’s give it a go. Often times the best way to figure something like …

Read more

Red Tape – Are Your Workflows Worth The Cost?

Who Am I? So, what is red tape? But first a little bit of background about me. My names Carlos and I’m a data guy. I started off as an analyst with a smaller firm that sold non passenger car tires. Think wheelbarrow tires. Next came Michael Kors, a luxury fashion brand valued somewhere around …

Read more

Removing Duplicate Elements From An Array in BigQuery

We’ll be removing duplicate elements from an array in BigQuery in this post. Let’s set the stage. It’s a chilly night in the office and marketing reaches out asking for data. PANIK. But they’ve given you a general location of where that data is located. CALM. You do a quick select * of the table …

Read more

Horizontally Scaling A WordPress Website – Part 1 – The Overview

Let’s start off with saying that horizontally scaling a wordpress website is very easy to do but is kinda expensive. I’d say it’s about $100+ a month to leave this infra consistently standing in AWS. Most of the cost is in the DB and EC2 instances. So, beware of forgetting about infra you stand up …

Read more

Apache Airflow DAG is Failing Silently

So your Apache Airflow DAG is failing silently. Are you running an ETL on a huge dataset? This is a symptom of an Airflow instance without sufficient memory. Dig into your instances logs and you’ll probably see an evicted worker if your running your instance’s workers on Kubernetes. You’ll see similar logs wherever you run …

Read more

Exit mobile version