Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and Hera | DoKC Town Hall

Data on Kubernetes Community

Player FM - Internet Radio Done Right

27 subscribers

Tilføjet four år siden

Indhold leveret af Data on Kubernetes Community. Alt podcastindhold inklusive episoder, grafik og podcastbeskrivelser uploades og leveres direkte af Data on Kubernetes Community eller deres podcastplatformspartner. Hvis du mener, at nogen bruger dit ophavsretligt beskyttede værk uden din tilladelse, kan du følge processen beskrevet her https://da.player.fm/legal.

On the Bus with Troy Vollhoffer

1
From Backroom Bars to Broadway with Dustin Lynch 36:27

for 23 dage siden36:27

Afspil senere

Lister

Liked

36:27

Not many artists actually hail from Tennessee, but the scenic valleys and rolling hills of The Volunteer State are part of Dustin Lynch’s DNA. In this episode of On the Bus, Country Thunder CEO Troy Vollhoffer sits down with Dustin to discuss his journey from playing fraternity parties and weddings across the southeast to being the first country artist with a club residency at the Wynn in Las Vegas. Plus, stick around for our new segment, Thunder Strike, where Troy features upcoming festival performer Riley Green’s hit song “Damn Good Day to Leave” to give you a taste of what’s to come at Country Thunder in 2025.…

for ca. et år siden 23:17

MP3•Episode hjem

Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and Hera
Presented by Matt Menzenski, Senior Software Engineering Manager, Payitgov

At PayIt, we’ve been deploying applications to Kubernetes almost since the beginning of the company. Our data workloads, however, have run instead in AWS Glue. This has worked well enough for the reporting use cases that have been the main focus of this team historically. However, at the beginning of 2022, the PayIt data team began building out a new data platform, and in the process, ran into a number of challenges with Glue. In this talk, I will share the difficulties that we encountered with building, deploying, and orchestrating ETL pipelines in AWS Glue, our decision process for moving those workloads into Kubernetes, and the ELT architecture that we’ve arrived at today. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w

243 episoder

#Tech #Demetrios