Based in Sydney, Australia, Foundry is a blog by Rebecca Thao. Her posts explore modern architecture through photos and quotes by influential architects, engineers, and artists.

Episode 34 - Data Engineering with Joe Crobak, Foundations of Smart Software

Episode 34 - Data Engineering with Joe Crobak, Foundations of Smart Software

 
IMG_3368.jpg
 

Now with extra mappers and reducers!

I talk to Joe Crobak of the Data Engineering Weekly newsletter about data engineering, and it's importance in modern software projects. We cover:
- The definition of data engineering
- Hadoop, MapReduce, and Spark
- How data engineering has evolved over the last decade.
- Joe's experiences working in government vs. startups. The US Digital service, and the paperwork reduction act.

Guest

Joe Crobak is software developer that has worked on several data-powered consumer products at Foursquare and other startups in NYC. He worked for two years in the federal government as a founding member of the United States Digital Service team at the Department of Health and Human Services. Joe's Data Eng Weekly newsletter, which covers distributed systems and big data, reaches over 11,000 subscribers. He lives in Washington, DC with his wife and two dogs.

Joe's site at dataengweekly.com
Joe’s Twitter
Joe’s Medium - check out The 10 most Surprising Things I Learned Working in the Government

Previous Episodes

Episode 13 on Applied Artificial Intelligence, where we also looked at organizational hurdles to creating AI product

Book Recommendations

Winning with Data - a business book about big data and infrastructure
Hadoop: The definitive Guide


Episode 35 - Camille Fournier and the Manager's Path

Episode 35 - Camille Fournier and the Manager's Path

Episode 33 – Ads, Bias, Censorship, Democracy

Episode 33 – Ads, Bias, Censorship, Democracy