Captain’s Log Star Date 2024.03.15

Study Progress This Week

• Data Pipelines Pocket Reference – Moving and Processing Data for Analytics
  • Chapter 4 – Data Ingestion: Extracting Data

Findings/Notes

I’ve kind of slowed down my technology research recently, I’ve been really busy with my other demands right now. The most important thing I studied this past week was understanding Kafka and Snowflake and how those technologies are used in modern data architectures. I know that Kafka is used for building real-time data pipelines and Snowflake is a cloud-based data warehousing platform. I am aware it’s a common coupling to have Kafka with Snowflake to ingest data and provide a means to analyze a data warehouse. I will most likely look at Kafka as it is open source, but given that Snowflake is commercial I’m not sure what I can get done with it for free. I would still like to learn how to use Apache Spark also. I am just seeing so much of it mentioned in the online communities I follow that I think they need some attention.

Other than that, I am transitioning out of a time intensive volunteer role so I can spend some more of my free time exploring technical topics and I am looking forward to that for sure. I am behind on my training plan and would like to make up for it.