2024: A retrospective
It’s been too long a year, I feel. Quite a lot happened in the last 365 days, and I’m here to discuss the ones that fit here.
Stuff I learnt
I learnt how different working in a big MNC and a growing startup is. And spoiler alert, its similar to living in 2 different universes. I can’t say which one makes more sense, it totally depends on what stage of life you’re at.
Its a ride for sure, doesn’t matter how long you stay in it, its a ride, day in and day out. Some interesting things I worked with in last 1 year:
- Databricks — Jobs, Workflows, Service principals, Cluster sizing, Spark metrics, CI/CD, Deployments, Fleet clusters, Analyze, Optimize, Vaccum, Z-Order
- AWS — S3, Glue, Athena, Lake Formation, Secrets Manager, Lambda, Sagemaker, Batch operator, SQS, SNS
- Checking up on quality of data — Great Expectations, Deequ
- Deploying stuff — ArgoCD, Jenkins
- Stuff on visualizing/UIs — Superset, Streamlit, ReactJS, Gradio
- The whole schebang of RBAC — Apache Hadoop, Apache Ranger, Hive Metastore, Okta
- Managing data — Data mart strategies, Data lakehouses, backfilling them, performing PITR, checkpointing
- Lakehouse tech — Delta, Iceberg, metadata manipulation, Deletion Vectors
- Machine Learning & more— NER, Fennel AI, Basics of Transformers & LLMs, Frontier models, Ollama
- Oncall :)
Stuff I read
Not everything that I consumed ideas from was in hard copy. And since you can read literally anything and everything, it makes more sense to mention books from completely different topics/walks of life. The top few books which taught me more than people were:
- Principles by Ray Dalio
- Predictably Irrational
- Hooked
- One Up The Wall Street
- Designing Machine Learning Systems
Stuff I watched
Quite a lot of time was spent in watching and re-watching content of all form. Ideally YouTube should do a Recap as well. Some great shows I caught up/discovered:
The Penguin
Monster (2004)
The Sympathizer
Patriot
Lastman
Station Eleven
The Little Drummer Girl
If I had a plug to Letterboxd, I’d share that before making movie suggestions. Regardless, some cinema I’d recommend that I saw last year:
Being John Malkovich
Dune 2
Dial M for Murder
Dead Poets Society
Dumb Money
Monkey Man
Raging Bull
The Banshees of Insherin
The Sting
Stuff I listened to
Always a mixed bag of artists from all over the world, but people who made a big dent in my Wrapped this year:
fred again…
Seedhe Maut
Hans Zimmer
David Bowie
Karan Aujla
Skrillex
Stuff I wrote/talked about
LinkedIn has quite a plethora of everything I wrote this year, you can find it here.
On Medium, I wrote about the things I’ve worked with in the past and it made sense to write in long form about them:
Also, I tried to give back to the community in form of career guidance on Topmate. Got some great reviews from folks who did join.
Stuff I want to do/try doing
Finally, things I want to do or at least try doing in the coming year:
- Get back to learning the keyboard and play at least one popular tune on it
- Dive deep into LLM architectures and learn enough to teach others in whatever way I can
- Build a fun side project in travel/food exploration, which allows me to create something from scratch and see it through, but this time using tools like v0 and Cursor
- Write more for myself
- Run more regularly
In retrospect, quite a lot happened this year that made me think about the choices I make in life, the things I choose not to do, and how they impact me in the long run.
Hopefully, the coming year will bring some new learning experiences.
fin