Shaurya's Blog

In this post, we'll explore different approaches for ingesting Kafka data into Iceberg tables, examine strategies for managing schema evolution, and discuss when to choose one method over another based on your specific use case.

A query engine or database's join algorithm is the mechanism through which datasets are unified, relationships are discovered and raw data is transformed into meaningful insights.

Two approaches have emerged to tackle the replication challenge: leader-based replication and leaderless replication. This article delves into the latter, exploring quorums, gossip protocols, sloppy quorums and hinted handoff.

Contextual bandits algorithms like LinUCB and Neural bandits used to optimise marketing campaigns

Streaming large files directly from SFTP to S3 without any intermediate disk storage, using Python's asyncio