Problem: I want to use Python to read messages from a Kafka Topic between two timestamps from all partitions. You can use tools like kcat to do this, but there is no simple way to do it from Python.

My Code:

https://github.com/vishalendu/python_kafka_read_messages_between_timestamps

#kafka #Python #Pandas

#kafka resizing a topic's replication factor

My script to help generate the remapping json

https://github.com/vishalendu/kafkaTopicIncreaseReplicationFactor

My notes on Kubernetes Troubleshooting and Ephemeral Containers:

https://medium.com/@vishalendu/kubernetes-ephemeral-containers-e73f2990885

#kubernetes #debugging

Quite nice set of Kubernetes tools and technologies mentioned in this blog for dev/ops/everyone doing k8s?: https://bit.ly/3QBmjdi Although Kubectl is irreplaceable, you can make your work almost pleasant 😉 and less error prone with some of these tools. #kubernetes #productivitytips

My latest article/collation of useful hacks for troubleshooting on hardened Kubernetes containers: https://medium.com/@vishalendu/my-favorite-linux-troubleshooting-tools-in-the-cloud-infra-era-7b8222476788 #linux #troubleshooting #cloud #containers

Introducing ksqldb (https://lnkd.in/dE7HYsFy) Using this database you can query Kafka with SQL syntax, you can do filtering, updating etc. This is a very powerful application that uses Kafka Streaming APIs. Python Code and tutorial can be found on my github: https://github.com/vishalendu/kafka-ksqldb

Following article gives a solution to a general problem of updating an excel with multiple DataFrames. In "openpyxl" its easier to create an Excel file with data, but updating it is tricky. So this is a possible "clean" and "generic" solution. https://medium.com/@vishalendu/writing-multiple-dataframes-to-an-excel-file-without-overwriting-it-605c631b7c95