Headline

Building Data Platform@zepto

About

P.S. This "About" was written in 2021. "Innovate. Automate. Elevate." I believe in the power of process: POC -> Automate -> Document -> Production -> Done. Why do the mundane when a computer can do it better? Let’s create intelligent systems that run themselves. Technologies I’ve Mastered: Big Data & Hadoop Ecosystem: HDFS, Apache Hive, Pig, Sqoop, Oozie, Hortonworks (Ambari, Ranger) Data Engineering & Streaming: Apache Spark (Python, Scala), Nifi, Kafka, Airflow, Trino Cloud Mastery: AWS: S3, EC2, EMR, Batch, RDS, Glue Azure: AKS, Blob Storage, SQL, Databricks, Active Directory, Cosmos DB, Cost Management GCP: GKE, BigQuery, Cloud Storage, Cloud SQL, Secret Manager, Billing Visualization: Apache Superset Programming Languages: Scala, Java (Core & EE), Python 3 Containerization & Orchestration: Docker, Kubernetes, Helm Charts CI/CD & Code Management: Jenkins, Git, Gitea, Bitbucket, GitHub If there's a smarter way to do it, I'll find it—or build it. Let's push the boundaries of what's possible.

Location

Bengaluru

AI stack

claude code

codex

cursor

opencode

Work experience

DE3, Data Platform Engineering at Zepto, Sep 2024 - Present · 2 yrs, Bengaluru, Karnataka, India

SDE3 Platform and Data engineering at Glance, Jan 2023 - Oct 2024 · 1 yr 10 mos, Bengaluru, Karnataka, India, Led modernisation of data pipelines and query engines using Spark, Trino, Airflow, improving performance and cutting query costs. Designed multi-cloud data infrastructure (AWS, GCP, Azure) with 99.99% uptime and high scalability. Built Debezium + Kafka CDC pipelines for near real-time OLTP → OLAP data sync and analytics. Migrated Spark workloads to Kubernetes, optimizing compute efficiency and cost. Set up Grafana, Superset, CI/CD for automated monitoring and deployments. Developed frameworks for data ingestion, transformation, and validation to improve data quality and reliability.

SDE2, Platform and Data Engineering at Glance, Sep 2021 - Jul 2023 · 1 yr 11 mos, India

Senior Data Engineer at Lumiq.ai, Mar 2020 - Sep 2021 · 1 yr 7 mos, Mumbai, Maharashtra, India, Worked as a Senior Data Engineer to build data lake for prestigious bank with aws cloud services like S3 ,EC2,EMR,AWS batch,AWS sagemaker,EKS and open source technologies like spark, airflow, elk stack, Nifi, kakfa, docker, kubernetes, helm etc.

software developer analyst at Accenture, Jun 2019 - Mar 2020 · 10 mos, Mumbai, Maharashtra, India, worked in Accenture as Software Engineering analyst with skillset of big data stack(spark, hive, sqoop, pig, kafka, storm,nifi,oozie)

Associate Software Engineer at Accenture, May 2017 - May 2019 · 2 yrs 1 mo, Mumbai Metropolitan Region

Education

Bachelor of Engineering (B.E.), Information Technology, Institute of Engineering & Technology DAVV, Indore

Associate’s Degree, network, Cisco Networking Academy

Public links

medium: https://medium.com/@shubhamkanungo