Shubham Gupta
Lead Engineer - Data Platforms at Blinkit - Zomato | Open Source Enthusiast. Views are personal
New Delhi
7+ years of work experience
Jan 20251 yr 8 mosPresent
Gurugram, Haryana, India
Oct 20213 yrs 4 mosJan 2025
Gurugram, Haryana, India
Event Gateway at Zomato: Built an extensive Golang based event gateway to enable realtime routing capabilities of event streams to pluggable destinations with custom transformations
> Built in-house capabilities to deprecate Rudder as a customer segmentation tool
Datalakehouse at Blinkit: Built a robust CDC based datalakehouse with over 1000 source tables (10TB data) getting replicated at 15 minutes lag
> Built support to power data-marts on our datalake using Trino Iceberg Connector with upsert capabilities in near real-time
> Built a robust FSM based Web UI to make the onboarding process of tables in the datalake self-served
> Reduced data lake querying cost by 90% ($12000 monthly) by identifying issues in the presto querying engine and inefficient querying patterns
> Leading deprecation of Redshift cluster by building the required capabilties on in-house Trino cluster to reduce cost by $25000 monthly
> Reduced the refresh frequency of lake tables to 15 minutes from existing 3 hours enabling near-realtime querying capabilities
> Migrated datalake source DBs from existing RDS to Amazon Aurora
> Upgraded Debezium from v1.3 to v1.9.4 and also migrated kafka connect to k8s from existing EC2
> Patched data in lake to fix data discrepancy in over 25 tables
> Zero downtime upgrade of Airflow cluster with over 1500 dags from v1.10 to v2.13
> Reduced Redshift storage by (1TB) by identifying the unused tables and automating the pruning process of tables
> Optimized existing Redshift cluster to scale to 100,000 queries/day saving around $15000/month upgrade cost
Sep 20201 yr 1 moSep 2021
Gurugram, Haryana, India
Data Governance at Blinkit: Introduced and drove the adoption of a robust data cataloging tool (Datahub by Linkedin)
> Setup one-click deployment for the tool using docker, k8s helm charts
> Integrated data catalog with ETL pipeline (Airflow) BI tool (Redash) to automate the catalog generation process
> Migrated Datahub with over 4000 datasets from version v0.6 to v0.8.18
Data Tools at Blinkit: Managing and building features on top of existing in-house/OSS tools
> Built data exporting functionality in Redash used by over 2000 active users daily
> Primary owner of in-house built ETL frameworks
> Built multiple slack-bots to automate reporting & monitoring process
> Reduced Data Infrastructure [ Aurora — Xplenty — Redshift ] cost by 10-15%
May 20194 mosAug 2019
Gurugram, Haryana, India
> Worked as a Big Data Developer Intern
> Built a scalable and configurable big data analytics platform
> Wrote scripts for data cleansing and preparation of the data warehouse
> Structured data warehouse to reduce the query time
> Communicated with front-end & back-end team to engineer design patterns
> Apache Hadoop, Hive, Pyspark, Elasticsearch, Kafka
Aug 20181 yrJul 2019
USICT
> Cogitans is the Web Development Club of USICT
> Worked and lead many live projects
> Conducted classes on web development
Feb 20185 mosJun 2018
New Delhi, Delhi
> Worked as Full Stack Developer here.
> Worked with AngularJS, responsive web designs
> Made backend APIS in PHP + MySql
Sep 20174 mosDec 2017
New Delhi, Delhi
> Worked as a Full MEAN stack developer cum freelancer
> Built a CMS with an extensive back-end using NodeJS
> Worked on multiple websites making reactive and responsive web designs
Shubham’s AI Stack
Education
Bachelor of Technology (B.Tech.), Information Technology
Guru Gobind Singh Indraprastha University
20164 yrs2020
12th (CBSE BOARDS)
N. C. Jindal Public School, Punjabi Bagh
20151 yr2016
Join Shubham and 10k+ other professionals on
Tal is the app where bosses meet engineers & builders directly, without middlemen
Download Tal Boss