Headline

Senior Data Engineer | AWS (Glue · Kinesis · Redshift · Lambda · Step Functions) | Databricks · Snowflake · PySpark · dbt | CDC · Streaming · Airflow | Vanguard · Kroger

About

🚀 Senior Data Engineer with 5 years building enterprise-grade, AWS-native data platforms for Fortune 500 clients in fintech and retail at production scale. I specialize in designing and delivering end-to-end cloud data solutions — from real-time ingestion to analytics-ready Gold layers — with a strong focus on AWS, Databricks, and Snowflake. I've built CDC pipelines for Vanguard ($9T AUM), a multi-region serverless API platform for Kroger (US retail), and streaming Medallion architectures for Qorvo — all with strict data quality, compliance, and uptime requirements. 𝗪𝗵𝗮𝘁 𝗜 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁 𝗮𝗻𝗱 𝗯𝘂𝗶𝗹𝗱: → Real-time CDC & streaming pipelines — AWS Glue, Kinesis, DynamoDB Streams, Lambda → Medallion Lakehouses (Bronze/Silver/Gold) — Databricks DLT, Auto Loader, Delta Lake, Unity Catalog → Cloud Data Warehousing — Snowflake (ELT, streams, tasks, dynamic tables) → Pipeline Orchestration — Apache Airflow (Astronomer/MWAA), Databricks Workflows → Data Transformation — dbt (models, tests, snapshots, incremental loads) → Serverless ETL/ELT — Lambda, Step Functions, API Gateway, AWS Glue Catalog → Data quality frameworks — DLT Expectations, automated reconciliation, SNS alerting → Multi-region Active-Passive AWS architectures for high availability 𝗖𝗼𝗿𝗲 𝗔𝗪𝗦 𝗦𝘁𝗮𝗰𝗸: AWS Glue · Kinesis · Lambda · Step Functions · Redshift · DynamoDB · S3 · SNS · API Gateway · IAM · CloudWatch · Glue Data Catalog · MWAA 𝗗𝗮𝘁𝗮 & 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴: Databricks · Snowflake · Delta Lake · dbt · Apache Airflow · PySpark · Apache Spark · Apache Kafka · Python · SQL · Unity Catalog 𝗔𝗹𝘀𝗼 𝘄𝗼𝗿𝗸𝗲𝗱 𝘄𝗶𝘁𝗵: Azure Data Factory · ADLS Gen2 𝗡𝘂𝗺𝗯𝗲𝗿𝘀 𝗜'𝗺 𝗽𝗿𝗼𝘂𝗱 𝗼𝗳: 📊 ~15% reduction in data inconsistency at Vanguard via automated reconciliation ⚡ 30–40% API throughput improvement at Kroger through decoupled serverless design 🔄 4,000+ annual transactions processed at 99.97% uptime (multi-region AWS) 📦 Millions of records migrated from DB2 mainframe to cloud — zero data loss

Location

Greater Delhi Area

Work experience

Senior Data Engineer at HashedIn by Deloitte, May 2024 - Present · 2 yrs 2 mos, Gurugram, Haryana, India, Building enterprise-grade cloud data solutions on AWS and Databricks for Fortune 500 clients across fintech and retail. 🔷 Project: Success Factor | Client: Qorvo (Semiconductor) • Architected end-to-end Databricks streaming Medallion pipeline (Bronze/Silver/Gold) ingesting JSON from AWS S3 via Auto Loader with metadata-driven schema inference • Built staging layer using Delta Live Tables (DLT) Expectations for automated data quality validation — failed records quarantined before Bronze layer • Implemented Silver transformations (SCD Type 2, multi-source joins, business-rule mappings) and Gold aggregations connected to Tableau dashboards • Orchestrated full pipeline via parameterized Databricks Workflows 🔷 Project: Digital Forms Signature | Client: Vanguard ($9T AUM) • Designed near real-time CDC pipelines using AWS Glue, Lambda, Step Functions, Kinesis & S3 — migrating legacy DB2 mainframe data to DynamoDB and Redshift at millions-of-records scale • Reduced data inconsistency by ~15% through automated source-to-target reconciliation with SNS-driven operational alerts 🔷 Project: Serverless API Platform | Client: Kroger (US Retail) • Deployed multi-region (Active-Passive) serverless AWS architecture with API Gateway, Step Functions, Lambda, JWT auth — processing 4,000+ annual transactions • Improved API throughput 30–40% through decoupled validation/submission workflows Skills: AWS Glue · AWS Lambda · Amazon Kinesis · Databricks · Delta Live Tables · PySpark · Apache Spark · CDC · ETL/ELT · Medallion Architecture

Data Engineer at EXL, Apr 2022 - Sep 2023 · 1 yr 6 mos, Gurugram, Haryana, India, Built and optimised data engineering solutions on Azure and Databricks for American Express. • Built and maintained 20+ Azure Data Factory (ADF) pipelines for ETL into ADLS Gen2 • Developed PySpark and SQL data processing solutions in Databricks — applying partitioning, coalescing, and join optimizations for performance tuning • Created Delta format Data Warehouse tables using MERGE operations; automated ADF workflows with triggers • Implemented monitoring and alerting for production pipelines on the American Express data platform Skills: Azure Data Factory · Databricks · PySpark · Delta Lake · ADLS Gen2 · SQL · Python

Data Analyst at OPK E SERVICES, Mar 2021 - Apr 2022 · 1 yr 2 mos, Gurugram, Haryana, India, • Designed full and incremental data load pipelines using Azure Data Factory • Developed ETL/ELT processes with Python and PySpark to ingest multi-source data into ADLS Gen2 for downstream analytics

Student at Applied AI Course, Jun 2020 - May 2021 · 1 yr

Data Analyst Intern at MyScoot, Sep 2019 - Mar 2020 · 7 mos, Gurugram, Haryana, India, • Got Letter of recommendation. • Performed analysis and presented results using SQL, Google sheets, Microsoft Excel, Google studio, and Python. • Imported, exported, and manipulated large datasets. • Performed insight analysis of users with Mixpanel, Google analytics.

Education

Bachelor of Technology - BTech, Civil Engineering, Dronacharya College of Engineering