Headline

Data Engineer | AdTech Expert @ GroundTruth | Distributed Systems & Scalable Pipelines | Apache Spark, Python, AWS

About

For the past six years, I've built the data backbone behind large-scale advertising technology — turning noisy, high-throughput ad-bidding and impression streams into fast, transparent, and cost-conscious pipelines. My focus sits at the intersection of distributed systems engineering and programmatic advertising, where milliseconds and data trust both translate directly into revenue. At GroundTruth, I'm leading the modernization of our data engineering stack: embedding rigorous CI/CD practices (Docker, GitHub Actions) into every release, adopting OpenLineage to give stakeholders full visibility into data lineage across the ad-serving pipeline, and introducing agentic AI workflows to cut manual toil out of engineering operations. Together, these shifts have shortened our deployment cycles and removed long-standing infrastructure bottlenecks that used to slow down ad-delivery and reporting systems. I build fault-tolerant, distributed architectures on AWS using Python and Apache Spark — designed to process ad-impression, click, and bid-stream data at scale without sacrificing speed or reliability. The goal is always the same: absorb the complexity of scaling adtech infrastructure so the business can ship faster, spend less on cloud, and trust every number in the dashboard. Core Arsenal: Python | PySpark | Scala | SQL | AWS | CI/CD | Docker | Agentic AI | Programmatic Advertising Data Systems

Location

Pune Division

AI stack

claude

cursor

gemini

openai

perplexity

Work experience

Data Engineer at GroundTruth, Oct 2022 - Present · 4 yrs, Gurugram, o Designed and developed scalable ETL/data pipelines using PySpark, Scala, SQL for processing large-scale structured and semi-structured datasets in distributed environments. o Built and automated production-grade data pipelines using AWS services such as S3, EMR, Lambda, Glue etc. for scalable data ingestion, processing, storage, and workflow orchestration. o Optimized Spark jobs using partitioning, caching, broadcast joins, and different DataFrame-based transformations, improving pipeline performance and reducing execution time for revenue-critical reporting workflows. o Collaborated with cross-functional engineering and analytics teams to troubleshoot production issues, optimize pipeline reliability, and improve operational efficiency.

Data Engineer at Wiser Solutions, Inc., May 2022 - Oct 2022 · 6 mos, Pune District, 1.Developed and optimized Hive queries for creating, managing, and querying large-scale datasets to support analytical and business reporting requirements. 2. Designed and scheduled Sqoop jobs for efficient data ingestion from traditional RDBMS systems into HDFS, including handling incremental data loads and data validation processes. 3.Built and optimized Spark applications using PySpark, Spark SQL, and DataFrames for large-scale data transformation, processing, and performance optimization.

Data solution engineer at Numerator, Jun 2021 - May 2022 · 1 yr, Pune Division, 1. Developed and optimized Hive queries for creating, managing, and querying large-scale datasets to support analytical and business reporting requirements. 2. Designed and scheduled Sqoop jobs for efficient data ingestion from traditional RDBMS systems into HDFS, including handling incremental data loads and data validation processes. 3. Built and optimized Spark applications using PySpark, Spark SQL, and DataFrames for large-scale data transformation, processing, and performance optimization. 4. Built interactive Tableau dashboards and SQL-based reports to monitor marketing KPIs and support business decision-making.

Data Analyst at IDeaS Revenue Solutions, Dec 2019 - Jun 2021 · 1 yr 7 mos, Pune Division, 1. Designed and optimized Managed and External Hive tables using partitioning and bucketing techniques, improving query performance and data retrieval efficiency. 2. Performed exploratory data analysis (EDA), statistical analysis, and trend identification to support revenue optimization and business decision-making initiatives. 3. Built and implemented Multiple Linear Regression models end-to-end to analyze customer and revenue patterns, enabling data-driven forecasting and strategic recommendations for clients. 4.Collaborated closely with R&D and business stakeholders to identify data quality issues, validate analytical outputs, and improve application performance and reporting accuracy. 5.Created analytical reports and dashboards to monitor KPIs, business trends, and operational metrics using SQL, Hive, and visualization tools.

Education

Post graduation (MA), Economics, Symbiosis College of Arts & Commerce

Bachelor's degree Science, Mathematics, Fergusson college