Headline

Data Engineer | PySpark · Azure Databricks · Airflow · Spark SQL · Hive · AWS EMR | ETL Pipeline | Pune

About

I build the pipelines that make data usable at scale. Around 2 years of hands-on experience as a Data Engineer, I specialize in designing and optimizing high-performance ETL pipelines using PySpark, Apache Spark, and Azure Databricks — processing millions of records daily from sources like Azure Blob Storage, AWS S3, and Snowflake. At Vodafone Idea (Vi), I engineered PySpark-based workflows on AWS EMR, automated job orchestration via AWS Step Functions, and consistently resolved production bottlenecks that kept data SLAs intact. At DGLiger, I've leveled up into Databricks + Airflow territory — building DAG-orchestrated batch pipelines and integrating data quality checks into every layer. What I bring to the table: → End-to-end ETL pipeline design (batch + incremental loads) → Spark performance tuning: partitioning, caching, predicate pushdown → Workflow orchestration with Apache Airflow → File format expertise: Parquet, Avro, ORC, JSON, CSV → Cloud-native data engineering on AWS (EMR, S3) & Azure (Databricks, Blob Storage) → Data warehousing with Snowflake and Hive I'm actively seeking Data Engineer / Big Data Engineer roles in Pune where I can contribute to building robust, scalable data infrastructure.

Location

Pune City

Work experience

Data Engineer at DGLiger Consulting, Feb 2026 - Present · 5 mos, Pune Division, • Built and maintained scalable ETL pipelines using PySpark on Azure Databricks, processing large-scale structured and semi-structured data from Azure Blob Storage and Snowflake with high reliability. • Designed and orchestrated end-to-end Airflow DAGs, reducing manual intervention in batch pipeline scheduling by automating dependency management and alerting on failures. • Integrated Databricks notebooks into Airflow workflows, enabling fully automated batch data processing cycles and improving pipeline observability. • Implemented Spark performance tuning techniques (partitioning, caching, broadcast joins, predicate pushdown), resulting in measurable reduction in job execution time. • Enforced data quality through validation logic at ingestion and transformation layers, reducing downstream data errors for business reporting teams. • Managed source code versioning, deployment, and collaboration using Atlassian Bitbucket.

Big Data Engineer at Vodafone Idea Limited, Sep 2024 - Oct 2025 · 1 yr 2 mos, Pune, Maharashtra, India, • Designed and implemented PySpark-based ETL pipelines on AWS EMR for large-scale data processing. • Used Spark SQL and DataFrames for data transformation, aggregation, and analytics. • Designed and implemented Spark jobs using Python. • Implemented Spark SQL queries for data querying and aggregation. • Developed custom Spark functions for complex data transformations. • Worked with Spark's data serialization formats (Avro, Parquet, JSON, etc.). • Debugged and optimized Spark code for performance bottlenecks. • Designed and implemented ETL processes using Spark. • Designed efficient PySpark workflows on AWS EMR for processing AWS S3 data. • Automated Spark job executions on AWS EMR clusters using AWS Step Functions.

Intern at Vodafone Idea Limited, Jan 2024 - Apr 2024 · 4 mos, Pune, Maharashtra, India

Education

Bachelor of Engineering - BE, Electronics and Telecommunication, Dr. D. Y. Patil Institute of Technology, Pimpri, Pune

Bachelor of Engineering - BE, Electrical, Electronics and Communications Engineering, Savitribai Phule Pune University (SPPU)