Atharva Jibhakate profile photo 1 on Tal

Atharva Jibhakate

Senior Data Engineer | Databricks | PySpark | Apache Spark | Airflow | Snowflake | AWS | ETL/ELT | Data Lakehouse | Data Warehousing | Data Platforms

Pune Division

5+ years of work experience

Publicis Groupe
Feb 20251 yr 6 mosPresent

Pune District, Maharashtra, India

Product: GrowthOS Campaign Manager Platform Working as a Senior Data Engineer in the GrowthOS product team, responsible for designing and developing large-scale cloud-native data platforms and analytics solutions for global marketing and advertising use cases. • Design and develop scalable ETL/ELT pipelines using Databricks, PySpark, Apache Spark, Airflow, Delta Lake, Snowflake, and AWS. • Build and maintain data products following Lakehouse and Medallion Architecture principles (Bronze, Silver, Gold layers) to support analytics and business intelligence workloads. • Develop metadata-driven frameworks and reusable pipeline components to improve scalability, maintainability, and onboarding of new client data sources. • Lead migration and modernization initiatives from legacy Redshift-based architectures to Databricks Lakehouse, improving performance, reliability, and operational efficiency. • Orchestrate end-to-end workflows using Apache Airflow and Databricks Workflows, ensuring reliable execution and monitoring of production data pipelines. • Implement data quality validation, performance optimization, and operational monitoring strategies to support enterprise-scale datasets. • Deliver outbound data integrations to downstream platforms including Snowflake and Redis, enabling faster data access and analytical consumption. • Collaborate with global cross-functional teams across Product, Engineering, Analytics, and Data Science to deliver scalable data solutions. • Contribute to AI-powered metadata enrichment initiatives using Claude Sonnet, enhancing dataset discoverability, documentation quality, and governance capabilities.
UST
Jun 20249 mosFeb 2025

Pune District, Maharashtra, India

Client: Blue Cross & Blue Shield Association • Designed and developed scalable ETL pipelines using AWS Glue, PySpark, Lambda, EC2, CodeBuild, and Spark, integrating data from S3, PostgreSQL, and Elasticsearch. • Built serverless data processing frameworks using AWS Lambda, Glue Workflows, and SNS, enabling automated monitoring, fault tolerance, and real-time alerting. • Optimized large-scale PySpark workloads using Broadcast Joins, Repartitioning, Salting, Caching, and skew handling techniques, reducing ETL execution time from 60 minutes to 25 minutes and lowering operational costs by 40%. • Developed a high-performance Change Data Capture (CDC) framework using Apache Iceberg, Spark SQL, and AWS Glue, improving incremental data processing efficiency and resource utilization by 30%. • Engineered and maintained data pipelines processing 10+ TB of data daily, delivering curated datasets to Elasticsearch for enterprise analytics and search workloads. • Led performance tuning initiatives using AWS CloudWatch and AWS X-Ray, identifying bottlenecks and improving pipeline execution performance by 50%. • Automated operational reporting and monitoring workflows using AWS Lambda and SNS, reducing manual effort and improving production visibility. • Implemented security and governance best practices using AWS KMS, IAM, encryption standards, and access controls to ensure compliance with enterprise data policies. • Collaborated with DevOps, QA, and business stakeholders during release cycles, supporting CI/CD deployments through AWS CodePipeline and ensuring production stability.
UST
Jan 20245 mosMay 2024

Pune District, Maharashtra, India

UST
Jan 20231 yrDec 2023

Pune District, Maharashtra, India

Google Developer Student Clubs MIT-WPU
Dec 20201 yr 6 mosMay 2022

Pune, Maharashtra

Applus+ IDIADA
Mar 20214 mosJun 2021

Pune Division, Maharashtra, India

● Develop, review, and optimize intra-company portal for file I/O management for CAE department to ease the process for mechanical engineers. ● Ensure maximum productivity by working on Utility Portal for managing various aspects of project management and to keep all project-related utilities at one place. ● Established end-to-end data pipeline and supplied data to frontend on interactive dashboards for live dynamic data visualization and added functionality for live data comparison of different modules. ● Deployed the live application for the department employees to use and received A grade feedback from the HOD.
HELO HEALTH
Nov 20203 mosJan 2021

Thane, Maharashtra, India

● Developed scalable Employee Identification & Diagnosis system for a back to office solution to provide real time insights of employees’ health-related data and also integrated facial attendance system. ● Devised and created a new algorithm which alerted administrator if health data patterns of an employee reach the danger level. ● Partnered with cross-functional UI development team for developing backed (Django & MongoDB) in a portal for administrator to manage all data in one place. ● This system was deployed across the metro cities of Mumbai & Delhi in various offices.

Education

MIT World Peace University

Bachelor of Technology - BTech, Computer Science Engineering

MIT World Peace University

20184 yrs2022

Join Atharva and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Atharva Jibhakate on Tal

Headline

Senior Data Engineer | Databricks | PySpark | Apache Spark | Airflow | Snowflake | AWS | ETL/ELT | Data Lakehouse | Data Warehousing | Data Platforms

About

Senior Data Engineer with 4+ years of experience designing and building scalable cloud-native data platforms and ETL/ELT pipelines. Currently working in the GrowthOS product team at Publicis Groupe, where I develop large-scale data solutions using Databricks, PySpark, Apache Spark, Airflow, Snowflake, Delta Lake, and AWS. My work focuses on data platform modernization, pipeline orchestration, data warehousing, metadata-driven engineering, and performance optimization. Previously at UST, I worked on enterprise healthcare insurance data platforms for Blue Cross Blue Shield Association, developing high-volume data pipelines on AWS using Glue, Lambda, Spark, Iceberg, and Redshift. Key expertise: • Databricks & Lakehouse Architecture • PySpark & Apache Spark • Airflow Orchestration • Snowflake & Data Warehousing • ETL / ELT Development • AWS Data Engineering • Data Modeling & Performance Optimization Databricks Certified Data Engineer Associate. Open to opportunities involving modern data platforms, large-scale analytics engineering, and cloud-native data architecture.

Location

Pune Division

Work experience

Data Engineer at Publicis Groupe, Feb 2025 - Present · 1 yr 6 mos, Pune District, Maharashtra, India, Product: GrowthOS Campaign Manager Platform Working as a Senior Data Engineer in the GrowthOS product team, responsible for designing and developing large-scale cloud-native data platforms and analytics solutions for global marketing and advertising use cases. • Design and develop scalable ETL/ELT pipelines using Databricks, PySpark, Apache Spark, Airflow, Delta Lake, Snowflake, and AWS. • Build and maintain data products following Lakehouse and Medallion Architecture principles (Bronze, Silver, Gold layers) to support analytics and business intelligence workloads. • Develop metadata-driven frameworks and reusable pipeline components to improve scalability, maintainability, and onboarding of new client data sources. • Lead migration and modernization initiatives from legacy Redshift-based architectures to Databricks Lakehouse, improving performance, reliability, and operational efficiency. • Orchestrate end-to-end workflows using Apache Airflow and Databricks Workflows, ensuring reliable execution and monitoring of production data pipelines. • Implement data quality validation, performance optimization, and operational monitoring strategies to support enterprise-scale datasets. • Deliver outbound data integrations to downstream platforms including Snowflake and Redis, enabling faster data access and analytical consumption. • Collaborate with global cross-functional teams across Product, Engineering, Analytics, and Data Science to deliver scalable data solutions. • Contribute to AI-powered metadata enrichment initiatives using Claude Sonnet, enhancing dataset discoverability, documentation quality, and governance capabilities.

Developer II - Enterprise Solutions at UST, Jun 2024 - Feb 2025 · 9 mos, Pune District, Maharashtra, India, Client: Blue Cross & Blue Shield Association • Designed and developed scalable ETL pipelines using AWS Glue, PySpark, Lambda, EC2, CodeBuild, and Spark, integrating data from S3, PostgreSQL, and Elasticsearch. • Built serverless data processing frameworks using AWS Lambda, Glue Workflows, and SNS, enabling automated monitoring, fault tolerance, and real-time alerting. • Optimized large-scale PySpark workloads using Broadcast Joins, Repartitioning, Salting, Caching, and skew handling techniques, reducing ETL execution time from 60 minutes to 25 minutes and lowering operational costs by 40%. • Developed a high-performance Change Data Capture (CDC) framework using Apache Iceberg, Spark SQL, and AWS Glue, improving incremental data processing efficiency and resource utilization by 30%. • Engineered and maintained data pipelines processing 10+ TB of data daily, delivering curated datasets to Elasticsearch for enterprise analytics and search workloads. • Led performance tuning initiatives using AWS CloudWatch and AWS X-Ray, identifying bottlenecks and improving pipeline execution performance by 50%. • Automated operational reporting and monitoring workflows using AWS Lambda and SNS, reducing manual effort and improving production visibility. • Implemented security and governance best practices using AWS KMS, IAM, encryption standards, and access controls to ensure compliance with enterprise data policies. • Collaborated with DevOps, QA, and business stakeholders during release cycles, supporting CI/CD deployments through AWS CodePipeline and ensuring production stability.

Cloud Infrastructure Associate II at UST, Jan 2024 - May 2024 · 5 mos, Pune District, Maharashtra, India

Big Data & Cloud Associate at UST, Jan 2023 - Dec 2023 · 1 yr, Pune District, Maharashtra, India

Google DSC Team Member at Google Developer Student Clubs MIT-WPU, Dec 2020 - May 2022 · 1 yr 6 mos, Pune, Maharashtra

Full Stack Developer Intern at Applus+ IDIADA, Mar 2021 - Jun 2021 · 4 mos, Pune Division, Maharashtra, India, ● Develop, review, and optimize intra-company portal for file I/O management for CAE department to ease the process for mechanical engineers. ● Ensure maximum productivity by working on Utility Portal for managing various aspects of project management and to keep all project-related utilities at one place. ● Established end-to-end data pipeline and supplied data to frontend on interactive dashboards for live dynamic data visualization and added functionality for live data comparison of different modules. ● Deployed the live application for the department employees to use and received A grade feedback from the HOD.

ML & SDE Intern at HELO HEALTH, Nov 2020 - Jan 2021 · 3 mos, Thane, Maharashtra, India, ● Developed scalable Employee Identification & Diagnosis system for a back to office solution to provide real time insights of employees’ health-related data and also integrated facial attendance system. ● Devised and created a new algorithm which alerted administrator if health data patterns of an employee reach the danger level. ● Partnered with cross-functional UI development team for developing backed (Django & MongoDB) in a portal for administrator to manage all data in one place. ● This system was deployed across the metro cities of Mumbai & Delhi in various offices.

Education

Bachelor of Technology - BTech, Computer Science Engineering, MIT World Peace University