Mihir Shah profile photo 1 on Tal

Mihir Shah

Senior Data Engineer | AWS, Snowflake, Databricks Specialist | Building Scalable ETL & Cloud Data Platforms

Pune Division

8+ years of work experience

Publicis Sapient
Feb 20251 yr 7 mosPresent

Pune, Maharashtra, India

Collaborated with senior engineers and clients to design end-to-end AWS-based ETL architecture leveraging S3, Glue, Lambda, Step Functions, IAM, and CloudWatch for scalable and maintainable data pipelines. Converted complex SQL stored procedures into AWS Glue Jobs using Slingshot to optimize time and space complexity, improving overall pipeline performance. Built and managed workflow orchestration using AWS Step Functions, enabling reliable multi-step data processing with error handling and retry mechanisms. Integrated diverse data sources such as S3, SQL Server, Workday, and HEMS, and designed ETL pipelines supporting 22 critical business reports. Implemented cost-optimized storage patterns on Amazon S3, ensured compliance with VPC configurations, and used AWS Secret Manager to enhance credential security. Partnered with cross-functional stakeholders to propose improvements, assess project risks, and develop mitigation plans for smoother project execution.
KPMG
Oct 20222 yrs 5 mosFeb 2025

Pune, Maharashtra, India

Designed and implemented relational database systems using 3NF to ensure strong data integrity, reduced redundancy, and improved query performance. Conducted multiple POCs to simplify complex pipelines—for example, using Databricks Autoloader for SCD Type 2 instead of traditional approaches. Optimized data processing by building efficient ETL pipelines and streamlining database structures; collaborated on ETL tasks using AWS Cloud Templates and ensured pipeline reliability. Migrated a terabyte-scale on-prem data warehouse to Amazon Redshift, achieving a 15% reduction in infrastructure costs. Developed and maintained scalable data ingestion and transformation pipelines using AWS Glue and Spark. Managed AWS resources including EC2, S3, and IAM to ensure secure and efficient operations. Built real-time data pipelines using Databricks Autoloader and integrated processed data into a unified platform for enhanced customer engagement. Implemented CI/CD using Databricks Asset Bundles to streamline deployments across environments. Designed ETL workflows with dynamic job parameters to route data either to a FHIR server or PostgreSQL based on business needs. Coordinated closely with Architects and Business Analysts to gather requirements and deliver end-to-end data solutions.
Fractal
Sep 20211 yr 4 mosDec 2022

Mumbai, Maharashtra, India

Conducted detailed evaluations of third-party data handling solutions to ensure compliance with internal standards and stakeholder requirements. Executed large-scale data conversions for seamless integration into AWS Glue and AWS Lambda pipelines. Collaborated on ETL processes, ensuring data integrity, smooth transformations, and stable end-to-end pipeline performance. Designed and implemented efficient database models and solutions to support reliable data storage and retrieval. Worked closely with system teams to ensure successful integration, performance tuning, and alignment with project goals.
Allianz Technology
Apr 20215 mosAug 2021

Pune, Maharashtra, India

Orchestrated large-scale software deployments, including feature testing, debugging, and code corrections to ensure smooth releases. Collaborated with business analysts, development teams, and infrastructure specialists to deliver high-availability, mission-critical applications. Worked with DevOps engineers and DBAs to build CI/CD pipelines using Jenkins, reducing deployment delays and improving delivery efficiency. Modernized legacy codebases by modularizing and optimizing components, resulting in reduced operational costs and improved system performance. Partnered with project managers, QA teams, and customers to resolve complex technical issues and ensure seamless project execution. Tested and documented troubleshooting approaches, contributing reusable solutions to the internal knowledge base. Performed expert code reviews to maintain coding standards, improve quality, and support team development.
Allianz Technology
Aug 20182 yrs 9 mosApr 2021

India

> Coding in PL/SQL > Monitored automated build and continuous software integration process to drive build/release failure resolution. > Versed in complete SDLC from preliminary needs analysis to enterprise-wide deployment and support. > Production support. Learnings: > PostgreSQL > Google Cloud Platform > JCL > Git, Github

Mihir’s AI Stack

antigravity
antigravity
opencode
opencode

Education

Modern Education Society's College of Engineering, Pune

Bachelor of Engineering, Computer Engineering

Modern Education Society's College of Engineering, Pune

20144 yrs2018

Join Mihir and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Mihir Shah on Tal

Headline

Senior Data Engineer | AWS, Snowflake, Databricks Specialist | Building Scalable ETL & Cloud Data Platforms

About

Diligent Senior Data Engineer with over 8 years of experience in developing scalable, data-driven solutions and a strong foundation in data engineering, showcasing expertise in designing and implementing complex data pipelines. Proficient in AWS cloud technologies, utilizing services such as S3, Glue, Lambda, Step Functions, and Redshift to create robust data architectures. Currently exploring Databricks and Snowflake to enhance capabilities in advanced analytics. Committed to transforming raw data into actionable insights that drive business growth and inform strategic decision-making.

Location

Pune Division

AI stack

antigravity

opencode

Work experience

Senior Associate Data Engineering L1 at Publicis Sapient, Feb 2025 - Present · 1 yr 7 mos, Pune, Maharashtra, India, Collaborated with senior engineers and clients to design end-to-end AWS-based ETL architecture leveraging S3, Glue, Lambda, Step Functions, IAM, and CloudWatch for scalable and maintainable data pipelines. Converted complex SQL stored procedures into AWS Glue Jobs using Slingshot to optimize time and space complexity, improving overall pipeline performance. Built and managed workflow orchestration using AWS Step Functions, enabling reliable multi-step data processing with error handling and retry mechanisms. Integrated diverse data sources such as S3, SQL Server, Workday, and HEMS, and designed ETL pipelines supporting 22 critical business reports. Implemented cost-optimized storage patterns on Amazon S3, ensured compliance with VPC configurations, and used AWS Secret Manager to enhance credential security. Partnered with cross-functional stakeholders to propose improvements, assess project risks, and develop mitigation plans for smoother project execution.

Consultant at KPMG, Oct 2022 - Feb 2025 · 2 yrs 5 mos, Pune, Maharashtra, India, Designed and implemented relational database systems using 3NF to ensure strong data integrity, reduced redundancy, and improved query performance. Conducted multiple POCs to simplify complex pipelines—for example, using Databricks Autoloader for SCD Type 2 instead of traditional approaches. Optimized data processing by building efficient ETL pipelines and streamlining database structures; collaborated on ETL tasks using AWS Cloud Templates and ensured pipeline reliability. Migrated a terabyte-scale on-prem data warehouse to Amazon Redshift, achieving a 15% reduction in infrastructure costs. Developed and maintained scalable data ingestion and transformation pipelines using AWS Glue and Spark. Managed AWS resources including EC2, S3, and IAM to ensure secure and efficient operations. Built real-time data pipelines using Databricks Autoloader and integrated processed data into a unified platform for enhanced customer engagement. Implemented CI/CD using Databricks Asset Bundles to streamline deployments across environments. Designed ETL workflows with dynamic job parameters to route data either to a FHIR server or PostgreSQL based on business needs. Coordinated closely with Architects and Business Analysts to gather requirements and deliver end-to-end data solutions.

Data Engineer at Fractal, Sep 2021 - Dec 2022 · 1 yr 4 mos, Mumbai, Maharashtra, India, Conducted detailed evaluations of third-party data handling solutions to ensure compliance with internal standards and stakeholder requirements. Executed large-scale data conversions for seamless integration into AWS Glue and AWS Lambda pipelines. Collaborated on ETL processes, ensuring data integrity, smooth transformations, and stable end-to-end pipeline performance. Designed and implemented efficient database models and solutions to support reliable data storage and retrieval. Worked closely with system teams to ensure successful integration, performance tuning, and alignment with project goals.

Senior Software Engineer at Allianz Technology, Apr 2021 - Aug 2021 · 5 mos, Pune, Maharashtra, India, Orchestrated large-scale software deployments, including feature testing, debugging, and code corrections to ensure smooth releases. Collaborated with business analysts, development teams, and infrastructure specialists to deliver high-availability, mission-critical applications. Worked with DevOps engineers and DBAs to build CI/CD pipelines using Jenkins, reducing deployment delays and improving delivery efficiency. Modernized legacy codebases by modularizing and optimizing components, resulting in reduced operational costs and improved system performance. Partnered with project managers, QA teams, and customers to resolve complex technical issues and ensure seamless project execution. Tested and documented troubleshooting approaches, contributing reusable solutions to the internal knowledge base. Performed expert code reviews to maintain coding standards, improve quality, and support team development.

Software Engineer at Allianz Technology, Aug 2018 - Apr 2021 · 2 yrs 9 mos, India, > Coding in PL/SQL > Monitored automated build and continuous software integration process to drive build/release failure resolution. > Versed in complete SDLC from preliminary needs analysis to enterprise-wide deployment and support. > Production support. Learnings: > PostgreSQL > Google Cloud Platform > JCL > Git, Github

Education

Bachelor of Engineering, Computer Engineering, Modern Education Society's College of Engineering, Pune