Headline

Senior Data Engineer at EY | GCP Certified Professional Data Engineer | Retail & BFSI Experience

About

I’m a Senior Data Engineer with 5+ years of experience designing and building scalable data pipelines, cloud-native data platforms, and data processing solutions. I specialize in transforming complex data into reliable, high-quality datasets that support analytics, business intelligence, and AI-driven applications. I’m a GCP Professional Data Engineer with hands-on experience across Python, SQL, PySpark, Apache Spark, Kafka, Airflow, BigQuery, Dataproc, Dataflow, and Cloud Composer. My experience includes designing ETL/ELT pipelines, optimizing large-scale data processing, implementing data quality and validation frameworks, and contributing to cloud migration and data platform initiatives. In my current role, I work on enterprise-scale data platforms and reusable data engineering frameworks, focusing on scalability, performance, reliability, and automation. I also have hands-on exposure to Generative AI, RAG, LangChain, prompt engineering, and AI-enabled data solutions, and I'm actively expanding my expertise in AI for Data Engineering. I enjoy solving complex data problems, exploring modern technologies, and building scalable solutions that create measurable business impact.

Location

Bengaluru

AI stack

anthropic apis

claude code

claude

codex

cursor

mcp

Work experience

Senior Data Engineer at EY, Mar 2025 - Present · 1 yr 8 mos, Bengaluru, Client: American Express •Designed the cloud architecture and reusable data platform for enterprise migration to GCP, reducing pipeline development effort by 90%, accelerating onboarding of new reports, and standardizing development across multiple engineering teams. •Built a reusable data engineering platform using Python, PySpark, and GCP services, enabling end-to-end data ingestion, transformation, validation, and loading while supporting the development of 100+ production data pipelines. •Built and exposed a Python-based REST API for data ingress/egress between Oracle and GCP, and to trigger Airflow pipeline runs. •Deployed containerized data services and pipelines, leveraging Kubernetes based pod orchestration for scalable, production grade rollouts. •Integrated centralized logging, automated failure detection, and built-in unit testing, reducing production defects by 80% and improving issue resolution time by 90%.

Systems Engineer (Data Engineer) at Tata Consultancy Services, Dec 2021 - Feb 2025 · 3 yrs 3 mos, Kolkata, •Led the design and implementation of ELT/ETL pipelines using GCP Dataflow to extract data (CSV, JSON, Parquet, Avro, XML) and perform data manipulation in BigQuery. •Streamlined ETL pipelines to load raw data from GCS into BigQuery using PySpark, DataProc and Cloud Composer, ensuring 99% data quality and removing corrupt and duplicate records improving processing efficiency by 20%. •Devised and implemented data partitioning and clustering strategies in BigQuery for a 2.5 TB dataset, optimizing query performance and reducing storage costs. •Collaborated with business stakeholders, data architects and analysts to understand their requirements and ensure timely delivery of accurate data. •Enhanced performance and efficiency of long-running Spark jobs by 25% through meticulous tuning and optimization, employing industry best practices for maximum effectiveness.

ASE - Trainee at Tata Consultancy Services, Jun 2021 - Dec 2021 · 7 mos, Kolkata

Coding Instructor at Toppr, Apr 2021 - Jun 2021 · 3 mos, Kolkata, West Bengal, India, Worked as a coding instructor teaching Python and Web Development to students.

Web Software Developer at Agitechavids, Apr 2020 - Jun 2020 · 3 mos, Kolkata, West Bengal, 1.Worked as a web developer on their upcoming product "Enlighty". 2.UI implementations of the website using HTML, CSS and JavaScript.

Education

Bachelor of Technology - BTech, Computer Science, Amity University Kolkata

CBSE, High School Diploma, Hariyana Vidya Mandir