Headline

Data Engineer @ M2P Fintech | PySpark · AWS · Azure · Kafka · ETL · SQL · API Integration · Data Migration | Turning Raw Data Into Business Intelligence

About

I build data pipelines that people actually rely on. Data Engineer with 1.5+ years of production experience across fintech and automotive tech. I specialise in ETL pipeline development, PySpark optimisation, and incremental data loading — working with Python, SQL, AWS, Azure, and Kafka to move and transform data at scale. At M2P Fintech I've built ingestion frameworks processing 10GB+ daily, applied Spark optimisation techniques (partitioning, salting, broadcast joins) to eliminate bottlenecks, and implemented Kafka-based incremental loads that replaced expensive full reloads. I care about clean, reliable, analytics-ready data — and I'm actively looking for my next Data Engineer role where I can go deeper on distributed systems and large-scale pipeline architecture. 📩 anirudhbihani27@gmail.com

Location

Bengaluru

AI stack

cursor

hugging face

langchain

ollama

Work experience

Data Engineer at M2P Fintech, Sep 2024 - Present · 1 yr 10 mos, On-site, Building and optimising end-to-end data pipelines that process 10GB+ daily across multiple fintech source systems — delivering clean, analytics-ready data for business reporting and decision-making. - Architected scalable ETL and data migration pipelines using PySpark, Python, SQL, and Pentaho — supporting batch and incremental ingestion workflows across 5+ source systems, reducing data latency by replacing full reloads with API-based incremental loading via Kafka. - Optimised Spark job performance through data partitioning, salting, broadcast joins, and skew mitigation — stabilising long-running jobs and eliminating processing bottlenecks in high-volume production environments. - Managed analytics-ready datasets on Amazon S3 using query-optimised, partitioned Parquet storage — enabling downstream reporting teams to run faster, more reliable queries on structured financial data. - Automated CI/CD pipeline deployments and orchestrated data workflows using AWS services — ensuring high availability, fault tolerance, and consistent environment parity across dev, staging, and production.

Software Engineer Intern at KPIT, Jun 2023 - Oct 2023 · 5 mos, Enhanced proficiency in Modern C++ through structured training and project work. Developed optimized, resilient code using advanced C++ features.

Trainee at KPIT, Jun 2023 - Sep 2023 · 4 mos, Pune, Maharashtra, India

Education

Bachelor of Technology - BTech, Information Technology, MIT ADT University

Master of Information Technology, MIT ADT University