Abhishek M profile photo 1 on Tal

Abhishek M

Data Engineer | Azure · Databricks · PySpark · AI agents | Building pipelines that cut processing time, not corners

Karnataka

4+ years of work experience

Komax CoDE India
Aug 20251 yr 1 moPresent

On-site

• Architected and deployed 11+ production AI document intelligence agents on Databricks Model Serving, reducing manual review effort by 70%. • Designed enterprise-scale Bronze-Silver-Gold lakehouse platform supporting 50+ mortgage attributes with lineage and governance controls. • Implemented SCD Type 1 & 2 strategies enabling audit-ready reporting and downstream analytical consumption. • Designed Photon-enabled serverless Spark architecture achieving 40% faster execution. • Established automated data quality framework across 30+ tables with schema monitoring and alerting.
Sigmoid
Jan 20258 mosAug 2025

Bengaluru

◦ ETL Pipelines: Developed and maintained scalable ETL workflows using Python, PySpark, Apache Airflow, and Pandas to process large-scale enterprise data. ◦ Cloud Platforms: Implemented data ingestion and transformation pipelines on Azure and AWS cloud environments leveraging services like Azure Data Factory, Databricks, and AWS S3. ◦ Performance Optimization: Enhanced ETL pipeline efficiency, reducing data processing time by 35% via code optimization and distributed processing. ◦ Data Integration: Ingested and transformed data from diverse sources including REST APIs and cloud storage; ensured data reliability and integrity. ◦ Version Control & DevOps: Utilized Azure DevOps and GitHub for source control, continuous integration, and deployment automation. ◦ Big Data Technologies: Worked with Apache Spark and Kafka for streaming and batch data processing scenarios. ◦ Tech Stack: Python, PySpark, Apache Airflow, Azure Data Factory, Azure Databricks, AWS S3, Kafka, SQL,Azure DevOps, GitHub
Wipro
May 20222 yrs 9 mosJan 2025

Pune, Maharashtra, India

◦ ETL Development: Built and optimized ETL pipelines using Azure Data Factory, PySpark, and SQL for data processing and analytics. ◦ Legacy Migration: Migrated SAS workflows to PySpark, significantly improving pipeline scalability and reducing operational costs. ◦ SQL: Developed complex SQL queries and optimized ETL datasets for reporting and analytics. ◦ Collaboration & Agile: Worked closely with cross-functional teams in Agile environments, managing code in Bitbucket and supporting CI/CD pipelines. ◦ Tech Stack: Azure Data Factory, PySpark, SQL Server, SAS, Python, GitHub, Azure DevOps
VST Infotech
Dec 20216 mosMay 2022

Remote

Abhishek’s AI Stack

antigravity
antigravity
codex
codex
cursor
cursor

Education

Smt.Kashibai Navale college of Engineering

Bachelor of Engineering - BE, Computer Science

Smt.Kashibai Navale college of Engineering

20184 yrs2022

Join Abhishek and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Abhishek M on Tal

Headline

Data Engineer | Azure · Databricks · PySpark · AI agents | Building pipelines that cut processing time, not corners

About

As a Data Engineer at Komax, I design and maintain scalable data pipelines on Azure. Previously at Wipro and Sigmoid, I've built ETL systems that handle millions of records, reduced processing time by 35%, and cut manual intervention by 75% through workflow parameterization. I work across Azure Data Factory, Databricks, PySpark, Airflow, and SQL with hands-on experience in both batch and streaming scenarios. Microsoft Certified (DP-203). I also write about data engineering on Medium to break down concepts I wish someone had explained to me earlier.

Location

Karnataka

AI stack

antigravity

codex

cursor

Work experience

Data Engineer at Komax CoDE India, Aug 2025 - Present · 1 yr 1 mo, On-site, • Architected and deployed 11+ production AI document intelligence agents on Databricks Model Serving, reducing manual review effort by 70%. • Designed enterprise-scale Bronze-Silver-Gold lakehouse platform supporting 50+ mortgage attributes with lineage and governance controls. • Implemented SCD Type 1 & 2 strategies enabling audit-ready reporting and downstream analytical consumption. • Designed Photon-enabled serverless Spark architecture achieving 40% faster execution. • Established automated data quality framework across 30+ tables with schema monitoring and alerting.

SDE (Data Engineer : Project-Based role - 8 months) at Sigmoid, Jan 2025 - Aug 2025 · 8 mos, Bengaluru, ◦ ETL Pipelines: Developed and maintained scalable ETL workflows using Python, PySpark, Apache Airflow, and Pandas to process large-scale enterprise data. ◦ Cloud Platforms: Implemented data ingestion and transformation pipelines on Azure and AWS cloud environments leveraging services like Azure Data Factory, Databricks, and AWS S3. ◦ Performance Optimization: Enhanced ETL pipeline efficiency, reducing data processing time by 35% via code optimization and distributed processing. ◦ Data Integration: Ingested and transformed data from diverse sources including REST APIs and cloud storage; ensured data reliability and integrity. ◦ Version Control & DevOps: Utilized Azure DevOps and GitHub for source control, continuous integration, and deployment automation. ◦ Big Data Technologies: Worked with Apache Spark and Kafka for streaming and batch data processing scenarios. ◦ Tech Stack: Python, PySpark, Apache Airflow, Azure Data Factory, Azure Databricks, AWS S3, Kafka, SQL,Azure DevOps, GitHub

Data Engineer at Wipro, May 2022 - Jan 2025 · 2 yrs 9 mos, Pune, Maharashtra, India, ◦ ETL Development: Built and optimized ETL pipelines using Azure Data Factory, PySpark, and SQL for data processing and analytics. ◦ Legacy Migration: Migrated SAS workflows to PySpark, significantly improving pipeline scalability and reducing operational costs. ◦ SQL: Developed complex SQL queries and optimized ETL datasets for reporting and analytics. ◦ Collaboration & Agile: Worked closely with cross-functional teams in Agile environments, managing code in Bitbucket and supporting CI/CD pipelines. ◦ Tech Stack: Azure Data Factory, PySpark, SQL Server, SAS, Python, GitHub, Azure DevOps

Software Programmer at VST Infotech, Dec 2021 - May 2022 · 6 mos, Remote

Education

Bachelor of Engineering - BE, Computer Science, Smt.Kashibai Navale college of Engineering