Headline

Data Engineer | Building Scalable Data Pipelines with Python, Spark, Kafka & Databricks on Azure

About

Data Engineer with 4 years of experience designing scalable data pipelines and modern data platforms for enterprise financial and operational datasets. Experienced in building batch and streaming ETL pipelines using Python, SQL, Spark, and Kafka within Microsoft Azure cloud environments. Skilled in implementing Databricks-based data processing, Airflow DAG orchestration, and automated data validation frameworks to ensure reliable and high-quality data delivery. Proven ability to optimize large-scale data pipelines, build resilient architectures, and deliver structured datasets for analytics, reporting, and decision-making.

Location

Bengaluru

AI stack

anthropic apis

claude code

claude

notion

claudecode

OpenAI Playground

Work experience

Senior Data Engineer at Capgemini, Jul 2023 - Mar 2026 · 2 yrs 9 mos, Bengaluru, Karnataka, India

Data Engineer at Capgemini, Mar 2022 - Jun 2023 · 1 yr 4 mos, Karnataka, India, • Designed and implemented scalable financial data pipelines on Azure, ingesting transactional and operational datasets from enterprise systems and APIs into Azure Data Lake for centralized analytics. • Built batch and streaming ETL pipelines using Python, Spark, and Databricks, processing 4M+ financial records daily and transforming raw datasets into structured analytics-ready tables. • Implemented Apache Kafka streaming pipelines to capture transaction events and enable near real-time monitoring of financial and operational metrics. • Developed Airflow DAG workflows to orchestrate ingestion, transformation, validation, and reporting pipelines while managing job dependencies and scheduling.

Data Analyst at Saveo.in, Oct 2021 - Mar 2022 · 6 mos, Bengaluru, Karnataka, India, • Developed automated data ingestion and transformation scripts using Python and SQL to process operational sales and inventory datasets. • Built integrated datasets combining procurement, inventory, and sales data to support operational analytics and reporting. • Created Power BI dashboards backed by Python-processed datasets to enable data-driven decision making for supply chain and operations teams.

Education

Master of Business Administration - MBA, Data science, AI and ML, Jain (Deemed-to-be University)

B.com, Accounting and Finance, S N BHAT COLLEGE OF COMMERCE AND BUSINESS MANAGEMENT