Headline

Data Engineer || Ex-Infocian || Azure Data Engineer,Databricks ,Microsoft Fabric ,PySpark,ADF | I Design Scalable ETL Pipelines| 3x Microsoft Certified | Open to Sponsored Roles in EU / UK / UAE /canada/USA

About

I’ve built data pipelines. I’ve fixed them at 3 AM. I’ve optimised them so they don’t break again. I’m a Data Engineer with 4 years of hands-on experience building and running large-scale data platforms on Azure — currently at ZS Associates, one of the top global analytics and consulting firms serving pharma and life sciences clients worldwide. What I build: 10+ production-grade pipelines processing 500K+ records daily across Medallion architecture (Bronze → Silver → Gold) using PySpark, Databricks, Delta Lake, ADF, and Synapse. I’ve improved ingestion and transformation performance by up to 60%, cut compute costs by 40%, and delivered end-to-end POCs across the full Azure data stack including Power BI. Spark tuning — partitions, caching, AQE — is bread and butter for me. What I’ve survived: Real production failures. Debugging distributed Spark jobs at scale, root-cause analysis with Splunk, SCD Type 2 MERGE logic across 4-table dimension chains, data quality issues mid-pipeline. 99%+ uptime wasn’t a metric we reported — it was one we maintained. That experience shapes how I design systems today: observability-first, failure-tolerant, built to last. Prior to ZS, I was at Infosys on the Capital Group account — a US-based global asset management firm managing ~$2.5T in AUM. Financial data pipelines, tight SLAs, no room for error. Certifications: Microsoft Fabric Data Engineer Associate (DP-700) · Azure Data Engineer Associate (DP-203) · Databricks Data Engineer (in progress) Stack: Python · PySpark · SQL · Databricks · Delta Lake · Azure Data Factory · Synapse Analytics · Microsoft Fabric · Airflow · Power BI 🌍 Open to Senior Data Engineer roles internationally — UK, EU (Netherlands, Germany, Ireland), UAE, and USA. Comfortable with visa sponsorship processes and available to relocate. If you’re hiring engineers who understand both how systems are built and how they fail — let’s talk. I move fast.

Location

Pune Division

AI stack

claude code

langchain

lovable

n8n

notebooklm

openai

Work experience

Senior Software Engineer - Data Engineering at Inteliment Technologies, Nov 2025 - Present · 9 mos, Pune District, Maharashtra, India, ZS Associates (via Intelliment) — Data Engineer The work here sits at the intersection of traditional data engineering and GenAI automation — building on top of a 5M+ records/month ERP data platform while prototyping agentic workflows that let finance stakeholders query data in plain English. • Prototyped LangChain + LangGraph agentic workflows on Databricks enabling natural language querying over 5M+ monthly finance records — replacing manual reporting cycles for business stakeholders • Built LLM-powered data assistants for automated insight generation and pipeline documentation, cutting analyst turnaround on ad-hoc requests by ~35% • Engineered end-to-end ERP data pipelines powering 400+ finance dashboards with automated validation and reconciliation checks, reducing manual verification effort by ~30% • Debugged schema mismatches, ingestion failures, and transformation-layer defects across pipelines serving 50+ business users • Implemented structured monitoring and alerting frameworks, cutting incident resolution time by ~25% Stack: Databricks · LangChain · LangGraph · Python · SQL · Azure Data Factory · Delta Lake

Technology Analyst at Infosys, Oct 2025 - Nov 2025 · 2 mos, Pune District, Maharashtra, India, Senior-most IC role at Infosys. Owned the most complex data engineering work on the Capital Group account — a US-based global asset management firm managing ~$2.5T in AUM. Designed, optimised, and delivered production systems at scale. • Architected high-throughput ingestion pipelines pulling heterogeneous financial data from multiple sources into Azure Data Lake using Databricks and Delta Lake — improving ingestion speed and reliability by ~60% • Led migration of legacy data architecture to Microsoft Fabric, achieving a 40% performance boost and improving data accessibility for downstream analytics consumers • Optimised Spark jobs and Delta table partitioning strategies, reducing compute costs and query runtimes by ~60%; applied Spark SQL indexing to accelerate BI query performance • Orchestrated complex multi-dependency workflows via ADF and Apache Airflow with modular Python/SQL pipelines, robust error handling, and task backtracking — maintaining 99%+ pipeline uptime • Built end-to-end ETL pipelines in Microsoft Fabric using Dataflow Gen2 (SharePoint → Lakehouse), supporting analytics at enterprise scale • Evaluated Azure Cosmos DB (MongoDB API) for real-time ingestion migration from MongoDB Atlas — led performance benchmarking and indexing strategy • Delivered interactive Power BI dashboards for investment strategy teams; monitored pipeline health via Splunk with proactive root-cause analysis on data anomalies Stack: Databricks · Delta Lake · Azure Data Factory · Apache Airflow · Microsoft Fabric · Python · PySpark · Spark SQL · Power BI · Splunk · MongoDB · ADLS Gen2

Senior System Engineer at Infosys, Oct 2024 - Oct 2025 · 1 yr 1 mo, Pune District, Maharashtra, India, Stepped up from data ops into full ownership of pipeline development. Took on progressively complex transformation and orchestration work as a hands-on data engineer. • Built 10+ production-grade pipelines processing 500K+ records daily using Databricks and ADLS Gen2 • Designed MongoDB and SQL Server schemas that improved query performance by 30% • Led ADF data integration refactoring that reduced pipeline redundancy by 35% and improved transformation speed by 25% • Owned Spark performance tuning — partition sizing, caching strategies, broadcast joins — to optimise job runtimes across production workloads • Began taking ownership of end-to-end pipeline delivery: design, build, test, and handover with documentation Stack: Databricks · PySpark · Azure Data Factory · Delta Lake · SQL Server · MongoDB · ADLS Gen2

System Engineer at Infosys, Oct 2022 - Oct 2024 · 2 yrs 1 mo, Pune District, Maharashtra, India, Transitioned from training into the Capital Group data platform team. Started in a DataOps capacity — monitoring, incident response, pipeline health — and progressively took on development work as I built familiarity with the production environment. • Monitored pipeline health and performed root-cause analysis on ingestion and transformation failures across Azure Data Lake pipelines • Supported Databricks and ADF pipeline runs — triage, diagnosis, and escalation of data quality issues • Wrote SQL and PySpark scripts for data validation, reconciliation, and ad-hoc data extraction • Gained hands-on exposure to Medallion architecture, Spark internals, and Delta Lake operations in a live financial data environment • Gradually shifted toward development tasks — fixes, enhancements, and small-scale pipeline builds — under senior engineer mentorship Stack: PySpark · SQL · Azure Data Factory · Databricks · Delta Lake · Splunk

Intern at Verzeo, May 2021 - Aug 2021 · 4 mos, Gained hands-on experience in Python, SQL, and machine learning. Developed multiple ML models (classification/regression) on real datasets, applying data preprocessing, feature engineering, and evaluation best practices.

Education

Bachelor of Engineering, Information Technology, Savitribai Phule Pune University

Public links

github: https://github.com/Naynx7