Headline

Azure Data Engineer | Building Scalable Lakehouses with Microsoft Fabric, Databricks & ADF | PySpark · Delta Lake · ETL/ELT

About

I build data platforms that engineers can maintain and stakeholders can trust — not just pipelines that run. With 3+ years in Azure data engineering, I specialize in designing and delivering cloud-native data platforms using Azure Data Factory, Azure Databricks, PySpark, Delta Lake, ADLS Gen2, and Microsoft Fabric. My focus is on turning raw, fragmented data into reliable, well-modeled datasets that power self-serve analytics, reporting, and forecasting. What I've built in production: Metadata-driven ELT pipelines with control-table orchestration and watermark-based incremental ingestion Medallion architecture (Bronze → Silver → Gold) with schema enforcement, deduplication, audit logging, and pipeline health monitoring Ingestion frameworks for structured and semi-structured sources with validation, error handling, and version-controlled deployments across dev, staging, and production Governance-aligned practices including RBAC, data lineage awareness, and managed environment promotion with validation gates and business acceptance What I'm exploring with Microsoft Fabric: I've been building case studies around modern data engineering patterns — trusted file ingestion, schema validation before load, upsert-based Lakehouse curation, quarantine handling for unsupported inputs, privacy-aware ingestion, and archive-before-delete lifecycle controls. These reflect how I approach data engineering: data quality boundaries, operational safety, and pipelines that are straightforward to monitor, explain, and scale. I'm actively looking for Data Engineering opportunities involving Azure, Microsoft Fabric, Databricks, Lakehouse architecture, and ETL/ELT platform modernization - l where building reliable, well-governed, and business-aligned data systems is the goal. 📬 Open to connecting with data engineering teams, hiring managers, and fellow practitioners

Location

Chennai

Work experience

Associate Data Engineer at Ensono, Jan 2023 - Present · 3 yrs 7 mos, Chennai, . Designed and implemented a Medallion Lakehouse architecture (Bronze → Silver → Gold) on Azure Databricks and Delta Lake for a US-based e-commerce platform, enabling reliable KPI delivery for analytics and reporting. . Built metadata-driven ELT pipelines in Azure Data Factory to ingest data from SQL Server, REST APIs, and SFTP into ADLS Gen2, with control-table orchestration and watermark-based incremental processing. . Enforced schema validation, deduplication, and upsert logic in the Silver layer using PySpark and Delta Lake features including Merge, Time Travel, and Schema Evolution. . Delivered curated Gold-layer datasets surfacing KPIs - Sales Revenue, Refund Percentage, and Customer Retention - contributing to a 25% improvement in sales forecast accuracy. . Implemented data quality controls including row count validation, checksum verification, null-threshold checks, and audit logging for pipeline traceability and operational reliability. . Established governance controls via Unity Catalog - including RBAC, data lineage tracking, pipeline SLA alerting via Logic Apps, and monitored job health across production workflows. . Deployed ADF pipelines and Databricks notebooks via Azure DevOps CI/CD; optimized Spark jobs through broadcast joins, partition tuning, and Z-Ordering - reducing pipeline runtime by 30% and compute cost by 15%.

Trainee at StackRoute, Mar 2022 - Sep 2022 · 7 mos, Bengaluru, Wipro-assigned Java Full Stack with AWS Cloud training program. Worked on hands-on projects using Java, JSP, Servlets, SQL, and HTML.

Project Trainee - Cloud AWS at Wipro, Feb 2022 - Sep 2022 · 8 mos, Chennai, Developed and deployed a job portal application using JSP, Servlets, MySQL, and JDBC. • Deployed on AWS Elastic Beanstalk with CloudWatch for monitoring and RDS Aurora for database management. • Managed source control and team collaboration via GitHub.

Education

Bachelor of Technology, Mechanical Engineering, Hindustan University