Headline

Azure Data Engineer | Databricks | PySpark | ADF | Delta Lake | Azure Synapse | ADLS Gen2 | ETL/ELT Pipelines | DP-203 (In Progress) | TCS

About

I'm an Azure Data Engineer with 4+ years of hands-on experience designing and scaling enterprise cloud data solutions at Tata Consultancy Services. My work sits at the intersection of cloud architecture, data modelling, and pipeline engineering — building systems that turn raw, fragmented data into reliable, analytics-ready assets. My core toolkit is Azure-native: Azure Data Factory (ADF), Azure Databricks, ADLS Gen2, Azure Synapse Analytics, and Delta Lake. I architect end-to-end ETL/ELT pipelines using PySpark and Spark SQL, apply Medallion Architecture (Bronze / Silver / Gold) patterns to organise lakehouse layers, and implement data quality and governance frameworks using Unity Catalog and Delta table validation. Some of what I've delivered at TCS: → Engineered 15+ ADF pipelines consolidating data from 15+ source systems into a unified ADLS Gen2 data lake, enabling Power BI self-service analytics → Optimised PySpark workloads to achieve a 35% gain in execution efficiency and reduced Azure compute costs → Re-architected SQL query patterns in Azure Synapse, cutting reporting latency by 40% → Maintained 99.8% pipeline reliability across production ADF/Control-M schedules using Azure Monitor alerting → Built CI/CD deployment workflows on Azure DevOps to standardise and accelerate release cycles I'm passionate about clean data architecture — particularly Medallion/Lakehouse patterns, schema-on-write governance, and building pipelines that are observable, testable, and maintainable at scale. I'm currently working towards my Microsoft Azure Data Engineer Associate (DP-203) certification. If you're building cloud data platforms, modernising legacy ETL infrastructure, or scaling analytics on Azure — I'd love to connect.

Location

Greater Kolkata Area

Work experience

Analytics and business intelligence engineer at Tata Consultancy Services, Sep 2023 - Present · 2 yrs 10 mos, Kolkata, ▸Architected end-to-end Medallion Architecture (Bronze / Silver / Gold) data lakehouse on Azure Databricks using PySpark, Spark SQL, and Delta Lake backed by ADLS Gen2 — processing 5TB+ of enterprise data daily with 99.8% pipeline reliability. ▸Engineered 15+ parameterised ADF pipelines with dynamic linked services, consolidating data from 15+ disparate source systems into a centralised landing layer and reducing manual ingestion effort by ~60%. ▸Optimised PySpark notebooks and Spark SQL workloads through cluster auto-scaling and caching strategies, achieving a 35% improvement in execution efficiency and measurable reduction in monthly Azure compute costs. ▸Re-architected legacy SQL queries and stored procedures into Azure Synapse Analytics patterns, delivering a 40% reduction in downstream reporting query latency for business analysts. ▸Implemented CI/CD deployment pipelines on Azure DevOps (Repos / Boards / Pipelines) with structured Git branching (feature/dev/main), standardising production release cycles and reducing deployment lead time. ▸Built multi-stage Delta table validation frameworks enforcing schema compliance, RBAC access controls, and PII data masking — achieving zero corrupted-data incidents in production data lakes. ▸Integrated Azure Key Vault for secrets management across ADF linked services and Databricks clusters, hardening enterprise security posture and meeting data governance compliance requirements. ▸Partnered with cross-functional stakeholders, solution architects, and product analysts within Agile/Scrum sprints to translate business analytics requirements into scalable, high-availability cloud data models.

Assistant System Engineer at Tata Consultancy Services, Sep 2022 - Sep 2023 · 1 yr 1 mo, Kolkata, West Bengal, India

Assistant system engineer- trainee at Tata Consultancy Services, Sep 2021 - Sep 2022 · 1 yr 1 mo, Kolkata, West Bengal, India

Education

Bachelor of Technology - BTech, Electrical, Electronic and Communications Engineering Technology/Technician, RNS Institute of Technology - India

Kendriya vidyalaya No. 2 AFS Hindon