Headline

Principal Data Platform Engineer | AWS & Azure | Data Lakehouse | Apache Iceberg | PySpark | Azure DevOps | Docker | Cloud Architecture | CI/CD | AI-Assisted Engineering | Ex Paytm

About

I’m a Data Engineer / Data Platform Engineer with 9+ years of experience building scalable Big Data and Cloud Data Platforms in FinTech and enterprise analytics domains. I specialize in designing modern Lakehouse architectures, building high-performance ETL/ELT pipelines, and delivering analytics-ready datasets that power business-critical dashboards, KPI reporting, and product decision-making. I have worked in high-scale environments at One97 Communications (Paytm) and Paytm Payments Bank, where I gained hands-on exposure to enterprise-grade data ecosystems, governance frameworks, and large-scale reporting platforms. Currently, I’m working at Algoworks Technology Pvt. Ltd., where I design and implement an AWS Lakehouse platform using Apache Iceberg on S3 following a Medallion Architecture (Bronze/Silver/Gold). I integrate complex data sources like OneStream (REST APIs), JDE, and TMS, and deliver curated Gold datasets for enterprise analytics and Power BI reporting. What I Do Best ✅ Data Platform Engineering Lakehouse Architecture ✅ Building scalable ingestion + transformation frameworks ✅ Batch + incremental pipeline design using Spark ✅ Data modeling, optimization & governance-ready datasets ✅ Data Quality, reconciliation & monitoring frameworks Core Skills: AWS • Azure • PySpark • Apache Iceberg • Spark • Python • SQL • Docker • GitHub Actions • CI/CD • PostgreSQL • Cloud Architecture Key Highlights: Improved query performance by 40% by driving migration and optimization using Trino. Reduced cluster overhead by 30% through optimized ETL execution and tuning. Supported migration of workloads from on-prem to AWS, improving scalability and cost efficiency. Delivered trusted KPI datasets powering business dashboards through strong reconciliation & validation frameworks Built an end-to-end Azure cloud deployment platform with automated CI/CD for a modern containerized application using Docker and Azure Container Apps. Career Goal I'm passionate about building scalable, secure, and cost-effective cloud platforms that combine modern data engineering, cloud infrastructure, and DevOps practices. I enjoy solving complex engineering challenges, mentoring teams, and adopting AI-assisted engineering to improve development productivity while maintaining high engineering standards. I'm always open to connecting with professionals in Data Engineering, Cloud Architecture, Platform Engineering, Data Platforms, and DevOps.

Location

Delhi

AI stack

opencode

claudecode

Work experience

Principal Engineer at Algoworks, Jan 2026 - Present · 8 mos, Remote, Designed and implemented an enterprise AWS Data Lakehouse using Apache Iceberg on Amazon S3 with Medallion Architecture (Bronze, Silver, Gold). Developed scalable ETL/ELT pipelines using AWS Glue, PySpark, and Spark SQL, integrating enterprise systems including OneStream, JDE, and TMS. Built incremental ingestion and MERGE/UPSERT frameworks for historical corrections, schema evolution, and analytics-ready datasets. Designed and implemented a cloud-native Azure deployment platform for a pilot application using Azure Container Apps, Azure Container Registry, Azure Storage, PostgreSQL, Docker, and GitHub Actions. Built end-to-end CI/CD pipelines to automate container build, deployment, and environment configuration. Leveraged AI tools (Claude) to accelerate infrastructure design, deployment automation, and CI/CD development while validating all generated solutions using engineering best practices.

Technical Lead at Paytm Payments Bank, Apr 2023 - Jan 2026 · 2 yrs 10 mos, Noida, Led modernization of enterprise data platforms supporting FinTech reporting and regulatory analytics. Migrated 200+ production workloads from Hortonworks to AWS, improving scalability and operational efficiency. Improved analytical query performance by approximately 40% through Hive-to-Trino migration and workload optimization. Optimized Spark and Hive workloads, reducing cluster utilization and infrastructure costs by 30–40%. Designed monitoring, partitioning, and performance optimization strategies to improve production SLA adherence.

Senior Data Engineer at Paytm Payments Bank, Nov 2020 - Apr 2023 · 2 yrs 6 mos, Noida, Uttar Pradesh, India

Data Engineer at Paytm Payments Bank, Oct 2019 - Nov 2020 · 1 yr 2 mos, Noida, Uttar Pradesh, India

Data Engineer at One97 Communications Limited, Aug 2016 - Sep 2019 · 3 yrs 2 mos, Noida, Designed scalable ETL pipelines for enterprise reporting and analytics using Spark, Hive, and SQL. Built reusable ingestion and transformation frameworks supporting multiple business domains. Optimized large-scale batch processing workflows for improved reliability and performance. Collaborated with analytics and business teams to deliver trusted datasets for reporting and decision-making. Participated in code reviews and platform improvements to enhance engineering quality.

Education

Master of Science - MS, Data Science, Liverpool John Moores University

Executive PG in Data Science, Deep Learning, International Institute of Information Technology Bangalore