Headline

Azure Data Engineer | PySpark | Azure Data Factory | Databricks | Delta Lake | ADLS Gen2 | SQL | Power BI | Python | ETL

About

Azure Data Engineer with 4+ years of hands-on experience in designing and implementing scalable data pipelines using Azure Data Factory, Databricks (PySpark), and Data Lake Gen2. Proficient in SQL, Python, and Power BI, with a solid understanding of ETL processes, Delta Lake architecture, and performance tuning. Committed to delivering high-quality, cloud-based data solutions that support business insights. Certified by AnalytixLabs in Excel, SQL, Power BI, and Python, I’ve built interactive dashboards and conducted deep customer segmentation, including identifying high-spending demographics. My internship at SkillForge strengthened my skills in financial analysis and credit behavior modeling. With a Bachelor of Commerce and a keen interest in Data Engineering/analytics, I blend business acumen with technical expertise. I’m always eager to drive growth through insights. Let’s connect and explore data-driven innovation together!

Location

Noida

Work experience

Azure Data Engineer at BLACKHUBBLE, Oct 2024 - Present · 1 yr 9 mos, Bengaluru, • Architected Azure-based financial data platform ingesting 5M+ daily transactions using ADF, Apache Spark, PySpark and Azure Databricks into ADLS Gen2 with Medallion Architecture, enabling automated P&L dashboards and reconciliation. • Built scalable PySpark and Spark SQL pipelines on Databricks, leveraging Delta Lake and Spark DataFrames to normalize multi-currency data, deduplicate late events for customer history tracking. • Designed real-time fraud detection pipeline using Apache Spark Structured Streaming and Delta Lake, processing transactions with sub-2-second latency and reducing chargebacks through anomaly detection on velocity, geo location, and transaction patterns. • Optimized large-scale data processing using Spark, PySpark, and SQL, reducing Delta Lake MERGE time significantly and resolving 500M+ row join issues with partitioning, broadcast joins, and efficient ETL/ELT strategies. • Developed robust data models using Star Schema in Azure SQL Database and MySQL, ensuring accurate financial reconciliation, data warehousing, and reporting with validation checks, alerting, and compliance-ready pipelines.

Azure Data Engineer at PowerLaw Pvt. Ltd., Jul 2022 - Sep 2024 · 2 yrs 3 mos, Ghaziabad, • Designed and implemented scalable ADF pipelines to ingest large-scale raw data (customer behaviour, vendor workflows) into Azure Data Lake Storage Gen2 using scheduled and event-based triggers. • Processed and transformed raw data using PySpark in Azure Databricks, applying data cleansing, joins, aggregations and schema mapping for structured, production-ready datasets. • Architected Delta Table storage in ADLS Gen2, enabling ACID transactions, time travel, versioning, and high performance downstream querying for analytics workloads. • Orchestrated end-to-end ELT workflows by integrating ADF pipelines with Databricks notebooks, contributing directly to ₹30CR ($3.5M) revenue growth and achieving #1 ranking across 18 Amazon product categories within 90 days. • Authored optimized SQL queries on Delta datasets to generate business-critical KPIs supporting executive dashboards and operational decision-making.

Data Science Intern at SkillForge, Mar 2022 - Mar 2022 · 1 mo, Bengaluru, Karnataka, India, ● Identified customers aged 38-47 as responsible for 24% of all credit card spending, highlighting a key demographic for higher expenditures. ● Developed a UDF to segment the top 10 customers in each city by repayment amounts, across credit card products (Gold, Silver, Platinum) and time periods. ● Enabled targeted financial strategies through detailed customer segmentation and spending analysis.

Education

Bachelor of Commerce - BCom, Accounting and Finance, Lalit Narayan Mithila University, Darbhanga

Data Science, Analytixlabs