About
I’m a Data Engineer passionate about building scalable, reliable, and efficient data solutions that transform raw data into meaningful, business-ready insights. I have hands-on experience and strong knowledge of Python, SQL, PySpark, Apache Spark, Azure Data Factory (ADF), Azure Storage, Databricks, ETL/ELT, and data pipeline development. I enjoy solving data engineering problems involving data ingestion, transformation, integration, orchestration, data quality, and pipeline optimization. My experience includes designing and developing end-to-end data pipelines to move and transform data from multiple sources into structured and analytics-ready datasets. I work with Azure Data Factory for pipeline orchestration, including Copy Data, Lookup, Stored Procedure, Execute Pipeline, ForEach, Get Metadata, parameters, variables, triggers, monitoring, and error handling. I’m also developing strong expertise in PySpark and Apache Spark for processing and transforming large datasets. My areas of interest include DataFrames, Spark SQL, transformations and actions, joins, aggregations, partitioning, caching, lazy evaluation, shuffles, and performance optimization. I believe a good data pipeline should be more than just functional. It should be scalable, maintainable, reliable, observable, and cost-efficient. I focus on understanding the source systems, defining efficient transformation logic, handling data quality issues, implementing appropriate error handling and logging, and designing reusable pipeline components. I continuously work on strengthening my knowledge of data modeling, incremental data loading, pipeline optimization, monitoring, and production-grade engineering practices. I enjoy taking complex data problems and breaking them down into simple, practical engineering solutions. Whether it’s building a new pipeline, troubleshooting a failed workflow, optimizing data processing, or improving data quality, I’m always focused on understanding the problem and delivering a reliable solution. I’m currently looking for opportunities as a Data Engineer, Azure Data Engineer, Cloud Data Engineer, or PySpark Data Engineer, where I can contribute to real-world projects, work with experienced engineering teams, solve challenging data problems, and continue growing as a data professional. Skills: Data Engineering | Azure Data Factory | PySpark | Apache Spark | Python | SQL | Databricks | Azure | ETL | ELT | Data Pipelines | Data Lake | Data Warehouse | Data Integration | Data Transformation | Data Modeling | Pipeline Optimization | Data Quality | Cloud Data Engineering
Work experience
Software Engineer at Accenture, Oct 2021 - Present · 5 yrs 1 mo, Designed and developed ETL/ELT data pipelines using Azure Data Factory. Built data ingestion pipelines for CSV and Excel files into Azure-based data platforms. Worked with Azure Data Lake Storage for scalable data storage and processing. Developed data transformation workflows using Databricks and PySpark. Used SQL for data extraction, transformation, validation, and analysis. Implemented parameterized and reusable ADF pipelines to improve pipeline flexibility. Worked with pipeline triggers, Integration Runtime, stored procedures, and linked services in ADF. Implemented logging, error handling, and monitoring for data pipelines. Applied ETL pipeline optimization techniques to improve performance and resource utilization. Designed data flows following Data Lake → Transformation → Curated/Analytics architecture. Worked with Parquet and structured data formats for analytics workloads. Built solutions following Data Engineering best practices for scalability, reliability, and maintainability
