Headline

Microsoft Fabric Data Engineer | Azure Data Engineer | Azure Databricks| Azure Synapse Analytics | Lakehouse | ETL Pipelines | ADLS Gen2 | Data Modeling | Delta Lake | Real-Time Data Processing

About

I am a Data Engineer with almost 3 years of professional experience, focused on building scalable and reliable data pipelines using modern cloud technologies. I have hands-on experience working with Microsoft Fabric and Azure Data Factory to design and orchestrate end-to-end ETL pipelines. My work involves ingesting data from multiple sources, transforming it using PySpark, and storing it in optimized formats such as Delta and Parquet within Lakehouse architectures. I have worked with Azure Data Lake Storage Gen2 and Azure Blob Storage for scalable data storage and integration. I am comfortable implementing Medallion architecture (Bronze, Silver, Gold layers) and applying data modeling techniques including fact tables and slowly changing dimensions (SCD). My experience includes building and monitoring pipelines, handling failures, optimizing performance, and supporting real-time and batch data processing workflows. I also have exposure to CI/CD practices and data governance concepts such as lineage tracking. I focus on writing efficient, maintainable data pipelines and solving real-world data engineering problems rather than just working with tools.

Location

Greater Hyderabad Area

Work experience

Data Engineer at PrimeSource IT Service & Consulting PVT LTD, Apr 2024 - Present · 2 yrs 3 mos, Hyderabad, Designed and developed end-to-end data pipelines using Microsoft Fabric and Azure Data Factory for ingesting, transforming, and loading data from multiple sources. Built and maintained ETL pipelines to process structured and semi-structured data, improving data availability and reliability for downstream analytics. Implemented data ingestion workflows from Azure Blob Storage to Lakehouse using scalable and optimized approaches. Performed data transformation using PySpark and Dataflow Gen2, ensuring clean, standardized, and analysis-ready datasets. Worked with Azure Data Lake Storage Gen2 for storing large-scale data and managing hierarchical data structures. Applied Medallion Architecture (Bronze, Silver, Gold layers) to organize and optimize data processing workflows in Lakehouse. Optimized data storage using Delta Lake and Parquet formats for improved performance and cost efficiency. Developed and managed pipeline orchestration including activities like Lookup, ForEach, and pipeline triggers for automation. Implemented monitoring and alerting mechanisms for pipeline failures, improving reliability and reducing downtime. Designed and implemented data models including fact tables and slowly changing dimensions (SCD) for analytical use cases. Worked with Fabric Data Warehouse for querying, CTAS operations, and creating views for reporting and analytics. Collaborated with cross-functional teams to understand data requirements and deliver scalable data solutions. Participated in deployment processes using CI/CD pipelines and maintained version-controlled data workflows.

Technical Developer at NxtWave, Sep 2023 - Mar 2024 · 7 mos

Education

Bachelor of Education - BEd, AIML, CHANDIGARH UNIVERSITY

Higher Secondary School Certification, Narayana Junior College - India