Headline

Senior Data Engineer | SnowPro Core & Databricks Certified | Azure Databricks • Snowflake • PySpark • ADF | Building Scalable Lakehouse & Data Engineering Solutions

About

I'm a Senior Data Engineer passionate about designing and building scalable cloud-native data platforms that transform complex data into reliable, business-ready insights. I specialize in architecting end-to-end ETL/ELT solutions using Azure Databricks, Azure Data Factory (ADF), ADLS Gen2, Snowflake, PySpark, and Delta Lake. My experience spans modern Lakehouse architectures, API integrations (REST, GraphQL, Webhooks), SQL databases, and both batch and near real-time data processing. Over the years, I've built production-grade data platforms across Healthcare, E-commerce, HR, Finance, and Marketing domains, delivering scalable Medallion (Bronze–Silver–Gold) architectures, metadata-driven pipelines, dimensional models, and high-performance data processing solutions. Some recent highlights: • Improved pipeline performance by up to 10× through Spark optimization and efficient pipeline design. • Built enterprise Lakehouse pipelines using Databricks Declarative Pipelines (DLT), Auto Loader, Structured Streaming, and Delta Lake. • Designed reusable ingestion frameworks for APIs and databases with incremental loading, CDC, schema evolution, and resilient error handling. • Optimized complex Spark workloads, significantly reducing processing time, duplicate records, and production failures. I'm a SnowPro Core Certified and Databricks Certified Data Engineer Associate professional who enjoys solving challenging data engineering problems, mentoring engineers, and building reliable, scalable data platforms that create measurable business impact. 🏆 Honored with multiple performance awards—including Aqua, Azure, Royal Blue, and True Blue Awards—for technical excellence, innovation, and consistently delivering high-impact solutions.

Location

Greater Hyderabad Area

AI stack

claude code

claude

cursor

gemini

openai

Work experience

Senior Data Engineer at ADA, Apr 2025 - Present · 1 yr 7 mos, Design and develop scalable cloud-native ETL/ELT solutions using Azure Databricks, PySpark, Snowflake, Azure Data Factory (ADF), and ADLS Gen2 across Healthcare, E-commerce, HR, Finance, and Marketing domains. Build reusable, metadata-driven data engineering frameworks supporting batch, streaming, API, and database ingestion with a strong focus on scalability, reliability, and maintainability. Architect modern Lakehouse solutions using Medallion Architecture, Declarative Pipelines (DLT), Delta Lake, and dimensional modeling to deliver production-grade analytical datasets. Develop robust ingestion frameworks for REST APIs, GraphQL, Webhooks, and SQL databases, implementing incremental loading, CDC, retry mechanisms, rate-limit handling, and schema evolution. Optimize Spark workloads through partitioning, join optimization, caching, parallelism, and query tuning, improving large-scale data processing performance by up to 10×. Drive production excellence through CI/CD, monitoring, incident management, SLA adherence (>99%), and observability using Azure DevOps, Git, New Relic, and Grafana, while mentoring engineers on data engineering best practices.

Data Scientist (currently aligned with Data Engineering projects) at ADA, Dec 2022 - Mar 2025 · 2 yrs 4 mos, Developed and maintained scalable ETL pipelines for structured and semi-structured datasets using SQL, PySpark, Java, and Snowflake. Designed normalized ERDs and dimensional models (Star & Snowflake) to support enterprise reporting and analytics. Built and enhanced API connectors by implementing authentication, pagination, incremental synchronization, and resilient error handling for reliable data ingestion. Performed complex JSON/XML processing, schema validation, and data quality checks to improve pipeline reliability and reduce transformation errors. Optimized SQL queries and Spark transformations for high-volume datasets while collaborating with cross-functional teams to deliver reliable data solutions.

Teaching Assistant at NxtWave, Oct 2022 - Nov 2022 · 2 mos, Remote, My responsibility as a TA is to resolve doubts of learners related to html,css and python.

MERN Stack Development Trainee at NxtWave, Mar 2022 - Dec 2022 · 10 mos

Education

Fellow at NxtWave’s CCBP 4.0 Intensive, Full Stack Development with mern specialisation, NxtWave

Bachelor of Technology - BTech, Electrical and Electronics Engineering, Jyothishmathi institute of technology and science