Headline

Data Engineer @eClerx | Databricks | AWS | Azure | Pyspark | SQL | Snowflake | ETL/ELT

About

I am a Data Engineer with 4.5+ years of experience designing, optimizing, and maintaining scalable, end-to-end data pipelines. My expertise lies in building cloud-native data architectures, optimizing ETL/ELT workflows, and managing high-volume big data environments. Throughout my career, including my time at eClerX, I have focused on reducing data processing latencies, lowering cloud costs, and ensuring high data reliability across distributed systems. Technical Core Stack: Languages & Frameworks: Python, PySpark, SQL, Hive Cloud Platforms: Microsoft Azure, Amazon Web Services (AWS) Data Warehousing & Big Data: Snowflake, Databricks, Hadoop, Databricks SQL I specialize in migrating legacy systems to modern cloud environments like Snowflake and orchestrating large-scale batch and streaming pipelines using PySpark. I am passionate about engineering clean, efficient, and production-ready data ecosystems that empower businesses to make data-driven decisions. Feel free to connect with me to discuss modern data stack trends, cloud migrations, or data engineering opportunities!

Location

Pune Division

AI stack

claude code

claude

codex

cursor

deepmind

deepseek

gemini

github copilot

notebooklm

openai

Work experience

Data Engineer at eClerx

Data Engineer at --, Nov 2023 - Present · 2 yrs 9 mos, •Designed and built scalable data pipelines using Azure Databricks (PySpark) for batch and streaming data ingestion •Implemented Delta Lake (Bronze, Silver, Gold layers) for reliable and governed data processing •Developed ETL/​ELT workflows with incremental loading and data validation frameworks •Integrated data from Azure SQL, Oracle, and Event Hubs •Optimized Snowflake tables, clustering, and warehouses to improve performance and cost efficiency •Loaded curated datasets into Snowflake and Azure Synapse Analytics for enterprise reporting and advanced analytics •Improved claims processing speed by ~40% while optimizing Azure and Snowflake compute and storage costs

Data Engineer at --, Dec 2022 - Oct 2023 · 11 mos, •Contributed to the automation of container demand forecasting and cargo capacity planning by modernizing manual Excel-based processes into a scalable Azure Lakehouse solution •Developed and maintained data pipelines using Azure Data Factory and Azure Databricks (PySpark) with Delta Lake to process OEM shipping data •Implemented automated ingestion using Azure Logic Apps and integrated Azure Machine Learning models for forecasting workflows •Delivered curated datasets to Couchbase for API consumption, Azure Blob Storage, and Azure Synapse Analytics for reporting and analytics •Utilized Delta Lake SCD Type 2 and Time Travel for historical data tracking and versioning •Applied performance optimization, monitoring, and logging best practices

Data Engineer at eClerx, Nov 2021 - Present · 4 yrs 9 mos, Hybrid

Data Engineer at eClerx, Nov 2021 - Present · 4 yrs 9 mos, Hybrid

Data Engineer at ---, Nov 2021 - Nov 2022 · 1 yr 1 mo, •Implemented an end-to-end AWS-based data pipeline using a Medallion Architecture to bifurcate Ireland-specific pharmaceutical sales and CRM data •Leveraged AWS Glue (PySpark) for ETL processing, Amazon S3 for data lake storage, and Amazon RDS (MySQL) for structured reporting •Integrated AWS Lambda for event-driven automation and used Amazon CloudWatch for monitoring and alerting •Enhanced CRM analytics capabilities, improved sales targeting in the Ireland region, contributing to an 18% increase in regional sales performance

Education

Bachelor of Science, Dr. Babasaheb Ambedkar Marathwada University