Headline

Senior Data Engineer | 8+ Years Exp | AWS, GCP, Apache Airflow, PySpark & Airbyte | Building Scalable ETL/ELT Pipelines

About

I am a Senior Data Engineer with over 8 years of experience architecting, deploying, and optimizing scalable data infrastructure and robust ETL/ELT pipelines. I specialize in bridging the gap between raw, unstructured data and advanced analytics platforms that drive critical business decisions. Throughout my career, I have successfully leveraged modern data stacks across both AWS and GCP ecosystems. I am highly proficient in Python, SQL, and Apache Spark, and I thrive on modernizing legacy data warehouses and orchestrating distributed batch workflows. Some of my recent technical focuses include: 🔹 Data Orchestration: Apache Airflow, Cloud Composer, AWS MWAA, Airbyte, GCP DTS 🔹 Cloud Infrastructure: AWS (S3, EMR, Athena, Redshift, Glue), GCP (GCS, BigQuery, Cloud Run) 🔹 Data Architecture: Data Lakehouse design, API-driven Ingestion, CDC, and ELT transformations I am passionate about writing clean, optimized code and building programmatic API integrations that significantly reduce operational compute costs and pipeline latency. I am always open to connecting with other data professionals and discussing modern cloud architectures!

Location

Bengaluru

AI stack

antigravity

claude code

codex

gemini

github copilot

openai

Work experience

Senior Data Engineer at Omnicom Global Solutions, Jul 2022 - Present · 4 yrs 3 mos, Lead a team of data developers to optimize PySpark/SQL workloads, driving a 30% increase in overall data processing efficiency. Architect and manage complex Airflow DAGs (via Cloud Composer and MWAA) to automate large-scale audience and ad-performance data ingestion across AWS and GCP. Spearhead data architecture modernization efforts utilizing Airbyte and AWS EMR, achieving a 25% reduction in operational compute costs.

Data Developer at Omnicom Global Solutions, Dec 2021 - Jul 2022 · 8 mos, Engineered foundational data ingestion workflows and optimized SQL/Python data pipelines to support large-scale audience analytics. Delivered critical infrastructure improvements that accelerated an internal promotion to a Senior-level role within eight months.

Software Developer (Systems Engineer) at Tata Consultancy Services, Jan 2018 - Dec 2021 · 4 yrs, • Engineered scalable data processing applications and ETL pipelines leveraging Python and Apache Spark to handle large volumes of structured and unstructured datasets. • Designed interactive dashboards and configured real-time alerts in Splunk for system health observability, significantly reducing MTTR for data pipeline failures. • Collaborated in Agile environments utilizing Bitbucket and Bamboo for version control and continuous integration, demonstrating rigorous debugging standards.

Microsoft Student Partners (MSP) at Microsoft, Dec 2015 - Dec 2016 · 1 yr 1 mo

Education

Bachelor’s Degree, Computer Science Engineering, Andhra university