Headline

Senior Software Engineer @blueshift

About

Senior Data Engineer | Apache Spark, AWS, Kafka | Building AI-Powered Data Platforms & Agentic Systems | 8+ Yrs Senior Data Engineer with 8+ years building scalable data infrastructure for high-growth SaaS and martech products. At Blueshift, I do data engineering for an AI-powered Customer Data Platform used to power personalised marketing at scale — architecting pipelines that process 100M+ events daily at 99.9% uptime. Recent wins include leading an AWS EMR migration that cut infrastructure costs by 33% (~$150K/year), and migrating legacy batch jobs to Apache Spark, reducing processing time by 70% (8 hrs → 2.4 hrs). Core stack: Apache Spark, Kafka, Apache Pulsar, Airflow, AWS (EMR, S3, Lambda, Glue), Python, SQL, HBase/ScyllaDB. Over the past year, I've expanded into applied AI — building a Text-to-SQL agentic system and contributing to an internal agentic automation platform (MCP servers, LLM-powered agents). I use AI-assisted development tools (Claude Code, GitHub Copilot) daily as part of my engineering workflow, not just as an experiment. I started in full-stack development before moving into data engineering, which gives me a practical view of how backend systems, data infrastructure, and product experience connect end-to-end. Always exploring how AI and modern data infrastructure intersect. Open to connecting with data engineers and to conversations about senior data/backend engineering roles in AI-driven products.

Location

Pune Division

AI stack

claude code

cursor

gemini

github copilot

metaai

ChatGPT

Work experience

Senior Software Engineer (Data Engineering) at Blueshift, Aug 2023 - Present · 3 yrs 1 mo, Pune District, Maharashtra, India, Senior Software Engineer (Data Engineering) | AI-Powered Customer Data Platform Leading data engineering for Blueshift's AI-driven Customer Data Platform, powering personalised marketing engagement across channels for 100M+ daily events. Partner closely with engineering and product teams to design, build, and scale reliable data infrastructure end-to-end — from ingestion through activation. Key Contributions: • Led migration of data infrastructure from on-premises Ambari to AWS EMR, cutting infrastructure costs by 33% (~$150K/year) through Spark and cluster optimisation • Migrated legacy MapReduce batch jobs to Apache Spark (Scala), reducing daily processing time 70% (8 hrs → 2.4 hrs) while processing customer engagement data across multiple marketing channels • Architected real-time streaming pipelines using Kafka and Apache Pulsar, processing 100M+ daily events at 99.9% uptime with zero data loss during outages, via fault-tolerant retry and failover mechanisms • Built and maintained end-to-end ETL pipelines on AWS, managing data flow from ingestion to activation for personalised, multi-channel marketing campaigns • Built a Text-to-SQL agentic system enabling natural-language data querying, and contributed to Blueshift's internal Agentic Launchpad platform — developing MCP servers and production AI agents • Optimised database queries and Spark job configurations, driving 40% additional compute cost reduction and improved processing throughput • Containerised and deployed data engineering services using Docker, improving deployment consistency and scalability • Monitored and troubleshot production data pipelines, maintaining platform reliability and performance across marketing automation workflows Impact: 33% AWS cost reduction ($150K/year) · 40% additional compute cost reduction · 70% faster batch processing · 99.9% uptime across 100M+ daily events

Software engineer ll at Blueshift, Apr 2022 - Aug 2023 · 1 yr 5 mos, Pune District, Maharashtra, India

Software Engineer at Blueshift, May 2021 - Apr 2022 · 1 yr, Pune Division, Maharashtra, India

Software Engineer at Josh Software, Inc., Jun 2018 - May 2021 · 3 yrs, Pune District, Maharashtra, India, Software Engineer | Full-Stack Development Built full-stack features for a sales management platform, focusing on automation, performance optimisation, and API design to support core business operations and third-party integrations. Key Contributions: • Developed end-to-end features for a sales management web application, reducing manual Excel maintenance by 80% and cutting daily reporting time from 3 hours to 30 minutes through automated inventory dashboards • Implemented performance optimisations — asynchronous job processing, database denormalisation, and query tuning — significantly improving page load times and user experience • Built a Redis caching layer that reduced response times and improved system throughput under load • Designed and implemented RESTful APIs supporting core ad-sales functionality and seamless third-party integrations with external systems Impact: 80% reduction in manual reporting effort · Reporting time cut from 3 hrs to 30 mins · Significant page load and API performance gains

Software Engineer at Josh Software, Inc., Jun 2018 - Nov 2020 · 2 yrs 6 mos, Pune/Pimpri-Chinchwad Area, Full-Stack Software Engineer | Ad-Sales Digital Platform Development Led end-to-end feature development for digital advertising platforms, working closely with business stakeholders to transform requirements into scalable technical solutions. Key Contributions: 1. Designed and implemented RESTful APIs to support core ad-sales functionality and third-party integrations 2. Developed clean, maintainable code following best practices for testing and documentation 3. Conducted proof-of-concept development to evaluate technical feasibility and recommend solutions for complex business requirements 4. Built responsive UI components and handled frontend-backend integration 5. Performed system optimization including database query tuning, page load improvements, background job processing, and caching strategies to enhance platform performance Impact: Contributed to building a comprehensive digital platform that streamlined ad-sales operations and improved system efficiency across the media domain.

Intern at Josh Software, Inc., Jan 2018 - Jun 2018 · 6 mos, Pune District, Maharashtra, India, Open Source Developer | Ruby on Rails Contributor Recognition Platform Contributed to an open source project focused on motivating and rewarding developers based on their contribution quality and community engagement activities. Key Contributions: 1. Developed RESTful APIs to track and evaluate contributor activities across repositories 2. Wrote clean, testable code following Rails conventions and open source development standards 3. Built user interface components to display contributor metrics and reward systems 4. Proposed and prototyped new features to improve user engagement and platform adoption within the open source community Impact: Helped create a platform that encourages quality contributions to open source projects by providing recognition and incentives to active contributors.

Education

Bachelor of Technology - BTech, Information Technology, Walchand College of Engineering(A Govt. Aided Autonomous Institute),SANGLI-M.S