Headline

Senior Software Engineer | Real-Time Data Platforms & Event Streaming | Kafka · Node.js · TypeScript · BigQuery | 300M+ events/day

About

Senior Software Engineer with 5+ years building high-scale, event-driven microservices and real-time data platforms at Tata Digital. I architected JARVIS — the company's first platform to stitch native-app and WebView journeys into a single unified customer journey across the Tata Digital ecosystem (Tata Neu, 1mg, BigBasket, Croma, Tata CLiQ) — processing 300M+ events/day in production. Since then, I've owned platform-wide engineering initiatives: - Led org-wide JSON→Avro migration for Kafka pipelines (~1.5B events) with Confluent Schema Registry — cutting storage footprint ~50% - Owned platform-wide BigQuery cost optimization — ~90% cost reduction, ~100 TB managed - Achieved 100% data parity between Databricks and BigQuery via reconciliation pipelines - Designed streaming and micro-batch ingestion on Databricks with Bronze/Silver/Gold medallion PySpark pipelines - Built a resilient backfilling and recovery framework — cutting pipeline recovery from days to hours Core stack: Node.js · TypeScript · Python · PySpark · Apache Kafka · Avro · Confluent Schema Registry · BigQuery · Databricks · Apache Spark · Kubernetes · GCP · Azure (AKS) Currently open to Senior Software Engineer / SDE-3 roles in backend, distributed systems, or platform engineering at product-first companies. Based in Bangalore, open to remote. 📩 aakashporwal111@gmail.com 🔗 Portfolio: https://aakashporwal1602.github.io/aakash-porwal-portfolio

Location

Greater Bengaluru Area

AI stack

claude code

claude

codex

deepseek

github copilot

Work experience

Senior Software Engineer at Tata Digital, Jun 2024 - Present · 2 yrs 3 mos, Owning platform-wide engineering initiatives across Kafka, BigQuery, and Databricks. -Owned the platform-wide BigQuery cost-optimization strategy end-to-end — partition-expiry TTLs, dataset lifecycle management, job governance, and purging of ~100 TB of orphaned data — cutting BigQuery spend ~90%; mentored 2 engineers. -Led org-wide JSON→Avro migration for Kafka event pipelines (~1.5B events) with Confluent Schema Registry — cutting storage footprint ~50% and enforcing producer-consumer schema contracts and safe schema evolution. -Achieved 100% data parity between Databricks (DWH) and BigQuery through reconciliation pipelines, automated validation jobs, and continuous monitoring. -Designed streaming and micro-batch ingestion on Databricks with layered Bronze/Silver/Gold (medallion) PySpark pipelines powering downstream analytics and ML workloads. -Built a resilient backfilling and recovery framework that automatically restores production pipelines after Kafka consumer lag, offset resets, upstream outages, and BigQuery↔Databricks drift

Digital Engineer at Tata Digital, Dec 2021 - Jul 2024 · 2 yrs 8 mos, Architected and scaled JARVIS — the company's first-party real-time analytics platform. - Architected JARVIS, the company's first real-time platform to stitch native-app and WebView journeys into a single unified customer journey across the Tata Digital ecosystem (Tata Neu, 1mg, BigBasket, Croma, Tata CLiQ) — 300M+ events/day in production. Built on Node.js, TypeScript, and Express over Azure Kubernetes Service. - Engineered a multi-sink event-distribution layer — streaming to BigQuery for product analytics (Mixpanel) and BI dashboards, fanning out to Meta, Firebase, Branch, and MoEngage for campaign attribution and audience building, and publishing to Kafka for real-time consumers. - Cut BigQuery costs ~70% via table partitioning, clustering, materialized views, and migration to BigQuery Storage API. - Owned BigQuery observability end-to-end — monitoring and proactive alerting across slot utilization, query cost, job health, and failure rates. - Migrated event ingestion from Azure EventHub to Kafka with exponential-backoff retries and dead-letter topics — cutting event-processing cost 7x; mentored 2 interns.

Java Consultant at GENPACT, Oct 2021 - Dec 2021 · 3 mos, Bengaluru, Karnataka, India

Senior Analyst-data strategy and engineering at Kantar, Jun 2021 - Oct 2021 · 5 mos, Bangalore Urban, Karnataka, India, ● Harmonized and modeled data for large-scale Market Return on Investment (MROI) projects across South-East Asia for a global Consumer Packaged Goods (CPG) major with a market cap of more than $150B. ● Analyzed competitor’s data and recommended significant advantages to the client through improvement in business performance and marketing effectiveness by utilizing deep domain knowledge and advanced analytics.

SDE intern at RoboMQ, Jan 2021 - Jun 2021 · 6 mos, Jaipur, Rajasthan, India, 1. Created various flows and recipes amongst different applications including top CRM, ERP, HRM, Database applications using company's inhouse Connect iPaaS product. 2.Automated Jira Ticket creation and updation corresponding to when any incidents are triggered or escalated in Pagerduty. 3.Developed Business logic in various existing microservices.

Volunteer at Teamwork Arts Pvt Ltd, Jan 2020 - Jan 2020 · 1 mo, Jaipur, Rajasthan, India, Volunteered at Jaipur Literature Festival 2020.

Education

Bachelor of Engineering - BE(CSE), Computer Software Engineering, Birla Institute of Technology, Mesra

Public links

github: https://github.com/aakashporwal1602