Headline

Data Engineering Leader

About

Strategic Data & AI Engineering Leader/Manager with 15+ years of experience driving large-scale digital transformations, migrating legacy systems, and architecting PetaByte-scale distributed data pipelines. Specialize in designing future-proof data ecosystems using Lambda/Kappa architectures, Data Mesh, and Data Fabric to deliver actionable insights directly to the executive boardroom. Proven track record of leveraging ETL processes, Core Java, and Python to build robust data solutions, optimize complex pipelines, and implement scalable systems for seamless global data integration. Beyond software engineering, I am a co-inventor with two granted international design patents for secure transaction cards. Core Expertise - Architectures: Lambda, Kappa, Data Mesh, Data Fabric, Agentic AI Solutions - Big Data & Streaming: Apache Spark, Apache Kafka, Apache Cassandra - Cloud : AWS (EMR, S3, Lambda, IAM, EC2) - Languages: Core Java, Python - Scale: PetaByte-scale volume applications, legacy-to-open-source migrations Key Leadership & Technical Impact * End-to-End Delivery: Engineered high-value data pipelines from the ground up, unlocking immediate business value and executive-level metrics. * Team Leadership: Managed and mentored a high-performing team of 10+ engineers, unblocking technical hurdles and fostering continuous growth. * System Optimization: Led the migration of legacy applications to open-source big data tech stacks, drastically improving processing efficiency and reducing technical debt. * Global Innovation: Co-owner of two design patents on "Transaction Card" (US Patents D909472 & D909473, granted Feb 2021). Also filed and granted across the UK, EU, Australia, Singapore, and China

Location

Bengaluru

AI stack

claude code

codex

github copilot

grok

langgraph

CrewAI

Work experience

Lead Consultant at Visa, Mar 2019 - Present · 7 yrs 7 mos, Bengaluru, • Championed the integration of Agentic AI frameworks into enterprise data pipelines, enhancing intelligent decision-making. • Designed and developed high-performance, petabyte-scale applications using Apache Spark and Kafka for efficient data processing. • Led data migration initiatives, transitioning legacy systems to modern open-source big data technologies. • Managed a cross-functional team of 10 engineers, providing technical leadership and mentorship.

Associate Technical Architect at Tavant, Jan 2017 - Mar 2019 · 2 yrs 3 mos, Bangaon Area, India, • Developing spark based applications using PySpark, deploy and run them in EMR. • Migrating the python based applications to PySpark applications and Scheduling the jobs using Azkaban • Optimizing the applications using spark memory configurations. • Writing Re:Dash/Presto queries as per the user requirements. • Building spark applications around the data science models using pickle objects. • Handling the team of 10 members. • Writing the data to S3 with the formats like ORC, Parquet, Json, CSV.

Principal Developer at Oracle India Pvt. Ltd, Feb 2015 - Jan 2017 · 2 yrs, Bangaon Area, India

Tech Lead at Tecnotree, Jun 2010 - Feb 2015 · 4 yrs 9 mos

Education

Masters, MBA, AV PG COLLEGE