Headline

Senior Forward Deployed Engineer @ Databricks | Building Production Multi-Agent Systems & RAG for Enterprise AI | Data + AI Architecture | 6x Databricks & 4x Azure Certified

About

I'm a Senior Forward Deployed Engineer at Databricks, where I help enterprises design and ship production-grade agentic AI and data platforms rather than one-off proofs of concept. My recent work centers on multi-agent systems built with supervisor-worker patterns, MLflow tracing, and persistent memory for financial services clients, alongside custom Deep Research frameworks, automated RCA engines, and RAG pipelines hardened with LLM gateway fallback routing, adversarial red-teaming, and semantic re-ranking. Beyond agentic AI, I focus on the foundations that make these systems trustworthy at scale: MLOps pipelines spanning batch, streaming, and real-time inference, and data governance frameworks built on Unity Catalog that enforce PII masking, access controls, and lineage across banking and insurance clients. I'm also a contributor to Databricks Overwatch and built UNIQ, a multi-agent migration framework that has cut project effort by three to six months for teams migrating off legacy platforms. Before Databricks, I spent time at Accenture and A.P. Moller-Maersk building large-scale ETL and cloud data platforms, which is where I developed the data engineering fundamentals I still lean on daily. I hold six Databricks certifications and four Microsoft Azure certifications, and I enjoy giving back to the community through partner talks, founding the Kolkata Databricks Meetup, and mentoring engineers navigating the shift from traditional data engineering into AI-native architectures. If you're working through how to move agentic AI from demo to production, or rethinking your data platform's governance and MLOps maturity, I'd love to connect

Location

Bengaluru

AI stack

claude code

claude

cursor

elevenlabs

hugging face

langgraph

Work experience

Senior Forward Deployed Engineer at Databricks, Jun 2026 - Present · 3 mos, Co-architected production Multi-Agent Systems (Supervisor-Worker patterns, MLflow tracing, persistent memory) for financial services clients, embedding responsible AI guardrails into automated enterprise decisioning. Engineered custom Multi-Agent Deep Research frameworks and automated RCA engines, building vector-ready datasets and governed data products for enterprise digital transformation. Hardened Multi-Agent deployments with LLM gateway fallback routing, layered memory, and adversarial red-teaming; applied semantic re-ranking and LLM-based evaluation to Vector Search-backed knowledge assistants.

Resident Solution Architect at Databricks, Feb 2025 - Jun 2026 · 1 yr 5 mos, Established end-to-end MLOps foundations (Feature Store, MLOps Stacks CI/CD, Lakehouse Monitoring) across batch, streaming, and real-time inference workloads. Architected data governance and lifecycle policies via Unity Catalog across global banking and insurance clients, enforcing PII masking, access controls, lineage, and quality standards. Designed high-throughput ELT and streaming pipelines using Lakeflow SDP and Apache Airflow across multi-terabyte data lakes, supporting Lakehouse and Medallion-pattern architectures. Core contributor to Databricks Overwatch (Labs OSS); authored UNIQ, a multi-agent migration framework cutting project effort by 3-6 months ($50K-$100K savings). Delivered partner tech talks to 125+ architects, founded the Kolkata Databricks Meetup, and received an MVP nomination at the global Databricks Tech Summit.

Senior Solutions Consultant at Databricks, Jul 2022 - Jan 2025 · 2 yrs 7 mos, Bengaluru, Delivered enterprise data governance and lifecycle policy solutions across global banking and insurance clients, embedding responsible AI guardrails into data platforms. Engineered custom Multi-Agent Deep Research frameworks and automated RCA engines, building vector-ready datasets and governed data products to support enterprise digital transformation. Established end-to-end MLOps foundations spanning batch, streaming, and real-time inference workloads for client deployments. Hardened production deployments with LLM gateway fallback routing, layered memory, and adversarial red-teaming, applying semantic re-ranking and LLM-based evaluation to Vector Search-backed knowledge assistants. . Open source contributor for Databricks-Overwatch. Github: https://databrickslabs.github.io/overwatch/

Senior Big Data Engineer at A.P. Moller - Maersk, Mar 2022 - Jun 2022 · 4 mos, Bengaluru, Karnataka, India

Big Data and Cloud Engineer at Accenture AI, Apr 2018 - Mar 2022 · 4 yrs, Greater Bengaluru Area, As ETL Lead for a multinational banking and financial services client, led a team of 10 data engineers managing Azure Data Factory pipelines, ingesting complex financial datasets into Azure Databricks using PySpark and Pandas. Orchestrated end-to-end data processing and analytics across Azure Synapse Spark pools and Databricks, delivering interactive Power BI dashboards for executive decision-making. Standardized data quality rules, PII protection, and CI/CD release management across pipelines using Git, Docker, and DataOps best practices. As Product Lead, architected an end-to-end patented ETL and ML automation product on Azure, integrating NLP and ML models into automated ELT workflows; ingested high-volume transactional data (SAP HANA, relational databases) into cloud data lakes and deployed Azure ML pipelines supporting real-time and batch inference for business operations.

Senior Software Engineer at Mashreq Bank, Oct 2016 - Mar 2018 · 1 yr 6 mos, Bengaluru, Karnataka, India, Working as Senior Software Developer in Credit and Debit Card Domain. Led Hortonworks-based big data engineering (MongoDB ingestion, HLD/LLD design) and built AWS pipelines (Oracle GoldenGate to Kafka/Cassandra, EMR, Lambda) across enterprise IT and banking clients. Built a Kafka streaming pipeline integrating Cassandra and Oracle Database for a banking client, with PySpark/Scala ETL and Hive UDF-based customer encryption.

System Engineer at Tata Consultancy Services, Oct 2012 - Sep 2016 · 4 yrs, Mumbai, Maharashtra, India, Built layered big data platforms (Spark/Hive, Oozie-automated workflows) for supply chain and risk-assessment analytics; migrated MapReduce jobs to Scala-Spark with optimized Hive/Sqoop pipelines. Implemented KMeans clustering and SVM/Logistic Regression classification models with Python-based deduplication, deployed via Jenkins automation.

Education

Bachelor's degree, Electronic and Communications Engineering Technology, Asansol Engineering College (AEC)

Higher Secondary, Science, Santinagar Vidyamandir

Public links

github: https://github.com/sourav692