Headline

Senior Engineering Leader @ Nutanix | Driving Cloud-Agnostic Solutions

About

Senior Engineering Leader with over two decades of experience in designing and delivering cloud-native platforms and AI-driven solutions. Currently at Nutanix, leading the development of a cloud-agnostic ETL Platform as a Service (PaaS) that seamlessly integrates generative AI for scalable data integration, advanced Text-to-SQL querying, and ELT workflows. With expertise in cloud architecture, generative AI, and multi-agent systems, they focus on enabling enterprise transformation through cutting-edge technology. By embedding data governance and leveraging advanced AI solutions, they drive innovation and empower organizations to achieve future-ready operational excellence.

Location

Bengaluru

AI stack

anthropic apis

claude

cursor

langchain

langfuse

langgraph

Work experience

Senior Engineering Leader | AI & Data Platforms | Generative AI (LLMs) | Cloud & PaaS Expert at Nutanix, Jun 2023 - Present · 3 y 2 mo, Bengaluru Prestige Tech Park, Leading the design and delivery of a cloud-agnostic ETL Platform as a Service (PaaS), driving scalable data integration, GenAI-powered querying, and modern ELT workflows with future-forward AI and Quantum-readiness. Architected an ELT-first Apache Spark framework supporting real-time and batch data pipelines across Snowflake, AWS RDS, and MySQL. Integrated Generative AI with a Text-to-SQL engine powered by Streamlit and vLLM for natural language query generation. Improved model accuracy from 61.11% to 85% through domain-specific fine-tuning of CodeLlama-7b-Instruct-hf, enabling enterprise-grade NLQ. Achieved 498.2 tokens/sec prompt throughput through GPU-optimized inference, accelerating GenAI application responsiveness. Embedded Model Context Protocol (MCP) to ensure secure, metadata-driven prompt routing and contextual accuracy. Orchestrated autonomous AI Agents for query planning, schema retrieval, and anomaly detection using an Agentic AI workflow design. Ensured platform resiliency via Kubernetes-managed Docker containers, enabling microservice scalability and self-healing capabilities. Enabled metadata cataloging and lineage traceability through a PostgreSQL-backed metadata layer, supporting RBAC and auditability. Currently exploring integration of Quantum-inspired algorithms and hybrid quantum-classical models for accelerated data processing and AI optimization (early research phase). Technologies: Apache Spark, Generative AI, Agentic AI, ELT Architecture, Kubernetes, Docker, Streamlit, vLLM, PostgreSQL, Snowflake, MCP, LLM Fine-Tuning, Quantum Computing (Research).

Senior Engineering Manager II at Walmart Global Tech, Sep 2021 - Jun 2023 · 1 yr 10 mos, Sunnyvale, California, United States, Led the development of a scalable, cloud-agnostic Unified Data Platform (UDP) for real-time and batch processing across hybrid environments. Built Apache Beam– and Spark–based pipelines with secure, high-throughput processing. Integrated multi-cloud and on-prem systems using prebuilt connectors. Delivered SDK and workbench for rapid pipeline design and deployment. Embedded data governance with lineage, schema registry, and audit logging. Automated CI/CD workflows for faster, reliable production rollouts. Enabled real-time monitoring, alerting, and resilient job recovery. Apache Beam, Spark, GCP Dataproc, Azure Databricks, Cloudbreak (HDP), Kubernetes, Airflow, CI/CD

Senior Engineering Manager I- Global Data Analytics Platform at Walmart Global Tech, Oct 2019 - Sep 2021 · 2 yrs, Sunnyvale, California, United States, Led the architecture and engineering of Walmart’s Global Data Analytics Platform, delivering scalable solutions across Finance, Supply Chain, Stores, and E-commerce. Integrated GCP, Cloudbreak (HDP), and Azure Databricks to enhance platform scalability. Enabled real-time and batch processing using Apache Beam, Kafka, and Cassandra. Streamlined workflows with Airflow and Automic for unified orchestration. Automated CI/CD pipelines using Kubernetes, Concord, and Looper. Reduced infrastructure cost through tool consolidation and automation. Optimized data delivery across domains ensuring high platform availability. GCP, Cloudbreak (HDP), Azure Databricks, Apache Beam, Kafka, Cassandra, Airflow, Kubernetes

Functional and Industry Analytics Manager - Big Data at Accenture, Mar 2019 - Oct 2019 · 8 mos, Washington, United States, Managed HPE’s Enterprise Analytics Platform (EAP) for Installed Base data, enabling advanced reporting and seamless data integration from Salesforce to Hive. Directed end-to-end data integration from Salesforce to Hive via IBSV. Transformed Hive data using Spark and Scala to create analytical DataFrames. Enabled dynamic reporting by migrating data to MongoDB and Qliksense. Developed Kafka pipelines for real-time clickstream ingestion into Hive and MongoDB. Scheduled batch processing using Oozie for timely data updates. Maintained high data accuracy across Hive tables supporting sales analytics. Spark, Scala, Hive, MongoDB, Qliksense, Kafka, Oozie, GitHub, Azure, Hortonworks

Big Data and Analytics Engineering Manager at TSYS, Jun 2017 - Feb 2019 · 1 yr 9 mos, Columbus, Georgia, United States, Led real-time data ingestion from mainframe systems into enterprise Data Lake clusters, architecting scalable big data zones and secure analytics workflows. Designed multi-zone Data Lake architecture using Solr, HBase, and HDFS. Directed CDC-based ingestion from IMS, DB2, and VSAM to Kafka streams. Built enrichment engines to transform JSON messages to Avro schema. Developed tokenization workflows for secure and compliant data processing. Enabled near real-time updates to OLTP systems via HBase and Elastic Search. Automated batch jobs to convert raw zone data into Parquet for analytics use. Kafka, HBase, Solr, HDFS, Avro, JSON, AWS, Oracle GoldenGate, Flink, Docker, Java, MAPR, Atlas, Bitbucket, SqData, Druid

Big Data Technical Manager at CDPHP, Oct 2016 - Jun 2017 · 9 mos, Albany, New York Metropolitan Area, Managed AWS-based big data infrastructure and analytics operations, leveraging EMR, Redshift, and Spark for scalable data processing and automation. Oversaw AWS services including EMR, S3, and Redshift to support data scalability. Processed high-volume datasets using Apache Spark for efficient computation. Utilized Tableau, R, SAS, and Python for advanced analytics and visualization. Automated deployments with Chef, Puppet, GitHub, and CloudFormation. Orchestrated workflows using Oozie and streamlined issue tracking via JIRA. Enhanced operational efficiency through robust cloud infrastructure oversight. AWS EMR, Redshift, S3, Apache Spark, Tableau, R, SAS, Python, Chef, Puppet, GitHub, CloudFormation, Oozie, JIRA

Education

IIT Delhi Online PG Diploma Advanced Comm Engg with Quantum AI Integration, IIT Delhi Online PG Diploma Advanced Comm Engg with Quantum AI Integration, Indian Institute of Technology, Delhi

Professional Certificate Programme in Agentic AI and Applications, Indian Institute of Technology, Madras

Public links

github: https://github.com/snbaskarraj

medium: https://medium.com/@snbaskar