Headline

Staff Engineer(Data & AI) at Nutanix

About

Shiv is a Staff Engineer / Senior Manager at Nutanix and works on all things data platforms. Shiv is responsible for Apache Pulsar, NATS, Druid and Debezium and works on availability, scalability, observability, use cases, architecture, wrapper libraries, maintaining internal source code fork, contributing upstream etc. The data platforms are self hosted in AWS cloud and offered as a service to n nutanix products. Shiv is also a regular speaker at open source conferences having talked on 16 topics including MySQL replication internals, Apache Pulsar, Apache Bookkeeper, Apache Zookeeper and Event Sourcing. Below is a list of Shiv’s blog posts that have been published by different companies based on his work in open source codebases: https://streamnative.io/blog/streaming-war-and-how-apache-pulsar-is-acing-the-battle https://www.datastax.com/blog/why-nutanix-beam-selected-apache-pulsar-over-apache-kafka https://dev.mysql.com/blog-archive/mysql-5-7-6-introducing-multi-source-replication/ https://dev.mysql.com/blog-archive/mysql-5-7-4-change-master-without-stopping-slave-altogether/ https://dev.mysql.com/blog-archive/mysql-5-7-6-it-is-easier-to-switch-master-now/ https://www.shivjijha.in/2013/09/mysql-57-monitoring-replication-with.html Shiv's interests include software architecture, web backend design, micro-service architecture and platform, distributed systems, data/stream stores and open source software in general.

Location

Bengaluru

AI stack

claude code

claude

cursor

langchain

langfuse

langgraph

Work experience

Staff Engineer (Nutanix Enterprise AI) at Nutanix, Apr 2026 - Present · 6 mos, Bengaluru

Staff Engineer / TLM (Data Platforms) at Nutanix, Aug 2022 - Apr 2026 · 3 yrs 9 mos, Bengaluru, As the founding engineer of the data platform team, I grew it from the ground up into a strong group of 10+, shaping its vision, mentoring engineers, and driving hiring. I specialize in designing and scaling distributed data platforms—providing internal platform-as-a-service offerings for messaging, streaming, analytics, and time-series workloads used across 12 Nutanix products. 1. Architecture & Enablement → Partnering with product teams and staff engineers to define data platform best practices, ensuring the right architecture choices for distributed systems. 2. Systems at Scale → Designing, debugging, and benchmarking critical workloads (e.g., running ClickHouse on low-memory Kubernetes, integrating Apache Iceberg with ClickHouse, and optimizing analytics workloads in PostgreSQL). 3. Open Source Engagement → Maintaining internal forks of Apache Pulsar, NATS, Apache Druid, and ClickHouse, contributing fixes and features upstream, and helping bridge cutting-edge open-source stacks into Nutanix products. 4. Community & Evangelism → Active in the open-source and deep-tech ecosystem: I’ve delivered 30+ talks at Apache, CNCF, and Postgres conferences; organized 10+ meetups at Nutanix bringing together 1000+ engineers in Bangalore; and regularly write/mentor to grow the ecosystem. 5. Partnerships → Led strategic collaborations with ecosystem partners to bring enterprise-grade support for critical data platforms into Nutanix. 6. Leadership & Mentorship → Building, mentoring, and scaling a high-performing team while driving cross-functional initiatives across engineering, product, and open-source communities.

Senior Member Of Technical Staff (SDE 5) at Nutanix, Aug 2020 - Aug 2022 · 2 yrs 1 mo, 1. Leading stream platform team backing multiple SAAS products at nutanix. The platform is built atop apache pulsar, apache flink and presto query engine. 2. Lead an existing SaaS platform apps team (including user management, account management, billing and subscription, authorization service, external integrations with customer communication and ticketing platforms etc). Tasks involved stabilizing, reducing cutomer issues, steading ship and turning it to align with the new product roadmap. 3. Lead the first milestone release of clutch platform based on the open source lyft/clutch for a single pane devops experience for microservice life cycle management, configuration as a service, secret store etc). 4. Apache Pulsar committer, SME and Developer Advocate at Nutanix. 5. Evangelist for Apache Pulsar - Multiple guest blogs / did talks (https://www.youtube.com/watch?v=Bx4csRi1b8Y&list=PLA7KYGkuAD071myyg4X5ShsDHsOaIpHOq) to give back to Apache Pulsar community. 6. Hiring, tech lead and mentor for two tech teams.

Data Platform Developer 2 at Swiggy, Jul 2018 - Jan 2019 · 7 mos, 1. Implemented Schema registry platform on top of Apache Atlas project. Provides data governance via visibility with data pipelines as first class citizen, fulltext search on all kafka schema and data / change ownership. 2. Designed CDC stream to provide changes on transactional DBs as kafka messages. Explored denezium and uber hudi.

Software Developer 2 at Swiggy, Aug 2017 - Jun 2018 · 11 mos, Bengaluru, Karnataka, India, Lead backend development (mentoring 2 engineers) for swiggy app’s help feature. Implemented in java a helpcenter chat bot based on decision tree.

Senior Software Developer at MoveInSync, Jul 2016 - Jul 2017 · 1 yr 1 mo, Bengaluru Area, India, Designed and implemented a communication platform from scratch (sms + mobile notifications + email, extensible) including service provider failover, channel level failover and configurable rules.

Software Developer at MoveInSync, Nov 2015 - Jul 2016 · 9 mos

Software Application Engineer at MoveInSync, May 2015 - Nov 2015 · 7 mos, Bengaluru Area, India, 1. Prototype an infrastructure to Instrument, Monitor and alert on API calls. Tools used: Dropwizard java library, Graphite DB, Grafaba for dashboards.Syren for alerts 2. Prototype ELK based log archival and cleanup framework to support micro-service based distributed architecture. Impact: no server crash because of disk space on server instances, no manual errors. 3. Research, test / verify and fill the scripting gaps to move all natively run postgres instance usage to amazon RDS instances.

Software Developer 2 at Oracle MySQL, Aug 2013 - Apr 2015 · 1 yr 9 mos, Bangalore, India, Developed following features for Oracle's MySQL replication team: (details on blog : http://shiv4289.blogspot.in/search/label/work) 1. performance_schema tables to show replication health and performance. 2. Relaxed the condition to stop slave before change master, stop only what needs to be configured. 3. Removed the manual wait function before switching masters making and failover in MySQL easier. 4. Re-factored the code for changing masters making it easier to extend. 5. Removed shortcomings of lag monitoring to make delay monitoring in replication easier. Also, proposed a generic transaction header format for MySQL binary logging infrastructure, contributed to multi-source replication in MySQL with code and code reviews and lead the MySQL Replication engineering team for 5 months leading to two development milestone releases.

Software Developer 1 at Oracle MySQL, Jul 2012 - Aug 2013 · 1 yr 2 mos

Software Development Intern at Indian Institute of Technology, Bombay, May 2010 - Jul 2010 · 3 mos, Bombay, We were a group of 5 people .We desgned a software that provides the facility to create slides and save them at the desired location on the hard disk. The software can open files saved with ppt,pdf,jpg,jpeg,txt and all other normal extensions and these can be viewed as slide show. The slides can also be reorganized and saved as another project on the hard disk. Guide: Mr. Deepak B. Phatak, Dept of CSE, IIT Bombay. Technologies Used: Javafx,Java,CSS

Education

Bachelor of Engineering (BE), Information Technology, Birla Institute of Technology, Mesra

kendriya vidyalaya