Headline

Senior Software Engineer | Distributed Systems | AI Infrastructure | Kafka • Spark • Python • AWS | LLM-integrated pipelines| ML | Ex-AWS | seeking core AI/ML engineering roles

About

I build real-time data systems that power AI products at scale — and I'm actively moving into the AI engineering layer itself. At HPE Aruba, I own the streaming ingestion platform (Apache Spark Structured Streaming + Kafka) that feeds 50+ telemetry topics from network devices into a datalake serving AIOps ML models and alerting systems — running across 15+ production clusters globally. Beyond the pipelines, I've worked directly on AI features: prototyped LLM-based auto site name generation, designed a stateful Kafka streaming system for real-time 802.1x alert clearing, and built the data backend for customer-facing recommender features. On the side, I built an agentic LLM tool — a GitHub Copilot extension that connects to Apache Airflow via tool-calling, so engineers can query DAG state, task logs, and failures in natural language without leaving their IDE. Before HPE, I was at AWS on the SageMaker Feature Store team — implemented Apache Iceberg as an offline store format (cutting S3 query time for ML consumers) and built a production Python backfill script that saved thousands of dollars in DynamoDB costs across prod accounts. What I'm looking for next: AI Engineer or ML Platform Engineer roles where I can work closer to the model layer — LLM pipelines, inference infrastructure, agentic systems, or ML feature platforms. I bring strong production instincts from operating streaming systems at scale, and hands-on LLM work from both personal projects and applied research at HPE. Skills: Python · Scala · Java · Apache Kafka · Apache Spark · Apache Iceberg · AWS (S3, SageMaker, DynamoDB) · Kubernetes · Airflow · Avro/Protobuf · LLM tool-calling · Agentic AI

Location

Bengaluru

AI stack

claude code

claude

github copilot

langchain

langfuse

langgraph

Work experience

Senior Software Engineer at Hewlett Packard Enterprise, Nov 2024 - Present · 1 yr 10 mos, Bengaluru, Karnataka, India · Hybrid, • Kafka streaming (aiops-sparklin): Develop and own high-volume, multi-topic Kafka ingestion on a Spark/Scala Structured Streaming platform serving enterprise AIOps analytics; built schema-governed consumers handling JSON, Avro, and Protobuf payloads via Confluent Schema Registry and descriptor-based deserialization, managing topic onboarding, consumer-group behavior, and backpressure. • LLM-integrated Split-Site Detection: Built a PySpark pipeline detecting improperly split network sites from telemetry and lineage data, integrating an LLM to auto-generate geographically accurate names for new sites. • Aruba AI Insights: Contributed ingestion pipelines to Aruba's AI-driven anomaly-detection product; led analysis and solution design for real-time 802.1X alert-clearing.

Software Engineer 2 at Hewlett Packard Enterprise, Sep 2023 - Nov 2024 · 1 yr 3 mos, Bengaluru, Karnataka, India · Hybrid, • Recommender backends: Built the data backend and dashboards for split-site and DFS recommender features surfacing recommendations to customers. • Reliability: Diagnosed and resolved Spark OOM/heap failures and SLA breaches across multi-cluster Kafka workloads; automated CVE remediation across Java, Python, and Go. • Orchestrated fault-tolerant workflows on Kubernetes via Airflow (cloud) and Cron (on-prem), writing curated outputs to AWS, Azure, and MinIO object storage.

Software Development Engineer at Amazon Web Services (AWS), Jun 2022 - Jul 2023 · 1 yr 2 mos, Bengaluru, Karnataka, India, • Worked on Amazon SageMaker Feature Store - the ML feature infrastructure powering model training and inference - across ingestion, offline-store storage, and retrieval. • Apache Iceberg integration: Added Iceberg table format as a storage option for the offline store's S3 data, cutting S3 query time for downstream ML consumers. Worked extensively across S3, DynamoDB, EC2, and CloudWatch. • Cost-saving backfill (Python): Built and ran a production backfill script against DynamoDB in the Feature Store's prod accounts, saving thousands of dollars in DynamoDB usage costs. • AWS Java SDK 1.x to 2.x migration: Migrated a Spring Boot / Maven codebase to AWS SDK v2; resolved multiple SEV-2 incidents as on-call.

Software Engineer at Hewlett Packard Enterprise, Sep 2020 - Jun 2022 · 1 yr 10 mos, Bengaluru, Karnataka, India, • Validated MPI/HPC software releases across enterprise hardware configurations. • Built the SFWT Tools App (React.js) to automate firmware-testing workflows.

Intern at Hewlett Packard Enterprise, Feb 2020 - Sep 2020 · 8 mos, Bangalore Urban district, India

Internship Trainee at Zeetron Networks Pvt. Ltd. India, Jun 2019 - Jul 2019 · 2 mos, Greater Jaipur Area

Education

Bachelor of Technology, computer science, Mody university of science and technology

Public links

linkedin.com: https://linkedin.com/in/sujata-nimiwal