Headline

Software Engineer III @ Walmart Global Tech | Data Platform & Backend | Kafka · Spark · Python · Java · BigQuery · Cassandra

About

I build data platform and backend systems at Walmart Global Tech: Kafka streaming, Spark/Scala batch pipelines, and Cassandra-backed GraphQL APIs on GCP, owned from design doc to production monitoring. Recent work: • Designed an always-on Python + Kafka → BigQuery ingestion pipeline across two regional clusters with at-least-once delivery; raised consumer throughput ~6× (1.3K → 8K rec/s) • Replaced a rate-limited REST integration with an async Kafka service (Java 17/Spring Boot) that sent 1.28M catalog items for English→French translation • Built the serving path for AI-generated image alt text on Walmart Canada item pages: Spark → Cassandra → federated GraphQL, with DGS BatchLoader removing N+1 reads • Built a 200M+ record Spark → Cassandra publisher on Dataproc and raised sustained write throughput by fixing client-side bottlenecks • Diagnosed a volume collapse in a 4.1M-record pipeline and restored output to ~200K publishable pages • Led zero-downtime migrations: JDK 8 → 17 across 7 services, and Kafka mTLS → SPIFFE identity Before that, 2.5 years on Walmart's advertising and marketing data platform: Airflow/Spark pipelines and GCP infrastructure as code. I like problems where correctness, throughput, and reliability all matter: delivery semantics, schema design, migrations that can't go down, and finding the real cause of an incident. Core stack: Python · Java · Scala · SQL · Apache Kafka · Apache Spark · BigQuery · Cassandra · Airflow · GraphQL (Netflix DGS) · Spring Boot · GCP Dataproc · Kubernetes

Location

Bengaluru

AI stack

anthropic apis

antigravity

claude code

claude

gemini

hugging face

Work experience

Software Engineer III at Walmart Global Tech India, Aug 2024 - Present · 2 yrs 4 mos, Bangalore Urban, Karnataka, India, • Real-time streaming ingestion: Designed and built an always-on Python + Kafka → BigQuery pipeline over two regional clusters, owning a 16-entry architecture decision log. At-least-once delivery via ack-after-write commits and idempotent insert IDs; raised throughput ~6× (1.3K → 8K rec/s) by tuning the Kafka TCP receive window. Client IP and cookies never persisted. • Incident ownership: Resolved a consumer crash loop caused by oversized BigQuery streaming inserts with byte-capped batching, wrote the incident review, and shipped health probes and Cloud Monitoring alerts to email and Slack. • Event-driven translation service: Replaced a rate-limited REST integration with async Kafka messaging, sending 1.28M catalog items for English→French translation. Java 17/Spring Boot service in three independently runnable phases with mTLS, completion tracking, and graceful shutdown. • Multi-market architecture: Sole owner of the epic unifying CA/MX/US alt-text pipelines into one config-driven Spark job (a new market needs config only), rolled out via dual-write, shadow validation, and cutover. • AI content serving: Built the production path for AI-generated alt text on Canada item pages: Spark → Cassandra (3.2M records, multi-tenant schema with composite partition keys) → federated GraphQL with DGS BatchLoader eliminating N+1 reads. • Pipeline diagnosis: Traced a topic-page volume collapse in a 4.1M-record Spark pipeline with a 7-stage validation funnel; restored output to 199,917 publishable pages and grew eligible keywords from 221K to 436K. • Distributed batch: Spark/Scala jobs on Dataproc up to 50 executors, including a 200M+ record Cassandra publisher; raised write throughput via prepared statements, batch tuning, and bounded in-flight writes. • Zero-downtime migrations: JDK 8 → 17 across 7 services and Kafka mTLS → SPIFFE identity. • JVM & Kubernetes: Root-caused production OOMKills via heap-dump and GC analysis; retuned heap, limits, and autoscaling.

Software Engineer II at Walmart Global Tech India, Jul 2022 - Aug 2024 · 2 yrs 2 mos, Bengaluru, Karnataka, India, • Advertising data platform: Built and productionized the DSP and EA advertising Python/Airflow pipelines across 62 PRs: dynamic DAG generation, dynamic Dataproc sizing, execution timeouts, change-managed releases. • Audience segmentation: Refactored the Spark audience-segmentation service feeding the DSP across 32 PRs, adding retries, SFTP delivery, job metrics, and Prometheus alerts to Slack. • Walmart Mexico welcome email: Delivered the BigQuery → GCS extraction and email-validation pipeline, then added unit tests to a previously untested Java service. • ASDA analytics & GCP IaC: Integrated config and secrets services across ASDA item, order, and marketing Scala/Spark aggregations; set up GCP infrastructure as code (IAM, service accounts, VPC Service Controls) across 120+ commits.

SDE at Walmart Global Tech India, Jan 2022 - Jun 2022 · 6 mos, Bengaluru, Karnataka, India, • Built and validated Looker dashboards for business stakeholders. • Improved Scala/Spark streaming pipelines for performance and scalability. • Added unit tests for Spark jobs, improving reliability and code coverage.

Campus Ambassador at HackerEarth, Apr 2020 - Jul 2020 · 4 mos

Logistics Executive at Equinox, IIIT Lucknow, Feb 2020 - Feb 2020 · 1 mo, IIIT LUCKNOW, It was an fantastic experience while working as an Executive of Techfest of IIIT Lucknow.

Coordinator Of Fine Arts Club at Indian Institute of Information Technology Lucknow, Nov 2019 - Apr 2021 · 1 yr 6 mos, Lucknow

Education

Bachelor of Technology - BTech, Information Technology, Indian Institute of Information Technology Lucknow

Public links

leetcode: https://leetcode.com/GiyuTomioka007

nightcodeowl.github.io: https://nightcodeowl.github.io/NightCodeOwl/