Headline

SDET @ Adobe | Evaluating AI Agents | Ex-Fivetran, ZS | GenAI Evaluation · LLM-as-Judge · Multi-Agent Testing

About

SDET building GenAI evaluation systems and AI-powered testing infrastructure at Adobe. Currently driving agentic quality for LLM-powered features — designing multi-layer evaluation pipelines with LLM-as-judge scoring, multi-agent testing across chained sub-agents and MCP tool calls, and security automation covering prompt injection, SQL injection, and multi-tenant isolation. At Adobe, I architected an 8-layer GenAI evaluation framework scoring 300+ golden queries across 8 business domains, built a 150+ test case security suite for LLM features, developed a Playwright-based automation framework integrated with Claude Code via custom skills and subagents, and ran infrastructure-level load tests. Previously built CI/CD optimizations at Fivetran (65% pipeline time reduction) and automation frameworks at ZS Associates (adopted by 10+ teams). Core: LLM evaluation, prompt engineering, multi-agent testing, MCP integration, Playwright, Python, TypeScript, AWS (EMR, DynamoDB, S3), Azure OpenAI, Amazon Bedrock Open to opportunities — especially AI-first companies. Open to relocation across APAC and EU.

Location

Bengaluru

AI stack

claude code

codex

cursor

gemini

langchain

langsmith

Work experience

Software Development Engineer in Test 2 at Adobe, Oct 2024 - Present · 2 yrs, Bangalore Urban, Karnataka, India, - Architected a GenAI evaluation framework with 8-layer LLM-as-judge scoring pipeline, evaluating 300+ golden queries across 8 business domains for accuracy, hallucination, and consistency - Engineered 150+ automated security test cases covering prompt injection, SQL injection, multi-tenant isolation, PII leakage, and guardrail bypass — integrated into CI for every release - Developed a Playwright-based UI and API automation framework integrated with Claude Code via custom slash-command skills and dedicated subagents (test planner, generator, healer) - Designed and ran infrastructure-level load tests on a 16-node AWS EMR cluster, identifying 2 production-blocking bugs in AWS API throttling and DynamoDB parameter validation before GA - Shipped an LLM-powered analytics feature across 2 release stages (feature-flagged → full GA) with 0 late-cycle defects and 0 customer-reported issues - Identified 300+ bugs across 4 release cycles; maintained 0 CSMs on owned scope for 3 consecutive releases - Built reusable automation framework modules — dynamic validation engine, partial-run execution, auto-save — unblocking 30+ previously failing test suites - Built an AI-assisted impact analysis tool mapping Git PRs and JIRA tickets to regression-prone areas using MCP integrations

Software Development Engineer in Test at Fivetran, Mar 2024 - Aug 2024 · 6 mos, Bengaluru, Karnataka, India, - Reduced CI pipeline execution time by 65% (120 → 45 minutes) through strategic parallelism and test segregation across Buildkite - Raised end-to-end test coverage to 80% by identifying automation gaps and fixing 30+ flaky E2E tests, restructuring the suite into unit, integration, and E2E tiers - Led cross-team testing for a critical shared library update impacting dependent teams

Software Developer in Test at ZS, Jul 2022 - Feb 2024 · 1 yr 8 mos, Bengaluru, Karnataka, India

Software Developer in Test Intern at ZS, Jan 2022 - Jun 2022 · 6 mos, Banglore

Education

Bachelor's, Computer Software Engineering, Shri ramdeobaba college of engineering and management