Headline

AI Product Manager - LLM evaluation & output quality for production AI

About

I build AI products in domains where being wrong has real consequences - tax law, compliance, regulated enterprise systems. When the rules are strict, the data is messy, and your users are the domain experts, "fast but plausible" isn't good enough. The output has to be right, and provably so. That's my specialty: LLM output quality and evaluation - the eval loops, rubrics, and benchmarks that make a production AI system trustworthy enough to ship where errors aren't an option. At Instead (AI-native tax research platform), I own output quality for the AI research engine: → Built the citation-accuracy evaluation loop and a four-dimension scoring rubric (answerability, accuracy, citation quality, actionability) - benchmarking model configs and catching regressions before release, sustaining ~95% citation accuracy on the eval set. → Ran a competitive model-benchmarking program scoring the platform head-to-head against three competing tax-AI research platforms across 50+ structured scenarios. → Diagnosed and fixed a systemic "knowledge–citation gap" failure mode - plausible-but-wrong citations, a precision problem - making outputs traceable and audit-ready. → Architected and QA'd a 270K+-record RAG corpus across 100+ legal source types. Before Instead, I shipped product and revenue from zero: built Keka's Background Verification module 0→1 (Checkr API) — 28 US enterprise clients, $2.7M MRR, Keka's first US-market expansion - and launched Multiplier's PosterElite compliance partnership end-to-end in 21 days. I think in ML terms about product problems - precision/recall, false positives, calibration, failure modes - and can talk through my own eval work in that language, cold. Focused on AI PM roles where output quality, evaluation, and reliability are the hard part. Let's talk: zoeb.nomi@gmail.com

Location

Bengaluru

AI stack

antigravity

cursor

Work experience

Product Manager at Instead, Sep 2025 - Present · 1 yr, Bengaluru, I own output quality, evaluation, and reliability for Instead's AI tax-research engine — a production LLM/RAG system in a domain where a wrong answer is a filing error. - Built the citation-accuracy evaluation loop (flag → classify by failure mode → weekly review) and a four-dimension scoring rubric — answerability, accuracy, citation quality, actionability — to benchmark model configs and catch regressions pre-release, sustaining ~95% citation accuracy on the eval set. - Ran a competitive model-benchmarking program scoring the platform head-to-head against three competing tax-AI research platforms across 50+ structured scenarios; designed a multi-step "pipeline-collapse" eval measuring chain coherence and self-QC on the full workflow. - Diagnosed a systemic "knowledge–citation gap" failure mode (plausible-but-wrong citations — a precision problem) and shipped a backend fix attaching document ID + sub-type to every cited source, making outputs traceable and audit-ready. - Architected and QA'd the 270K+-record RAG corpus across 100+ legal source types (Tax Court, IRS publications, state case law); led remediation of ~139K scraped documents to raise retrieval reliability. - Led legal-support research across 160+ tax strategies (Legal Support Matrix from Tax Court cases, IRS rulings, audit guidance) and shipped the Source Explorer surface.

Product Manager at Multiplier, Mar 2025 - Sep 2025 · 7 mos, Multiplier is a global employment platform enabling companies to hire and manage distributed teams compliantly across 150+ countries. I owned the Value-Added Services vertical — procurement, background verification, ITSM, and product partnerships — turning ancillary capabilities into a standalone revenue-generating product line. - Generated $100K incremental revenue and 47% operational efficiency gain by building SOPs for the VAS vertical from scratch — defining the operating model for a new product area - Launched PosterElite compliance partnership in a record 21 days — owned the full cycle from partner due diligence through technical integration and go-to-market - Built and shipped the BGV module across all employee types, expanding the platform's compliance capabilities for global workforce management - Drove product-market fit for VAS by identifying high-value partner integrations and optimizing adoption through operational workflow design

Associate Product Manager 2 at Keka HR, Jul 2024 - Feb 2025 · 8 mos, On-site, Keka is one of India's leading HR technology platforms, serving enterprise clients with a $25M+ ARR CoreHR product suite. Promoted to lead product strategy across SaaS App Management, Employee Onboarding & Offboarding, Document Management, Asset Management, and Integrations. - Built the Background Verification (BGV) module from 0→1 — integrated Checkr API, onboarded 28 US enterprise clients, and generated $2.7M MRR, driving Keka's first expansion into the US market - Led the Exit Module revamp: shipped 10+ features in a single quarter, reducing employee offboarding time from 7 to 5 days (~28% faster) and increasing CSAT by 80% - Executed go-to-market for enterprise features — product training and client webinars reaching 1,300+ attendees, generating 200+ upsell requests and 30% feature adoption increase - Owned the CoreHR product roadmap aligned to company OKRs, driving cross-functional collaboration across engineering, design, and customer success with 100% on-time delivery - Built NPS analysis framework and customer feedback loops — identified product improvements that increased client satisfaction by 15% and reduced churn

Associate Product Manager at Keka HR, Jul 2023 - Jul 2024 · 1 yr 1 mo, Hyderabad, Owned product development for Employee Onboarding & Offboarding, Document Management, Asset Management, and Integrations modules within Keka's CoreHR platform. - Led the employee login revamp — shipped 6 SSO integrations, login history mapping, and Azure AD/Google Workspace integration, driving 27% adoption increase and $121K in upsell revenue - Reduced product development cycle time by 35% through process optimization, accelerating time-to-market for key features across multiple modules

Senior Product Analyst at Keka HR, Dec 2022 - Jun 2023 · 7 mos, Hyderabad, Conducted market research, competitive analysis, and user research to inform product strategy across Keka's HRMS platform. - Led competitive analysis that shaped the asset management module roadmap — delivered employee asset requests and asset reports, increasing user engagement by 37% and driving 11% upsell growth - Created PRDs and user stories aligning cross-functional teams on feature development priorities - Partnered with data teams to introduce user engagement dashboards, improving visibility into product usage and decision-making quality

Business Analyst at HurixDigital, Sep 2021 - Sep 2022 · 1 yr 1 mo, HurixDigital is an enterprise content and learning technology company serving global EdTech and corporate learning markets. - Reduced delivery times by 43% by implementing agile processes for the enterprise Learning Management System, transforming the product development workflow - Drove 31% growth in customer conversions through end-to-end deployment of video learning solutions - Owned the 2022 LMS product roadmap, driving feature parity and performance optimization across engineering and design teams

Content Writer at Ghaas, Oct 2019 - Dec 2019 · 3 mos, Pune Area, India, - Developed content strategies for clients in the talent and tech domains, increasing their web traffic by 15%. - Collaborated with cross-functional teams to build effective content marketing funnels and vision documents.

Education

General Management Program, Cohort 6, Stoa

Marathwada Institute of Technology Engineering College, Satara Road.