Vishal Dewani profile photo 1 on Tal

Vishal Dewani

Lead AI QA Engineer | AI Quality Engineering | GenAI & LLM Testing | AI Agent & Agentic AI Testing | LLM Evaluation | Python, JavaScript, Playwright, Selenium | DeepEval | LangChain | CI/CD

Bengaluru

12+ years of work experience

Lloyds Banking Group
Sep 20263 mosPresent

Hyderabad

S&P Global
Jan 20233 yrs 9 mosSep 2026

Bengaluru

• Defined test strategies, automation roadmaps, and test plans for enterprise and AI-powered applications, aligned with sprint goals and release objectives. • Architected two automation frameworks — Playwright (Python) with GitHub Actions CI/CD, and Playwright (JS/TS) with Azure Pipelines — achieving 40% faster execution, 30% greater reliability than legacy Selenium; drove UI (95%), API (90%), DB (35%) coverage, cutting regression effort 60% and release cycle time 25%. • Led GenAI and Agentic AI testing adoption, validating multi-agent LLM workflows on LangChain and LangGraph — covering tool-calling, memory/state persistence, agent hand-offs, and reasoning-path validation. • Piloted RAG evaluation using RAGAS metrics (faithfulness, answer relevancy, context precision/recall) and DeepEval, benchmarking retrieval and generation quality for production adoption. • Built AI observability infrastructure using Langfuse for Gemini content generation — automated evaluators, scoring frameworks, dashboards, alerting; validated 2000+ non-deterministic outputs including guardrail effectiveness (Google Model Armor), improving reliability 30% and cutting manual validation effort 90%. • Performed end-to-end functional, regression, integration, API, database, accessibility, and performance testing ensuring high-quality releases. • Championed Shift-Left testing across automation and AI validation workflows, reducing post-production defects by 40%. • Monitored and optimized Continuous Testing pipelines, maintaining 90%+ automated test pass rate. • Mentored QA engineers on automation and AI-specific testing, guiding the shift from deterministic to probabilistic testing, improving team productivity 25%. • Participated in Agile ceremonies (backlog grooming, sprint planning, stand-ups, retrospectives), contributing to estimation and release planning. • Led RCA investigations for critical defects, including AI system failures, driving corrective actions and quality improvements.
S&P Global
May 20202 yrs 8 mosDec 2022

Ahmedabad, Gujarat, India

S&P Global
Aug 20172 yrs 10 mosMay 2020

Ahmedabad Area, India

Ecosmob Technologies Pvt Ltd
Apr 20175 mosAug 2017

• Collaborated with business clients and project stakeholders to distill technical and functional user requirements. • Contributed to test planning by creating requirement traceability matrices and designing manual/automated test cases for UI, API, and database testing, ensuring 100% coverage and enhancing defect detection by 60% • Designed, executed, and automated test cases for UI, API, and database testing, improving defect detection by 60% across VOIP web/mobile applications, IVR systems, and Android TV apps through functional, integration, system, localization, exploratory, and UAT testing. • Developed a scalable, data-driven automation framework using Java, Selenium WebDriver, TestNG, and Page Object Model (POM), cutting regression execution time by 70% and enhancing CI/CD integration with Git and Jenkins. • Actively contributed to Agile ceremonies (daily stand-ups, sprint planning, retrospectives), optimizing test planning and sprint execution through data-driven insights. • Mentored and coached junior QA engineers in automation best practices, Agile methodologies, and framework implementation, boosting team efficiency by 30%.
Ecosmob Technologies Pvt Ltd
Jun 201611 mosApr 2017

eClinicalWorks
Jun 20142 yrs 1 moJun 2016

Ahmedabad Area, India

• Contributed to test planning by creating requirement traceability matrices and designing manual/automated test cases for UI, API, and database testing, ensuring 100% coverage and enhancing defect detection by 60% • Assisted in developing and maintaining test data for the eCW Enterprise Patient Portal (EPP) using ADT/CCD interfaces, leveraging XSL/XML/JSP to validate GUI changes, improving test accuracy and reducing defects by 30%. • Supported the configuration & maintenance of internal/client test environments for integrated testing, product demonstrations, and training, reducing 25% time. • Assisted in automation initiatives, developing and maintaining Selenium-based regression test scripts using Selenium WebDriver, Java, and TestNG, helping reduce regression execution time by 30%. • Performed initial troubleshooting and RCA for server-side issues, assisting in client escalations and reducing resolution time by 40% while collaborating with senior engineers and R&D teams.
StayinFront
Jul 20128 mosFeb 2013

GitHub
@vish1202

Vishal’s AI Stack

claude
claude
github copilot
github copilot
langchain
langchain
langfuse
langfuse
githubcopilot
githubcopilot
Google ADK
Google ADK

Education

Information Technology

LDRP institute of technology and research

20094 yrs2013
St. Mary's high school

Bachelor of Science (B.S.)

St. Mary's high school

20072 yrs2009

Join Vishal and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Vishal Dewani on Tal

Headline

Lead AI QA Engineer | AI Quality Engineering | GenAI & LLM Testing | AI Agent & Agentic AI Testing | LLM Evaluation | Python, JavaScript, Playwright, Selenium | DeepEval | LangChain | CI/CD

About

Lead QA Engineer | AI-Driven Test Automation Specialist with 12+ years of experience designing, developing, and scaling test automation frameworks using Python, Selenium, Playwright, and JavaScript. Proven track record of improving software quality, optimizing test coverage, and accelerating delivery across Healthcare IT, Telecom, and Financial Analytics domains. Hands-on expertise in GenAI, Agentic AI, and LLM testing — including multi-agent workflow validation (LangChain, LangGraph), prompt and hallucination testing, RAG evaluation (RAGAS, DeepEval), AI-generated content validation, and observability using Langfuse. Strong background in Agile delivery, API and UI automation, CI/CD integration, quality strategy, and team mentorship, with a focus on building scalable, high-quality testing solutions for enterprise systems.

Location

Bengaluru

AI stack

claude

github copilot

langchain

langfuse

githubcopilot

Google ADK

Work experience

Senior AI Quality Engineer at Lloyds Banking Group, Sep 2026 - Present · 3 mos, Hyderabad

Lead Quality Engineer at S&P Global, Jan 2023 - Sep 2026 · 3 yrs 9 mos, Bengaluru, • Defined test strategies, automation roadmaps, and test plans for enterprise and AI-powered applications, aligned with sprint goals and release objectives. • Architected two automation frameworks — Playwright (Python) with GitHub Actions CI/CD, and Playwright (JS/TS) with Azure Pipelines — achieving 40% faster execution, 30% greater reliability than legacy Selenium; drove UI (95%), API (90%), DB (35%) coverage, cutting regression effort 60% and release cycle time 25%. • Led GenAI and Agentic AI testing adoption, validating multi-agent LLM workflows on LangChain and LangGraph — covering tool-calling, memory/state persistence, agent hand-offs, and reasoning-path validation. • Piloted RAG evaluation using RAGAS metrics (faithfulness, answer relevancy, context precision/recall) and DeepEval, benchmarking retrieval and generation quality for production adoption. • Built AI observability infrastructure using Langfuse for Gemini content generation — automated evaluators, scoring frameworks, dashboards, alerting; validated 2000+ non-deterministic outputs including guardrail effectiveness (Google Model Armor), improving reliability 30% and cutting manual validation effort 90%. • Performed end-to-end functional, regression, integration, API, database, accessibility, and performance testing ensuring high-quality releases. • Championed Shift-Left testing across automation and AI validation workflows, reducing post-production defects by 40%. • Monitored and optimized Continuous Testing pipelines, maintaining 90%+ automated test pass rate. • Mentored QA engineers on automation and AI-specific testing, guiding the shift from deterministic to probabilistic testing, improving team productivity 25%. • Participated in Agile ceremonies (backlog grooming, sprint planning, stand-ups, retrospectives), contributing to estimation and release planning. • Led RCA investigations for critical defects, including AI system failures, driving corrective actions and quality improvements.

Senior Quality Engineer at S&P Global, May 2020 - Dec 2022 · 2 yrs 8 mos, Ahmedabad, Gujarat, India

Quality Engineer II at S&P Global, Aug 2017 - May 2020 · 2 yrs 10 mos, Ahmedabad Area, India

Senior Test Engineer at Ecosmob Technologies Pvt Ltd, Apr 2017 - Aug 2017 · 5 mos, • Collaborated with business clients and project stakeholders to distill technical and functional user requirements. • Contributed to test planning by creating requirement traceability matrices and designing manual/automated test cases for UI, API, and database testing, ensuring 100% coverage and enhancing defect detection by 60% • Designed, executed, and automated test cases for UI, API, and database testing, improving defect detection by 60% across VOIP web/mobile applications, IVR systems, and Android TV apps through functional, integration, system, localization, exploratory, and UAT testing. • Developed a scalable, data-driven automation framework using Java, Selenium WebDriver, TestNG, and Page Object Model (POM), cutting regression execution time by 70% and enhancing CI/CD integration with Git and Jenkins. • Actively contributed to Agile ceremonies (daily stand-ups, sprint planning, retrospectives), optimizing test planning and sprint execution through data-driven insights. • Mentored and coached junior QA engineers in automation best practices, Agile methodologies, and framework implementation, boosting team efficiency by 30%.

Test Engineer at Ecosmob Technologies Pvt Ltd, Jun 2016 - Apr 2017 · 11 mos

Software Specialist at eClinicalWorks, Jun 2014 - Jun 2016 · 2 yrs 1 mo, Ahmedabad Area, India, • Contributed to test planning by creating requirement traceability matrices and designing manual/automated test cases for UI, API, and database testing, ensuring 100% coverage and enhancing defect detection by 60% • Assisted in developing and maintaining test data for the eCW Enterprise Patient Portal (EPP) using ADT/CCD interfaces, leveraging XSL/XML/JSP to validate GUI changes, improving test accuracy and reducing defects by 30%. • Supported the configuration & maintenance of internal/client test environments for integrated testing, product demonstrations, and training, reducing 25% time. • Assisted in automation initiatives, developing and maintaining Selenium-based regression test scripts using Selenium WebDriver, Java, and TestNG, helping reduce regression execution time by 30%. • Performed initial troubleshooting and RCA for server-side issues, assisting in client escalations and reducing resolution time by 40% while collaborating with senior engineers and R&D teams.

Project Trainee at StayinFront, Jul 2012 - Feb 2013 · 8 mos

Education

Information Technology, Information Technology, LDRP institute of technology and research

Bachelor of Science (B.S.), St. Mary's high school

Public links

github: https://github.com/vish1202