Headline

Building reliable and safe AI systems | Applied ML Researcher | LLM Evals, Uncertainty & Financial AI

About

I’m an applied ML researcher and Senior Member Data Scientist at Perfios, focused on building and evaluating LLMs, AI agents, and multimodal systems for financial applications. I specialize in technical AI safety and reliability, with work spanning agent evaluation, uncertainty quantification, model calibration, guardrails, and failure-mode analysis. I develop evaluation frameworks for financial agents, covering task- and process-level metrics, automated evaluators, LLM-as-judge bias and calibration, and inference-time scaling. In parallel, I’m building a BFSI-specific benchmark to measure calibration and cultural and demographic bias in financial LLMs, while testing steering, context conditioning, and fine-tuning as mitigation methods. I also work on guardrails for LLM-based banking applications aligned with RBI’s FREE-AI recommendations. My production work includes measurable impact across latency, accuracy, and reliability: → Reduced multimodal document-classification latency by 97.5%, from 8 seconds to 200 milliseconds → Improved generalized table-detection accuracy by 27.6% → Built a document-quality service that identifies low-quality inputs at 92% precision before downstream processing → Adapted PaliGemma2 through LoRA for financial document understanding, achieving a 0.85 TEDS score Outside my core role, I collaborate with Georgia Tech’s Financial Services Innovation Lab and HCAI Lab on financial AI research spanning benchmark generation, model behavior, and role steering. My work has been presented at the EMNLP 2025 FinNLP Workshop and the AAAI 2026 Workshop on Agentic AI in Financial Services. I also placed first globally among 150+ teams in the Reasoning Dataset Creation Challenge and have released open-source artifacts on Hugging Face with more than 59K downloads. I’m especially interested in technical AI safety, agent evaluations, scalable oversight, uncertainty and calibration, model behavior, and interpretability. I’m open to research collaborations and roles such as Applied Scientist, ML Research Engineer, or Senior Data Scientist.

Location

Bengaluru

AI stack

claude code

codex

deepseek

hugging face

notebooklm

Google ADK

Work experience

Research Volunteer at Georgia Institute of Technology, Aug 2025 - Present · 1 yr 1 mo, Remote, - Developing generalized financial reasoning models and validating their performance through rigorous ablation studies across multiple model scales and difficulty bands.

Senior Data Scientist at Perfios, Apr 2025 - Present · 1 yr 5 mos, Bengaluru, • Received the Circle of Excellence Award for research and contributions to generative AI and financial document intelligence. • Leading uncertainty-quantification research for financial LLMs by building a BFSI-specific benchmark, auditing cultural and demographic biases, measuring calibration across numerical-precision conditions, probing cues associated with miscalibration, and testing steering, context conditioning, and fine-tuning as mitigation methods. • Developing guardrails for LLM and agent applications in banking by mapping vulnerability surfaces, constructing adversarial and failure-mode evaluations, and testing output controls aligned with RBI’s FREE-AI recommendations for safe, fair, explainable, and robust AI. • Developed and now monitor a reference-free document image-quality assessment service using ViT regression; the production system identifies low-quality documents at 92% precision before they enter HDB’s downstream applications. • Adapted PaliGemma2 via LoRA on financial documents, achieving a 0.85 TEDS score on internal benchmarks for structured document understanding.

Data Scientist at Perfios, Apr 2024 - Mar 2025 · 1 yr, Hybrid, • Resolved inference latency bottlenecks (8s → 200ms) by compressing a large multimodal classifier into a lightweight student architecture via Distillation and quantization techniques, preserving statistical parity in F1-scores. • Boosted generalized table detection accuracy by 27.6% through the careful, semi-synthetic data curation for finetuning model and rigorous evaluation of YOLOv8 models. • Augmented vision-only Table Structure Recognition (TSR) systems with a semantic row-detection module. Fine-tuned specialized text-encoders to capture lexical dependencies in bank-statements, with real-time inference (<40ms).

Applied Scientist - I at Perfios, Jun 2023 - Mar 2024 · 10 mos, Bengaluru, Karnataka, India, Joined Perfios full-time as an Applied Scientist, building foundational ML systems for financial document intelligence. This role marked the beginning of production-scale research at the intersection of computer vision, NLP, and FinTech. • Developed early-stage pipelines for Table Structure Recognition (TSR) in complex financial documents, laying the groundwork for the semantic row-detection module later scaled in production. • Researched and benchmarked multimodal classification approaches for document type identification, informing the model compression strategy that achieved a 97.5% latency reduction in subsequent work. • Contributed to data curation workflows for semi-synthetic financial document datasets used in YOLOv8-based table detection fine-tuning. • Collaborated cross-functionally with the document processing and engineering teams to integrate research prototypes into Perfios's core data extraction pipelines.

Undergraduate Student Researcher at Indian Institute of Technology, Kharagpur, Aug 2022 - Apr 2023 · 9 mos, The research project titled "Impact of Social Engagement on Crowdfunding Campaigns", is currently being done under the guidance of Dr. Swagato Chatterjee, COEAI, IITKGP. The project tries to identify the influencing factors of social media posts related to crowdfunding campaigns, and tries to correlate them with the funds raised through the campaigns. This is achieved in 3 steps: •Identifying the characteristics that inspire social media users to participate in crowdfunding campaigns •Identifying the relationship between social media engagement and the amount of funds raised for the initiative •Identifying the connection between the funds raised and the uncovered motivating factors

Data Science Intern at Perfios Software Solutions Pvt. Ltd., May 2022 - Jul 2022 · 3 mos, Bengaluru, Karnataka, India, During the 3 months of this Internship, I have worked on 2 projects, involving basic DBMS, Pandas, Flask. The projects I have worked on are as follows, 1. Evaluation of BI tools 2. Banking product recommendations for current account holders.

Education

Master of Science - MS, Data Science, International Institute of Information Technology Hyderabad (IIITH)

Bachelor's degree, Indian Institute of Technology, Kharagpur

Public links

github: https://github.com/akhil-theerthala

medium: https://medium.com/@akhiltvsn