Shiva Yada profile photo 1 on Tal

Shiva Yada

Data Engineer | Ex-Razorpay | 3+ YOE | Building Scalable ETL Pipelines on AWS (Airflow, dbt, Redshift, Python)

Hyderabad

3+ years of work experience

dataAlpha.ai
Dec 20257 mosPresent

Hyderabad, Telangana, India

Working on Financial Data Engineering and Security Master platforms supporting hedge fund clients across multiple asset classes, including Equities, Bonds, Options, Futures, and Commodities. • Own and support Security Master workflows, processing Bloomberg market data for instrument classification, reference data enrichment, and downstream financial applications. • Serve as Subject Matter Expert (SME) for client-specific valuation and reference data platforms, driving issue resolution, production support, and enhancement delivery. • Design and maintain valuation exception reporting and monitoring solutions to identify pricing anomalies across OTC and listed instruments. • Optimize financial data processing for complex attributes such as coupon schedules, paydown rates, CPI-linked instruments, and security reference data. • Resolve critical data quality issues involving security identifier mapping, exchange normalization, pricing accuracy, and reference data integrity. • Deliver production enhancements and scalable solutions supporting hedge fund operations, valuation workflows, and enterprise financial data platforms. Skills: Financial Data Engineering, Security Master, Bloomberg, Market Data, Reference Data, Valuation, Capital Markets, Hedge Funds, SQL, Python, Data Quality, Production Support.
Razorpay
Jul 20256 mosDec 2025

Hybrid

At Razorpay POS, I engineer scalable ELT pipelines and reporting solutions that ensure high data availability, accuracy in financial operations, and faster decision-making across payments and merchant services. My work focuses on automation, reliability, and data accessibility, enabling teams to move from manual effort to strategic impact. I have automated end-to-end workflows using Airflow, dbt, Python, and AWS Redshift, boosting data availability from 85% → 99% and reducing manual reporting by 90%. I also develop robust transformation pipelines to streamline billing accuracy, merchant settlements, and compliance reporting, while strengthening pipeline monitoring, anomaly detection, and authentication security (migrated Airflow from app passwords to OAuth 2.0). On the analytics side, I’ve built and maintained 20+ interactive dashboards in Metabase and Tableau, covering payments, refunds, and merchant KPIs. These dashboards are actively used by business and operations teams, improving decision-making efficiency by 15% and saving 20+ analyst hours/month through automation. Key Achievements: Engineered scalable workflows for real-time transaction ingestion & settlements with 99.9% uptime. Automated compliance & sales reporting by ingesting Gmail & SFTP sources via Airflow DAGs. Led root-cause investigations on settlement discrepancies; built alerting logic to flag anomalies proactively. Optimized financial reporting queries in ART, improving reconciliation, inventory, and performance reporting. Leveraged AWS S3 & Redshift Spectrum for external tables & long-term data archiving, strengthening governance. Reduced workflow downtime by 40% through improved monitoring and faster issue resolution.
Razorpay
Apr 20232 yrs 4 mosJul 2025

Bengaluru, Karnataka, India

Automated end-to-end ELT pipelines with Airflow, dbt, and Python, reducing manual report generation time by 90%. Engineered scalable workflows for real-time transaction ingestion and merchant settlements, ensuring 99.9% uptime and improving billing accuracy. Built Metabase dashboards powered by Redshift-backed models for payments, refunds, and merchant KPIs. Developed dynamic Airflow DAGs to ingest and clean data from Gmail and SFTP sources, automating compliance and sales reporting. Led root-cause investigations on settlement discrepancies; implemented alerting logic to automatically flag anomalies. Optimized financial reporting queries in ART, supporting inventory, performance, and reconciliation reports. Partnered with product and finance teams to define KPIs, deliver custom dashboards, and automate recurring data tasks. Leveraged AWS S3 and Redshift Spectrum for external table setup and data archiving, enhancing data governance and accessibility.
MountBlue Technologies
Jan 20234 mosApr 2023

Bengaluru, Karnataka, India

Chegg Inc.
May 20222 mosJun 2022

Telangana, India

Education

Bachelor of Technology - BTech, Information Technology

St. Martin's Engineering College

20184 yrs2022

Join Shiva and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Shiva Yada on Tal

Headline

Data Engineer | Ex-Razorpay | 3+ YOE | Building Scalable ETL Pipelines on AWS (Airflow, dbt, Redshift, Python)

About

I’m a Data Engineer with 3+ years of experience building scalable data pipelines, automating reporting workflows, and improving data reliability for business-critical systems. At Razorpay, I engineered pipelines that reduced reporting time by 90% for merchant settlement data. I work across Python, SQL, Airflow, dbt, AWS, and Redshift to turn fragmented data into clean, high-quality models that power analytics, decision-making, and product insights. My focus is on performance, accuracy, and automation. Open to Data Engineer, Analytics Engineer, and BI Engineer roles where I can build efficient, resilient, and data-driven solutions. Let’s connect.

Location

Hyderabad

Work experience

Software Engineer - Data at dataAlpha.ai, Dec 2025 - Present · 7 mos, Hyderabad, Telangana, India, Working on Financial Data Engineering and Security Master platforms supporting hedge fund clients across multiple asset classes, including Equities, Bonds, Options, Futures, and Commodities. • Own and support Security Master workflows, processing Bloomberg market data for instrument classification, reference data enrichment, and downstream financial applications. • Serve as Subject Matter Expert (SME) for client-specific valuation and reference data platforms, driving issue resolution, production support, and enhancement delivery. • Design and maintain valuation exception reporting and monitoring solutions to identify pricing anomalies across OTC and listed instruments. • Optimize financial data processing for complex attributes such as coupon schedules, paydown rates, CPI-linked instruments, and security reference data. • Resolve critical data quality issues involving security identifier mapping, exchange normalization, pricing accuracy, and reference data integrity. • Deliver production enhancements and scalable solutions supporting hedge fund operations, valuation workflows, and enterprise financial data platforms. Skills: Financial Data Engineering, Security Master, Bloomberg, Market Data, Reference Data, Valuation, Capital Markets, Hedge Funds, SQL, Python, Data Quality, Production Support.

Data Engineer consultant at Razorpay, Jul 2025 - Dec 2025 · 6 mos, Hybrid, At Razorpay POS, I engineer scalable ELT pipelines and reporting solutions that ensure high data availability, accuracy in financial operations, and faster decision-making across payments and merchant services. My work focuses on automation, reliability, and data accessibility, enabling teams to move from manual effort to strategic impact. I have automated end-to-end workflows using Airflow, dbt, Python, and AWS Redshift, boosting data availability from 85% → 99% and reducing manual reporting by 90%. I also develop robust transformation pipelines to streamline billing accuracy, merchant settlements, and compliance reporting, while strengthening pipeline monitoring, anomaly detection, and authentication security (migrated Airflow from app passwords to OAuth 2.0). On the analytics side, I’ve built and maintained 20+ interactive dashboards in Metabase and Tableau, covering payments, refunds, and merchant KPIs. These dashboards are actively used by business and operations teams, improving decision-making efficiency by 15% and saving 20+ analyst hours/month through automation. Key Achievements: Engineered scalable workflows for real-time transaction ingestion & settlements with 99.9% uptime. Automated compliance & sales reporting by ingesting Gmail & SFTP sources via Airflow DAGs. Led root-cause investigations on settlement discrepancies; built alerting logic to flag anomalies proactively. Optimized financial reporting queries in ART, improving reconciliation, inventory, and performance reporting. Leveraged AWS S3 & Redshift Spectrum for external tables & long-term data archiving, strengthening governance. Reduced workflow downtime by 40% through improved monitoring and faster issue resolution.

Data Analyst (Data engineering ) at Razorpay, Apr 2023 - Jul 2025 · 2 yrs 4 mos, Bengaluru, Karnataka, India, Automated end-to-end ELT pipelines with Airflow, dbt, and Python, reducing manual report generation time by 90%. Engineered scalable workflows for real-time transaction ingestion and merchant settlements, ensuring 99.9% uptime and improving billing accuracy. Built Metabase dashboards powered by Redshift-backed models for payments, refunds, and merchant KPIs. Developed dynamic Airflow DAGs to ingest and clean data from Gmail and SFTP sources, automating compliance and sales reporting. Led root-cause investigations on settlement discrepancies; implemented alerting logic to automatically flag anomalies. Optimized financial reporting queries in ART, supporting inventory, performance, and reconciliation reports. Partnered with product and finance teams to define KPIs, deliver custom dashboards, and automate recurring data tasks. Leveraged AWS S3 and Redshift Spectrum for external table setup and data archiving, enhancing data governance and accessibility.

Software Engineer at MountBlue Technologies, Jan 2023 - Apr 2023 · 4 mos, Bengaluru, Karnataka, India

Subject Matter Expert at Chegg Inc., May 2022 - Jun 2022 · 2 mos, Telangana, India

Education

Bachelor of Technology - BTech, Information Technology, St. Martin's Engineering College