Venkata Sai Palepu profile photo 1 on Tal

Venkata Sai Palepu

Data Engineer | Databricks | PySpark | Delta Lake | AWS | ETL/ELT | Data Warehousing | SQL | Apache Airflow | Lakeflow Connect

Bengaluru

2+ years of work experience

Idexcel
Oct 20242 yrs 1 moPresent

Bengaluru, Karnataka, India

Working on enterprise-scale data engineering solutions using Databricks, PySpark, Delta Lake, AWS, and Apache Airflow to build scalable, high-performance data platforms. Key Responsibilities & Achievements Designed and implemented a Bronze–Silver–Gold Medallion Architecture on Databricks using Delta Lake for enterprise analytics. Built CDC ingestion pipelines using Databricks Lakeflow Connect with PostgreSQL WAL-based logical replication for reliable incremental data loading. Developed scalable ETL/ELT pipelines using PySpark, AWS Glue, Lambda, S3, Athena, and Apache Airflow. Implemented SCD Type 1 & Type 2 strategies, MERGE-based upserts, and Delta Lake optimization techniques including Z-ORDER. Designed an AI-powered document extraction solution using Databricks ai_parse_document for Accounts Receivable invoice processing. Built enterprise natural-language query solutions using Databricks Genie and AWS Bedrock, integrating them into business applications through REST APIs. Developed Tableau AI Extensions and Apache Superset dashboards for self-service analytics and business reporting. Implemented CI/CD pipelines using GitHub Actions and collaborated with cross-functional teams to deliver production-ready data solutions.
Ai Variant
Aug 20236 mosJan 2024

Online

Supported data quality assurance, created SQL queries for data validation, and assisted in building Tableau dashboards for stakeholder reporting. Key contributions: • Analyzed data flow and identified optimization opportunities in ETL processes • Developed SQL queries for data extraction, transformation, and validation • Created and maintained Tableau dashboards for business intelligence

Venkata’s AI Stack

antigravity
antigravity
claude code
claude code
codex
codex
deepseek
deepseek
kiro
kiro

Education

NBKR Institute of Science and Technology

Bachelor of Technology - BTech, Computer Science

NBKR Institute of Science and Technology

20194 yrs2023

Intermediate, MPC

Narayana Junior College - Gudur

20172 yrs2019

Join Venkata and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Venkata Sai Palepu on Tal

Headline

Data Engineer | Databricks | PySpark | Delta Lake | AWS | ETL/ELT | Data Warehousing | SQL | Apache Airflow | Lakeflow Connect

About

Data Engineer with 2 years building production-grade ETL/ELT pipelines and Medallion Architecture (Bronze-Silver-Gold) lakehouses on Databricks, Delta Lake, and AWS (S3, Glue, Athena, Lambda). Specialized in AI-powered data engineering: LLM-based document extraction (ai_parse_document, ai_query), natural-language query agents (Databricks Genie, AWS Bedrock), and REST API integrations that embed AI directly into business applications and BI dashboards. AWS Certified Cloud Practitioner with a track record of eliminating manual SQL dependency and cutting manual reporting effort by 60%+.

Location

Bengaluru

AI stack

antigravity

claude code

codex

deepseek

kiro

Work experience

Data Engineer at Idexcel, Oct 2024 - Present · 2 yrs 1 mo, Bengaluru, Karnataka, India, Working on enterprise-scale data engineering solutions using Databricks, PySpark, Delta Lake, AWS, and Apache Airflow to build scalable, high-performance data platforms. Key Responsibilities & Achievements Designed and implemented a Bronze–Silver–Gold Medallion Architecture on Databricks using Delta Lake for enterprise analytics. Built CDC ingestion pipelines using Databricks Lakeflow Connect with PostgreSQL WAL-based logical replication for reliable incremental data loading. Developed scalable ETL/ELT pipelines using PySpark, AWS Glue, Lambda, S3, Athena, and Apache Airflow. Implemented SCD Type 1 & Type 2 strategies, MERGE-based upserts, and Delta Lake optimization techniques including Z-ORDER. Designed an AI-powered document extraction solution using Databricks ai_parse_document for Accounts Receivable invoice processing. Built enterprise natural-language query solutions using Databricks Genie and AWS Bedrock, integrating them into business applications through REST APIs. Developed Tableau AI Extensions and Apache Superset dashboards for self-service analytics and business reporting. Implemented CI/CD pipelines using GitHub Actions and collaborated with cross-functional teams to deliver production-ready data solutions.

Data Analyst Intern at Ai Variant, Aug 2023 - Jan 2024 · 6 mos, Online, Supported data quality assurance, created SQL queries for data validation, and assisted in building Tableau dashboards for stakeholder reporting. Key contributions: • Analyzed data flow and identified optimization opportunities in ETL processes • Developed SQL queries for data extraction, transformation, and validation • Created and maintained Tableau dashboards for business intelligence

Education

Bachelor of Technology - BTech, Computer Science, NBKR Institute of Science and Technology

Intermediate, MPC, Narayana Junior College - Gudur