Headline

Data Engineer (Associate Solutions Engineer) @ RightData | PySpark • Databricks • ETL Pipelines

About

As Data Engineer at RightData since January 2025, I specialize in building scalable ETL pipelines using PySpark on Azure Databricks, complemented by expertise in Python, FastAPI, and SQL. My background includes crafting automated data-quality validation workflows and integrating Power BI dashboards with Delta Lake tables to ensure seamless data monitoring. Certified as a Data Scientist, I bring a strong foundation in AI/ML principles and hands-on experience in designing data-reconciliation pipelines for enterprise-grade analytics. At RightData, I collaborate with cross-functional teams to deliver robust data solutions that enhance operational efficiency and compliance. By leveraging tools like Jenkins, AWS S3, and innovative microservices, I contribute to dynamic reporting and automated notifications, enabling data-driven decision-making across the organization. Focused on continuous learning, I aim to foster a culture of collaboration and innovation, ensuring impactful contributions to data ecosystems.

Location

Hyderabad

AI stack

claude code

hugging face

langchain

notion

openai

githubcopilot

Work experience

Data Engineer at RightData, Jan 2025 - Present · 1 yr 11 mos, • Build end-to-end ETL pipelines using PySpark on Azure Databricks for large-scale enterprise data processing • Design and automate data quality validation workflows using FastAPI and SQL-based frameworks • Develop data reconciliation pipelines to detect mismatches across multiple data sources • Integrate Power BI dashboards with Delta Lake tables for operational KPIs • Implement cost optimization for Databricks and AWS S3 compute/storage

Python Developer at RightData, Jan 2022 - Feb 2025 · 3 yrs 2 mos, Developed Python-based automation solutions for data validation, reconciliation, and workflow orchestration. Built REST APIs using FastAPI to support data quality and metadata management platforms. Designed reusable frameworks for rule execution, reporting, and monitoring. Automated testing and operational workflows, reducing manual effort and improving release quality. Collaborated with cross-functional teams to deliver enterprise data management solutions

Assosciate QA Tester at RightData, Feb 2021 - Jan 2022 · 1 yr, Executed functional, regression, integration, and data validation testing for enterprise data platforms. Developed automated test suites using Python and Selenium, improving testing efficiency. Validated data quality and ETL processes across multiple data sources. Worked closely with developers and business users to identify and resolve defect