Headline

Data & AI Engineer | Python | Automation | Dashboards | AI Integrations | AI chatbot | LLM Integration | Open for Remote Projects

About

Senior Data Engineer with 6+ years of experience designing scalable data platforms and distributed processing systems using Microsoft Fabric, PySpark, Kafka, AWS, and Azure. Experienced in building event-driven and Lakehouse-based architectures for enterprise analytics, real-time processing, and data migration initiatives. Worked on hybrid cloud environments leveraging Kafka, Spark Structured Streaming, AWS S3, and Microsoft Fabric OneLake/Lakehouse to support scalable analytics and reporting workloads. Skilled in ETL/ELT architecture, distributed data processing, API-driven integrations, dimensional data modeling, and cloud-based analytics platforms. Strong exposure to Medallion Architecture, metadata-driven pipelines, data quality validation, and real-time data workflows. Currently focused on evolving toward Data Architecture and AI-enabled analytics platforms involving Microsoft Fabric, GenAI integration patterns, RAG-based systems, and scalable enterprise data solutions. Core Skills: • Microsoft Fabric (OneLake, Lakehouse, Data Factory) • PySpark & Spark Structured Streaming • Kafka & Event-Driven Architectures • AWS (S3, Lambda, Glue/EMR) • Azure Data Engineering • FastAPI & Python • Data Modeling & Medallion Architecture • Power BI & Semantic Modeling • Real-Time Analytics & Distributed Systems

Location

Bengaluru

AI stack

antigravity

github copilot

llmapi

Granola

Haystack

Udio

Work experience

Senior Data Engineer at Tech Mahindra, Oct 2025 - Present · 1 yr 1 mo, •Architected a hybrid multi-cloud data platform (AWS + Microsoft Fabric), using S3 for staging and Lakehouse for analytics, enabling petabyte-scale data processing •Designed event-driven pipelines using Kafka and PySpark (AWS Glue/EMR) to process 100K+ events/sec with high reliability and low latency •Implemented Lakehouse architecture (Medallion – Bronze/Silver/Gold) and designed dimensional data models (SCD Type 1/2) for scalable analytics and reporting •Developed FastAPI-based microservices on AWS Lambda (API Gateway) to expose data services, and implemented real-time data validation and anomaly detection pipelines •Optimized distributed workloads, achieving 30% cost reduction, and ensured 99.9% uptime through CI/CD automation (Azure DevOps) •Collaborated with business stakeholders to translate requirements into scalable data solutions for analytics and decision-making

Senior Executive at Syngene International Limited, Jan 2024 - Oct 2025 · 1 yr 10 mos, Bengaluru, Karnataka, India, •Serve as a liaison between research scientists, lab personnel, and IT teams to gather comprehensive requirements and translate intricate experimental workflows into scalable IT solutions. •Spearheaded API-based integrations between laboratory instruments, Oracle databases, Electronic Lab Notebook (ELN) platforms (BIOVIA, Benchling, Dotmatics, PerkinElmer Signals Notebook), and Laboratory Information Management Systems (LIMS), enabling seamless data exchange and reducing manual effort. •Designed and configured interactive BI dashboards (Power BI, Tableau, Spotfire) to deliver actionable insights on scientific data, enhancing decision-making for researchers and leadership. •Provided comprehensive support to end-users with LIMS and ELN applications, including troubleshooting, customization, and structured user training programs to enhance adoption and usability. •Orchestrated Agile delivery for three scientific applications, reducing JIRA tickets by 33% through streamlined workflows and backlog refinement, empowering scientists to focus on core research activities. •Ensured strict compliance with ITIL change management processes and regulatory requirements while deploying solutions into live lab environments, mitigating risks, and ensuring data integrity. •Collaborated with vendors and internal stakeholders to drive digital transformation projects within Pharma R&D, ensuring alignment with critical scientific goals and stringent regulatory standards.

Junior Software Engineer at Covalense Global, Jan 2021 - Dec 2023 · 3 yrs, Bengaluru, Karnataka, India, •Collaborated with stakeholders and cross-functional teams to ensure cloud services met business requirements and Service Level Agreements (SLAs), translating complex business needs into actionable technical solutions. •Analyzed Extract, Transform, Load (ETL) and data workflows to identify bottlenecks and recommend process improvements; extracted and analyzed SAP and enterprise data for reporting and strategic initiatives. •Developed Power BI dashboards to track KPIs (SLA compliance, Customer Satisfaction (CSAT), resource utilization) to support data-driven decision-making processes. •Monitored incidents and service requests within ServiceNow, ensuring ITIL compliance and proactively proposing preventive measures to minimize business impact and enhance service reliability.

Software Engineer Trainee at ScriptBees, Feb 2020 - Dec 2020 · 11 mos, Hyderabad, Supported internal tools and client applications; resolved technical issues. Assisted in Python scripting, backend development, and database configuration. Contributed to ETL testing, automation tasks, and issue documentation. Researched new tech for process optimization and performance improvement. Collaborated on feature updates using user feedback and team inputs.

Trainee Quality Engineer at Hindustan Aeronautics Limited, Sep 2018 - Aug 2019 · 1 yr, Koraput, Odisha, India

Education

Master in Data science and Business analyst, Data science, Simplilearn Alumni

Data Science and Machine Learning, Data science, NATIONAL INSTITUTE OF ELECTRONICS & INFORMATION TECHNOLOGY (NIELIT)