Headline

Data Engineer | Azure | Google Cloud (GCP) | Databricks | PySpark | Spark | Data Lakehouse | AI Data Engineering | DP-203 | Databricks Certified

About

Hi, I'm Karthikeya 👋 I'm a Data Engineer with 3.5+ years of experience designing scalable cloud data platforms and building enterprise-grade ETL pipelines. I specialize in Microsoft Azure and Databricks while actively expanding my expertise in Google Cloud Platform (GCP) and Agentic AI to build next-generation intelligent data solutions. 🚀 What I do: • Design scalable ETL/ELT pipelines • Build Data Lakehouse architectures • Develop PySpark applications on Databricks • Optimize enterprise data platforms • Automate cloud data workflows 💡 Technologies: ☁️ Azure Data Factory ☁️ Azure Databricks ☁️ Azure Synapse Analytics ☁️ Azure Data Lake Storage Gen2 ☁️ Google Cloud Platform (BigQuery, Cloud Storage, Dataproc, Dataflow, Cloud Composer) ⚡ PySpark ⚡ Apache Spark ⚡ SQL ⚡ Python ⚡ Delta Lake ⚡ Data Lakehouse 🤖 Currently Learning: • Google Cloud Data Engineering • Agentic AI • LangChain • Retrieval-Augmented Generation (RAG) • Large Language Models (LLMs) • AI Agents for Data Engineering 📈 Career Highlights: ✅ 70+ production ETL pipelines ✅ 45% improvement in processing performance ✅ 30% reduction in execution time ✅ 25% increase in operational efficiency On LinkedIn, I share: 📌 Azure Data Engineering 📌 Google Cloud (GCP) 📌 Databricks 📌 SQL & PySpark tips 📌 Data Engineering interview preparation 📌 AI & Agentic AI learning 📌 Real-world project architectures Let's connect and grow together!

Location

Hyderabad

AI stack

antigravity

cursor

Work experience

Data Engineer at Lloyds Technology Centre India, Jul 2026 - Present · 2 mos, Hyderabad, Telangana, India, • Working as a Data Engineer at Lloyds Technology Centre, Hyderabad. • Building and supporting scalable data engineering solutions and modern data pipelines. • Working with cloud technologies, data processing, and enterprise data platforms. • Continuously enhancing expertise in Data Engineering, Cloud Computing, and Generative AI to deliver business value through data-driven solutions.

Big Data Consultant at Impetus, Apr 2025 - Jun 2026 · 1 yr 3 mos, Hyderabad

Azure Data Engineer at Accenture, Dec 2022 - Mar 2025 · 2 yrs 4 mos, Hyderabad, Telangana, India, -Designed the high-level data flow and get it reviewed with the Architect team Experienced in creating pipelines in Azure Cloud ADFv2 using different activities like Move & Transform, Control Flow etc. -Understand the business requirement. -Developed and Optimized 70+ data pipelines in Azure Data Factory, boosting data integration and real-time processing within Azure Data Lake. -Built real-time workflows using Databricks Lakehouse, leveraging Python Spark, and SQL to speed up machine decisioning by 45%. -Planned, and develop scalable data pipelines using Azure Data factory(ADF), Databricks, Azure Synapse. Implement ETL processes to Ingest, Optimize and load data from various sources into data lakes and warehouses. -Processed and Aggregated raw data with Azure Synapse Analytics, resulting in a 30% reduction in data processing time. -Design Azure Data Factory (ADF) pipelines and implement logic in the Azure. -Databricks notebook in a generic manner for reusability. -Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs. -Automating the data pipeline using Azure data factory, Azure Synapse Analytics and increasing operational efficiency by 25%. -Strong Analytical thinking and problem-solving skills in Azure. -Develop and maintain documentation for data engineering processes and best practices.

Education

Master of Computer Applications, Data Modeling/Warehousing and Database Administration, Jain (Deemed-to-be University)

Bachelor of Computer Applications, Information Technology, Krishna Chaitanya Institute of Science & Technology, Kakatur

Public links

github: https://github.com/karthikeya0904