Headline

Data Engineer | GCP | Real-Time Pipelines. Kafka. BigQuery. Spark|

About

At Lloyds Banking Group, I built and maintained cloud data infrastructure on GCP that powers real-time decision-making in production banking environments, from Kafka streaming pipelines to BigQuery and Cloud Spanner warehouses optimized for cost and performance. My work spanned end-to-end: pipeline development, Spark optimization, and CI/CD automation with Terraform and Jenkins. Before that, at Teesside University, I worked on an AI-driven student induction platform integrating face detection, event recommendations, and Text Analysis to turn qualitative feedback into actionable insights visualized in PowerBI. I enjoy the intersection of engineering rigour and real business impact building data platforms that are reliable, scalable, and actually used. If you're working on cloud data or analytics infrastructure, let's connect. 📩 vishwas.simha98@gmail.com

Location

Hyderabad

AI stack

gemini

github copilot

hugging face

openai

Work experience

Cloud Data Engineer at Lloyds Banking Group, Sep 2022 - Sep 2024 · 2 yrs 1 mo, Greater London, • Maintained Spark data pipelines on GCP to ingest and transform complex JSON datasets into BigQuery, improving data usability for analytics teams and reducing manual preparation. • Migrated legacy Hive and HBase datasets into BigQuery and Cloud Spanner, redesigning schemas and applying partitioning and clustering that improved analytical query performance by 20% across multiple reporting workloads. • Developed Kafka and Spark Streaming pipelines to process high volume transactional data, reducing data availability delays from hours to near real time for downstream consumers. • Implemented real-time data replication using CDC workflows between transactional systems and Google Cloud services by configuring Kafka and Spark, supporting continuous data synchronization and traceability for reporting and analytics. • Optimized Spark ETL jobs by refining joins, partition logic, and caching strategies, reducing pipeline runtimes by 25% and significantly improving pipeline reliability. • Contributed to CI/CD automation using Jenkins and GitHub Actions, creating deployment jobs and version-controlled workflows that reduced manual steps during releases. • Collaborated with analysts and product teams to translate data requirements into transformation logic and model structures, ensuring consistent outputs for reporting and audit needs.

Research Intern at Teesside University, Jan 2021 - May 2021 · 5 mos, Middlesbrough, England, United Kingdom, • Led the development of a student induction app as part of a research group, integrating AI and ML features such as face detection and event recommendations to enhance user engagement. • Analyzed qualitative feedback data using Text Analysis Azure API and created data visualizations in PowerBI, providing actionable insights and improving the student onboarding experience. • Developed a cross-platform web application with a focus on user experience and data-driven recommendations, applying innovative Machine Learning techniques to improve service delivery.

Student Intern at National Remote Sensing Centre, May 2018 - Oct 2018 · 6 mos, Hyderabad

Intern at Chronos Infinity, May 2017 - Jul 2017 · 3 mos, Hyderabad

Education

Master's degree, Teesside University

Bachelor of Technology, Computer Science, ICFAI Foundation for Higher Education, Hyderabad