Headline

Data Engineer at S2BI Limited

About

Data Engineer with 7+ years of IT experience specializing in data engineering, ETL development, and large-scale data processing. Proficient in PySpark, Apache Spark, Spark SQL, Hive, Hadoop, SQL, and Python, with hands-on experience in building data pipelines, data transformation, validation, reconciliation, and performance optimization. Currently focused on Azure databricks, ADF and cloud-based data solutions. Experienced in automation, production support, troubleshooting, Git/GitHub, and Agile delivery, with a strong focus on building reliable, scalable, and high-quality data solutions.

Location

India

AI stack

claude

gemini

github copilot

ChatGPT

Work experience

Data Engineer at S2BI Limited, Jan 2022 - Feb 2026 · 4 yrs 2 mos, London, • Designed and maintained scalable batch ETL workflows using PySpark, Spark SQL, Hive, and SQL for enterprise data processing. • Implemented incremental data ingestion from relational databases into Hadoop using Sqoop and supported automated batch execution through Hue. Developed Spark applications to process high-volume datasets in distributed environments, focusing on reliability and efficient resource utilization. • Created Hive tables, SQL views, and curated datasets to support downstream analytics and reporting requirements. • Optimized Hive and Spark workloads through partitioning, join optimization, SQL tuning, file compaction, and Parquetbased storage. • Developed Python utilities and scripts for ETL automation, data validation, reconciliation, and operational monitoring. • Performed source-to-target validation and reconciliation to verify completeness, consistency, and accuracy of processed and migrated data. • Participated in lift-and-shift modernization of Hadoop workloads to cloud platforms while preserving existing business logic. • Used GitHub for source-code management, peer reviews, collaborative development, and release coordination. • Supported requirement analysis, testing, deployment, production issue resolution, and Agile/Scrum delivery activities

Associate Consultant at Infrasoft Solutions Inc., Jul 2019 - Sep 2019 · 3 mos, Hyderabad, • Provided L2 support for Azure-based infrastructure and applications, ensuring high availability and performance. • Diagnosed and resolved issues related to Azure Virtual Machines, Networking, Storage, and Active Directory. • Assisted in implementing security best practices, including IAM, RBAC, and security monitoring. • Automated repetitive tasks using PowerShell and Azure CLI to improve efficiency and reduce manual efforts. • Managed and supported Active Directory, including user management, group policies, and authentication troubleshooting. • Handled on-call support for critical issues, ensuring timely resolution of incidents affecting business operations. • Documented technical issues, solutions, and best practices to improve knowledge sharing within the team. • Created and managed user accounts and licenses based on organizational policies. • Administered firewall environments in alignment with IT security standards. • Supported users on change control and system updates. • Worked on tickets & resolved tickets within given SLA. • Provided remote support to engineers and end-users during installations.

System Engineer at Sify Technologies Limited., Sep 2018 - Jun 2019 · 10 mos, Hyderabad, • Created & Managed User Profiles, Organizational units, Distribution Lists & Group Policies on on-premises AD. • Addition & Deletion of users in Distribution Lists, providing send access permissions for groups in on-premises AD. • Creating & managing Office 365 user accounts. Assigning Office 365 licenses to the users like Exchange online, Skype for Business, One drive for business, etc • Enabling, Troubleshooting & resolving Multi factor authentication issues for higher management users. • Resolving outlook related problems. • Delete & restore the user accounts in office 365. • Generating Office 365 reports, AD Reports and share with Management Team. • Create & manage Office 365 mailboxes. • Create & adding/removing members from Distribution Group in Exchange online. • Maintained Azure AD & Virtual Machines. • Worked on Radia Client for updating the PC's of the Client zonal offices , Regional offices & Branch Offices. • Logging and managing helpdesk tickets using Service Now. • Setting up desktop computers, laptops for onsite and remote users. • Mixture of first and second line support. • Performed Patch Management, Anti-Virus Management to ensure that client systems are up to date. • Assist to update IT Policies. • On-call support for the client

Software Engineer at Pasham Technologies, Jun 2017 - Sep 2018 · 1 yr 4 mos, Hyderabad, • Created SOP's for the clients. • Worked as 1st line support for Clients. • Resolving outlook related problems. • Delete & restore the user accounts in office 365. • Worked on Tickets. • Setting up hardware, installing and configuring software and drivers. • Maintaining and repairing technological equipment (e.g. routers) or peripheral devices. • Installing well-functioning LAN/WAN and other networks and managed components (servers, IPs etc.)

Education

Masters, Information systems, Kingston University

Bachelor of Technology - BTech, Computer Science, Mallareddy institute of technology