Headline

Ex-Dish Network | Spark | Python | AWS | Apache Kafka | SQL | Iceberg | Databricks

About

An Enthusiastic Engineer working on Data Engineering with 5+ years of experience in building data and web-intensive applications, tackling challenging architectural and scalability problems in various domains.technologies like Python, Hadoop, Kafka, Spark, GIT, SQL, HQL, AWS, etc. Roles & Responsibilities • Building scalable big-data pipelines/applications/data warehouses with Hadoop, Kafka & SQL, Pl/SQL • Knowledge of relational database design concepts, coding, and experience in Apache HAWQ, and PostgreSQL(creating/modifying tables, views, sequences, aggregate functions, etc.) • Proficient in Agile and Waterfall Methodology. • Experience in interfacing with Onsite Stakeholders and supported at onsite too. • Knowledge in installing, configuring, and using Hadoop ecosystem components like Hadoop Map Reduce, HDFS, Hive, Spark, and Kafka. • Used Kafka to collect, aggregate, and store the web log data from different sources like web servers and pushed to HDFS. • Used Hive to process data and Batch data filtering. • Implemented a distributed messaging queue using Apache Kafka. • End-to-end ownership for developing, unit testing, and migrating the ETL code with quality and minimal supervision. • Capable of quickly, regularly, and independently learning new technologies under the pressure of consistent high-profile project deliverables. • Expertise in writing complex SQL queries to perform table partitioning, and data cleansing. • Handled slowly changing dimensions (SCD 1 and SCD 2). • Involved in Batch scheduling and processing (Full and Incremental) using Informatica to ensure on-time delivery of extract reports even during non-business hours to BA. Skill Set: Data Engineering tool: - Big Data-Apache Hadoop, Databricks File System: Hadoop Distributed File System – HDFS Computing Paradigm:- MapReduce & Apache Spark Data Store: Hive Streaming: Kafka RDBMS: MySQL, Apache HAWQ, PostgreSQL Schema Evolution & File Formats: Parquet, ORC, CSV Languages: Python, PHP, C

Location

Bengaluru

AI stack

antigravity

claude code

codex

Work experience

Senior Data Engineer at Oportun, Sep 2025 - Present · 1 yr 2 mos, Bengaluru, Karnataka, India

Lead Data Engineer at GrowthArc, Nov 2024 - Aug 2025 · 10 mos, Bengaluru, Karnataka, India, Designed and implemented an LLM-powered solution using the OpenAI API to interpret complex business logic and automatically generate summarized reporting stored procedures, reducing manual effort and significantly accelerating data analysis and business decision-making. Re-architected a Kotlin-based data pipeline from Snowflake to Elasticsearch using Apache Spark, reducing processing time for 262 million records from 32 hours to 12 hours, enabling faster reporting cycles and enhancing overall operational efficiency. Enhanced another pipeline transferring data from Oracle to Elasticsearch via Logstash , reducing processing time from 8 hours for 1.4 million records to just 14 minutes with Apache Spark, leading to reduced infrastructure costs and faster data availability for downstream systems. Led the successful delivery of a Proof of Concept (POC) for scalable ETL pipelines leveraging Snowflake and DBT, resulting in improved data accuracy, performance, and maintainability.

Senior Engineer at DISH Network Technologies, Oct 2022 - Nov 2024 · 2 yrs 2 mos, Bengaluru, Karnataka, India, Architected and executed a 5G data pipeline following Medallion Architecture using AWS (Lambda, DynamoDB, S3, Glue Crawler, Parquet), enhancing storage efficiency and data processing. Engineered and optimized a Gold Layer with AWS EMR Serverless & Iceberg , eliminating AWS Glue , cutting cloud costs by 25% , and boosting query performance by 40% , enabling real-time analytics. Optimized AWS Lambda memory usage and streamlined logging, cutting cloud costs by 12% and improving execution efficiency. Designed and deployed a data quality framework, detecting 40% of schema changes and monitoring multi-source data arrivals, reducing data inconsistencies by 20%. Collaborated with stakeholders to define and implement scalable data solutions, accelerating project delivery by 20% .

Senior Data Engineer at Enquero, Dec 2019 - Oct 2022 · 2 yrs 11 mos, Bengaluru, Karnataka, India, Engineered complex Databricks scripts replicating PostgreSQL stored procedures to process 1 TB of data daily, delivering accurate data transformations and enabling a 20% improvement in report generation speed. Orchestrated a 30% speed increase in databricks data processing by cache and data partitioning, using salting and broadcasting for efficient data distribution, which reduced latency.

Back End Developer at Talocity : Touchless HiRing, Sep 2019 - Nov 2019 · 3 mos, Mumbai, Maharashtra, India, Developed and provided scalable APIs for an AI-powered interview platform, enabling seamless user authentication, onboarding 10+ clients, and data processing. Integrated 2 DAGs in Apache Airflow to automate workflows, improving platform efficiency and reliability.

Education

Master of Computer Applications (MCA), Vivekanand Education Society's Institute Of Technology

BCA, Computer Programming, Specific Applications, LACHOO MEMORIAL COLLEGE,JODHPUR