Suryansh Tiwari profile photo 1 on Tal

Suryansh Tiwari

Big Data Developer @ TransUnion | Apache Spark, Python, SQL, DSA, ETL, Big Data

Pune City

8+ years of work experience

TransUnion
May 20224 yrs 4 mosPresent

Pune District, Maharashtra, India

TransUnion
May 20215 yrs 4 mosPresent

Data Pipeline Architecture: Design, develop, test, and implement new ETL pipelines for aggregating non-financial consumer profile data from multiple domains into the TLOxp product using Bolt (a Spark-like C++ processing engine). Cloud Migration (GCP): Spearheading the migration of archival and vendor data from on-premise load boxes to Google Cloud Storage (GCS) using gcloud storage, enabling cost-effective scaling and seamless analytics access. PySpark Optimization (TU CIBIL Project): Led a massive SQL to PySpark migration project for processing workflows, resulting in a 50% performance improvement in data analytics capabilities. System Modernization: Upgrading proprietary data processing codes on the OneTrue CDP mounted on top of GCP DataProc. Monitoring & KPI Development: Built custom robust applications utilizing Pandas, NumPy, XlsxWriter, and MariaDB to continuously monitor KPIs for large datasets generated by TLOxp. Performed qualitative analysis on new data sources to guide stakeholder implementations. Production Support: Providing reliable L2 support, troubleshooting pipelines, and deploying immediate fixes ensuring minimum downtime.
TransUnion
May 20211 yr 1 moMay 2022

Pune Division, Maharashtra, India

Charles Schwab
Jan 20192 yrs 5 mosMay 2021

Pune Division, Maharashtra, India

Fulltime with Wipro
Wipro
Oct 20182 yrs 8 mosMay 2021

Log Processing Pipelines: Created and maintained batch processing jobs to extract, cleanse, and transform large-scale Splunk log data using PySpark before loading it optimally into Teradata. Data Lake Ingestion: Developed custom Big Data applications to ingest raw data from disparate heterogeneous sources, loading into Apache Hive using Spark DataFrames and RDDs. Performance Tuning: Actively troubleshot and optimized existing Spark and Hive data pipelines to remove bottlenecks and improve SLA compliance. Tech Stack: Teradata, Apache Hive, Spark-SQL.
Wipro
Oct 20184 mosJan 2019

India

Suryansh’s AI Stack

claude code
claude code

Education

Visvesvaraya Technological University

Bachelor of Engineering, Mechanical Engineering

Visvesvaraya Technological University

20144 yrs2018

Join Suryansh and 10k+ other professionals on

Tal Boss

Tal is the app where bosses meet engineers & builders directly, without middlemen

Download Tal Boss

Suryansh Tiwari on Tal

Headline

Big Data Developer @ TransUnion | Apache Spark, Python, SQL, DSA, ETL, Big Data

About

Big Data Developer with over 7 years of experience in architecting and developing end-to-end data pipelines and applications. Skilled in the Hadoop ecosystem, PySpark, SQL, Python, and Google Cloud services (GCS, DataProc, BigQuery). Currently driving data innovation at TransUnion on the TLOxp/TruLookup product. Passionate about expanding into Machine Learning and Gen-AI, as demonstrated by my recent projects building a 90%+ accurate NLP Gender Prediction Model and developing a comprehensive web-based PDF Utility suite. Adept at turning complex data problems into optimized, scalable solutions.

Location

Pune City

AI stack

claude code

Work experience

Developer ll at TransUnion, May 2022 - Present · 4 yrs 4 mos, Pune District, Maharashtra, India

Big Data Developer at TransUnion, May 2021 - Present · 5 yrs 4 mos, Data Pipeline Architecture: Design, develop, test, and implement new ETL pipelines for aggregating non-financial consumer profile data from multiple domains into the TLOxp product using Bolt (a Spark-like C++ processing engine). Cloud Migration (GCP): Spearheading the migration of archival and vendor data from on-premise load boxes to Google Cloud Storage (GCS) using gcloud storage, enabling cost-effective scaling and seamless analytics access. PySpark Optimization (TU CIBIL Project): Led a massive SQL to PySpark migration project for processing workflows, resulting in a 50% performance improvement in data analytics capabilities. System Modernization: Upgrading proprietary data processing codes on the OneTrue CDP mounted on top of GCP DataProc. Monitoring & KPI Development: Built custom robust applications utilizing Pandas, NumPy, XlsxWriter, and MariaDB to continuously monitor KPIs for large datasets generated by TLOxp. Performed qualitative analysis on new data sources to guide stakeholder implementations. Production Support: Providing reliable L2 support, troubleshooting pipelines, and deploying immediate fixes ensuring minimum downtime.

Developer l at TransUnion, May 2021 - May 2022 · 1 yr 1 mo, Pune Division, Maharashtra, India

Senior Software Engineer at Charles Schwab, Jan 2019 - May 2021 · 2 yrs 5 mos, Pune Division, Maharashtra, India, Fulltime with Wipro

Big Data Developer at Wipro, Oct 2018 - May 2021 · 2 yrs 8 mos, Log Processing Pipelines: Created and maintained batch processing jobs to extract, cleanse, and transform large-scale Splunk log data using PySpark before loading it optimally into Teradata. Data Lake Ingestion: Developed custom Big Data applications to ingest raw data from disparate heterogeneous sources, loading into Apache Hive using Spark DataFrames and RDDs. Performance Tuning: Actively troubleshot and optimized existing Spark and Hive data pipelines to remove bottlenecks and improve SLA compliance. Tech Stack: Teradata, Apache Hive, Spark-SQL.

Project Engineer at Wipro, Oct 2018 - Jan 2019 · 4 mos, India

Education

Bachelor of Engineering, Mechanical Engineering, Visvesvaraya Technological University