Headline

Software Engineer II - Golang, Data

About

I’m a Software Engineer with 2.5 years of experience building reliable and scalable data ingestion and backend systems. My work focuses on designing high-performance data workflows and infrastructure components that enable accurate, efficient, and compliant data movement across platforms. I’ve worked with technologies such as Golang, Apache Airflow, Apache Iceberg, Apache Spark, and containerized microservices, contributing to systems that handle large-scale data processing and integration. I’m deeply interested in the evolution of data platforms, especially around areas like data discovery, quality, and governance frameworks that enhance how organizations manage and trust their data. Areas of Focus: * Building robust and scalable data ingestion workflows * Data quality and observability improvements * Metadata, cataloging, and lineage systems * Distributed computing and modern data architectures I’m passionate about solving engineering challenges that make data systems more reliable, discoverable, and impactful for analytics and decision-making.

Location

Pune District

AI stack

anthropic apis

claude code

claude

cursor

github copilot

mcp

notebooklm

obsidian

openai

Work experience

Software Engineer II at Pattern®, Oct 2025 - Present · 10 mos, Pune District · Hybrid, Leaned into the data realm more. 1. Built data pipeline using airflow, spark, kafka. 2. worked on Iceberg and Glue catalogs. 3. Caterpillar - data ingestion pipeline (https://github.com/patterninc/caterpillar)

Software Engineer at Pattern®, Sep 2023 - Sep 2025 · 2 yrs 1 mo, Pune District · Hybrid, 1. Built large scale data intensive web scraping services responsible for bringing in data to Snowflake. 2. Developed and maintained Chrome extension and backend service for scraping 3. Used Fivetran to load data from postgres. 4. Terraform for IaC

Education

Postgraduate Diploma, Advanced Computing, Centre for Development of Advanced Computing (C-DAC)

Bachelor of Engineering - BE, Computer Science and Engineering, Government Engineering College Raipur

Public links

github: https://github.com/divyanshu-tiwari

leetcode: https://leetcode.com/divyanshu-tiwari