Headline

Data Engineer II | GCP | Apache Beam | Dataflow | BigQuery | Airflow | Cloud Composer | Pub/Sub | Kubernetes | PySpark | ETL | ELT | CI/CD | AI Data Pipelines | Python | 4 Years Experience

About

I build data pipelines that power AI systems at scale. At Infocusp Innovations (4 years), I've owned end-to-end pipeline engineering across two production domains: ▸ Legal Document Intelligence Architected Apache Beam workflows ingesting millions of documents daily across 10+ formats into structured, LLM-ready datasets. Cut downstream AI data prep time by 60%. Built annotation extraction and PDF rendering libraries enabling AI systems to semantically understand document structure and hidden metadata. ▸ Geospatial & Environmental Data Engineered petabyte-scale ingestion pipelines for satellite imagery from Google Earth Engine 100GB+ per run. Reduced pipeline run time from hours to under 40 minutes. Built fully automated Cloud Scheduler → Cloud Workflows → Cloud Batch pipelines powering weekly ML training cycles. 𝗖𝗼𝗿𝗲 𝗦𝘁𝗮𝗰𝗸 Python · Apache Beam · Apache Airflow · GCP Cloud Composer · BigQuery · GCS · Pub/Sub · Cloud Batch · Kubernetes · Docker · Terraform · PySpark 𝗢𝗽𝗲𝗻 𝗦𝗼𝘂𝗿𝗰𝗲 : IntelliScraper (PyPI) async anti-bot scraping library 5500+ downloads pdfnotate (PyPI) extracts hidden PDF annotations for LLM pipeline

Location

Pune Division

AI stack

antigravity

claude code

gemini

geminicli

Work experience

Senior Data engineer at Infocusp Innovations, Jan 2026 - Present · 7 mos, Hybrid, Architected end-to-end Apache Beam data pipelines for a legal document intelligence platform — ingesting millions of files (PDFs, emails, docs, sheets) and transforming them into structured, LLM-ready datasets with full metadata preservation. Designed and built core annotation extraction and PDF rendering libraries, enabling downstream AI systems to semantically understand document structure and content. Owned orchestration and scheduling architecture using Apache Airflow + GCP Cloud Composer, establishing reliable deployment patterns for production-scale enterprise pipelines. Defined data schemas and pipeline contracts with AI/ML teams, ensuring clean, correctly formatted data delivery for model consumption.

Data engineer at Infocusp Innovations, Jul 2023 - Present · 3 yrs 1 mo, Hybrid, Built scalable ingestion and processing pipelines for petabyte-scale geospatial datasets satellite imagery, GeoTIFF files, spectral data, and environmental metrics from Google Earth Engine. Handled datasets exceeding 100GB per run. Designed an automated export pipeline using Cloud Scheduler + Cloud Workflows + Batch Jobs to deliver fresh geospatial datasets on regular intervals for downstream ML model training cycles. Achieved 20% efficiency improvement by optimizing ingestion algorithms and refactoring pipeline logic using OOP design principles. Containerized distributed processing workloads using Kubernetes + Docker; provisioned all GCP infrastructure reproducibly via Terraform. Used GCS, Pub/Sub, and Cloud Logging for reliable, observable pipeline runs.

Intern at Infocusp Innovations, Jan 2023 - Jun 2023 · 6 mos, On-site, Processed large-volume healthcare device data using Apache Spark (PySpark) on a distributed cluster (AWS Glue + Terraform), extracting structured insights from raw sensor streams. Built self-serve Streamlit dashboards for data visualization, reducing manual data analysis time for the data science team.

Education

Bachelor's degree, Information Technology, Pimpri Chinchwad Education Trust'S. Pimpri Chinchwad College Of Engineering

Public links

github: https://github.com/omkarmusale0910

medium: https://medium.com/@omkarmusaleich