Headline

Software Architect @ CommerceIQ | Crawlers, GCP, DataBricks, AWS, Big Data & Data Engineering

About

As a Software Architect (L7) at CommerceIQ, I lead the Platform Engineering team and design and develop large scale Big Data solutions using open sources, cloud computing, managed services and agile methodologies. I have over 19 years of experience in the E-Commerce, Mobility, SaaS, VAS and Telecom domains, with primary skills in Big Data, Cloud Computing, backend Platforms and Java/J2EE technologies. I also hold a PG Diploma in Machine Learning and Artificial Intelligence from IIIT Bangalore, and I am a student mentor for Big Data courses. I am passionate about solving complex business problems using data-driven approaches and leveraging the latest technologies and frameworks. My mission is to deliver high-quality, reliable, and scalable products that empower our customers and partners to grow their businesses.

Location

Bengaluru

AI stack

claude code

claude

cursor

langgraph

langsmith

notion

Work experience

Software Architect at CommerceIQ, Sep 2023 - Present · 3 yrs, Bengaluru, Karnataka, India, • Leading technical leadership in platform engineering across multiple platforms including Crawler Platforms, Data Platforms, and DataBricks Compute Platforms. • Migrating crawler platforms from GCP to AWS. This also includes unification of 3 different crawler platforms (~80M crawls/day across ~1K retailers). • Involved in senior leadership hiring (VP, Directors, Architects) for key engineering roles resulting in a high-performing team. • Achieved ~35% cost reduction in 2024 through cost optimization initiatives. • Migrated 100TB data from SnowFlake to DataBricks. • Lead the architecture of a new unified Data Platform.

Principal Architect ( Data Engineering ) at API Holdings, Apr 2021 - Sep 2023 · 2 yrs 6 mos, Bangalore Urban, Karnataka, India, - Leading Data Engineering at API Holdings / PharmEasy. - Primarily involved in Design and Architecture of Big Data Platform. - Tech stacks & service : Amazon EMR, Spark, Hudi (Lakehouse), Debezium, Druid, etc - Data Quality, Data Profiling, Data Auditor - Tech hiring

Sr Architect ( Data Engineering ) at API Holdings, Aug 2018 - Mar 2021 · 2 yrs 8 mos, Bengaluru, Karnataka, India, • Heading the Data Engineering team at API Holdings / PharmEasy. • Primarily involved in Design and Architecture of Big Data Platform. • Hiring and managing the core team. • Involved in setting up the infra, monitoring, and alerting stack. The Data Platform has been built using AWS, HDP Ambari, Qubole, Spark, kafka, Redshift, Kubernetes, StatsD, InfluxDB, Chronograf • Built Batch Processing pipelines, ETL & CDC pipelines, Data Lake, Data Warehousing. • Currently working on Real time data / Stream processing.

Principal Engineer at Ola (ANI Technologies Pvt. Ltd), Apr 2017 - Aug 2018 · 1 yr 5 mos, Bengaluru, Karnataka, India, Worked on an Org level initiative : 'Kafka As A Service' (KaaS). Authentication, Authorisation, Quota management, Metric Centre, Config Management, KaaS Client , multi-tenancy are some of the key features of 'KaaS'. Contributed to architecture and design of in house built tools like Data Auditor / Profiler, and Notification As a Service. Built an ETL (CDC- Change data capture) pipeline called 'Foster' as a DataPlatform initiative. 'Foster' drives data lakes, and data marts in DataPlatform. Key components includes open sources like Secor, Maxwell, and Kafka along with in house built Meta Service, UI, Snapshot (De-duplication and Reconciler), and Config Service. The pipeline runs on AWS with data residing in S3. Hive tables built on top of the Snapshot data is consumed by Data Analysts.

Education

PG Diploma in Machine Learning & Artificial Intelligence, International Institute of Information Technology Bangalore

B.E., Telecommunications, Visvesvaraya Technological University