Data & AI engineering for regulated finance · Gurugram, India
Prashant Singh
I build data platforms and AI systems that banks and asset managers can put in front of an auditor.
Lead Software Engineer at Nomura Asset Management, owning curated data products and regulatory disclosure pipelines. Before that: investment data pipelines on PySpark and AWS at Macquarie, with the Airflow framework they ran on, and ten years at Indian Overseas Bank building credit-scoring and fraud models, AML pipelines, and the bank's Snowflake warehouse.
- stack
- python · sql · snowflake · airflow · spark · aws · pytorch · langgraph
- writing
- 13 posts across 4 categories
- Financial machine learning from zero, part 2: tensors, time-ordered batching, and why leakage hides in .shift()machine-learning
- AWS security for data engineers: seventeen recipes, worked throughdata-engineering
- How to read the Spark UIdata-engineering
- Financial machine learning from zero, part 1: the vocabulary, the traps, and the metrics that mattermachine-learning
- How granular should a pipeline task be?data-engineering
Skills
Generative AI & LLM
- Agentic AI (LangGraph, MCP)
- RAG & Vector Search (ChromaDB, BGE)
- LLM Integration & Evaluation
- Prompt Engineering & Guardrails
- Knowledge Graphs (Neo4j)
Bankopedia's RBI circular summarizer runs as a LangGraph agent over a RAG pipeline. See the project →
Machine Learning
- Predictive Modeling (credit, fraud)
- scikit-learn / XGBoost
- Feature Engineering & Evaluation
- PyTorch
Built credit-scoring and fraud models at Indian Overseas Bank; now writing that up as a series. Financial ML from zero →
Languages & Frameworks
- Python
- SQL
- Java
- PySpark / Scala
- FastAPI / Django
Six Python packages on PyPI, from NSE market data to a quantitative-finance library. Open source →
Data Platforms & Pipelines
- Snowflake (Snowpipe, Cortex, Dynamic Tables)
- Apache Spark
- Databricks
- dbt
- Kafka
Built a bank's enterprise Snowflake warehouse; wrote the Spark UI walkthrough my team onboards with. How to read the Spark UI →
Orchestration & Cloud
- Apache Airflow
- AWS (Glue, EMR, MWAA, Lambda)
- Docker & CI/CD
- Temporal
Designed an environment-aware Airflow DAG factory at Macquarie and moved Control-M and AutoSys workloads onto it. One DAG, every environment →
Databases & Storage
- PostgreSQL
- SQL Server / T-SQL
- Redis (caching, pub/sub)
Warehouse procs with an explicit error path: TRY…CATCH, ROLLBACK, THROW. The T-SQL post →
Experience
Nomura Asset Management International
Dec 2025 — PresentLead Software Engineer
Own curated data products and regulatory disclosure pipelines for US funds; prototyping AI and analytics automation with Snowflake Cortex ML and Dynamic Tables.
Macquarie Asset Management
Oct 2024 — Dec 2025Senior Data Engineer, Assistant Manager
Built investment data pipelines for positions, benchmarks, and performance attribution with PySpark on AWS; designed a spec-driven, environment-aware Airflow DAG factory and a metadata-driven ETL framework, and moved legacy AutoSys and Control-M batch workloads onto them. Evaluated LLM, RAG, and MCP patterns for workflow automation.
Indian Overseas Bank
Jun 2014 — Oct 2024Python Developer → Senior Data Engineer, Manager
Built credit-scoring and fraud ML models, AML analytics pipelines, and the bank's enterprise Snowflake warehouse; led a six-member engineering team.
The longer story, certifications, and education are on the about page →
AI Projects
Agentic AI · Live
An agentic AI application that ingests Reserve Bank of India circulars and publishes Claude-generated summaries on bankopedia.co.in. Orchestrated as a LangGraph agent workflow over a RAG pipeline — document chunking, BGE embeddings stored in ChromaDB — with prompt engineering, output validation, and guardrails keeping every summary faithful to the source regulatory text.
Project Synapse
Knowledge Graph
A knowledge graph of a codebase in Neo4j, modeling code structure and relationships, that translates a production error directly into the pinpointed source of the issue and a suggested fix. Combines graph traversal with semantic retrieval over BGE embeddings in ChromaDB, linking error signatures to the relevant code and fix context.
Open Source
Python packages published on PyPI — market data, quantitative finance, and tooling for regulated-AI workflows.
pybhav
pip install pybhav ↗Download NSE bhavcopy (end-of-day market data) for Indian equities and derivatives.
last release · Jul 2026
quantra
pip install quantra ↗An open-source Python library for quantitative finance — covering time value of money, credit lifecycle, risk analytics, derivatives pricing, and portfolio optimization.
last release · Jun 2026
atomingest
pip install atomingest ↗A metadata-driven data ingestion framework for AtomSQL with fintech-grade security and compliance features.
last release · Mar 2026
atomsql
pip install atomsql ↗A lightweight Object-Relational Mapping (ORM) library for Python with support for SQLite and PostgreSQL.
last release · Feb 2026
regchain
pip install regchain ↗Governance, audit, evaluation, and controls for AI workflows in regulated finance.
last release · Dec 2025
qfinbox
pip install qfinbox ↗A Python library for quantitative finance, offering tools for risk management, portfolio optimization, and financial modeling.
last release · Oct 2025
Get in Touch
Whether it's a role, a data-platform problem, or a question about something I wrote — drop me a line and it lands straight in my inbox.