Data & AI engineering for regulated finance · Gurugram, India
Prashant Singh
I build data platforms and AI systems that banks and asset managers can put in front of an auditor.
Twelve years of data engineering across a public-sector bank and two global asset managers: data platforms, pipelines, and machine learning in environments where every number has to stand up to review.
- stack
- python · sql · snowflake · airflow · spark · aws · pytorch · langgraph
- writing
- 13 posts across 4 categories
- Financial machine learning from zero, part 2: tensors, time-ordered batching, and why leakage hides in .shift()machine-learning
- AWS security for data engineers: seventeen recipes, worked throughdata-engineering
- How to read the Spark UIdata-engineering
- Financial machine learning from zero, part 1: the vocabulary, the traps, and the metrics that mattermachine-learning
- How granular should a pipeline task be?data-engineering
Skills
Generative AI & LLM
- Agentic AI (LangGraph, MCP)
- RAG & Vector Search (ChromaDB, BGE)
- LLM Integration & Evaluation
- Prompt Engineering & Guardrails
- Knowledge Graphs (Neo4j)
Project Synapse pairs a Neo4j knowledge graph with semantic retrieval over BGE embeddings. See the project →
Machine Learning
- Predictive Modeling (credit, fraud)
- scikit-learn / XGBoost
- Feature Engineering & Evaluation
- PyTorch
A from-scratch series on credit-risk modeling, built on public loan data. Financial ML from zero →
Languages & Frameworks
- Python
- SQL
- Java
- PySpark / Scala
- FastAPI / Django
Six Python packages on PyPI, from NSE market data to a quantitative-finance library. Open source →
Data Platforms & Pipelines
- Snowflake (Snowpipe, Cortex, Dynamic Tables)
- Apache Spark
- Databricks
- dbt
- Kafka
A walkthrough of the five Spark UI screens that matter, on a job you can run on a laptop. How to read the Spark UI →
Orchestration & Cloud
- Apache Airflow
- AWS (Glue, EMR, MWAA, Lambda)
- Docker & CI/CD
- Temporal
How a spec-driven DAG factory keeps one codebase across dev, test, and prod. One DAG, every environment →
Databases & Storage
- PostgreSQL
- SQL Server / T-SQL
- Redis (caching, pub/sub)
Warehouse procs with an explicit error path: TRY…CATCH, ROLLBACK, THROW. The T-SQL post →
Experience
Nomura Asset Management International
Dec 2025 — PresentLead Software Engineer
Data products and pipelines for fund operations on Snowflake.
Macquarie Asset Management
Oct 2024 — Dec 2025Senior Data Engineer, Assistant Manager
Investment data pipelines with PySpark, AWS, and Apache Airflow, including migration of legacy batch scheduling to Airflow.
Indian Overseas Bank
Jun 2014 — Oct 2024Python Developer → Senior Data Engineer, Manager
Data engineering and analytics across lending, risk, and compliance; led a six-member engineering team.
The longer story, certifications, and education are on the about page →
AI Projects
Project Synapse
Knowledge Graph
A knowledge graph of a codebase in Neo4j, modeling code structure and relationships, that translates a production error directly into the pinpointed source of the issue and a suggested fix. Combines graph traversal with semantic retrieval over BGE embeddings in ChromaDB, linking error signatures to the relevant code and fix context.
Open Source
Python packages published on PyPI — market data, quantitative finance, and tooling for regulated-AI workflows.
pybhav
pip install pybhav ↗Download NSE bhavcopy (end-of-day market data) for Indian equities and derivatives.
last release · Jul 2026
quantra
pip install quantra ↗An open-source Python library for quantitative finance — covering time value of money, credit lifecycle, risk analytics, derivatives pricing, and portfolio optimization.
last release · Jun 2026
atomingest
pip install atomingest ↗A metadata-driven data ingestion framework for AtomSQL with fintech-grade security and compliance features.
last release · Mar 2026
atomsql
pip install atomsql ↗A lightweight Object-Relational Mapping (ORM) library for Python with support for SQLite and PostgreSQL.
last release · Feb 2026
regchain
pip install regchain ↗Governance, audit, evaluation, and controls for AI workflows in regulated finance.
last release · Dec 2025
qfinbox
pip install qfinbox ↗A Python library for quantitative finance, offering tools for risk management, portfolio optimization, and financial modeling.
last release · Oct 2025
Get in Touch
Whether it's a role, a data-platform problem, or a question about something I wrote — drop me a line and it lands straight in my inbox.