series

AI in financial risk, from statistics to LLMs

My notes and code from a thirteen-week course on AI in financial risk, written up in my own words as I go. The first half compares the pre-AI way of measuring market and credit risk with bagging and boosting models, and ends in a capstone that backtests both honestly. The second half builds up how large language models work and where they fit in portfolio management and surveillance, and ends in a capstone that combines FinBERT and a knowledge graph.

for
People who know some Python and want to see where machine learning and LLMs actually change risk work, and where they don't.
published
1 part · 16 min of reading
planned
10 more
Start with part 1

parts

  1. 01

    Financial data and its distributions

    The first part of a series on where machine learning and LLMs change risk work. Before any model, ten years of daily S&P 500 closes go through one inspectable step at a time: prices become returns, the returns are compared with a normal distribution of the same mean and volatility, and the gaps are counted. There are 36 days beyond three standard deviations where the normal expects about 7, a one-day 99% cut-off breached 51 times instead of 25, and volatility that clusters in a way the normal says it cannot. Removing 2020 makes the tails thinner, not normal.

    16 min readcode ↗

  2. 02

    Machine learning versus statistical modelsplanned

    What changes when the goal is prediction rather than inference, the variance and bias trade-off on a risk problem, and the confusion matrix read with costs attached.

  3. 03

    A map of the algorithmsplanned

    Supervised and unsupervised learning, bagging and boosting, sampling for rare events, and clustering, each with the risk question it answers.

  4. 04

    Python for riskplanned

    Returns, volatility, and historical and parametric value at risk computed from scratch, one inspectable step at a time.

  5. 05

    Market risk before and after AIplanned

    Historical and GARCH value at risk as the baseline, a boosted model against it, and a backtest that counts the breaches instead of quoting a fit.

  6. 06

    Credit risk before and after AIplanned

    A logistic scorecard as the baseline and a bagged forest against it, compared on ranking, calibration and stability rather than accuracy.

  7. 07

    Capstone 1: a pre-AI versus AI risk model, backtestedplanned

    One public market dataset and one public credit dataset, a classical baseline and an ensemble model for each, a time-ordered backtest, and a written verdict on whether the extra complexity paid for itself.

  8. 08

    What an LLM is, built up from tokensplanned

    Tokenization, embeddings, vector search, the transformer, and retrieval-augmented generation, built from the smallest piece upward on financial text.

  9. 09

    LLMs in portfolio managementplanned

    Where language models can feed a portfolio process, from reading filings to summarising research, and how to test whether they add anything.

  10. 10

    FinBERT, knowledge graphs and insider-trading semanticsplanned

    Domain sentiment with FinBERT, a graph of insiders, companies and events, and how the two together flag trades that came before material news.

  11. 11

    Capstone 2: surveillance signals and a balanced portfolioplanned

    Public insider filings and company disclosures scored with FinBERT and linked in a knowledge graph to flag suspicious timing, then the same signals tested as an input to portfolio balancing against a plain baseline.

Follow new parts via RSS.