R.K. HendersonAI leader · data scientist

Interactive lab / 03

credit risk & loss

Credit risk & loss lab

A fictional property-management lender, from a SQL warehouse to a lifetime loss reserve that is defended to an external reviewer.

The decisionHow much a resident lending book will lose over its life, how each loan is graded, and whether the answer holds up in front of an external validator.

Lending 7 steps Synthetic or public data

Executive brief

The decision, in two minutes.

Two-minute read · one-page PDF with the call, the evidence, the limits and a 90-day plan
Try the live demo Download the brief (PDF)
Decision brief · Credit risk & loss

A fictional property-management lender, from a SQL warehouse to a lifetime loss reserve that is defended to an external reviewer.

Credit risk & loss lab Synthetic data 7 decisions
0.848out-of-time AUC, ahead of the logistic benchmark
10/10validation tests recomputed on every load
4audit findings, each closed with evidence
7decisions, told step by step
6algorithms and methods used
The decision

How much a resident lending book will lose over its life, how each loan is graded, and whether the answer holds up in front of an external validator.

The problem

A loss reserve sits on the financial statements, so the models behind it are tested by auditors and examiners, not only by data scientists. A boosted model that ranks well is not enough: it has to behave like credit policy, give exact reasons, stay calibrated on loans it never saw, and be reproducible to the cent.

The answer

Ship the boosted model only with monotone constraints and a standing logistic challenger, and fix the out-of-time miss with an overlay fitted on development data, not on the test.

What it changed
1 Gradient boosting at 0.848 out-of-time AUC against 0.836 for the benchmark, with every monotone constraint holding and TreeSHAP reason codes.
2 A probability-weighted lifetime reserve with baseline, adverse and severe scenarios and a sensitivity on every input an auditor will ask about.
3 Calibration failure found out of time (Hosmer–Lemeshow p = 0.016) and remediated without touching the test vintage: p = 0.053, all seven grades green.

Results come from synthetic data with known answers. They show the method, not a client result.

Inside the lab
A screen from the Credit risk & loss lab app
How it was proven
  • Synthetic only: 19,200 loans, monthly performance and a synthetic unemployment series generated with known relationships. Calder Lane is fictional.
Gates cleared · 2 of 4
What it can’t tell you
  • It runs on synthetic data and books no real reserve, so adoption is not measured here.
Built with

PythonSQL (SQLite)XGBoostscikit-learnSciPypytestPlotlyStreamlit

The call: Ship the boosted model only with monotone constraints and a standing logistic challenger, and fix the out-of-time miss with an overlay fitted on development data, not on the test. richardhenderson.io/projects/credit-risk · All data synthetic · R.K. Henderson

Try it

The live demo.

The real app, running on synthetic data. Change the inputs and watch the decision move.

Interactive, synthetic data. If nobody has used it for a while it takes about 30 seconds to wake.

Open in a new tab ↗

The methods

How it was built.

7 decisions, told as chapters. Swipe or step through; any chapter opens that step of the live demo.

  1. Chapter 01 of 07 · Portfolio

    What have we lent, and how is it performing?

    Vintage curves, roll rates and delinquency, every number queried in SQL from a small warehouse.

    Open this step
  2. Chapter 02 of 07 · PD model

    How likely is each loan to charge off within a year?

    Monotone, calibrated XGBoost beats a logistic benchmark on a vintage it never saw, with reason codes.

    Open this step
  3. Chapter 03 of 07 · Loss & reserve

    How much will the open book lose over its life?

    Default timing by term, loss given default by property class, amortizing exposure: lifetime loss per loan.

    Open this step
  4. Chapter 04 of 07 · Scenarios

    What happens if the economy turns?

    Unemployment sensitivity estimated from the book, weighted scenarios with reversion, and a tornado of every input.

    Open this step
  5. Chapter 05 of 07 · Backtest

    Did the predictions come true?

    Calibration failed out of time; a macro overlay fitted on development data alone fixes it, grade by grade.

    Open this step
  6. Chapter 06 of 07 · Audit defense

    Can we defend it to an external reviewer?

    A live test plan, an inventory record, an assumptions log and findings answered with root cause and evidence.

    Open this step
  7. Chapter 07 of 07 · Ownership

    Who built it, and what did it take?

    One person, no direct reports, every layer from the data design to the audit response.

    Open this step

The call: Ship the boosted model only with monotone constraints and a standing logistic challenger, and fix the out-of-time miss with an overlay fitted on development data, not on the test.

Data and provenance

Synthetic only: 19,200 loans, monthly performance and a synthetic unemployment series generated with known relationships. Calder Lane is fictional.

Methods, in order

  1. SQL vintage & roll-rate analysis
  2. Monotone XGBoost PD
  3. Logistic benchmark
  4. Fractional-logit LGD
  5. Lifetime ECL (CECL-style)
  6. Macro scenarios
  7. Binomial backtests
  8. Model validation

Algorithms and stack

  • XGBoost (monotone)
  • Logistic regression
  • Fractional logit
  • Discrete-time hazard
  • Hosmer–Lemeshow
  • SQL window functions
  • Python
  • SQL (SQLite)
  • XGBoost
  • scikit-learn
  • SciPy
  • pytest
  • Plotly
  • Streamlit

Open conversations

Questions about
how it was built?

Decision Lab · live demo
Starting the demo…