I Built an NLP Bad-Debt Early Warning System for a Bank — the Lessons Retail Can Use on Stocks Today

The five-line version

• I solved this inside a bank: built my workplace's first NLP Early Warning System, using WangchanBERTa to read Thai text and flag a borrower in trouble before the numbers said so
• Why it had to exist: every number is a lagging indicator — financial statements report the past, and an arrears line means you are already late
• Investors sit on the identical problem: the quarterly tells you what already finished; the live signal is buried in the language of Form 56-1 (One Report) and the MD&A, which nobody has time to read end to end
• The three most expensive lessons: false positives kill a system faster than inaccuracy does · target leakage builds a model that is accurate because it cheats · unexplainable means it never clears audit (I used SHAP)
• That whole skill set now sits in a tool retail can run without writing code — Boom Leverage Terminal searches Form 56-1 (One Report) and MD&A across 916 companies by meaning and returns verbatim text plus the source link at the SEC (Thailand's Securities and Exchange Commission) · right now, logging in gets you the DELTA pack free through 30 September 2026 (4 years of history · 150 credits/day · no card)

You know the sequence too well. A position starts to slide and you tell yourself "wait for the numbers" — which sounds like discipline. Fundamental investors decide on figures, not feelings.

The statements land. Earnings shrank exactly as feared. You sell.

The price had been sitting on the floor for two weeks already.

Not one step of that was wrong. The tool does exactly one thing: report the past. Every figure in a financial statement is a lagging indicator — a record of what already finished, not a clue about what is coming.

Trading on reported numbers alone is driving by the rear-view mirror — total clarity on the road behind you, and nothing at all on the road ahead.

I know the shape of this because I sat on the other side of the same problem, with the bank's money at stake.

The Job That Cured Me of Trusting Numbers Alone

One question haunted every year I spent in model risk: by the time the numbers say a borrower is in trouble, it is usually already too late.

Arrears, financial ratios, payment behaviour — all of them show up after the story has started going wrong. So I inverted the question. If part of the signal sits in text — news, interviews, reports — before it ever becomes a number, can it be pulled out and used as a warning in advance?

That was the origin of the NLP Early Warning System I built, and the first time I brought deep learning into this kind of work at my job. The core was WangchanBERTa, a pretrained Thai-language model, for two blunt reasons: the text to be read was overwhelmingly Thai, and at the time it was the best pretrained Thai model actually available. I could fine-tune it directly instead of training from scratch.

This is not a memoir. The same problem sits in the hands of every retail investor — different chair, same wall.

Same Problem, Different Chair: The Bank Hunts Bad Debt, the Investor Hunts Rotten Stocks

Inside the bank I sat on the creditor side. The question was whether this borrower could still pay over the next 6–12 months. A fundamental investor sits on the owner side. The question is whether the business is getting worse.

Both questions run on the same data and hit the same wall.

Read the last row again. This is not an intelligence problem, it is a volume problem. The data that describes the future exists, and it is public. It arrives in a form no single human can finish reading.

Why a Numbers-Only Signal Always Arrives Late

Traditional credit-risk models run mostly on numeric data. That is clean to measure and easy to model. The weakness is that it reflects what already happened, not what is about to.

By the time the statements are filed and the arrears surface in the system, the story behind them has usually been running for months. On a risk desk, being one quarter late costs you several options — and on the investor side, that lag is the price spread you hand to whoever knew first.

The problem I set out to solve reduces to three lines:

• Numeric signals are lagging indicators — always behind the event
• The data that arrives first is unstructured text (unstructured)
• Human reading does not scale; the volume outruns the team

The model-validation view: a numbers-only signal is a lagging indicator — it reports what already happened, not what is about to. In risk work, being even one quarter late is expensive

The system rests on one premise: trouble gets talked about before it reaches the books. Negative press, an abrupt management change, a dispute, stress in the supply chain — all of it is language, not numbers.

NLP's job is turning that language into a signal you can act on. I designed it as a complement, never a replacement, from day one. Proposing to displace the incumbent process wholesale is the fastest way to make the team that has to use it stop trusting the new tool.

The Architecture: Data → NLP → A Signal You Can Act On

The NLP conveyor the bank ran — and today you use its output without building one

  ①  RAW TEXT
      News · interviews · disclosure documents (56-1 / MD&A)
                    │
                    ▼
  ②  Match "who is this text about"
      Clean it, then bind it to the correct legal entity
                    │
                    ▼
  ③  WangchanBERTa  ← the Thai-language NLP core
      Positive or negative · how heavy · about what
                    │
                    ▼
  ④  THE FILTER ─── cut the false positives
      └─ Bank side: SHAP says "why the model fired"
      └─ Your side today: Citation Gate — no page number in the source PDF = dropped
                    │
                    ▼
  ⑤  ACTIONABLE SIGNAL
      Reaches the decision-maker with the source text attached to check

  ⚠️ Strip out ④ and what is left is "an AI that sounds credible" — worse than nothing

Every stage has its own problem, and the hardest one is not the model.

Stage one, collection — the challenge here is not modelling, it is cleanliness and continuity of data, full stop. (Collecting consistently beats collecting a lot in bursts.)

Stage two, signal extraction — identify who the text is about, whether it reads positive or negative, and whether it carries enough weight to care about. This is the core, and it is also the easiest place to fool yourself.

Stage three, delivery — turn the output into something a person opens and can act on, not a pile of scores nobody knows what to do with.

The Lessons That Cost More Than the Model

1. False positives are the real enemy, not inaccuracy

A system that fires wrong repeatedly gets ignored by the team within weeks, however clever it looks on paper. The people who have to chase every alert stop believing it first. That same false-positive problem is deciding outcomes again right now, as vendors put AI agents directly onto anti-money-laundering investigations inside banks — faster means nothing if it only means people believe a wrong signal sooner.

What I learned: few alerts, all of them right beats many alerts that are sometimes right.

The rule from risk measurement: false positives are the real enemy, not accuracy — a system that fires wrong repeatedly is ignored within weeks. Few alerts, all correct, wins every time

The investor version of this lesson: a screener that spits out 80 tickers every week has not helped you. It moved the reading pile onto your desk.

2. Trust is harder to build than the model

A good risk tool has to feel like it is helping the user, not grading their homework. I spent as much time sitting with the team, hearing what they wanted to see and reshaping the alerts around it, as I spent on the algorithm.

3. Measure whether it is useful, not whether it is accurate

• Accuracy on a test set does not mean it helps the actual job
• The good metric is whether the team acts on the signal
• If nobody opens it, pretty numbers are still a failure

The Real Gate Is Audit: Target Leakage and Being Able to Explain Yourself

What I did not anticipate at the start became the hardest gate at deployment: convincing audit — and that is not settled with an accuracy figure. Two things took disproportionate time.

• Handling target leakage — a model that looks too accurate is usually not good, it is quietly seeing data that had not yet arrived at the decision point. Where the consequences are real credit exposure, sealing every one of those leaks matters more than squeezing out another point of score. A model that is accurate because it cheats collapses the moment it meets live data.
• Explaining with SHAP values — saying the model fired is not enough; you have to say why. I used SHAP to show which factor pushed the prediction in which direction, so the team and audit could both see where the signal came from, instead of a black box demanding to be believed.

The line you do not cross: a model that looks too accurate is usually not good, it is seeing data that had not arrived at the decision point — closing target leakage matters more than another point of score, because a model that is accurate because it cheats collapses on contact with live data

Investors should read this part slowly. Target leakage is not a banking phenomenon. It is the same thing that happens when you pull up the chart of a stock that already blew up and say it was obvious at the time. It was obvious because you know the ending. What was actually in your hands on that day is the far more expensive question.

The discipline of having to prove where a signal came from is the same spine as the three-gate method for checking AI-generated numbers — that lesson followed me straight out of the audit room.

Translating the Bank Lessons Into Investor Language

How Retail Gets an Early Warning System of Its Own

When I built this, the time sink was never the idea. It was plumbing, experiments, and iterating code until something worked in production — headcount, budget and time that only an institution can pay for.

But the most important input in the equation is public: the Form 56-1 (One Report) and MD&A that every listed company must disclose under SEC rules. Free for anyone to download. Access is not the problem. Nobody has time to read the whole market.

So I compressed the entire skill set into a tool retail can run without writing a line of code — Boom Leverage Terminal. You type a question in plain Thai, say "เลื่อนการชำระหนี้" [they went back to the lender and asked for more time] or "ต้นทุนวัตถุดิบสูงขึ้น" [input costs repriced and the pass-through has not happened yet], and the system matches on meaning, not exact strings, against the management discussion of listed companies — 916 companies, FY2021–FY2026 — and returns the closest passages in seconds.

The same questions I used to put to the bank's EWS — today you type them yourself into one box. Every hit quotes the real text, with the page number and a link to the source document to check.

How Those Three Lessons Are Wired Into a Tool Like This

1. Beat the lagging indicator by reading language instead of waiting for numbers — the MD&A is text that is hard to lie in and easy to dodge in. Management rarely lies outright; they choose words, choose emphasis, choose silence. The change in wording from one year to the next is the signal. (a real read across years — the PSL case)

2. Attack false positives at their most expensive root: AI fabrication — I state this plainly because credibility depends on it: semantic search can still return results that miss, like every search tool ever built. What this system never does is invent text or manufacture a number. Every result is real text from the real document (verbatim), with company name, year, page number, and a link to the file at the SEC — so you kill a wrong hit in one click instead of guessing where the AI got a figure.

3. Solve explainability by pointing at the position on the real page — the system draws a box over the sentence that answers your question, on the source page, where you can see it. Measured live against the index built on 14 August 2026: 99.14% of served passages can be proven to a location (359,413 of 362,524 chunks · by domain: the auditor's report 99.48% · the financial statements 99.06% · risk factors 99.23% — across the searchable FY2021–FY2026 window), and the 3,111 passages that cannot be proven are quarantined and never displayed. I publish that number instead of claiming 100% because it is the same standard I had to meet handing a model to audit.

The One Thing I Want Remembered: the Bank's SHAP Is Your Citation Gate

Handing a model to audit, the brutal question was never "how accurate is it" — it was where did this signal come from. No answer, no approval, at 99% accuracy or any other number. A system that cannot explain itself is a system nobody can be held accountable for.

SHAP was my answer then. It shows which factor pushed the prediction which way — not a black box demanding to be believed.

Building the retail tool did not relax that requirement by a single degree. It only changed form.

Put as plainly as possible: retail does not need to train an NLP model and does not even need to understand SHAP. You need exactly one thing from it — show me the evidence — and that is what the source button delivers in one click, with no knowledge of the conveyor behind it.

For why searching by meaning differs materially from hitting Ctrl+F, I broke it out separately in how semantic search differs from exact-match search. For how to turn it into actual alpha, read mining the MD&A as alternative data.

Bottom Line: This Is No Longer Institutional-Only Kit

The interesting question now is not whether it can be done. It is how useful you can design it to be — and after doing this inside a bank and outside one, my answer has not moved: few alerts and all of them right, a provable source on every line, and a user who can always click through to the original.

Test it on a question from your own book: open Boom Leverage Terminal and type the thing you are most afraid of about a stock you hold — "สภาพคล่องตึงตัว" [cash tight enough that they had to put it in writing], "พึ่งพาลูกค้ารายใหญ่" [revenue hanging on a handful of customers], "เลื่อนการชำระหนี้" [debt service pushed out; the original schedule broke] — and see who in the market has said it, in management's own words. Log in today for the DELTA pack free through 30 September 2026 — pulling a full 4 years of history · 150 credits/day · no card required (billing is switched off entirely for the duration; when it ends the account reverts to the free pack on its own, with nothing charged) — why I am giving it away, and the full terms · for what the paid packs actually change, read the straight guide to picking a pack · team and institutional (seats · Excel export · API) at the Enterprise page or contact@boomleverage.com

To be explicit: this article recounts lessons from building a credit-risk early warning tool and from testing whether its output can be trusted. For education only. Not investment advice, no security is being recommended, and no outcome is guaranteed · Investing carries risk; past results do not guarantee future ones