Our AI Deletes Its Own Sentences — Here Is the Validator That Decides Which, and the Mistakes It Still Lets Through

Here is one sentence from PTT's management discussion and analysis for fiscal year 2025, page 2 of the document as filed with the SEC (open it on the SEC's site):

Verbatim — PTT, MD&A FY2025, page 2: "Thai Baht appreciated against US$, with the average foreign exchange rate strengthening from 35.5 Baht/US$ in 2024 to 33.0 Baht/US$ in 2025, resulting in lower revenues denominated in US$."

Two numbers, one direction, one consequence. Now give that sentence to a language model with the instruction "summarise what companies said about the strong baht", and you get back a line like "PTT said the baht strengthened from 35.5 to 33.0 per dollar, lowering its US-dollar revenues [1]". Correct. Or you get back "…from 35.5 to 31.0…". Or "…in 2566…". Or the right sentence with the footnote pointing at somebody else's filing. All four look identical on a screen. All four are written in the same confident register.

Since 11 October 2026, Boom Leverage AI has been open to every signed-in account on the Terminal, and the AI summary on a search result, a Health Check, or a single year of one company runs the same machinery. Before any of that text reaches you, a piece of code I call the citation validator reads it line by line and decides what is allowed through. This article is the validation write-up of that control: what it checks, where the rules came from, what it costs you when it fires — and, the part most vendors leave out, what it does not catch.

Why a validator, and not just a better prompt

I spent years in model validation at Thai banks. The first thing that job teaches you is that a model's instructions are not a control. A prompt that says "only use the sources, never invent numbers" is a policy statement. It describes what you hope the model does. A control is something that observes what the model actually did and acts on it, independently of the model, every time.

The prompt behind Boom Leverage AI does say those things. It tells the model to use only the numbered sources it is given, to end every line that carries a number, a company name, or a fact with a citation like [1] or [2][3], to write years exactly as the source labels them, and to say in one plain sentence when the sources do not answer the question. Models follow instructions like these most of the time. "Most of the time" is not a number you can publish next to a figure someone might trade on.

So the prompt is the first line of defence and the validator is the second, and the validator does not trust the first.

Rule from risk work: a control that the model can talk its way past is not a control. The validator never asks the model whether it complied. It reads the text, compares it to the sources, and decides on its own.

The unit of checking is a line — and that was a production lesson

Before you can check a sentence, you have to find it. In English that is mostly a matter of full stops. In Thai it is not: written Thai rarely ends a sentence with punctuation, so the validator splits an answer at line breaks, bullets, and Western terminal punctuation, and treats each resulting line as one claim whose citations cover all of it.

The first version did something that looked reasonable: it also split after every [n]. On the very first question asked in production, on 10 October 2026, a line of the shape "…21.9% [1] ซึ่งเป็นข้อมูลทั้งปี" ("…which is the full-year figure") was cut into a cited head and an uncited tail, the tail was judged an uncited claim, and an honest answer was withheld. The fix was to stop splitting at citations.

Two more splitting rules came from the same two days:

• A period after a Thai letter is an abbreviation, not a sentence end. "ลบ." (million baht), "บ.", "พ.ศ." — splitting there turns "revenue 120 ลบ. up from…" into two fragments.
• A table row is one unit, whatever punctuation is inside its cells. In an end-to-end test on production on 11 October, a summary table row containing "0.8 ลบ. (จาก -0.3 ลบ. ใน Q2)" was broken into three fragments, two of them "uncited", and the row was displayed torn. A row's citation cell now covers the whole row.

None of these is glamorous. All of them are the difference between a validator that cuts invented numbers and a validator that cuts honest answers — and a validator that cuts honest answers gets switched off by the people it annoys.

The six checks every line goes through

Each line is first classified: is it a claim that needs a source, or framing that may stand without one? A line is a claim if it contains a number, names a ticker that appears in the sources, or is long enough (60 characters or more) to be asserting something. The first line of an answer, a line ending in a colon, and a markdown heading are treated as headings — they restate the question and assert nothing. Then:

1. A claim must carry a citation. No [n] on a fact-bearing line → the line fails as an uncited claim. One exception: a line that says the sources do not cover something ("the sources do not mention hedging policy") may stand uncited, provided it states no number. That is the model doing exactly what it was told to do when evidence runs out, and punishing it for that would teach it to fill the gap instead.

2. Every [n] must point at a source that exists. If the answer was given twelve sources and a line cites [13], the line fails. This sounds trivial; it is the cheapest way to catch a model that is pattern-completing citations rather than reading them.

3. A number in the line must appear in a source the line cites. This is the core check, because numbers are what models invent. The validator pulls every number from the line — Arabic or Thai digits, thousands separators and decimals normalised — and looks for it in the quoted passage of each cited source and in that source's neighbouring page context. Years and period labels are set aside ("FY2025", "Q3", "2Q25" are which period, not what happened). If the numbers are nowhere in the cited sources, the line fails — and the validator then asks one more question: is the number in some other source? If yes, it is a wrong footnote (repairable). If it is in no source at all, the model made it up.

4. A Thai Buddhist-era year must be explainable. The number check exempts years, which left a hole that production found on 11 October: on a Health Check summary for CPALL, sources labelled FY2026 Q1 came back in the answer as "ปลายไตรมาส 1/2566" and "1Q66". The model had "translated" 2026 into 2566. The correct Thai year is 2569. Both the prompt and the validator changed the same day. The prompt now forbids converting between Western and Thai years at all; the validator now cuts any 25xx year that neither appears in a cited source's text nor matches the Thai face of a cited source's fiscal-year label (with one year either side, so a prior-period comparison still passes).

5. A line with no number must still resemble the source it cites. A line with no digits can carry a wrong footnote straight past checks 1 to 4. So for numberless claims, the validator measures overlap: the share of the line's four-character letter sequences found in the cited source. Thai has no spaces between words, which is why the unit is a character sequence rather than a word; four letters is long enough that common particles do not count. If the overlap with the cited source is 8% or less while some other source overlaps 35% or more, the line is flagged as miscited — it plainly came from a different filing. The two thresholds are deliberately far apart, for a reason I come back to below.

6. History does not leak old footnotes. In a chat, when you change the scope and the system searches again, the new source list starts at [1] again. The earlier answers in the thread still carry chips pointing into the old list, and on 11 October the model copied them — a "[2]" on a line whose text matched the new source [6]. Now, whenever a turn re-searches, the citation numbers are stripped out of every earlier answer in the history the model sees, with a one-line note telling it why. The only numbers it can cite are the ones it was just given.

What happens to a line that fails

This is where the design choice matters most, so here is the policy exactly as the code implements it.

A failing line is dropped, not the whole answer. The lines that passed are shown, and a note under the answer says how many lines were cut because they could not be traced to a source. The answer is withheld entirely only when no cited claim survives — and then you see the sources on their own, with a note that the answer was not shown and nothing was charged for it.

Before dropping, there is at most one repair pass — and only for citation errors. If every reason the validator found is a footnote problem (a missing citation, an out-of-range citation, a number that sits in a different source than the one cited, a miscited numberless line), the answer goes back to the model once, at temperature zero, with an instruction to fix the citations only and add nothing new. Then it is validated again. If any line contains a number found in no source at all, there is no repair pass for that answer: the invented line is simply cut. Asking a model to "find a citation" for a number it made up would only teach it to cite harder. The repair is also skipped when there is not enough time left in the request to finish it safely.

The AI portion is charged only when at most half of the lines were cut. An answer that was mostly wrong is free, even if some of it was shown. The model is not paid for guessing, and neither are we. On the /scan summary, the summary's own credits are taken only after the validator has run, under the same rule; the search credits you already spent to get the result set are a separate thing you bought before the AI was involved.

Every shown answer carries the same sentence underneath it: "สรุปโดย AI อาจผิดพลาดได้ — โปรดตรวจกับแหล่งอ้างอิงด้านล่างก่อนนำไปใช้งาน" — summarised by AI, may be wrong, check against the sources below before using it. Every [n] opens the source card: the verbatim quote, the page, the link to the SEC original, and for Opportunity Day sources, the recording at the minute it was said.

And every verdict is kept. For each AI summary I can see, on an internal review page, the verdict, which lines were cut and why, whether credits were taken, the token counts, and the full text beside its sources. A withheld answer raises an alert. The chat stores its verdict with each answer. A control whose outputs nobody reviews degrades quietly; this one leaves a trail.

What I will not do: quietly loosen the validator because an honest answer got cut. Every relaxation listed in this article — headings, index names, durations, Thai abbreviations, table rows — was made because a specific production answer was wrongly withheld, and each one is pinned by an automated test so the next change cannot widen it by accident.

Watching it work: constructed test lines against the real code

I did not want to describe the validator from memory, so for this article I ran the actual validator code against two real sources: [1] the PTT passage at the top of this page, and [2] this passage from CM's MD&A for Q3 of fiscal 2025, page 1 (SEC link):

Verbatim — CM, MD&A Q3 FY2025, page 1: "appreciation of the baht, which strengthened from the average exchange rate of foreign sales of 34.87 baht per dollar to 32.17 baht per dollar, or an average appreciation of more than 7.7%"

The lines below are test lines I wrote by hand to probe the validator. None of them is real AI output, and none is a statement about either company. The verdicts are what the code returned.

That is the control doing its job. The next table is the reason this article exists.

What the validator does not catch — measured, not guessed

Same two sources, same code. These lines all passed:

Read those rows carefully, because they define how you should use the product.

The number check fails a line only when every number in it is missing from the cited sources. A line that pairs one real figure with one wrong or invented one passes. That is the most important limitation in this article, and I am publishing it rather than waiting until it is fixed, because you should know it today. Tightening it is not free — legitimate answers compute differences and ratios from cited figures (the prompt explicitly allows "33.0 vs 35.5, a fall of…" written beside its sources), and a strict "every number must appear verbatim" rule would cut those. Until there is a check that can tell a derived number from an invented one, treat every figure in a line as needing its own look at the source card.

The validator checks traceability, not meaning. A line that cites the right source and reverses what the source says contains no number and shares most of its vocabulary with the source, so nothing fires. Negation, direction, cause and effect — "lower" versus "higher", "despite" versus "because of" — are invisible to overlap and number checks. This is the class of error a human reader catches in two seconds and a string-matching control never will.

The miscite check only catches footnotes that point at an unrelated source. Between two sources on the same topic — two companies both writing about the strong baht — vocabulary overlaps so heavily that a line citing the wrong one sails through. And a Thai paraphrase of an English filing overlaps almost nothing with any source, which is exactly why the thresholds are set so far apart: if the check fired on low overlap alone, it would cut every correct Thai summary of an English MD&A.

Context is checked, not shown. A number can pass because it appears in the neighbouring page text the model was given for understanding, rather than in the quoted passage on the card. It is real and it is in the filing, but you will not see it in the quote box — you will see it when you open the page.

Nothing checks what was left out. The validator judges the lines that were written. It cannot see the source the model ignored, the company that said the opposite and was not mentioned, or the quarter that was skipped. Selection is the oldest bias in research and no line-level control touches it. The cross-sectional search underneath — every company that wrote about the theme, not the ones the model chose — is the part of the product that addresses this, which is why the sources sit under every answer rather than behind a click.

And the sources themselves are the ceiling. The validator proves a line came from the filing. It cannot prove the filing is right. A company's MD&A is what management chose to write; an Opportunity Day passage is an AI transcript of what management said. I wrote about what that difference means in the Opp Day piece and about verification gates more generally in AI fabricates financial numbers better than you think.

The model-validation view: if you validate this tool the way a bank validates a model, the finding is "effective against invented figures and wrong footnotes; not designed to detect misstatement of meaning, partial numeric errors, or omission." That is a usable control with a documented boundary — which is the honest description of almost every control that exists.

Two smaller rules that came from the same two days

The model is not allowed to label transcripts. On the first run of the per-angle summary table on production, on 11 October, the model wrote "(ถอดเสียง)" — "transcript" — on 18 of 24 rows. None of the cited sources was an Opportunity Day transcript. The label now comes only from the source's own metadata and is attached to the [n] chip; anything the model writes in that form is stripped before the validator runs and before the answer is stored. A provenance label that a model can invent is not a provenance label.

The model only sees what you can see. Rows locked behind a pack you have not bought are excluded from the source list before the model is called. It cannot summarise a passage you were not shown, so every [n] in your answer is one you can open.

How to read an AI answer on the Terminal, given all of this

A workflow you can follow tonight, built from the limitations above rather than despite them:

1. Read the note under the answer first. "N lines cut" tells you the model was struggling on this question; treat the surviving lines as leads and the sources list as the real answer.
2. For every number you intend to use, open its [n]. Not the line — the number. The validator guarantees at least one figure on a passing line is in the source; it does not guarantee all of them.
3. For every line that states a direction (up, down, because, despite), read the quote and check the verb. This is the error class nothing automated is catching for you.
4. Scan the source list for companies the answer did not mention. If eight companies wrote about the theme and the answer discusses five, the other three are where your research starts.
5. Change the question, not the trust level. If an answer is withheld or heavily cut, narrow the scope — fewer companies, a tighter year range, one axis — rather than asking the same question again. There is more on phrasing in ask the filings a theme, not a year.

The same checks run on every surface: the chat, the AI summary over a search result, and the Health Check summary for a single company.

Why this matters beyond one product: the Bank of Thailand already expects financial institutions to be able to explain and validate the AI they rely on (I wrote that up here). An individual investor has no validation team. A published, testable description of what the tool checks — and what it does not — is the closest thing to one I can give you.

Overview

Every line Boom Leverage AI shows you has passed six automated checks: it is cited if it states a fact, its citations exist, its numbers appear in what it cites, its Thai years can be explained, its words resemble its source when it has no numbers, and its footnotes are not left over from an earlier search. Lines that fail are cut and counted in front of you; an answer with nothing traceable left is withheld and not charged; the AI portion is charged only when at least half the lines survived. What it does not catch — a wrong number sitting next to a right one, a reversed meaning, a footnote swapped between two similar sources, and anything the model chose to leave out — is listed above, from a test run against the code itself.

If you run a research, risk or compliance function and need to put this in front of a model-risk or validation process — the verdict logs, the test suite, the incident history behind each rule — that conversation is at boomleverage.com/th/enterprise or contact@boomleverage.com.

If you want to try it yourself, sign in at terminal.boomleverage.com, ask a question, and then do what this article recommends: open the [n] behind every number before you believe it.

Disclaimer: This article is written for education and to explain how an AI summarisation control works and where its limits are. It is not investment advice, it recommends no security, and it guarantees no outcome. · The PTT and CM passages are quoted verbatim from filings published on the SEC's website; the test lines in the tables were written by hand to probe the validator and are not statements about either company or output from the AI · Validator behaviour is described as of 11 October 2026 and will change as the control is tightened · AI summaries can be wrong even when every line passes the validator — always check the original source before relying on any figure · Investing carries risk; past results do not guarantee future ones