# Phase E — the mailbox, and one arrival of money

> Append-to for [Wallet Statements](/docs/modules_handbook/manage/payments/wallet-statements/readMe.md) (reading) and [Transfer Receipts](/docs/modules_handbook/manage/payments/transfer-receipts/readMe.md) (resolving). Phase D read statements and showed them; Phase E fetches them by itself and lets their money reach the ledger.

## The convergence rule

> **One arrival of money is one `purchase_histories` row, carrying up to two identity columns that different lanes own. The lane that arrives first creates the row; the lane that arrives second never creates — it writes its own identity onto the row a human named. When the machine cannot be certain which row that is, it HOLDS and asks.**

```
payment_reference    ← the CLAIM's code (TNG-DUFB26).   Globally unique.
provider_payment_id  ← the STATEMENT LINE's dedupe_key. Unique per provider.
```

⚠️ **Why this was necessary.** `resolve()` used to create a row keyed on the claim's reference; an ingester would have created one keyed on the statement line's. Those are different strings, so the unique index never fires and **one RM299 transfer becomes two ledger rows — RM598 of recorded revenue for RM299 of money**, double-counted by the Sales dashboard, by `AdPerformanceService` (and therefore ROAS), and potentially reported to Meta twice.

[`TngLedger`](/src/Payment/Services/TngLedger.php) is the **only** place a Touch 'n Go row is created. Both lanes call it, so the column payload cannot drift into two versions of the same fact.

### The state machine

| Ordering | Who creates the row | Result |
|---|---|---|
| **Statement first**, claim later | the ingester (`tng:ingest-lines`) | An unattributed ACTIVE row: no payable, no lead, `provider_payment_id` set. Not revenue. When the admin resolves the receipt **onto that line**, the row is UPDATED and `adoptClaim()` adds the claim's reference. **One row, both identities, one grant.** |
| **Claim first**, statement later | `resolve()`, via `TngLedger::record()` | An attributed row with `payment_reference` set and `provider_payment_id` **NULL** — "unbacked". When the statement arrives, the line finds a candidate and is **HELD**. Zero rows created; a human confirms. |
| Statement only, nobody claims | the ingester | An unattributed row forever. Invisible to every revenue figure — they filter on a payable and join through the lead. |
| Claim only, statement never comes | `resolve()` | Stays unbacked. **Never auto-revoked**: `likelyFiltered()` means absence from a statement is not evidence of absence. |
| The same money in two overlapping statements | nobody | `TngStatementRepository::store()` already does `firstOrCreate` on `dedupe_key`, and the ingester only reads `STATE_NEW`. Structurally a no-op. |
| A payment whose row was **deleted**, re-ingested | nobody | The line keeps its `purchase_history_id`; the ingester answers `already` and looks it up `withTrashed()`. A deletion is a decision. |

## ⚠️ Why the machine may never match a line to a payment

Measured on this business's own ledger:

> **294 of 332 payments (88.6%) share an amount, a currency and a day with at least one other payment**, in 47 groups. The worst: **26 identical MYR 500 rows on 2026-06-05**, and 17 identical MYR 2,388 rows on 2024-09-30.

Amount and date are not a discriminator here. Auto-matching on "exactly one candidate in the window" would file real money against the wrong customer — and `counterparty_name`, `payment_provider` and `payment_reference` are all **create-only**, so that mis-file is unrecoverable.

So a line with any candidate becomes `STATE_HELD` and waits. **Holding costs nothing that matters**: in the only ordering that produces a hold (the statement arriving after the receipt was resolved) the buyer already has their product and the money is already in the ledger. What waits is the bookkeeping fact that the wallet corroborates it.

## The five refusals in `resolve()`

Server-side, every one — the picker renders from a snapshot and the queue is shared between screens, so "the option was not offered" is not a guard.

| Refusal | Prevents |
|---|---|
| A **second receipt from the same person** for the same item and amount within `config('tng.match.duplicate_claim_days')`, unless `confirm_duplicate` | ⚠️ **The double-record hole.** The public form's double-tap window is per-browser and ten minutes long, so a customer submitting from their phone and then their laptop files two claims. Reproduced before the guard: **two ledger rows, RM598 recorded for RM299 of money.** |
| The line is not `direction=in` + `is_customer_payment` + `is_successful` | attaching a receipt to a toll payment or a reload |
| The line's row is **soft-deleted** | resurrecting a deletion decision as a second grant |
| The line's row already has a `payment_reference` | two customers' receipts on one wallet transfer |
| The line's row already has a `payable_type` | another admin got there first |

⚠️ **Exactly one of `statement_line_id` / `no_statement_yet` must be answered.** A bare `nullable` would make *"I did not look"* indistinguishable from *"I looked and it is not there"*, and those are different facts about money.

⚠️ **With a line chosen, the WALLET's figures win** — amount, currency and `purchased_at` come from the line, not from what the admin typed. They are frozen by `READ_ONLY_FACT_PROVIDERS`, so accepting a typed amount would silently discard it behind a success message. The form disables those inputs and says why.

## The mailbox

The owner's real inbox auto-forwards Touch 'n Go mail to a dedicated Gmail, and the poller reads only that. An App Password on his own inbox would hand this system his entire mail; a forwarding rule limits the blast radius to statements.

- **Composer-only.** [`webklex/php-imap`](https://github.com/Webklex/php-imap) — **PHP 8.4 removed `ext/imap` from core** (moved to PECL; its c-client library has been unmaintained since 2007), and neither the dev box nor `scripts/server-setup.sh` has it. Zero new apt packages, the rule Phase D set.
- **Behind a contract.** [`TngMailboxReader`](/src/Payment/Contracts/TngMailboxReader.php) — every rule about which message counts, what a duplicate is and when a message may be flagged is tested with **no network, no Google account and no password** ([`FakeTngMailboxReader`](/tests/Support/Tng/FakeTngMailboxReader.php)). The IMAP class only moves bytes.
- **Identity is the RFC 5322 Message-ID, not the IMAP UID.** A UID is unique only within one folder and one `UIDVALIDITY` generation; the server may reset it, and a moved message gets a new one.
- **Plus the attachment's own sha256**, because the same statement forwarded twice arrives under two Message-IDs.
- **Every message gets a record** (`tng_statement_mails`), including "no PDF" and "could not read". Otherwise *"no statements arrived"* and *"three arrived and all three failed"* are the same silence — and only one of them means nobody paid.
- **`\Seen` is set only AFTER the statement is committed**, and never on an unreadable one. A message flagged first and then lost to a crash is a statement nobody will look at again.
- **Bytes, never a temp file.** A statement carries a person's whole month of spending.

### Scheduling

```php
$schedule->command('tng:poll-mailbox --ingest')->hourly()->withoutOverlapping()
    ->when(fn () => TngSetting::current()?->credential(TngSetting::CRED_MAILBOX_USER) !== null);
```

⚠️ **Hourly, and the reason is not load.** The owner triggers each statement BY HAND in the app, so freshness depends on how often *he* remembers — polling every minute against a boss who exports weekly buys nothing but 1,440 pointless Google logins a day. Noticing that gap is Phase F.

⚠️ **A failure is logged ONCE, not every run.** `ops:scan-logs` emails every new warning line every five minutes ([`Kernel.php`](/app/Console/Kernel.php)), so a chatty poller becomes an email every quarter hour until somebody fixes the mailbox — and a channel that noisy is one nobody reads. `PollTngMailbox` remembers the failure in the cache and reports the first occurrence only.

⚠️ **Reading and recording are separate commands** (`tng:poll-mailbox`, `tng:ingest-lines`) so they fail independently and either can be re-run alone — a statement uploaded by hand must reach the ledger with nobody polling mail. On the schedule they run together, because a statement read and then left unrecorded is money the screens do not know arrived.

## The picker — the one screen that decides whether money is counted twice

[`StatementLinePicker.vue`](/resources/js/Pages/Manage/Payment/Claims/Partials/StatementLinePicker.vue), mounted between *what the customer typed* and *what we are recording*: the one moment a person holds both halves of the evidence.

1. **Nothing is ever preselected**, not even with exactly one candidate — prefilling turns a required judgement into a default a tired admin clicks past. Same reason the buyer is not prefilled from the typed email.
2. **"Not in a statement yet" is always present and is the starting state.** The safe answer has to be the cheap one.
3. **Ambiguity is stated, not ranked away.** The `names agree` / `different name` label orders and labels the list; it never selects and never writes.
4. **The raw cells are one click away** — the balance chain proves the date, the amount and the direction and says *nothing* about the sender, which is the only thing that tells two identical amounts apart.
5. **Candidates come from the server**, so the rule is testable rather than living in a component.
6. Picking a line **visibly locks** amount, date and currency, and says they came from the wallet.

## Related files

- [src/Payment/Services/TngLedger.php](/src/Payment/Services/TngLedger.php) — the single writer: `record()`, `ingestLine()`, `candidatesForLine()`, `candidatesForClaim()`, `isRecordable()`
- [src/Payment/Support/Tng/TngLedgerResult.php](/src/Payment/Support/Tng/TngLedgerResult.php) — carries the REASON, not a boolean
- [src/Payment/Services/TngMailboxPoller.php](/src/Payment/Services/TngMailboxPoller.php) · [ImapTngMailboxReader.php](/src/Payment/Services/ImapTngMailboxReader.php) · [Contracts/TngMailboxReader.php](/src/Payment/Contracts/TngMailboxReader.php)
- [src/Payment/TngStatementMail.php](/src/Payment/TngStatementMail.php) + repository + facade + migration — what has already been read
- [src/Payment/Repositories/PurchaseHistoryRepository.php](/src/Payment/Repositories/PurchaseHistoryRepository.php) — `adoptClaim()`, a narrow setter for the two create-only columns the claim lane owns. **Never widen `update()` instead**: `payment_reference` is a live Stripe row's correlation key.
- [app/Console/Commands/Tng/PollTngMailbox.php](/app/Console/Commands/Tng/PollTngMailbox.php) · [IngestTngLines.php](/app/Console/Commands/Tng/IngestTngLines.php)
- [config/tng.php](/config/tng.php) — `match.window_days`, `match.claim_window_days`, `match.duplicate_claim_days`. These bound a SEARCH, not a decision; a wider window means more questions, never a wrong answer.
- [tests/Feature/Payment/TngConvergenceTest.php](/tests/Feature/Payment/TngConvergenceTest.php) — every ordering, each asserting the row count **and** the grant count; four guards mutation-verified
- [tests/Feature/Payment/TngMailboxPollTest.php](/tests/Feature/Payment/TngMailboxPollTest.php) — polling, with no network

## ✅ Proven on a real statement (2026-08-07)

The first genuine encrypted statement went end to end: forwarded by the owner's Gmail rule,
fetched over IMAP, decrypted, parsed, and two payments recorded. What it settled — all of
which had been inference until this moment:

| Question | The answer, from a real file |
|---|---|
| **What is the PDF password?** | TNG's own email says it: *"your registered TNG eWallet mobile number (without spaces, symbols and the country code)"* — `+6011-23456789` → `1123456789`. `expectedPdfPassword()` already computed exactly that. |
| **Which encryption?** | Revision 3 / V2 — **RC4-128**. The hand-written RC4 (pinned to RFC 6229) is what opens it; `openssl_decrypt('rc4', …)` would have returned false on Ubuntu 24.04. |
| **What does a money-in row's sender look like?** | A **real full name** — `LEOW CHUN HO`, `CHEW HENG TSENG`, `YONG KEAN KWAN`. Not a masked phone, not a flat "TNG eWallet User". The picker's name label is therefore genuinely able to tell two identical amounts apart. |
| **`occurred_at_precision` on a credit?** | **`day`.** Money-in rows carry NO time. `purchased_at` lands at midnight and is then frozen by `READ_ONLY_FACT_PROVIDERS` — so the Meta 7-day reporting wall is measured from midnight, and the candidate window must widen to the whole day (it does). |
| **Is the type allow-list right?** | Decisively. 79 rows: `Reload` ×36 (money in, **not** a sale — the owner topping up), `Receive from Wallet` ×3 (**the only real customer payments**), `Cashback` ×1, and every debit type recognised. ⚠️ `str_contains($type, 'Received')` would have **missed all three** — `Receive from Wallet` has no "d". |
| **Does the balance chain hold on a real file?** | 70 links, **0 broken** — cleaner than the public sample, which had 5 gaps from a filtered export. Both behaviours are real; `likelyFiltered()` distinguishes them. |
| **Reference format?** | A 37-digit numeric with a `YYYYMMDD` prefix (`2026071411121700010300171233832627163`), so the free date cross-check works. |

Of the 3 customer payments, **2 reached the ledger** (`source=both`, chain-verified) and **1 did not**
(`type_only` — the balance could not corroborate it), which is the asymmetry working as designed:
recording money in is the dangerous action, so anything uncorroborated waits for a person.

### And one bug only a real file could show (2026-08-08)

The same statement first read as **97 rows**, not 79. The extra 18 were the **page footer** —
two lines of small print at the bottom of all nine pages. It starts at the left margin, which
is the *date* column's anchor, so `*This is a system generated email…` landed in the date cell,
and `isAnchorRow()` asked only whether that cell was non-empty. Each one became a transaction
carrying a 1970 date and a zero amount, quarantined as `row_undeterminable` — noise a person
has to read past to find the rows genuinely in doubt, and a row count a fifth higher than the
statement's own.

Two things let it through, and both are fixed:

- **The furniture list was guessed.** `config('tng.statement.furniture')` held
  `'This email is a system generated email'`; the document says `'*This is a system generated
  email'`. A string that never matched anything looked exactly like one that had nothing to
  match.
- **The fixture agreed with the bug, not the document.** `TngPdfBuilder::footer()` printed the
  *config's* wording, at an x that was not the date anchor. Every test passed. ⚠️ **A fixture
  copied from the code it is testing cannot fail** — the footer text there is now verbatim off
  a real statement, drawn at the date anchor, because that is what makes the test meaningful.

The wording is no longer the only defence: `TngTableRow::isAnchorRow()` now requires the date
cell to be **date-shaped** (`9/7/2026` passes, a sentence does not) when there is no amount — so
a reworded footer cannot sprout transactions either. A row with a garbled date but a real
**amount** is still kept and reported (`ROW_UNREADABLE_DATE`); what is refused is a band with
neither, which has nothing in it to record. And a layout change that puts *every* fragment
outside the columns still ends at `NO_TABLE_BODY` — a named refusal, never a quiet zero.

Both defences are mutation-verified in
[TngStatementParserTest](/tests/Feature/Payment/TngStatementParserTest.php)
(`the_page_footer_never_becomes_a_transaction`, `a_sentence_in_the_date_column_is_not_a_row_whatever_it_says`).

**A statement can hold more rows than the database stores, and that is correct.** The same file
has two genuinely indistinguishable pairs — `Reload RM50 → bal 50.00` and `Transfer to Wallet
RM50 → bal 0.00`, each done twice on the same day — which share a `dedupe_key` and store as one
row apiece, flagged `row_ambiguous_key`. 79 rows → 77 lines. Read it with
`php artisan tng:inspect-mail --rows`, which prints every row and every identity two of them
share, and stores nothing.

### What a duplicate is protected against, and what it was not (2026-08-08)

An adversarial pass over the claims in this document found two of them false. Both are fixed;
both are recorded here because the shape of the mistake recurs.

**The same file read twice used to mint a second statement.** The LINES were always safe —
`firstOrCreate` on the `dedupe_key` means no transaction and no money can double, and that is
the guarantee that matters. But `store()` ran `TngStatement::create($header)` unconditionally,
with counters taken from the **parsed file**. Measured: the identical PDF uploaded twice leaves a
second statement claiming *3 transactions, 2 money in* with **zero lines beneath it**, because
every line already belonged to the first — a list that sums to twice the money that exists and a
Show page rendering an empty table under a header that says otherwise. `store()` now returns the
existing statement; callers check `wasRecentlyCreated`.

⚠️ **The check is on `status = STATUS_PARSED`, not on the hash alone.** A file refused because
the wallet number was wrong MUST be readable again once the operator fixes it, or the fix has
nowhere to take effect — the same lesson `TngStatementMailRepository::TERMINAL_OUTCOMES` learned
on the mailbox side, arrived at independently in a second place.

**The overdue watchdog was measuring the wrong date.** `statementOverdue()` took the newest
statement's `created_at` — when a file was *read*. An export covering March, uploaded in August,
was therefore perfectly current. On a manual-only install (no mailbox, so `markPolled()` never
runs and `last_statement_at` stays null) re-uploading one old PDF silenced the alert outright,
with no new money having arrived — precisely the quiet failure the command's own docblock says
it exists to catch. It now measures `max(period_end)`: the last day the wallet has actually told
us about. Three dates look interchangeable and only one is the question:

| Date | Answers |
|---|---|
| `last_polled_at` | We checked the mailbox. Says nothing about whether anything was in it. |
| `created_at` | We read a file today. An old export scores full marks. |
| **`period_end`** | **The last day the wallet has told us about. Money after it is invisible.** |

All three guards are mutation-verified
([TngStatementUploadTest](/tests/Feature/Payment/TngStatementUploadTest.php),
[TngWatchdogTest](/tests/Feature/Payment/TngWatchdogTest.php)).

### Eleven findings from an external review (2026-08-08) — all real, all fixed

An independent review returned twelve findings. Verified one by one against the code: **eleven
were present, one had just been fixed, none were wrong.** Three of them were losing or doubling
money, and every one of the three was invisible to a green test suite. The pattern behind all
three is the same and worth naming: **a test that supplies the value the UI is supposed to
produce proves the branch, never the path.**

| | Finding | What it did |
|---|---|---|
| **C1** | `candidatesForClaim()` filtered `whereNull('purchase_history_id')` while `refuseLine()` deliberately allowed an ingested-but-unattributed line | The whole **statement-first** ordering was unreachable from the UI. `tng:poll-mailbox --ingest` runs hourly, so a clean money-in line was hidden from the picker within the hour; the admin was told "no wallet transfer of RM299 has been read around this date", answered "not in a statement yet" because the form makes them answer, and the CREATE branch wrote a **second row** — `provider_payment_id` null, so the unique index saw no collision. RM299 → RM598, and a second grant one ordinary click later. |
| **C2** | `confirm_duplicate` existed in the form state, the controller and a test — but **no control anywhere set it** | The server refuses a look-alike and says to tick "record it anyway". A customer who genuinely paid twice inside the 14-day window had a claim **no sequence of clicks could resolve**. Now the look-alike is sent as `duplicate_of` and shown *before* submitting, so the decision is made with both payments on screen. |
| **C3** | `TngDirectionResolver` merged its verdict with `array_merge`, and the verdict always carries `quarantine_reason` — null on the happy path | Every quarantine the **mapper** set was erased. A row whose date could not be read fell back to `1970-01-01` and was **ingested automatically**, with that fallback date baked into its identity. `ROW_UNREADABLE_DATE` / `ROW_UNREADABLE_AMOUNT` / `ROW_REFERENCE_MISMATCH` had no reader anywhere in the repo — nothing ever survived to be read. A quarantine is a reason to STOP: first one wins, review only accumulates. |
| **C4** | "No registered name → every incoming row is reviewed" was asserted by a warning banner **and by the comment sitting next to the predicate**, and implemented by neither | `is_self_transfer` is a comparison against the Registered Name, so with no name it is always false — and the owner's own top-ups (type `Receive from Wallet`, which the allow-list marks a customer payment because *by type* it is one) were ingested as customer money. ⚠️ And `isIngestable()` did not read `needsReview` **at all**, so the flag meant nothing to the one decision it exists for. Both clauses added. |
| **C5** | `ConfirmModal` receives `confirmText`/`processing`/`variant`; the Claims index passed `confirm-label`/`:loading`/`tone` | The "Not a payment?" dialog — whose body says *nothing is deleted* — rendered the component's default red **"Delete"** button, and `:loading` being ignored left it double-clickable while submitting. |
| **C6** | `SystemHealthService` emitted `tng`; the Health page's `SERVICES` list did not include it | The page's `overall` reads **every** key the server sends, but only listed keys get a row — so an unconfigured mailbox turned the header amber with nothing on the page explaining why. A check the page cannot show must not speak for it. |
| **M1** | Notify throttle `1440` against a comment saying "once a day" | Seconds, not minutes: 24 minutes, on the one alert whose whole design argument is that repeating it destroys it. |
| **M2** | `wallet_number` validated `max:32`, column is `string(30)` | A 31-character number passed validation and was truncated — a payee number a digit short of the one the buyer is told to transfer to. |
| **M3** | Column key `submitted_at`, sort whitelist `created_at` | Every header click sent a key the whitelist dropped; the sort fell back to the default and nothing on screen moved. There is no `submitted_at` **column** — the whitelist takes the name the header sends and translates it. |
| **M4** | `:count` passed to a `FilterDrawer` declaring `draftCount` / `activeFilterCount` | "Clear all" permanently greyed out. |
| **M5 / M6** | Raw `<a href>` for internal nav; hardcoded back-href on Statements Show | §13 and §14 deviations — a full page reload, and being dropped on page one of the list every time. |

C1, C3, C4 and the `needsReview` clause are each mutation-verified: remove the guard, the named
test goes red.

## The 20-page statement: the reader was the filter (2026-08-08)

⚠️ **THE WORST BUG THIS MODULE HAS HAD, and the diagnosis was wrong before it was right.** A
real statement named `tng_ewallet_transactions_20260709_20260807.pdf` read as **79 rows ending
28 July** — ten days short of what its own header claimed. First diagnosis: a filtered export
(the app's *"last 90 days OR the current search result"* trap). That diagnosis even shipped a
warning. **It was wrong.** The owner insisted the PDF held ~20 pages, and a raw-byte probe —
PDF structure names are never encrypted, so `/Type /Page` can be counted without the password —
proved it: **20 pages in the file, 9 in our reading.**

Two reader bugs stacked:

| Bug | Mechanism |
|---|---|
| **The page-tree root was guessed.** | iText (TNG's generator) nests pages into groups of ten once a document passes ten pages, and writes the LEAF groups before the root. The reader took the *first* `/Pages` object in the file as the root — right for every flat tree, and flat was all the fixture builder ever produced — so it enumerated only the first group: **10 of 20 pages**. The root now comes from the **Catalog**, which is what the spec says. |
| **`rtrim` ate ciphertext.** | A stream is encrypted bytes, so its last byte can be anything — including `0x0A`. Trimming "line endings" off the tail ate real data whenever it was, a 2-in-256 flip per stream that across ~45 streams hits every third document. It hit page 19: one byte short, FlateDecode failed, the page **silently vanished** — 10 pages became 9. The stream's own `/Length` is now what gets read, exactly. |

**What was invisible:** eleven pages, everything after 28 July, and **five separate RM299
customer payments** (3× `DUITNOW_RECEIVEFROM`, 2× `Receive from Wallet`) — on every screen, a
quiet fortnight.

Three defences came out of it:

- **The page count is now an invariant.** Enumerated-and-decoded pages must equal the count the
  page tree itself declares, or the whole file is refused with both numbers in the message. A
  refusal with a number in it beats a silent half-read — this alone would have caught either bug.
- **The tail-gap warning stays** (`ParsedStatement::missingTailDays()`, threshold
  `tng.statement.max_quiet_tail_days`). It was built on the wrong diagnosis and was still the net
  that surfaced the right one — and a genuinely filtered export produces exactly the same symptom.
- **A parser version bump now re-reads stored statements in place.** `VERSION` went to
  `tng-geom-2`; a statement (or polled mail) stored under an older version no longer counts as
  "already read". The re-read updates the SAME row and `firstOrCreate` on `dedupe_key` leaves
  every existing line — id, state, payment link — untouched, only adding what the old reader never
  saw. Without this, the fix could never reach any install that had already polled the mail.

Measured after the fix, on the same file: **20 pages, 185 rows, newest 7 Aug, tail gap 0**; the
statement upgraded in place from 79 to 185 rows, the CHEW HENG TSENG line kept its link to
payment #341, and the five new RM299 transfers auto-recorded as unattributed payments (the price
book contained RM299). ⚠️ The fixture lesson repeated for the third time: footer wording, flat
page trees, and month-first date parsing were all fixtures agreeing with the code instead of the
document. `TngPdfBuilder` now builds nested trees (`nestedPageTree()`).

## Overlapping statements — verified end to end

The owner re-sends overlapping periods as a matter of course. Demonstrated with physically correct
balances: a 9 Jul–28 Jul statement (2 rows) then a 16 Jul–14 Aug one (2 rows, one shared) stores
**3 lines**, and the shared transaction stays attached to the statement that saw it first.

⚠️ **This rests on `balance_after_cents` being stable across exports**, since it is part of
`dedupe_key`. It is: a filtered export still prints the TRUE running balance, so a given
transaction's balance-after is the same figure in every file that contains it. A fixture that
computes balances backwards from a different closing figure will produce a false duplicate — that
is the fixture being physically impossible, not the rule failing.

## The recognition rule — only money we sell something for (2026-08-08)

> **A wallet transfer becomes a payment automatically only when its amount is the price of
> something this business sells BY TRANSFER. Everything else money-in waits for a person.**

⚠️ **Why it was needed.** The wallet is ONE wallet for every product and for the owner's own
life. Before this rule, every settled customer transfer into it became a Payment History row —
so a RM500 transfer, for a project sold through a card gateway, sat in the ledger beside the
RM299 that really was an AI Member sale. Both were real money; only one was this system's
business, and nothing on screen distinguished them.

The price book is built from **`MODE_OFFLINE` payment links that are ACTIVE** — nothing else.
A gateway-hosted item is paid by card on the gateway's own page; the same figure turning up in
the wallet is a coincidence, not a sale this lane can recognise.

⚠️ **IT IS A RECOGNITION RULE, NOT A FILTER — and the difference is the whole design.** Money
matching no price is **not dropped and not hidden**. It stops being recorded *automatically* and
appears on the Wallet Statements waiting list instead, where the admin is asked *"this amount is
not the price of any payment item — was it a customer paying for something?"* and can still
record it by hand. Three ordinary things land there:

- a buyer who mistypes (RM290 instead of RM299)
- a price that changed after somebody paid the old one
- a transfer for something sold through a gateway

Money that arrives and silently vanishes is the one outcome this whole feature exists to prevent,
so none of those may disappear.

### Re-judging what an older rule recorded — `tng:recheck`

⚠️ **Re-reading the statement will not do this, deliberately.** A line that already produced a
payment answers `already` for ever, so that a deletion is a decision re-polling cannot reverse.
That guard is right, and it means a rule change cannot reach the rows the old rule created.
[`tng:recheck`](/app/Console/Commands/Tng/RecheckTngPayments.php) is the one door that can.

| Property | Why |
|---|---|
| **Dry run unless `--apply`** | On a command about money, a default that writes is one somebody runs by accident. |
| **Only touches money nobody has looked at** | Skips any payment with a buyer, a product, a customer receipt reference, or an existing deletion. A person's answer outranks the rule. |
| **Soft-deletes, never destroys** | The row lands in Payments → Deleted with a Restore button, so a wrong call costs a click rather than the money. |
| **Releases the line** | `purchase_history_id` is cleared and the state reset, so the transfer returns to the waiting list instead of being pinned to a deleted payment for ever. |
| **Refuses on an empty price book** | With nothing sold by transfer, every recorded transfer matches nothing and one command would wipe the whole Touch 'n Go ledger. |

Measured on the real ledger: `1 payment(s) no longer match a price you collect by transfer —
MYR 500.00 LEOW CHUN HO 2026-07-14 (statement line 35)`, while the RM299 was left alone.

## Two more surfaces (2026-08-08)

- **`Sync Touch 'n Go` on Payment History.** Statements arrive when the OWNER remembers to export
  one and the poller runs hourly, so the gap between "he sent it" and "I can see the money" was an
  hour of not knowing whether anything had broken. Synchronous (the person is looking at the
  screen) and behind a `Cache::lock` (two admins, or an admin and the scheduler, would otherwise
  race on the same rows). It reports what happened **even when nothing did** — "checked, nothing
  new" and "the button did not work" are indistinguishable otherwise, and this button exists for
  people who suspect the second.
- **Transfer Receipts and Wallet Statements are hidden when Touch 'n Go is not collecting.** Both
  exist ONLY for this lane — one is the queue of customer transfer screenshots, the other reads
  the wallet's own PDF — so on an install that does not collect by transfer they are two
  permanently empty pages that still cost a click to find out. Gated on a shared `features.tng`
  flag (lazy, admin-only: `share()` runs on every Inertia render including the anonymous pay
  page). ⚠️ **Unreconciled is NOT one of them** — that is gateway webhook events.

### The six holes the module owned up to, and how each was closed (2026-08-08)

The handbook listed six known gaps rather than pretending they were not there. All six are now
closed. Three of them share one shape: **a fact the database knew and no screen could show.**

| Hole | Why it mattered | Closed by |
|---|---|---|
| **A deleted payment could never come back.** `TngLedger::ingestLine()` looks for its payment with `withTrashed()` on purpose — a deletion is a decision, and re-reading a statement must not reverse it. Correct, and it left a payment deleted by mistake unreachable from every screen. | The money is simply gone, and re-polling the wallet a hundred times will not bring it back. | A **Deleted view** (`?deleted=1` → `onlyTrashed()`) with a **Restore** action. `restore()` is bookkeeping only — deleting never revoked what the payment granted, so restoring must not grant it twice. Its own **button**, never a filter chip: the person who needs it just deleted RM500 by mistake and would not think to open a drawer. |
| **The mailbox had no screen at all.** Every message was recorded with labels and colours ready; nothing anywhere read the table. | On a lane whose only failure mode is silence, "we checked the mailbox and here is what was in it" existed only in the database. | `Partials/MailboxLog.vue` on Wallet Statements — the last 15 messages, each with its outcome and, load-bearingly, whether it **will be tried again**. |
| **An unreadable statement wrote a fresh red row every hour.** `storeFailure()` called `create()` on each attempt, and an unreadable statement is retried by design. | The classic wrong-wallet-number case produced seventy-odd "Could not read" rows for one broken file, burying the statements that did read. | One row per file, **refreshed** — so the reason stays current when fixing one cause reveals another. |
| **"Fix it and the next poll picks it up" had an unwritten 3-day deadline.** The window looks back three days past the last run. | Fix the wallet number on the fourth day and the email was outside the window forever; the money in it never existed as far as this system was concerned. | The window **stretches to cover the oldest message not finished with** (`TngStatementMailRepository::oldestUnfinished()`), bounded by `tng.mailbox.max_retry_days` so a statement that will never parse cannot make every poll walk back forever. |
| **Only the first PDF in a message was read**, and the message was then marked fully handled. | The owner exporting two months at once loses the second month, silently. | Every attachment is read. One record per message (its identity is `message_id`), so the note summarises all of them. ⚠️ **Any** unreadable attachment makes the whole message unreadable — pessimistic on purpose: a message wrongly retried costs one read, a message wrongly finished with costs a statement nobody sees again. |
| **Four customer-facing refusals were invisible.** The pay page is a standalone card with no FlashToast, so a closed link, a switched-off wallet, a failed upload and an expired token gave a bare browser error or a silently reset form. | **Everyone who hits these has already transferred the money.** | All four now redirect to the pay page with a `refused` prop, keyed by link — the same pattern `busy` already used. The 419 case is caught in `Handler::convertTokenMismatchExceptionToResponse()` for `pay/*/claim` specifically. ⚠️ It leaks nothing a plain GET does not: a stranger still cannot tell a switched-off wallet from an expired link. |

### Two claims that did NOT survive review, and are true as stated below

Neither is a bug — both are this document being imprecise about a thing that matters.

- **"Payment items have nothing to do with what gets polled."** True of polling, parsing and
  storing: `TngMailboxPoller` has no `PaymentLink` awareness and every parsed row is stored
  whatever it is. **False of matching.** A claim's amount and currency are frozen from its
  payment link (`PayClaimController`), and `TngLedger::candidatesForClaim()` filters wallet lines
  on `amount_cents` and `currency` by **exact equality** — so a link priced RM 299.00 will never
  be offered a wallet line of RM 300.00. Change the link's price and a different set of wallet
  rows becomes matchable. The link does not decide what is *seen*; it decides what can be
  *matched to a receipt*.
- **"Customer payments are separated from the owner's own activity by transaction type."** Only
  half the rule. The type allow-list handles reloads, cashbacks and gift packets. But the owner
  moving money in from their **own second wallet** carries the type `Receive from Wallet` — which
  `config/tng.php` marks `customer_payment => true`, because by type it is indistinguishable. The
  thing that actually separates them is `TngRowMapper::isSelfTransfer()`, a case-insensitive
  string comparison between the counterparty and the statement's **Registered Name**, and it gates
  `isMoneyIn()`, `TngLedger::isRecordable()` and `waitingQuery()` as its own separate clause. This
  is why a statement with no registered name warns that *every* incoming row must be reviewed — a
  missing NAME could not degrade a type-only classification at all.

---

# Phase F — noticing the silence

## The problem it exists for

⚠️ **Every way this lane breaks is quiet, and they all look identical.** Touch ’n Go has no
API and no webhook, so money is only ever discovered by reading a PDF the owner emails
himself. When the owner forgets to export, when the mailbox password expires, when a
statement will not parse, when a transfer is waiting on a question nobody answered — every
screen in this system shows the same thing: **no new payments**. Which is exactly what a
slow week looks like.

That is the only failure mode in the whole feature nobody would notice on their own.

## The waiting queue

**Payment → Wallet Statements**, above the upload box. Cross-statement, deliberately: a held
line used to live inside one statement's own page, which meant the person who had to answer
it needed to already know which statement to open. **Money waiting on somebody who cannot
see it is not waiting — it is lost.**

Two questions share the list because they are one job for the admin:

| | The question | Answers |
|---|---|---|
| **HELD** | "Is this the same money as a payment already recorded?" | *same money* → attaches the wallet line's identity to that payment, records nothing new, runs no fulfilment. *separate money* → a new unattributed row. |
| **NEW, not recordable** | "This arrived, but the balance could not corroborate it / the type is unrecognised." | *record it* → an unattributed row. *not a customer payment* → ignored. |

Both answers are re-checked on the server: the list renders from a snapshot and two admins
can be looking at it, so an option being offered is not the same as it still being true.

## ⚠️ One definition of "waiting", or the alert cries wolf

[`TngLedger::waitingQuery()`](/src/Payment/Services/TngLedger.php) is the **only** definition,
shared by the screen, the watchdog and the System Health check. Two copies would drift, and
then the alert and the page it sends people to would disagree about whether anything needs
doing.

**A RELOAD IS NOT WAITING.** Money coming in that the type table says is not a customer
payment — the owner topping up, a cashback, a gift packet — will never be recorded whatever
anybody decides, so putting it in a queue asks for a decision that does not exist. Measured
on the first real statement: the original predicate reported **5 transfers waiting, of which
4 were the owner's own top-ups**. An alert that is 80% noise is one nobody reads by the
second week — and then it cannot warn about anything. Pinned, and mutation-verified, by
`TngWatchdogTest`.

What genuinely waits:

- a line **HELD** because it might already be recorded;
- an **unrecognised type carrying money in** — which is what a renamed Touch ’n Go
  transaction looks like, on the side where money is at stake;
- a **known customer payment the running balance could not corroborate**.

## The watchdog

`tng:watch`, **daily at 09:30**. Not hourly, and the reason is not load: these are QUEUES,
not events. The same three transfers are still waiting an hour later, and a message that
repeats hourly is one nobody reads by Tuesday.

| Alert | Fires when | Why it is not noise |
|---|---|---|
| `payment.tng_needs_attention` | anything in `waitingQuery()` | Each one is money that arrived and is not in the ledger. |
| `payment.tng_statement_overdue` | the newest statement READ is older than `config('tng.watch.statement_overdue_days')` (10) | ⚠️ Measured from the newest statement **read**, never from the last poll — a poller running happily every hour against a mailbox nobody sends to is precisely the failure this must catch. *"We checked"* is not *"we heard"*. A wallet switched on minutes ago is never nagged. |
| `payment.tng_mailbox_broken` | the poller's failure flag is set | Statements are piling up unread and the system looks as though no money is coming in. |

**Unbacked payments are deliberately NOT alerted on.** A payment recorded from a customer's
receipt that the wallet has not corroborated is often perfectly real: an export can be a
filtered subset (`likelyFiltered()`), so absence from a statement is not evidence. Alerting
would accuse honest customers on a schedule. It is a count on screen, nothing more.

## System Health

`Setting → System Health` gains a **tng** check, which distinguishes the four states that
otherwise all present as silence:

| Status | Meaning |
|---|---|
| `off` | Not set up, or switched off. Silence is expected. |
| `warn` (no mailbox) | Collecting, but every statement must be uploaded by hand. **Healthy, not broken** — that is a legitimate way to run this. |
| `warn` (overdue / waiting) | Statements have stopped arriving, or transfers are waiting on somebody. |
| `down` | The mailbox cannot be read. Statements are piling up unread. |

## Related files (Phase F)

- [src/Payment/Services/TngLedger.php](/src/Payment/Services/TngLedger.php) — `waitingQuery()`, `attachLineTo()`, `recordLineAnyway()`
- [app/Console/Commands/Tng/WatchTngStatements.php](/app/Console/Commands/Tng/WatchTngStatements.php) — `tng:watch`
- [src/Common/Services/SystemHealthService.php](/src/Common/Services/SystemHealthService.php) — `touchNGo()`
- [resources/js/Pages/Manage/Payment/Statements/Partials/WaitingLines.vue](/resources/js/Pages/Manage/Payment/Statements/Partials/WaitingLines.vue)
- [config/notify.php](/config/notify.php) — the three `payment.tng_*` events, all throttled to once a day
- [config/tng.php](/config/tng.php) — `watch.statement_overdue_days`
- [tests/Feature/Payment/TngWatchdogTest.php](/tests/Feature/Payment/TngWatchdogTest.php) — 16 tests; the "is a reload waiting?" predicate is mutation-verified

## ⚠️ Open items after Phase F

1. ~~**Nobody has still seen a real money-in row.**~~ **ANSWERED 2026-08-07 — see the section above.** Real senders are full names, so the picker's name label works as designed. The remaining risk is the opposite one: a sender whose name matches a *different* customer's. Only a person can catch that, which is why the raw cells sit one click away.

   *(Original note, kept for the reasoning:)* `tng_statements`, `payment_claims` and `purchase_histories WHERE payment_provider='tng'` are all empty; all 332 live payments are Stripe. Everything about `counterparty_name` on a `Receive from Wallet` row is inference. **Have someone transfer RM1, export that single day, and run `php artisan tng:parse <file> --all`.** If the sender turns out to be a masked phone or a flat "TNG eWallet User", the picker must render *"these cannot be told apart"* rather than a ranking — build that as a prop, not a rewrite.
2. **`occurred_at_precision` on a real credit is unknown.** If it is `day`, `purchased_at` lands at midnight, is frozen by `READ_ONLY_FACT_PROVIDERS`, and past `ReportPurchaseToMetaAction::MAX_EVENT_AGE_DAYS = 7` the sale is un-reportable to Meta forever.
3. **The watchdog Phase F owes:** "no statement for N days", "held lines waiting", "unbacked payments older than N days", and a System Health check for the mailbox. `TngSetting::last_polled_at` and `last_statement_at` are already stamped for it.
4. **Notify event keys contain dots**, so `config('notify.events.payment.no_statement')` returns null — read via `NotifyEvent`.
5. **No retention sweep exists** for `payment_claims` or `tng_statement_lines`. Both repositories have the purge methods; nothing calls them. The retention window is a decision, not a bug.
6. **A held line has no screen yet.** `STATE_HELD` is set and reported by `tng:ingest-lines`, but the Wallet Statements page does not surface a "Same money?" tab. That is the first thing Phase F should build — a held line is money waiting on a person who cannot see it.