# Property Finder ↔ Bayut matching — "Bayut Actual Location" (`catalogue:match-pf-bayut`)

> 📍 Part of the project catalogue doc set — the map is
> [start-here.md](/docs/modules_handbook/shared/project-catalogue/start-here.md). Inputs:
> [propertyfinder-new-projects.md](/docs/modules_handbook/shared/project-catalogue/propertyfinder-new-projects.md)
> and [bayut.md](/docs/modules_handbook/shared/project-catalogue/bayut.md).

## What it does

New-project information comes from Property Finder; Bayut is the market-evidence source and the
building-accurate GPS. This command finds, for every Property Finder project, the Bayut project(s)
it is, and writes the Property Finder file back out with a `bayut_match` block:
`bayut_actual_location` (lat/lng), `bayut_ids`, `relation` (`same` / `building_group`),
`gps_distance_km`, `rule_confidence` (0–100) + `rule_reason`, the top 8 `candidates`, `status`
(`auto_rule` / `ai_queue` / `pf_only`) and an `ai` slot that `catalogue:judge-pf-bayut` fills. Reads two files, writes
three; touches no database.

```bash
php -d memory_limit=6G artisan catalogue:match-pf-bayut      # ~15 s; newest PF file + newest bayut_full_*.json
```
Output in `storage/app/catalogue/pf-bayut-matching/`: `pf_matched_<stamp>.jsonl.gz`,
`pf_match_review_<stamp>.csv` (every project, one row, best candidate), `_summary.json`.

## How it works (`PfBayutMatcher`)

1. **Candidates**: Bayut projects in the same emirate within twice the pin's reach (min 3 km) of
   Property Finder's pin, plus any with the same name or base name anywhere.
2. **Names are compared in several forms** (`PfBayutMatcher::variants`) and two names are the same
   when any form agrees: as written without the developer tail ("Portside by Samana" → "Portside"),
   "Phase 2" read as "2" and roman numerals as digits ("Granada II" = "Granada 2"); the community
   after "at" dropped or moved to the front ("Montura at Grand Polo" = "Montura", "East Residence 3
   At Serenia District" = "Serenia District East Residence 3"); either side of a spaced dash
   ("Damac Lagoons - Lagoon Views 12"). A form that is only a community from Property Finder's own
   location tree never counts ("Palm Jebel Ali - Frond A" is not the Palm Jebel Ali master project).
   A number after the developer stays the project's ("Tara Park by Modon 2" is "Tara Park 2"),
   except for a developer whose own name ends in one ("… By Object 1"). **Phase numbers** include
   digits glued to a word ("Flow25") and a trailing building letter ("Frond A" ≠ "Frond B",
   "Zenith D3" ≠ "Zenith D1"). **Pins outside the UAE count as no pin** (`inUae`): 16 Bayut projects
   carry their latitude twice ("25.047, 25.047" — 3,000 km out) and 7 Property Finder pins are off
   the map; such a Bayut project can never lend its location. Developers are compared without noise words
   (PJSC, LLC, properties, developments, realty, homes, group, …) with spaced initials joined
   ("S R G" = "SRG").
3. **Rule confidence** (0–100) = name 50 + developer 25 + GPS 15 + handover year ±1 5 + bedroom
   mix 5, plus evidence **signals**: Property Finder's tower name equals the Bayut project or its
   location leaf +10, starting price within 15 % of Bayut's +6, a shared community in the
   location chain +4, description word overlap up to +5 — and −15 when the phase numbers differ.
4. **GPS check scales with Property Finder's pin level** (`REACH_KM`): tower 0.5 km, sub-community
   2 km, community 4 km, city 25 km — a correct match sits 1.8 km off when Property Finder pinned
   the community (Talay ↔ Saadiyat Island).
5. **Auto-match** (`auto_rule`), in this order:
   - **Building group** — an UNNUMBERED Property Finder project whose name is the base (or the same
     word set) of one or more Bayut buildings (Sobha One ↔ Sobha One Tower A/B/C), GPS within reach,
     same developer → `building_group`, location = their centre. **Unless Bayut has the exact name
     itself** within reach (Greenspoint ↔ Greenspoint, not Greenspoint + Greenspoint 2).
   - **Exact name** + same developer + GPS within reach.
   - **Clear winner** — confidence ≥ 85, a 10-point lead, GPS within reach, same developer, same
     phase numbers, **the name itself ≥ 0.75** (`MIN_NAME`: bonus points alone carried "Senses at
     The Fields" onto "Elie Saab Vie at the Fields") **and its core ≥ 0.75** (`name_core`: the name
     without the words both share with Property Finder's own location — "Orania The Valley" looked
     83 % like "Talia the Valley" through "the Valley" alone; a name made only of its community,
     "Sobha Sanctuary", has no core and never wins on resemblance).
   - A building group of ONE member must hand over in the same year (±1): "The Haven" (2025) is not
     Bayut's lone "The Haven III" (2027).
   - **Decisive evidence** — GPS within reach + same numbers, and either Property Finder's TOWER
     names exactly one candidate, or only one candidate shares both the starting price and the
     community (and has the developer and a name ≥ 0.5).
6. **Conflicts** (`MatchPfToBayut::resolveConflicts`): one Bayut project auto-matched by two
   Property Finder projects — the stronger keeps it; the other moves to its next candidate if that
   one passes the auto rules, otherwise to the AI queue.
7. Everything else → `ai_queue` with the reason (numbers differ, GPS off, developer differs,
   runner-up too close, name not close); nothing nearby → `pf_only`.

## Accuracy (2026-10-09, hand-checked)

Random samples of 50 auto-matches, each error turned into a rule and pinned by a test:
**phase numbers must agree** (La Rosa 6 ≠ La Rosa 2; v3 found Zenith D3 auto-matched to Zenith D1), **a numbered project is never a building
group** (Olfah 12 ≠ Olfah 1 + 2), **an exact name beats a group** (Greenspoint) and **the name must
be close on its own** (Senses at The Fields). After the fixes, 49–50 / 50 per sample.

| Run | auto_rule | ai_queue | pf_only |
|---|---|---|---|
| v1 (name / developer / GPS) | 1,414 | 1,560 | 16 |
| v2 (+ signals, developer noise, reassignment) | 1,797 (1,610 same + 187 groups) | 1,178 | 15 |
| v3 (+ name forms, letters as numbers, nearest exact record) | 1,874 | 1,102 | 14 |
| **v3.1 (+ name core, number after developer, UAE-only pins)** | **1,867** | **1,109** | **14** |

v2 auto-matches by rule: exact name 1,288 · clear winner 208 · building group 187 · tower name 111
· reassigned 3. Pins: median 32 m from Bayut's, p90 ~390 m. AI queue by reason: runner-up close
287 · GPS off 250 · numbers differ 229 · name not close 173 · developer differs 139 · Bayut project
already taken 66 · group with GPS or developer off 34. Known soft spot: a co-developer name
("Sea View Residence Club House" by Sama Ezdan ↔ "Sea View Residence by Sama Mayas") auto-matches
on name + GPS; the AI audit sample is what measures this class.

**Gotcha:** read these files line by line with `fopen('compress.zlib://…')` + `fgets`, never PHP's
`gzfile()` — it splits long lines, and a Property Finder record (whole page under `raw`) runs to
tens of kilobytes. The first run read 6 of 2,990 records that way.

## The AI stage (`catalogue:judge-pf-bayut`, `PfBayutAiJudge`)

No person reviews matches (owner, 2026-10-09: *"i dun wan perso involve. i rather ask AI to do for
me"*). Every `ai_queue` project is put to **Gemini Pro** (`PfBayutAiJudge::MODEL`, prompt key
`pf_bayut_match`, body `resources/prompts/pf_bayut_match.md`) with an evidence packet: Property
Finder's title, developer, location + pin level, handover, starting price, unit types and a
description excerpt, beside each of the rule stage's **8 candidates** (`CANDIDATES_KEPT`) — name,
developer, community chain, completion year, bedrooms, starting price, distance, and the rule
flags (within reach, same price, same community, tower name, phase numbers). The model answers
`match` (one or more `bayut_ids` — several = a building group), `none` or `unsure`, with its own
confidence and a reason.

1. **Facts pass.** Accepted (`PfBayutAiJudge::decide`) only when the confidence is ≥ 80
   (`ACCEPT_CONFIDENCE`) **and a hard fact agrees** for a chosen candidate
   (`PfBayutMatcher::hardFacts`: GPS within the pin level's reach, starting price within 15 %, or
   a shared community) → `auto_ai`, with `hard_facts` recorded. A confident `none` → `pf_only`.
   An id the rules did not offer is discarded (the answer becomes `unsure`).
2. **Web pass** — anything not settled is asked again with Google Search grounding
   (`'grounding' => true`), same acceptance rule.
3. **Still unsettled → `pf_only`** with `ai.outcome = unsettled`: the page keeps Property Finder's
   approximate pin and shows "-" for Market Value / Rental.

Each record keeps the rule result and adds the AI's: `rule_status` (what the rules said),
`status` (final: `auto_rule` / `auto_ai` / `pf_only`), `rule_confidence` + `rule_reason`,
`ai_confidence` + `ai_reason` (the deciding pass), `ai` = `{judge, web?, outcome}` (`matched` /
`none` / `unsettled`). Output: `pf_judged_<stamp>.jsonl.gz`, `pf_judged_review_<stamp>.csv`,
`_summary.json` (status counts + AI outcomes).

**Resumable and parallel.** Each project's verdicts are cached in
`pf-bayut-matching/ai-verdicts/{pf_id}.json` the moment they arrive, so a rerun only asks what is
missing (a failed call is simply asked again) and never pays twice. `--shard=k/n` splits the queue
by `crc32(pf_id) % n`: run n processes side by side, then once **without** `--shard` to write the
merged file (a shard run does not write it). `--limit=N` for a trial; `--no-web` skips pass 2.

**Accuracy check — `catalogue:audit-pf-bayut`.** An independent model (Claude, prompt key
`pf_bayut_match_audit`) re-checks a random `--sample` (default 30) of each group: `rule_exact`,
`rule_other`, `ai`, `ai_web`, `ai_no_match`, and reports accuracy per group, the wrong ones listed in
`pf-bayut-matching/audits/audit_<stamp>.json`. A group under ~95 % is the next rule to fix.

**The 2026-10-09 run — judged in a Claude Code session, not on wk.** The owner chose to have
Claude judge the queue instead of Gemini on the API. Eight parallel sub-agents read the same
evidence (compact: facts + flags, 300-character description) with the same rules, searched the
web for the ones they were unsure of (~130 searches), and their answers went into
`ai-verdicts/{pf_id}.json` in this command's cache format (`model: claude-opus (Claude Code
session)`); `catalogue:judge-pf-bayut --no-web` then assembled the file with the same acceptance
rule. About 2.2 M sub-agent tokens for 1,085 projects.

| Final (2,990 Property Finder projects) | projects |
|---|---|
| `auto_rule` — matched by the rules | 1,867 |
| `auto_ai` — matched by the AI (confident + a hard fact) | 364 |
| `pf_only` — AI said none (561), unsettled or no usable Bayut location (184), nothing near (14) | 759 |

- **Conflict review.** After assembly, 67 Bayut projects were claimed by two Property Finder projects.
  About 60 were harmless: Property Finder lists one building twice ("Saba 4" / "Marriott
  Residences By Saba", "Tulip Oasis X" / "10"), or an umbrella listing beside its own part ("The
  Woods" / "The Woods Altura"). Both get the right location. The rest were rule bugs (now fixed
  above: Tara Park, Orania, Sobha Sanctuary, The Haven) and two weak AI calls, overruled in the cache
  (Lunaya Terraces, Aria by Beyond).
- **Spot check:** 30 random `auto_ai` matches all read right; three are the convention that an
  unnumbered listing beside a numbered one is phase 1 (Capri → Capri 1).
- **Pins that disagree.** Eight matches sit more than 10 km from Property Finder's pin. Six are
  Property Finder pin errors (The Willows pinned to "The Sanctuary" in MBR City; Arim pinned to the
  city), which is what the Bayut location corrects. In two the agents judged BAYUT's pin off (Expo
  Golf Villas 6, The Serene): their `gps_distance_km` says so, and an importer should not trust a
  Bayut location more than ~10 km from a TOWER-level Property Finder pin.

**Cost and time with Gemini on the API instead (estimate, not measured).** ~1,100 facts calls + a few hundred web calls on
Gemini 3.1 Pro (thinking on), roughly USD 40–80 all in plus grounding fees; 15–40 s a call, so
~1.5–3 h with 6 shards. The audit is ~15 Claude calls.

### Runbook (on a server with the AI keys — wk)

There is no AI key in the development container, so the AI stage was built against a stand-in
`AiClient` and runs on wk. Needs the global **Gemini** key (and **Anthropic** for the audit) on
Setting → AI Providers; in gateway client mode the Hub picks the model and grounding is relayed.

```bash
# 1. Inputs: the Property Finder crawl (storage/app/catalogue/propertyfinder-projects/
#    pf_new_projects_*.jsonl.gz) + the Bayut full artifact. On wk keep the Bayut file OUT of
#    inbox/bayut/ and pass --bayut= to all three — Bayut's new-project data is not imported there.
B=--bayut=storage/app/catalogue/pf-bayut-matching/input/bayut_full_20261009_052316_692835.json
php -d memory_limit=6G artisan catalogue:match-pf-bayut $B                 # ~15 s → pf_matched_*
php -d memory_limit=6G artisan catalogue:judge-pf-bayut $B --limit=20      # trial: read the CSV
for k in 1 2 3 4 5 6; do nohup php -d memory_limit=6G artisan catalogue:judge-pf-bayut $B --shard=$k/6 \
  > storage/logs/pf-judge-$k.log 2>&1 & done                                # wait for all six
php -d memory_limit=6G artisan catalogue:judge-pf-bayut $B                 # merged → pf_judged_*
php -d memory_limit=6G artisan catalogue:audit-pf-bayut $B                 # accuracy per group
```

## Related files

- `app/Services/Property/Matching/PfBayutMatcher.php`, `app/Console/Commands/MatchPfToBayut.php`
- AI stage: `app/Services/Property/Matching/PfBayutAiJudge.php`, `app/Console/Commands/JudgePfBayut.php`,
  `app/Console/Commands/AuditPfBayut.php`; prompts `resources/prompts/pf_bayut_match.md`,
  `resources/prompts/pf_bayut_match_audit.md` (keys on `Src\Ai\AiRequest`, `config/ai_prompts.php`)
- Tests: `tests/Unit/Matching/PfBayutMatcherTest.php`, `tests/Unit/Matching/PfBayutAiJudgeTest.php`
