# Hong Kong Catalogue Mapping

Column map from the petaV2 production `ih_hk_*` reference tables (inspected
read-only, 2026-07-13; representative redacted payloads committed under
`tests/Fixtures/hk/`) into the canonical catalogue. The adapters live in
`app/Services/Property/Ingestion/Adapters/` and run through the standard
`catalogue:sync` pipeline; the petaV2 prod scrapers (HkScrapeAll /
HkImportCentanet / House730Scrape* / MidlandScrape*) remain the external
producers (gate G3) and PetaV3 reads over the read-only `reference`
connection (gate G2).

## Providers and identity

| Registry key | data_providers.code | Source tables | External identity |
|---|---|---|---|
| `house730` | `house730` | `ih_hk_newproj` (248) | `estateId` |
| `centanet` | `centanet` | `ih_hk_new_projects` (511) + `ih_hk_projects` (514) | `projectId` / `sourceId` (one centanet new-property id space; matching rows coalesce before ingestion) |
| `centanet-estates` | `centanet` | `ih_hk_estates` (17,855 — SUBSALE candidates) | `estateCode` |

The two Centanet registry entries retain distinct `source_scope` values even
though they share one provider identity. Each may therefore run as a complete
snapshot without deactivating records belonging to the other feed.

`ih_hk_listings` / `ih_hk_transactions` / `ih_hk_investment` / indices /
schools / hma_profiles are market evidence, never canonical projects.

**Explicit cross-provider identity (T14):** `ih_hk_projects.matchedEstateId`
and `ih_hk_name_map.code` equal `ih_hk_estates.estateCode` — deterministic
curated keys that feed the mapping REPORT (`catalogue:hk-map-sources`);
auto-attach stays behind gate G5, otherwise admins confirm through the
existing attachSource UI. `ih_hk_name_map` also carries Tc/Sc/En names and
districts for translation enrichment.

## Provider sentinels (normalized to null by `HkReferenceAdapter`)

- epoch `-62135596800` (.NET DateTime.MinValue) in `intakeInfo`,
  `saleProcess[].itemValue`, raw timestamps;
- `"0000-00-00"` dates in `completedDate`;
- empty strings for developer/management fields.

## house730: ih_hk_newproj -> canonical (segment [new_project])

| Source | Canonical |
|---|---|
| name / nameEn / nameSc (fallback raw.estateNameZH) | project_name + name_translations {yue-Hant-HK, en, zh-Hans} |
| district / region / address | area / state / address |
| lat, lng (strings) | latitude, longitude (floats) |
| priceFrom / priceTo | price_min / price_max |
| unitTotal (fallback unitCount) | non_landed_units |
| saleStatus | sale_status |
| completedDate | completion_date (sentinel-safe) |
| parkingInfo / managementFee | parking_info / management_fee |
| propertyRightYears > 0 | tenure ("N years") |
| saleProcess (json) | sale_process (itemValue sentinel-nulled) |
| facilities (id=>label object) | facilities (label list) |
| nearby + primarySchoolNet | amenities {nearby, primary_school_net} |
| updated_at | data_scraped_at |
| coverImage | media hero |
| gallery[].url | media gallery |
| attachments[].url (brochures, price lists, plans) | media document |
| floorPlans[] (urls) | media floor_plan (project level) |
| unitsData blocks -> units (unit, sfa, price, planImg) | catalog_floor_plans (name, sqft, price) + plan-scoped media floor_plan |

Media `source_key` = sha1(url) — house730/centanet CDN asset URLs are
immutable content paths, so the hash is a stable per-source media identity
(`ih_hk_asset_map` maps house730 file hashes to URLs for provenance).

## centanet: ih_hk_new_projects + ih_hk_projects -> canonical (segment [new_project])

| Source | Canonical |
|---|---|
| name / nameEn | project_name + name_translations |
| district / address / lat / lng / developer | area / address / latitude / longitude / developer |
| priceRange json {minPrice,maxPrice} (ref table) | price_min / price_max |
| priceFrom (detail table) | price_min |
| estimatedKeyDate (ref table) | completion_date |
| saleStatus | sale_status |
| unitCount | non_landed_units |
| medianSalePsf (rounded) | psf_median |
| grossYield | rental_yield |
| primarySchoolNet / secondarySchoolScope | amenities |
| thumbnail / imageUrl | media hero |
| brochureLink / priceListLink / transactionLink | media document |
| projectDescription, bedrooms, propertyStatus, firstPriceListDate, matchedEstateId | source fields only (provenance / AI input / T14 mapping) |

## centanet-estates: ih_hk_estates -> canonical (segment [subsale])

| Source | Canonical |
|---|---|
| estateName + phaseName | project_name (one canonical per estate/phase row; umbrella grouping is explicit admin action via parent_catalog_project_id) |
| district / address / lat / lng / developer | area / address / latitude / longitude / developer |
| unitCount | non_landed_units |
| minOpDate year | completion_year |
| statSalePsf (fallback saleUnitPrice) | psf_median |
| schoolNet / mtrInfo / walkMtrTime / hma | amenities {school_net, mtr, walk_mtr_minutes, hma} |
| thumbnail | media hero |
| statRentPsf, stat*Chg*, buildingCount, estateType, managementCompany | source fields only |

## Freshness

Adapter watermark = max(updated_at) of its source tables; runs whose
watermark exceeds `project_catalogue.staleness_threshold_hours` are flagged
`source_stale` — a stopped prod scraper can never look like a fresh sync.

## Upstream producer scheduling (G3 verification, read-only, 2026-07-13)

Inspected the petaV2 prod crontab and `app/Console/Kernel.php`: the only
scheduled HK entry is `hk:warm-assets` (image-proxy cache warmer, daily
04:30). **The Hk scraper/import commands (HkScrapeAll, HkScrapeTransactions,
HkImportCentanet, House730Scrape*, MidlandScrape*) are NOT scheduled — they
run manually today.** Gate G3 therefore needs the owner to assign producer
scheduling (who runs them, at what cadence) before the PetaV3 HK sync can be
flipped on; until then the watermark staleness flag is the guard that keeps
manual-run gaps visible.