# Property Research (Main · User Portal)

**Portal:** Main · **Routes:** `main.portal.research.*` at `/property/research` · **Nav:** sidebar → **Property Research** (visible while the feature flag is enabled) · **Gated by:** `auth`, `main`, `contact.verified`, and `PROPERTYLAB_RESEARCH_ENABLED`.

**Delivery status — 8 September 2026:** implemented in an isolated checkout of the existing Laravel portal and exercised against an isolated MySQL database. OpenAI is the selected conversation provider; the configured local credential was validated during integration. No public or production deployment has occurred. Browser/provider integration checks are being recorded by the integration owner; this handbook does not certify every design milestone or launch acceptance test as complete.

## What it does

A buyer can ask a property question, inspect the supporting map and charts, change the assumptions, and save a private comparison. The conventional scheme explorer and deterministic evidence endpoints remain usable when AI is unavailable.

| Surface | Implemented behavior | Important boundary |
|---|---|---|
| Conversation | Free-text planner, allowlisted research tools, contextual selected schemes, clarification, streamed results/answers, cancellation and retry | AI numerical statements come from deterministic allowed claims; unsupported requests do not become invented evidence |
| Map | Full 18,917-scheme index, type filters, five styles, 2D/3D, rail/amenities, selection and chart linking | Fresh view uses Satellite Streets, pitch 70°, zoom 16, bearing −20°, KLCC center (Suria KLCC: 101.712205, 3.157376); the unified Map settings panel starts collapsed |
| Price evidence | Source aggregate price/PSF, aligned monthly trends, missing-value gaps, sale-row drilldown, two-to-five-scheme comparisons | Source medians are historical observations, not live available units or formal valuations |
| Asking price | Buyer price/area conversion, declared cohort, explicit exclusions and compatible-sample statistics when available | All imported sale area bases are unknown; the current archive cannot justify an asking-PSF benchmark or price verdict |
| Daily-life evidence | Recorded amenity proximity, open-rail filtering, explicit criterion states, Mapbox destination candidates and walking/driving estimates | Proximity is straight-line; routing needs a confirmed destination. Scheduled/traffic-aware and public-transit journeys are unsupported |
| Saved work | Lead-owned conversations/results, editable brief, shortlist notes, checklist, named scenarios, result/scene restore and undo | Changing a saved scenario does not mutate an earlier result; broader scenario and user acceptance gates remain open |
| Export | Authorized JSON decision brief with shortlist identities and source scheme aggregates | Conversation text, destinations, private notes, offers and nested scenario state are excluded; export does not publish a site |

Three primary tabs appear in order: **Property Explorer** (default), **Heat map**, and **AI Advisor**. Each has its own browsing or conversation controls. Explorer has a compact search field and Filters button. Search automatically focuses exact property or recorded area matches. Repeated names such as Sunway produce source-backed location choices; a nearby match includes mapped properties within an approximate 3 km area and is explicitly labelled as approximate, rather than an administrative boundary. Other location choices remain selectable. The popup stages property-type checkboxes and a historical-price cap until Apply; Escape/close discards uncommitted edits. Price and PSF each sort low-to-high or high-to-low, with missing values last. Sorting, map-area filtering, compact property cards and an explicit five-property comparison tray remain in the list. Undo, Save and Shortlist toolbar controls appear only in Advisor. Map settings combines the colored property-type checkboxes with appearance, rail and amenities; its type filters stay synchronized with Explorer. Viewing a marker is independent of comparison membership, has no five-item limit and does not move the camera or cancel an AI answer. Clicking a card’s body or title focuses the map, marks the card blue, and opens price/PSF evidence in a sliding right panel. Marker selection also opens that panel without moving the camera. The list stays available; mobile uses a closable full-width details panel. Requests are serialized with latest-selection-wins behavior, including revision updates from superseded responses and coordination with pending saves. The information popup is anchored below the selected or hovered marker using Mapbox Popup. Opening evidence never restricts the map to the answer property IDs. Desktop panes can be resized; Explorer starts with a 360px property list (resizable from 300–480px), giving the remaining width to its map. Advisor independently retains its 42% reading pane. Answers include their price/PSF cards, charts, tables and sources inline in the conversation. Direct map comparisons appear there too, replacing the welcome screen. Saved answers can load their own charts on demand; metric changes use the properties in that answer. Entering AI Advisor from Explorer or Heat map always starts a fresh chat. New chat also clears the active conversation; saved chats reopen only through the searchable History drawer. History shows the first question and updated date, scopes records to the current lead, and excludes evidence-only Explorer workspaces. Explorer search, filters, comparison and map state are restored on return. Ask AI from the comparison tray explicitly carries those selections into a fresh conversation. Compact Properties / Map and Chat / Map subviews preserve state within their current mode; this differs from entering Advisor through the primary tab. Tables and deliberate select/show-on-map actions accompany canvas/WebGL interaction; full WCAG/mobile conformance remains a release check.

The Heat map tab uses fixed geographic cells for recorded scheme PSF, recorded scheme median price, or mapped scheme concentration. Price cells show the unweighted median of all available positive finite scheme values, including a single property. One or two valid properties are labelled Limited coverage; their value is not an area benchmark. Concentration counts unique mapped schemes, not units or transaction volume. State, property types, approximate 500 m / 1 km / 2 km cells and opacity are configurable. A fixed light green → green → yellow → orange → red → dark red legend retains the same thresholds while panning or filtering. Grey occupied cells have no valid price data; empty areas stay unfilled. Selecting a cell reveals its exact values and constituent properties, which open in Explorer. The cell list provides a keyboard alternative to map clicks. See [the heat map contract](heatmap-contract.md) for source grain, exclusions, geometry and interpretation. Heat-map camera and filters are local to that tab. It starts at KLCC in 3D Satellite Streets at area zoom; View from above provides an optional 2D overview.

The Research page integrates portal-menu access into its own masthead and omits the duplicate shell header on all viewport sizes. Other pages retain their existing shell header behavior. Ordinary markers use 5px radius at zoom 15 (formerly 8px); selected and hovered rings are reduced, while the existing 8px hit-query tolerance is retained.

## Data and analytical meaning

The source was collected on **7 September 2026**. The imported monthly and displayed-sale periods run **January 2021 through March 2026**. Default analytical windows cover the final 24 months supported by the snapshot, unless the query explicitly supplies another period. These dates are historical coverage, not a live feed or refresh promise.

| Record set | Verified count | Meaning |
|---|---:|---|
| Original neutral archive | 643,210 records across 18 datasets | Retained read-only as provenance; only the research subset is normalized |
| Canonical map schemes | 18,917 | Existing first-16-hex SHA-256 name IDs preserved and collision-checked |
| Scheme snapshot values | 18,917 | Source median, reported PSF, aggregate `n`, and history-link status |
| Monthly observations | 186,259 | Scheme-month aggregates; price and PSF remain independent |
| Displayed source sale observations | 109,648 | Source record/page/row identities, not verified unique transactions |
| Sale observations linked to a map scheme by exact name | 78,606 | Additional date/value/area eligibility filters still apply |
| Sale observations outside the map universe | 31,042 | Retained with `scheme_not_in_map`; excluded from scheme comparisons |
| Amenities | 52,544 | Recorded categories/names/coordinates |
| Rail records | 398 | 361 open, 33 planned, 4 under construction; proximity results include open stations |
| Scheme history links withheld | 1,249 | Unverified/wrong-name history responses do not become sale evidence for that scheme |
| Canonical source crosswalks | 18,917 | Source scheme record mapping and identity audit |

The initial verified snapshot ID is `15799fa91b8e90287d0807cb5053733dba79464bafe467bfef66f5d7d42b265c`. The archive SHA-256, dataset/record IDs, source URL, page content hash and row position remain inspectable in source references. A source observation's identity is not a claim of a unique sale.

Every imported sale has `area_basis=unknown`. A buyer supplying `built_up` does not establish that the source sample used built-up area. The asking-PSF calculation can be shown, while the unsupported benchmark remains null with an explicit reason. Unknown/missing values never become zero. Source `n` has an unconfirmed aggregation window and is not added to displayed-row counts.

Cohort eligibility is applied to the declared category/radius/period universe before display limits. Same-scheme and nearby-scheme evidence are separate. Quartiles use linear interpolation at `(n-1)p`; fewer than five compatible observations have no distribution band. Legitimate outliers are retained. The module does not reinstate modeled flood depth, infer demographics or school quality, forecast appreciation, or advertise current unit inventory.

## How it works

Explorer cards always follow the visible map. `mapVisibility.js` resolves rendered property dots and the leaves of rendered clusters, deduplicating identities. Updates run after movement, on throttled rendered frames and at idle to include newly loaded source data without waiting for all background imagery; generation checks discard stale cluster callbacks. Type/price filters are shared by markers and cards. A manual map gesture clears an earlier text/area search, while search-driven camera moves keep it until the user browses. The old optional map-area checkbox is replaced by an automatic-update label. Bounds are only a fallback before the map reports its rendered identities.

Property history defaults to the first through last available monthly records for the selected schemes, rather than the snapshot-wide latest 24 months. Explicit date requests keep their requested window. Price and PSF share the available record span; missing monthly values remain gaps. Each trend labels its displayed period and number of observed values. Monthly observations render as unconnected dots with MMM-YYYY axis, tooltip and table dates. The period comparison uses the median of the earliest three and latest three retained recorded months, skipping gaps and displaying actual date ranges. At least six retained months are required to avoid overlapping groups. It is a full-period comparison, never annualised. Both adjusted and unfiltered comparison results are shown when exclusions occur, with their respective dates.

`historyAnalysis.js` screens each property's metric independently using modified Z-scores of residuals from a Theil–Sen trend (median pairwise slope, median joint intercept). Absolute scores above 3.5 flag candidate outliers. This exploratory product rule needs seven observations and nonzero MAD; automatic exclusion is withheld if more than 25% would be flagged. Missing/nonpositive values are not observations. Excluded candidates are removed from the default dots, fitted line and comparison, but raw rows remain in the source table with an exclusion label. A Show excluded control reveals hollow dots without changing the adjusted calculation. No database records are modified. The method does not establish that flagged source values are erroneous.

A dashed overall trend uses a least-squares line fitted to retained positive monthly values against elapsed calendar months, with equal weight per recorded month. It spans the first through last observation without extrapolation, requires at least three observations at distinct dates, and excludes missing months from the fit. Source dots remain unconnected. The line is explicitly an estimated period summary, not a forecast or moving average. Fewer than eight observations receive a limited-observations note. The table and tooltips label trend estimates separately. Property comparison legends toggle both that property's dots and trend. Individual displayed sales remain separate from monthly aggregate trends.

- [ResearchRepository](/src/PropertyLab/Research/ResearchRepository.php) supplies structured tool schemas, deterministic outputs and the complete map index. The map cache is keyed by immutable snapshot ID; it omits bulky per-marker provenance already available from the evidence endpoints.
- [Research services](/src/PropertyLab/Research/README.md) compute cohorts, prices, null-aligned trends, proximity and supported route estimates. Results include stable entity references, chart series, typed metrics, allowed claims, coverage, exclusions and limitations.
- [Advisor orchestration](/src/PropertyLab/Advisor/Services/AdvisorOrchestrator.php) resolves tool plans and grounds the answer. It scopes a 45-second turn deadline around tool execution; each provider call checks the remaining time rather than consuming an independent full timeout.
- [WorkspaceRepository](/src/PropertyLab/Advisor/Repositories/WorkspaceRepository.php) owns transactional user-state writes, revisions, cancellation and immutable saved results. Ownership follows the existing portal **lead**. Per-result and per-conversation lookups are scoped to that owner.
- [Controller](/app/Http/Controllers/Main/Portal/PropertyLabResearchController.php) renders Inertia and exposes authenticated JSON/SSE endpoints through [the route group](/routes/propertylab-research.php). Mutation requests use the portal's session/CSRF flow and route throttles.
- [Advisor.vue](/resources/js/Pages/Main/Portal/Research/Advisor.vue), [PropertyMap.vue](/resources/js/Components/PropertyLab/Maps/PropertyMap.vue) and [EvidenceChart.vue](/resources/js/Components/PropertyLab/Research/EvidenceChart.vue) share canonical scheme references. Hover is transient and does not call the model, move the camera or change a saved shortlist.

The two additive migrations are [research tables](/database/migrations/2026_09_08_000001_create_propertylab_research_tables.php) and [private workflow tables](/database/migrations/2026_09_08_160100_create_propertylab_advisor_workflow.php). No source archive tables or existing portal credentials are renamed.

## Configuration and credentials

See [config/propertylab.php](/config/propertylab.php), the [shared AI module](/docs/modules_handbook/shared/ai/readMe.md), and the [AI gateway handbook](/docs/modules_handbook/ai-gateway/readMe.md).

| Setting | Default / use |
|---|---|
| `PROPERTYLAB_RESEARCH_ENABLED` | `false`; route gate and sidebar visibility. Enable only for the intended deployed audience |
| `PROPERTYLAB_RESEARCH_MODEL` | Optional model override. Otherwise use a compatible OpenAI prompt pin, then the OpenAI catalog default |
| `PROPERTYLAB_ROUTING_ENABLED` | `false`; enables configured Mapbox routing and, unless separately configured, address geocoding |
| `MAPBOX_TOKEN` | Existing Mapbox configuration; browser-safe token is passed through the normal portal services config. Never use an AI provider secret here |
| `PROPERTYLAB_GATEWAY_OPENAI_VERIFIED` | `false`; gateway clients remain unavailable until their route to OpenAI has been verified |
| `INERTIA_USE_SCRIPT_ELEMENT_FOR_INITIAL_PAGE` | Set to `true` in the isolated preview; the existing Vue adapter 3.3.0 needs script-element initial-page data from Laravel adapter 2.0.24 in this setup |
| `API_PREFIX` | Set to `api` when bootstrapping the empty preview environment so the existing Dingo provider has its required prefix |
| `propertylab.geocoding.enabled` / `.token` | Optional application-config overrides; otherwise inherit routing enablement/token and `services.mapbox.token` fallback |

**Existing-stack preview integration:** leaving `INERTIA_USE_SCRIPT_ELEMENT_FOR_INITIAL_PAGE` at `false` produced null initial-page data and a blank screen in the isolated preview. Setting the existing option to `true` resolved that bootstrap mismatch. `API_PREFIX=api` was also required by the existing Dingo provider in the empty environment. These are environment settings, not dependency upgrades or a claim that every deployed Inertia/SSR path has been certified.

The provider is explicitly **OpenAI**. Credentials use the portal's existing `ai_credentials` model and `AiKeyService` / `AiCreditService`, including global/lead ownership, gateway policy and credits. `api_key` is encrypted at rest and hidden from serialization. Configure it through the existing authorized AI settings flow. Keep the deployment's Laravel encryption key stable so stored credentials remain readable. Do not put provider key text in this handbook, design files, frontend props, source control or diagnostic output.

Mapbox geocoding returns alternatives with precision/address context and always asks for confirmation. The AI orchestrator verifies that route coordinates match a destination already confirmed in the saved brief. Geocoding requests `permanent=true` because research results persist; a provider refusal does not fall back to caching temporary responses. See [Mapbox storage requirements](https://docs.mapbox.com/api/search/geocoding/#storing-geocoding-results). Walking/driving routes travel from scheme coordinates to the confirmed destination, with estimated duration and timestamp; these are not verified entrance-to-door journeys. Explicit departure/arrival schedules return unsupported rather than an untimed result disguised as a commute forecast.

## Import and deployment

1. Prepare an isolated staging environment with the portal's existing dependencies, authorization conventions and separate database/cache configuration. Audit pending migrations before running the portal's normal migration process. The local implementation used an empty scratch MySQL instance and did not migrate a live database.
2. Apply the additive migrations through the normal reviewed portal release process. Keep the feature disabled until schema and import verification succeed.
3. Convert the neutral archive read-only into a fresh output directory, then import the verified snapshot:

```sh
python3 scripts/propertylab/prepare_research_import.py \
  --sqlite /approved/archive/propertylab.sqlite \
  --output storage/app/propertylab-research-2026-09-07

php artisan propertylab:import-research \
  --source=storage/app/propertylab-research-2026-09-07 --activate
```

[The converter](/scripts/propertylab/prepare_research_import.py) uses Python's standard-library SQLite reader; PHP SQLite support is unnecessary. [The Artisan command](/app/Console/Commands/PropertyLab/ImportResearch.php) delegates writes to [ResearchImportRepository](/src/PropertyLab/Research/ResearchImportRepository.php). It checks file hashes and counts, imports transactional chunks into an invisible staging snapshot, and switches the active pointer only after verification. A repeated matching import is idempotent; changed content cannot overwrite an existing snapshot identity. Omit `--activate` to prepare data while retaining the current active snapshot. The converter requires a fresh directory if an output manifest already exists.

4. Build the portal assets with its normal build pipeline. Configure OpenAI through encrypted AI settings and configure the browser-safe map token through the existing services flow. Verify same-session cancellation, SSE buffering and provider timeout behavior through the actual staging proxy before enabling the feature.
5. Enable `PROPERTYLAB_RESEARCH_ENABLED` for the intended audience, refresh configuration using the deployment's standard process, and verify authenticated research, source evidence, save/restore and conventional exploration. Monitor provider failures and latency without logging raw private prompts or destination details.

The current source archives are a manually supplied snapshot. No scheduled source refresh, new data licence, production queue topology, or provider quota has been established by this implementation. The ordinary portal build/test commands and existing modules remain authoritative; this feature does not silently upgrade Inertia adapters or replace shared infrastructure.

## Rollback without losing saved evidence

Disable `PROPERTYLAB_RESEARCH_ENABLED` and reload the deployment's configuration first. The route gate hides the feature and its navigation while keeping additive research/workflow records intact. Deploy the preceding compatible application/assets release if necessary. Keep the previous verified dataset and release artifact available.

To return to a prior dataset, rerun `propertylab:import-research` against that retained, unchanged prepared directory with `--activate`. Activation is transactional; the map cache uses a different snapshot key. Saved result payloads retain their original snapshot references and are not rewritten to look current. Do not use schema reversal as the feature-disable procedure once users have saved research. Any later schema retirement is a separate reviewed maintenance change with verified backups and private-data retention handling.

## Validation and open release work

See the [local validation record](validation.md) for exact test counts, browser observations, known baseline warnings and the pending OpenAI data-transfer approval.

Locally verified research fixtures cover unit conversion, quartiles, duplicate-looking source identities, exact-name guards, full-universe filtering before pagination, monthly null gaps, unknown criteria, open-rail selection, immutable import/reimport, rejected staging activation, route direction, geocoding precision/confirmation and provider deadline behavior. Provider adapter tests fake HTTP and prohibit unexpected network calls. The complete real import reconciled to the counts above. A first map-index retrieval measured approximately 1.23 seconds and 4.73 MB of JSON on the scratch environment; this is one local measurement, not a production percentile guarantee.

The temporary local PHPUnit configuration targets the isolated test database. The repository's default PHPUnit configuration points at its normal test infrastructure; do not assume it selects the scratch instance. Run the relevant scoped tests only after checking the actual target. Example from this implementation:

```sh
php vendor/bin/phpunit -c phpunit.propertylab.local.xml \
  tests/Unit/PropertyLab/ResearchCalculationsTest.php \
  tests/Unit/PropertyLab/ResearchRoutingTest.php \
  tests/Unit/PropertyLab/ResearchGeocodingTest.php \
  tests/Unit/PropertyLab/ResearchRepositoryIntegrationTest.php

PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover \
  -s tests/Unit/PropertyLab -p 'test_*.py' -v
```

The local XML is an environment-specific, uncommitted file. Create an equivalently isolated configuration in CI/staging instead of relying on that filename to exist. The integration fixtures use transactions and preserve the imported snapshot.

**Outstanding release gates:** the balanced 60-question English/Bahasa Malaysia/Mandarin evaluation; at least five representative-buyer usability sessions; full accessibility, keyboard/touch and Safari/Chrome/mobile coverage; actual proxy/concurrency/cancellation and throughput measurements; provider quota/billing/storage permissions; deployment-specific authorization/tenant review; source licensing/refresh/retention decisions; production backup/restore and rollback rehearsal; and any capability still described only as proposed in the design documents. Inertia Laravel `2.0.24` and Vue adapter `3.3.0` coexist in the existing lockfiles: no dependency upgrade was made here, and release navigation/SSR compatibility remains a specific verification item.

## Design references and delivery tracking

- [Development specification v1.1 with delivery annotation](design/propertylab-chatbot-development-spec.md)
- [Agreed scope summary with delivery annotation](design/propertylab-chatbot-summary.md)
- [Original implementation backlog](design/propertylab-chatbot-backlog.csv)
- [Design-copy provenance and milestone status](design/readMe.md)

These documents preserve the agreed scope and acceptance criteria. Their original milestones are a plan, not a completion ledger. Local code and fixture coverage do not by themselves close M0–M4 or authorize production/public release.

Explorer UI: `resources/js/Components/PropertyLab/Advisor/ExplorerPanel.vue`; regression checks: `ExplorerPanel.test.js` and the independent-inspection case in `useResearchScene.test.js`.