Revamping the HDB Price Predictor

Building on the series that started with Feature Impact on HDB Predictions and From Model to HDB App , this post documents a significant revamp of the HDB Price Predictor - now running entirely in the browser with no server, zero cold starts, and offline capability.

Three things changed materially:

1. A second model covering private non-landed properties (including condominiums, apartments, and executive condominiums represented in the transaction data) sourced from URA caveats
2. Two prediction modes that make explicit which engine is running and what its error means
3. Honest measurement - chronological holdout instead of random split, and a clear separation between what the model measures and what it assumes

Background: Why the Revamp

The original predictor was trained on HDB resale data and deployed as a Streamlit app backed by KServe. That worked well for demonstrating the ML pipeline, but it had practical limitations - a server that cold-starts, a single property class, and no way to project beyond the training window.

The revamp sets a different constraint: everything runs in the browser. The model, scaler, and inference logic are all shipped as static files and evaluated in JavaScript. No API call, no waiting, no cost per inference.

That constraint forced every design decision to be explicit: how to represent the model compactly, how to scale features identically between Python training and JS inference, and - most importantly - how to handle time.

The Two Models

Model 1: Public HDB Resale (600 Trees)

Model 2: Private Non-Landed (77 Trees)

Both models use XGBoost with max depth 8 and early stopping. The private model stopped after 77 trees versus 600 for HDB. Likely contributors are its shorter training history and the strong project-level encoding, which concentrates much of the price signal into a small number of features.

Feature Engineering

HDB Features

The 60 features fall into four groups:

Temporal - sale_year, sale_month capture the market level at the time of sale. These are the features that make the model time-aware - but also the ones that hit a wall at the training boundary, which is why forward projection needs a separate mechanism.

Flat characteristics - floor_area_sqm, storey_avg, remaining_lease_years, plus one-hot encodings of flat_type (7 categories) and flat_model (20 categories). Remaining lease is computed as (lease_commence_year + 99) - sale_year, capped at 99 years.

Location - One-hot encodings of town (26 towns). This gives the model a discrete location signal without requiring geocoordinates, which are unavailable for all historical records.

MRT proximity - mrt_distance_km (continuous, straight-line to nearest exit) and mrt_walkable (binary flag at ≤ 400 m). These were added in a later iteration after measuring a S$2,745 MAE improvement even with ~7% of blocks having averaged distances due to shared block keys.

FEATURE_NAMES = [
    'floor_area_sqm', 'storey_avg', 'sale_year', 'sale_month',
    'remaining_lease_years',
    'flat_type_1 ROOM', ..., 'flat_type_MULTI-GENERATION',
    'flat_model_2-room', ..., 'flat_model_Type S2',
    'town_ANG MO KIO', ..., 'town_YISHUN',
    'mrt_distance_km', 'mrt_walkable'
]

All features are MinMax-scaled before tree traversal. The scaler parameters are exported as JSON so the browser can apply the identical transformation without calling Python.

Private Condo Features

The private model’s strongest feature is project-level price encoding - a mean PSF (price per square foot) per development, computed on the training set only and smoothed against the global mean for developments with fewer than five transactions. In a variance-decomposition check on the training data, grouping transactions by project accounted for approximately 87% of observed PSF variance, compared with approximately 47% when grouping by district.

Additional features include: area, floor_mid, remaining_lease, sale_year, sale_month, project_txn_count, street_psf (street-level encoding as a fallback), tenure flags (is_freehold, is_99_yr, is_999_yr), property type dummies, market segment dummies (CCR/RCR/OCR), and 28 district dummies.

Why Chronological Validation

This is the most important methodological decision, and it’s the one most often skipped in tutorials.

A random train/test split lets the model see later sales of the same block during training, then be tested on earlier ones. In a market that rose ~31% between 2017 and 2026, this leaks both price level and trajectory. The result is an accuracy figure that looks good but does not tell you how well the model will perform on a sale that hasn’t happened yet - which is exactly the task.

The shipped models use chronological holdout: all complete calendar years through end-2025 for training, and subsequent unseen future sales (2026 onwards) for evaluation. This ensures that historical comparable valuations (up to 2025) are fully learned by the tree structures, while true forward generalization is evaluated on future transactions. The difference is significant:

The 4.63% figure represents immediate forward generalization on unseen 2026 transactions (and 6.27% for private condos). When extending the validation window across multiple unseen years (2025+), error naturally widens to 7.22% for HDB (and 7.52% for private non-landed). Rather than citing only the more flattering single-year metric, the application deliberately incorporates the conservative multi-year bounds (±7.2% for HDB, ±7.5% for condo) when displaying indicative real-world confidence intervals in the UI.

The Two Prediction Modes

This is the design decision that prompted the most rethinking.

A common pattern in property-price demos is to train a model, let users pick any date, and display a number. The problem is that standard tree ensembles do not extrapolate a continuous trend beyond the range represented by their learned splits. Once a temporal feature such as sale_year moves beyond the relevant training thresholds, additional increases do not automatically produce further price growth—any query beyond the cutoff simply follows the same branches to the same terminal leaves. The unadjusted “future estimate” is actually just the training-range endpoint, presented without qualification.

Initially, one might consider three conceptual modes: historical back-testing, current comparable valuation, and forward projection. But looking closely at the tree mechanics and the codebase, that third split was artificial:

In the inference engine (getForecastAdjustment() in app.js), there are fundamentally only two operational regimes:

1. Within training cutoff (sale_year <= cutoff): pure XGBoost tree traversal
2. Beyond training cutoff (sale_year > cutoff): XGBoost base at cutoff clamped + index compounding

Whether a user queries 2022 (past) or 2025 (latest complete year in training data), the engine executes the exact same code path, feeds the actual sale_year and sale_month into the trees, sets growthFactor = 1.0, and carries the exact same measured holdout error. Splitting that into two separate modes in the UI created unnecessary complexity.

The revamp aligns the UI directly with the engine into two clear modes:

Mode 1 - Historical / Comparable Valuation (Within Cutoff)

When: user selects any year and month within the model’s training range (sale_year <= latest_complete_year, e.g. 2017–2025 for HDB, 2021–2025 for private condos).

What the engine does: passes sale_year and sale_month directly to the XGBoost model. growthFactor = 1.0. No macroeconomic index is applied.

What the error means: the published chronological holdout MAPE provides the empirical error benchmark for this mode: 4.63% for HDB and 6.27% for Private Non-Landed on the evaluated unseen transactions.

Use cases:

• “What did a comparable flat sell for back in 2022?” (historical back-test)
• “What is a comparable property worth within the established market data?” (comparable valuation)

Mode 2 - Index-Adjusted Forward Projection (Beyond Cutoff)

When: user selects a year beyond the model’s training cutoff (e.g. 2026 through 2032).

What the engine does:

1. Clamps sale_year to the model’s last complete training year (2025).
2. Sets modelMonth to a neutral mid-year month (June) to prevent within-year tree splitting thresholds from causing seasonal price cliffs.
3. Compounds the baseline price using an annualized historical growth rate derived from official published indices: growthFactor = (1 + annual_growth) ^ years_projected.

Rates are calculated from the relevant official index levels over the model’s selected historical window using compound annual growth, rather than treated as official forecasts. The HDB Resale Price Index (base period 1Q2009 = 100) yields ~6.73% annualized over the public training period, while the URA Property Price Index yields distinct annualized rates when segmented by region for private property:

• Core Central Region (CCR): 3.86%/yr
• Rest of Central Region (RCR): 5.92%/yr
• Outside Central Region (OCR): 7.84%/yr

Segment-specific rates are crucial: from 2021-Q1 to 2026-Q2, CCR grew ~20%, RCR ~33%, and OCR ~46%. A single blended rate overstates a prime CCR six-year forecast by about 24% (~S$876k on a S$3M condo).

What the error means: the base valuation inherits the model’s measured error benchmark (with the app’s conservative ±7.2% / ±7.5% multi-year margin), while the projection adds an explicit macroeconomic assumption. If market policy, BTO supply, or interest rates pivot, the future trend may diverge. The app explicitly displays the assumed rate and horizon (e.g., • index-adjusted forward valuation: +2.5 yr @ 6.7%/yr) rather than pretending it is an empirical measurement.

// app.js - getForecastAdjustment()
function getForecastAdjustment(scaler, saleYear, saleMonth, segment) {
  const cutoffYear = scaler && scaler.latest_complete_year
    ? scaler.latest_complete_year
    : (scaler && scaler.latest_year ? scaler.latest_year : saleYear);

  if (!(saleYear > cutoffYear)) {
    return { modelYear: saleYear, modelMonth: saleMonth,
             growthFactor: 1.0, yearsProjected: 0, growthRate: 0 };
  }

  const segmentGrowth = (segment && scaler && scaler.segment_growth)
    ? scaler.segment_growth[segment] : undefined;
  const growth = (typeof segmentGrowth === 'number')
    ? segmentGrowth
    : ((scaler && typeof scaler.annual_growth === 'number')
        ? scaler.annual_growth
        : FALLBACK_ANNUAL_GROWTH);

  const month = Math.min(12, Math.max(1, saleMonth || 6));
  const yearsProjected = Math.max(0, (saleYear - cutoffYear - 1) + month / 12);
  const modelMonth = NEUTRAL_FORECAST_MONTH;

  return {
    modelYear: cutoffYear,
    modelMonth: modelMonth,
    growthFactor: Math.pow(1 + growth, yearsProjected),
    yearsProjected: yearsProjected,
    growthRate: growth
  };
}

The UI dynamically updates the year label (Sale Year (Historical) in green vs Target Sale Year (Forecast)) and the guidance badge based on the active property class’s cutoff.

In-Browser XGBoost Inference

The model is exported as a compact JSON tree structure after training:

# pipeline_public/03_train_model.py
def export_web_model(booster, best_iteration, scaler, output_path):
    trees = []
    for tree_dump in booster.get_dump(dump_format='json'):
        tree = json.loads(tree_dump)
        packed = pack_tree(tree)   # compact {f, c, l, r, w} arrays
        trees.append(packed)
    web_model = {"b": base_score, "t": trees}
    with open(output_path, 'w') as f:
        json.dump(web_model, f, separators=(',', ':'))

In the browser, inference is a typed Float32Array tree walk:

function evaluateHdbModel(inputFeatures) {
  // 1. MinMax scale: x_scaled = x * scale + min
  const scaled = new Float32Array(featureCount);
  for (let i = 0; i < featureCount; i++) {
    scaled[i] = inputFeatures[i] * scalerData.scale[i] + scalerData.min[i];
  }
  // 2. Traverse 600 decision trees
  let totalScore = modelData.b;
  for (const tree of modelData.t) {
    let node = 0;
    while (tree.l[node] !== -1) {
      node = scaled[tree.f[node]] < tree.c[node] ? tree.l[node] : tree.r[node];
    }
    totalScore += tree.w[node];
  }
  return totalScore * adj.growthFactor;
}

Inference time: < 1 millisecond for 600 trees. The model runs reactively on every slider drag or dropdown change.

What the Literature Shows

The broader machine learning literature—including the XGBoost house price prediction study —demonstrates that gradient-boosted tree models are widely used and well suited for tabular property price prediction. The purpose of citing this work is not to claim that it guarantees a specific error rate for Singapore property, but to show that the underlying modelling approach is well established. The empirical results here (4.63% chronological holdout MAPE for HDB and 6.27% for private condos) align naturally with those published benchmarks.

The forward-projection disclaimer in the README is not a contradiction of that literature. The studies referenced in the literature evaluate against historical observations with known outcomes rather than genuinely unknown future transactions. In contrast, this project adds a UI that lets users query arbitrary future dates—taking the output beyond what any historical validation can measure. The distinction between an empirically measured model error and an unvalidated forward macroeconomic assumption is made explicit rather than hidden.

Results

Final Thoughts

The key design principles that carried through this revamp:

• Honest measurement over flattering numbers - chronological holdout, not random split
• Explicit assumptions - forward projection is labelled with the rate and horizon it assumes
• Two modes, cleanly delineated - within-training historical/comparable valuation vs forward projection beyond the cutoff carry different confidence and serve different questions
• Offline-first - the model, scaler, and all inference logic ship as static files; no server needed
• Segment-specific growth rates - a single blended rate for CCR/RCR/OCR private property can be misleading

The full source is on GitHub: hdb-price-predictor . The Python ETL and training pipeline is fully reproducible with a single uv run python pipeline_public/run_pipeline.py command.

Next up: integrating this valuation engine into the Autonomous HDB DeepAgents pipeline so natural-language queries return both geospatial results and XGBoost price estimates side by side.