Revamping Autonomous HDB Agents

In last week’s post, Revamping the HDB Price Predictor , I overhauled my machine learning models into client-side, browser-evaluated engines with chronological holdouts and multi-year projection modes. I concluded that post with a roadmap promise:

“Next up: integrating this valuation engine into the Autonomous HDB DeepAgents pipeline so natural-language queries return both geospatial results and XGBoost price estimates side by side.”

This week, that integration is complete. With these changes, a user can submit an open-ended natural-language query to retrieve relevant historical transactions, enrich them with straight-line distances to MRT station exits, and evaluate model-estimated price benchmarks alongside the original transaction data. The local data and valuation pipeline requires no external database setup, while natural-language interpretation uses the configured LLM provider.

Turning a standalone model into an agentic intelligence tool meant revisiting the foundation I laid back in December 2025 in Autonomous DeepAgents for HDB Insights .

In this revamp, four major architectural upgrades transform the project from a heavy database-bound prototype into a leaner intelligence pipeline:

1. Removing the External Database Runtime: Replacing Dockerized PostgreSQL, PostGIS, and MCP Toolbox with an in-memory SQLite table and native Python geodetics.
2. Integrating XGBoost-Based Price Estimates: Inserting an ML valuation node into the LangGraph state machine to compute model-estimated prices, model-error-based indicative ranges, and valuation gaps for retrieved transactions.
3. Preventing Misleading Historical Price Comparisons: Enforcing temporal alignment checks to distinguish elapsed market inflation from relative price signals.
4. Deterministic Tables, LLM-Generated Analysis: Adopting a strict separation of concerns—Python deterministically renders every tabular figure, while the LLM focuses purely on qualitative interpretation.

1. Removing the External Database Runtime

In the original 2025 architecture, the pipeline required running Docker Compose to spin up a PostgreSQL instance with the PostGIS extension, alongside an MCP Toolbox server. While PostGIS is the gold standard for enterprise spatial queries, running a dedicated database daemon added noticeable operational overhead: startup delays, container resource contention, and network hops.

For ~985,000 historical HDB resale transactions (dating from 1990 to 2026), full-blown database infrastructure is unnecessary.

In-Memory SQLite on First Use

The revamped local_store.py loads the raw seed CSVs directly from db/init/data/ into an in-memory SQLite database:

# autonomous_hdb_deepagents/agent/local_store.py
RESALE_GLOB = "resale_flat_price_*.csv"

def _ensure_db():
    global _conn
    if _conn is not None:
        return _conn
    with _lock:
        if _conn is not None:
            return _conn
        conn = sqlite3.connect(":memory:", check_same_thread=False)
        _init_schema(conn)
        _load_resale_csvs(conn)
        _create_indexes(conn)
        _conn = conn
        return _conn

On first query, parsing and indexing ~985k CSV rows into memory takes approximately 4 seconds. Once the indexes are built, queries like:

SELECT block, street_name, flat_type, floor_area_sqm,
       resale_price, month, flat_model, lease_commence_date, storey_range
FROM resale
WHERE town LIKE :town AND flat_type = :flat_type AND resale_price <= :max_price
ORDER BY month DESC LIMIT 30;

It is worth scoping what “fast” actually means here, measured end to end rather than assumed. list_hdb_flats – the tool the pipeline actually calls – originally matched town with a plain substring LIKE '%...%' (to tolerate loose station-to-town resolution), and EXPLAIN QUERY PLAN showed that pattern could not use the idx_resale index: a full scan over ~985k rows plus a temp B-tree sort, measured at a median of ~143 ms. Every real caller, though, already passes a canonical, upper-cased town name (mrt_resolve_node’s station-code lookup, intent_node’s LLM extraction) – a genuine fragment like "BISH" never actually occurs in the pipeline, only in a test that deliberately checks the tool still tolerates one. So list_hdb_flats now tries an exact match against a generated, indexed town_norm column first, and only falls back to the original LIKE scan when that returns nothing. The common path is index-backed and measures a median of ~17.9 ms over 200 repeated queries rotated across five towns – about 8x faster – while the rare fragment case still works, just at the old ~124 ms. Geospatial enrichment across MRT exits for a 30-flat batch measures ~3 ms total (~0.1 ms per flat), and XGBoost inference for the same batch measures ~15 ms total (~0.5 ms per flat). The whole pipeline still finishes well under the network latency of the downstream LLM API call either way, but the index fix removes SQLite as the dominant cost in the common case.

Python Geodetics Near Singapore’s Latitude

In PostGIS, I relied on ST_DWithin and ST_Distance on geography/geometry columns. In Python, because Singapore is situated virtually on the equator at approximately 1.35° N, an equirectangular projection centered on this latitude offers exceptional accuracy:

KM_PER_DEG_LAT = 110.57
KM_PER_DEG_LON = 111.32

def _dist_km(lat1: float, lon1: float, lat2: float, lon2: float) -> float:
    dlat = (lat1 - lat2) * KM_PER_DEG_LAT
    dlon = (lon1 - lon2) * KM_PER_DEG_LON
    return math.sqrt(dlat * dlat + dlon * dlon)

Benchmarking the Approximation: In tests/test_local_store.py, this equirectangular calculation was benchmarked against the full spherical Haversine formula across the first 500 real HDB block coordinates loaded from hdb_property_info.csv (out of ~13,200 blocks with usable coordinates, spanning Singapore’s geographic bounding box, approx. 1.20°N to 1.47°N, 103.60°E to 104.05°E), each evaluated against all 613 official LTA MRT/LRT station exits loaded from the seed data.

Across this compact ~50 km × 27 km operating territory, cosine distortion is negligible:

• The maximum observed discrepancy is 7.9 metres (with a median difference under 1 metre). The checked-in test asserts a looser 10-metre budget rather than pinning this exact figure, so it stays green if a future data refresh shifts the worst case by a metre or two; 7.9 m is what that budget currently measures.
• Nearest station exit rankings are identical, diverging only on rare centimeter-scale ties between adjacent exits.
• For pedestrian radius filtering (typically 400 m to 800 m), a 10-metre budget is well within the real-world variance of walking paths.

The payoff: zero external C-extensions, zero database daemons, and instant execution.

2. Integrating XGBoost-Based Price Estimates

In the previous system, the pipeline stopped at retrieving historical resale transactions and measuring distance to the nearest MRT station. A user reviewing a recent transaction at S$580,000 had no quantitative baseline to assess whether that transaction aligned with broader market pricing patterns.

The new pipeline embeds the trained XGBoost model (hdb_model.bst / .ubj) and scaler metadata (hdb_scaler.json) directly from the sibling hdb-price-predictor project.

The LangGraph Orchestrator

The LangGraph state machine now places valuation_node between spatial enrichment and summarization:

# autonomous_hdb_deepagents/agent/pipeline.py
graph = StateGraph(PipelineState)
graph.set_entry_point("intent")

graph.add_node("intent",      intent_node)
graph.add_node("mrt_resolve", mrt_resolve_node)
graph.add_node("resale",      resale_node)
graph.add_node("mrt",         mrt_node)
graph.add_node("valuation",   valuation_node)
graph.add_node("summary",     summary_node)

graph.add_edge("intent",      "mrt_resolve")
graph.add_edge("mrt_resolve", "resale")
graph.add_edge("resale",      "mrt")
graph.add_edge("mrt",         "valuation")
graph.add_edge("valuation",   "summary")
graph.add_edge("summary",     END)

[User Query]
     │
     ▼
[intent_node]       ──> Extracts town, MRT, flat type, budget
     │
     ▼
[mrt_resolve_node]  ──> Resolves MRT to nearest HDB town
     │
     ▼
[resale_node]       ──> Fetches recent transactions from SQLite
     │
     ▼
[mrt_node]          ──> Calculates straight-line distance to MRT exits
     │
     ▼
[valuation_node]    ──> Feeds features into XGBoost model
     │
     ▼
[summary_node]      ──> Generates Markdown table + LLM analysis
     │
     ▼
   [END]

Dynamic Feature Mapping & MinMax Scaling

The valuation node dynamically maps database fields to the exact 60-feature vector expected by the model:

# autonomous_hdb_deepagents/agent/valuation.py
def value_flat(
    town: str,
    flat_type: str,
    floor_area_sqm: float,
    storey: float,
    lease_commence_year: int,
    target_year: int,
    target_month: int = 6,
    flat_model: str = "Model A",
    mrt_distance_km: float = 0.513,
) -> dict:
    booster, meta, feature_names, feature_idx = _model()
    adj = _forecast_adjustment(target_year, target_month)
    remaining_lease = max(1.0, min(99.0, (lease_commence_year + 99.0) - target_year))

    raw = np.zeros(len(feature_names), dtype=np.float32)
    raw[feature_idx["floor_area_sqm"]]        = floor_area_sqm
    raw[feature_idx["storey_avg"]]             = storey
    raw[feature_idx["sale_year"]]              = float(adj["model_year"])
    raw[feature_idx["sale_month"]]             = float(adj["model_month"])
    raw[feature_idx["remaining_lease_years"]]  = remaining_lease
    raw[feature_idx["mrt_distance_km"]]        = mrt_distance_km
    raw[feature_idx["mrt_walkable"]]           = 1.0 if mrt_distance_km <= 0.4 else 0.0

    # Categorical one-hot encodings
    ft_col = f"flat_type_{flat_type.upper()}"
    if ft_col in feature_idx: raw[feature_idx[ft_col]] = 1.0
    fm_col = f"flat_model_{flat_model}"
    if fm_col in feature_idx: raw[feature_idx[fm_col]] = 1.0
    town_col = f"town_{town.upper()}"
    if town_col in feature_idx: raw[feature_idx[town_col]] = 1.0

    # MinMax scale: x_scaled = x * scale + min
    scaled = raw * np.array(meta["scale"], dtype=np.float32) + np.array(meta["min"], dtype=np.float32)
    dmat = xgb.DMatrix(scaled.reshape(1, -1), feature_names=feature_names)
    raw_pred = float(booster.predict(dmat)[0])

    price = max(150_000.0, raw_pred * adj["growth_factor"])
    return {
        "estimated_price": round(price),
        "range_low":       round(price * (1 - _mape_hdb)),
        "range_high":      round(price * (1 + _mape_hdb)),
        "psf":             round(price / (floor_area_sqm * 10.7639)),
        "mode":            adj["mode"],
    }

Deconstructing the Valuation Pipeline

Several critical design choices in this excerpt warrant detailed technical examination:

1. Effective Dates & The Forecast Adjustment

The _forecast_adjustment(target_year, target_month) helper distinguishes between pure ML inference and macroeconomic index projection:

• Training Cutoff Date: The model was trained on historical data up to end-2025 (latest_complete_year = 2025).
• Within-Cutoff Mode (target_year <= 2025): The trees run in pure historical or comparable mode (growth_factor = 1.0), where price levels are evaluated directly by the tree splits for that year and month.
• Forward Projection Mode (target_year > 2025): Because decision trees cannot extrapolate trends beyond the boundary of their training data, tree inputs are clamped to the cutoff year and a neutral mid-year month (model_year = 2025, model_month = 6, avoiding seasonal boundary cliff effects). The tree prediction is then multiplied by an index-growth multiplier: $$\text{growth\_factor} = (1 + g)^{\Delta t}$$ where \(g\) is the annualized growth rate exported in hdb_scaler.json (currently 6.73%, sourced from the HDB Resale Price Index) and \( \Delta t \) is elapsed time. If hdb_scaler.json omits the field, a conservative 2.5% fallback constant is used instead.

Distinguishing these two components is vital: the output combines a learned structural valuation (capturing location, size, lease, and amenities) with an explicit, transparent macroeconomic growth assumption.

2. Feature Vector Contract & Reproducibility

To ensure exact parity between Python agent inference and the browser-based predictor:

• Direct Artifact Coupling: _load_model() reads feature_names, scale, and min directly from hdb_scaler.json. This guarantees that feature ordering and array indices never drift from the compiled model (hdb_model.bst).
• Handling Unrepresented Categoricals: _normalise_flat_model cleans casing and maps data quirks (such as legacy "MAISONETTE" entries to "Model A-Maisonette"). If an unknown town, flat type, or model appears that was absent from training, all one-hot columns remain zero. This is a deliberate fallback: rather than crashing, the model prices the unit as a baseline generic flat.
• Auditing Assumed Inputs: The pipeline tracks missing or defaulted fields in assumed_fields (e.g. missing storey range or floor area in older records) and surfaces caveat notes directly under the generated table so the user knows when an estimate carries reduced confidence.
• Parity Testing: tests/test_valuation.py validates that feature vector construction and scaling math replicate the sibling project’s exported formulas exactly.

3. Indicative Range vs. Statistical Confidence Intervals

The valuation dictionary outputs:

"range_low":  round(price * (1 - _mape_hdb)),
"range_high": round(price * (1 + _mape_hdb)),

This produces a model-error-based indicative range scaled by the holdout Mean Absolute Percentage Error (MAPE), currently 4.63%, read live from the sibling project’s metrics.json at load time (falling back to a conservative 7.22% constant only if that file is absent).

It is important to clarify that this is not a formally calibrated prediction interval or statistical confidence interval (such as those generated via conformal prediction or quantile regression). It does not quantify individual flat uncertainty; rather, it offers a pragmatic, transparent envelope reflecting average model error across the unseen holdout dataset.

4. MRT Proximity: Correlative Feature, Not Causal Premium

The pipeline passes the unrounded floating-point distance (flat["dist_km"]), which avoids discretization artifacts from rounded display strings like "1.1km".

However, in an XGBoost tree model, MRT proximity is an observational feature that interacts non-linearly with town, flat type, and remaining lease. A lower distance correlates with higher prices in historical transactions, but the model does not estimate an isolated causal premium (i.e. it does not prove that moving 100 metres closer independently generates a specific dollar increase). It reflects the complex, learned associations of the Singapore property landscape.

3. Preventing Misleading Historical Price Comparisons

Before analyzing price differences, an essential domain distinction must be emphasized: historical transaction records are not active property listings. The Singapore public resale dataset records completed, settled sales with fixed historical registration dates—not asking prices open to negotiation today.

In machine learning, temporal leakage strictly refers to future information leaking into model training or evaluation sets. Here, we face an inference-time temporal alignment problem: treating past completed transaction records as if they were current market prices.

If a query retrieves a transaction registered in 2021 for S$450,000, and my XGBoost model values the flat in 2026 at S$560,000, is the flat selling at a 20% discount?

No. That difference reflects five years of market appreciation, not a below-market bargain. Presenting historical transaction prices as “underpriced” relative to current valuations conflates market trajectory with a discount.

To prevent this, valuation_node enforces an explicit temporal comparability check:

sale_year, _sale_month = _parse_sale_period(flat.get("month"))
listed = flat.get("resale_price")

if listed is not None and sale_year == target_year:
    listed = float(listed)
    est = valuation["estimated_price"]
    valuation["listed_vs_estimate_pct"] = round((listed - est) / est * 100, 1)
    valuation["comparable_to_listed"] = True
else:
    valuation["comparable_to_listed"] = False
    valuation["listed_sale_year"] = sale_year

If the sale occurred in an earlier year, the table verdict explicitly marks:

sold 2021, not comparable

and the LLM context prompt instructs the agent:

“A row reading ’not comparable’ sold in an earlier year than the valuation target, so the difference there is market growth over time, not a discount. Never present it as one.”

Refining Comparability: Why Same-Year Is Not Always Contemporaneous

While checking sale_year == target_year prevents comparing a 2017 transaction against a 2026 valuation, it remains a coarse filter. For instance, a transaction registered in January 2026 and an evaluation performed in September 2026 are eight months apart; in an active property cycle, market levels can shift materially within the same calendar year.

A production refinement is to compare the transaction month against a defined rolling tolerance window (e.g., within 3 to 6 months of the target period), categorizing anything outside that window as historical. Gating by calendar year provides an immediate guardrail against multi-year index distortion, but treating same-year transactions as strictly contemporaneous should always be understood with this window in mind.

Qualifying “Fair Value” vs. Model Estimates

It is equally essential to qualify what the valuation actually represents:

• Historical transactions are not asking prices: The data reflects settled sales, not live listings on the market.
• Model estimates are statistical benchmarks: An XGBoost prediction is a central tendency based on historical features (holdout MAPE 4.63%), not an independently certified appraisal.
• Valuation gap is a potential value signal: The percentage difference (listed_vs_estimate_pct) is a comparison with the model’s estimate, highlighting potential relative anomalies rather than definitive proof of a bargain.

4. Deterministic Tables, LLM-Generated Analysis

Anyone building LLM-based financial or real estate agents encounters the same headache: LLMs struggle with numerical precision. When asked to format a table of 10 properties with prices, square footages, remaining leases, and percentage differences, LLMs will routinely:

• Transpose digits (e.g., reporting S$542,000 as S$524,000)
• Inaccurately calculate percentage differences
• Confuse row alignments between distance and price

Prompting techniques like “Be extremely accurate with arithmetic” or “Double check your calculations” act as gentle suggestions, not guarantees.

The Architectural Solution

In summary.py, I eliminated this problem by stripping the LLM of transcription duties entirely. Python renders the Markdown table with every number formatted and verified:

def _flat_row(index, flat):
    val = flat.get("xgb_valuation") or {}
    inputs = val.get("inputs") or {}
    pct = val.get("listed_vs_estimate_pct")

    if not val:
        gap, verdict = "—", "not valued"
    elif not val.get("comparable_to_listed"):
        gap, verdict = "—", f"sold {val.get('listed_sale_year')}, not comparable"
    elif pct is None:
        gap, verdict = "—", "—"
    else:
        gap = f"{pct:+.1f}%"
        verdict = "above estimate" if pct > 0 else "below estimate"

    area = inputs.get("floor_area_sqm")
    storey = inputs.get("storey_avg")
    lease = val.get("remaining_lease")
    psf = val.get("psf")

    return [
        ("#", str(index)),
        ("Block / Street", f"{flat.get('block', '?')} {flat.get('street_name', '')}".strip()),
        ("Sold", flat.get("month") or "—"),
        ("Listed", _fmt_price(flat.get("resale_price"))),
        ("ML estimate", _fmt_price(val.get("estimated_price"))),
        ("Gap", gap),
        ("Verdict", verdict),
        ("MRT", flat.get("dist_formatted") or "N/A"),
        ("Area", f"{area:g} sqm" if area else "—"),
        ("Floor", f"~{storey:g}" if storey else "—"),
        ("Lease", f"{lease:g} yrs" if lease else "—"),
        ("Model", inputs.get("flat_model") or "—"),
        ("PSF", f"S${psf:,}" if psf else "—"),
    ]

The column is literally named "Listed" in the code above – a holdover label that, per the domain distinction in Section 3, undersells what it actually holds: resale_price is a completed, registered transaction, not an active asking price. The rendered table below refers to it by its more accurate meaning, transaction price.

Verdict is a mechanical comparison, not a valuation judgment: it labels a row “above estimate” or “below estimate” purely from the sign of listed_vs_estimate_pct, nothing more. It is not a claim that the flat is a bargain or overpriced – a transaction priced below the model’s estimate could just as easily reflect a lower floor, an older lease, or a data gap covered by the caveats under the table, as it could reflect genuine relative value.

The output table produced by Python looks like this:

Keeping Table Arithmetic Outside the LLM

The prompt sent to the LLM supplies this pre-rendered table as read-only context. While the LLM receives the figures to interpret overall market patterns, it has zero responsibility for calculating or formatting the table cells:

prompt = f"""Write a short analysis of HDB resale flats near {location}.

The table below is already written and is shown to the user above your text.
Every figure the answer needs is in it.

{table}{context}

Rules:
- Refer to flats by row number.
- Do NOT restate a flat's price, estimate, gap, lease, floor, area, model or MRT distance -- the reader has the table. Describe what the numbers mean.
- Do not introduce any figure that is not in the table.
- No tables of your own.

Write 3-4 short paragraphs: the overall picture, which rows look like value or look expensive and why, any pattern worth noting across the rows, and one caveat a buyer should keep in mind.
"""

Measurable Reliability: What This Solves (and What It Doesn’t)

By offloading all table formatting and arithmetic to Python, I remove an entire class of deterministic errors:

• Eliminated: Dropped digits, transposed numbers (e.g. S$542k into S$524k), miscalculated percentages, and row-column misalignment in markdown syntax.
• Remaining Risks: The LLM can still misinterpret qualitative context, reference the wrong row index (e.g. referring to Row 2 when discussing Row 3), or make unsubstantiated claims if not strictly grounded by the rendered table. Furthermore, upstream data errors (such as missing attributes or geocoding approximations) will propagate directly into the table.

Separating data calculation from semantic prose does not make the agent infallible—it makes the failure modes inspectable and keeps numerical precision strictly in compiled code.

5. Automated Data Refresh & OneMap Geocoding Verification

A data-driven assistant is only as good as its underlying datasets. I added unified CLI scripts under db/ to maintain the data layers:

• refresh_data.py: Central orchestration to pull the latest 2026 resale CSVs, LTA MRT exits, and NParks coordinates.
• geocode_schools.py: Reverse geocoding of MOE schools via OneMap.

Handling Shared Postal Code Ambiguity & Data Freshness

During school geocoding, I discovered an interesting real-world data anomaly: postal code collisions.

A naive postal-code lookup returns the first building registered to that postal code. In Singapore, several campuses share a postal code with adjacent facilities:

Boys’ Town is not a former or renamed identity of Assumption English School – the two are distinct institutions (a MOE-aided secondary school and a separate Catholic residential/vocational facility for at-risk youth) that happen to sit side by side on the same Upper Bukit Timah Road compound and share one postal code. If uncorrected, Assumption English School’s coordinates would be silently pinned to Boys’ Town’s building instead of its own.

db/geocode_schools.py resolves this automatically: when the building name returned by OneMap does not match the school name, it falls back to querying the school’s name, confirming the result only if it resolves to the identical postal code.

When neither the postal code nor the verified name search yields a confident match on OneMap (or coordinates fall outside Singapore’s geographic bounding box), the pipeline explicitly rejects and excludes the school rather than writing unverified or centroid-defaulted coordinates. The unresolved entity is logged, preserving spatial integrity.

uv run python db\geocode_schools.py --apply

Data Freshness Cutoff: The seed datasets include transactions through September 2026. Because data.gov.sg publishes resale data with a 1-to-2 month registration lag, db/refresh_data.py polls and downloads genuinely new months directly from the data.gov.sg API. Because local_store.py globs resale_flat_price_*.csv, adding new monthly files requires zero code changes—the in-memory SQLite store automatically indexes them on process restart.

6. Running the Revamped System

The repository is fully managed using uv, enabling fast, deterministic dependency resolution.

1. Installation & Environment Setup

git clone https://github.com/seehiong/autonomous-hdb-deepagents.git
cd autonomous-hdb-deepagents

# Install dependencies with uv
uv sync

Copy .env.example to .env and provide your OpenRouter API key:

OPENROUTER_API_KEY=sk-or-v1-...
LLM_MODEL=anthropic/claude-3.5-haiku
# Optional: point to sibling model directory if not using default sibling folder
# HDB_MODELS_DIR=F:/github-proj/hdb-price-predictor/models

2. Running via CLI

uv run -m autonomous_hdb_deepagents.agent.cli "Find flats near Bukit Batok MRT"

[MRT-RESOLVE] Resolving MRT station: Bukit Batok
[MRT-RESOLVE] BB -> BUKIT BATOK
[RESALE] Fetching 4 ROOM in BUKIT BATOK <= 600000...
[RESALE] Retrieved 30 flats
[MRT] Enriching 30 flats (radius=800)
[MRT] Example: BT BATOK ST 21 -> BUKIT BATOK MRT STATION (380m)
[VALUATION] Loading XGBoost model  : .../hdb_model.bst
[VALUATION] Loading scaler metadata: .../hdb_scaler.json
[VALUATION] Valued 30/30 flats (target=2026-09, mode=forward projection)
[SUMMARY] Summarizing 30 flats

=== FINAL RESPONSE ===

### Flats near Bukit Batok MRT

| # | Block / Street | Sold | Transaction price | ML estimate | Gap | Verdict | MRT | Area | Floor | Lease | Model | PSF |
...

3. Running the Gradio Web Chat Interface

For interactive conversational exploration:

uv run python -m autonomous_hdb_deepagents.ui.gradio_app

The Gradio UI provides a full-screen chat interface where users can click quick query buttons or type open-ended questions like “Show me 5-room flats near Redhill MRT under 950k”.

4. Running the FastAPI HTTP Server

For headless integration into web or mobile apps:

uv run python src/autonomous_hdb_deepagents/api/api_server_launch.py

You can test the health and query endpoints directly:

Invoke-RestMethod -Uri "http://localhost:8000/query" -Method Post `
  -Body '{"query":"Find flats near Bishan MRT under 800k"}' `
  -ContentType "application/json"

7. Architectural Reflection: Is This Still a “Deep Agent” and “Autonomous”?

A fair question arises when examining the LangGraph state machine:

If the execution pipeline is an engineered graph (intent -> mrt_resolve -> resale -> mrt -> valuation -> summary), is this still an autonomous deep agent, or is it just a workflow pipeline?

This touches on one of the most critical debates in current AI engineering: the illusion of the unconstrained ReAct loop versus engineered autonomy.

The Fallacy of the “Wild-West” ReAct Loop

In early agent tutorials, autonomy was often equated with an unconstrained ReAct loop: handing an LLM five tools and letting it wander in a Thought -> Action -> Observation cycle until it decides to stop.

In domain-critical tasks involving machine learning and spatial data, unconstrained loops fail consistently:

1. Skipped dependencies: An XGBoost model requires exact spatial distance and MinMax feature scaling before inference. An open-ended LLM frequently calls valuation before distance resolution, or skips valuation entirely.
2. Infinite looping & token burn: The LLM re-queries the database when a specific flat isn’t found, exhausting context windows and API budgets.
3. Loss of deterministic guarantees: Asking an LLM to decide the execution graph on every turn turns simple search into a stochastic gamble.

What Makes It a “Deep Agent”?

In modern multi-agent systems, “Deep” refers to hierarchical decomposition and sub-agent abstraction, not circular prompt loops.

In our codebase, the orchestrator compiles directly as a sub-agent using the deepagents framework:

# autonomous_hdb_deepagents/agent/pipeline.py
from deepagents import CompiledSubAgent

orchestrator_subagent = CompiledSubAgent(
    name="orchestrator",
    description="Intent -> MRT Resolve -> Resale -> MRT Enrichment -> XGBoost Valuation -> Summary",
    runnable=compiled_orchestrator
)

This allows the entire multi-tier system—spanning intent parsing, SQLite querying, geodetics, XGBoost inference, and natural-language synthesis—to be encapsulated as an atomic agent that can be plugged into higher-level supervisor agents (for instance, a master wealth advisory or property relocation agent).

Constrained Autonomy in Production

What does autonomy actually mean in production?

It means task-level autonomy. A user submits an ambiguous, open-ended natural-language goal:

“Find flats near Bukit Batok MRT under 600k with good value”

Without human intervention, the system:

• Autonomously extracts semantic parameters and assigns intelligent defaults
• Autonomously maps station names to administrative towns
• Autonomously orchestrates the data retrieval, spatial measurement, and ML valuation
• Autonomously synthesizes qualitative takeaways from the quantitative results

Anthropic’s research on Building Effective Agents (published December 2024) distinguishes predefined workflows from agents that dynamically direct their own processes and tool use. It also emphasizes choosing the simplest architecture that meets the task’s needs, with evaluation and iteration guiding additional complexity.

My implementation applies that engineering perspective through a predefined LangGraph execution path, with LLM-based intent extraction and qualitative interpretation at selected stages. By pairing deterministic execution with LLM reasoning at the boundaries, I built a constrained autonomous agent that can take a natural-language property search from intent extraction through retrieval, spatial enrichment, and model-based valuation.

The architecture makes important parts of the execution predictable and auditable. It does not eliminate uncertainty in the underlying data, model estimates, or generated interpretation—and that distinction is especially important when the output informs real-world property decisions.

8. Reproducible Test & Validation Benchmarks

To ensure the system behaves as documented, the project includes an automated test suite under tests/ covering data parsing, spatial distance, valuation parity, and pipeline orchestration. The table below reports key empirical benchmarks measured across the codebase:

Reproducing the Test Suite

All tests can be executed locally from the repository root:

uv run --extra dev pytest -v tests/

Evaluation Details & Scope:

• Query Benchmarks: Timed by calling list_hdb_flats (the function the pipeline actually calls, with a LEFT JOIN and ORDER BY month DESC LIMIT 30) 200 times over an in-memory SQLite connection holding ~985,000 resale rows, rotating across five canonical town names – the only kind of town value the real pipeline ever passes. EXPLAIN QUERY PLAN confirms this now seeks idx_resale_norm rather than scanning, since town_norm is an exact match. The original substring-LIKE path is still there and still measures ~124 ms, but only a caller passing a genuine fragment (never the pipeline itself) reaches it; tests/test_local_store.py::test_list_hdb_flats_exact_town_uses_the_index pins the index-seek plan so a future change can’t silently reintroduce the scan for the common case. Geospatial and valuation timings were measured the same way, over the same 30-flat batch, calling geospatial_query and value_flat directly.
• Valuation Baseline & Error Metrics: The reported holdout MAPE is read live from the sibling hdb-price-predictor project’s published models/metrics.json, so it always reflects the currently trained model rather than a value baked into this repo. The current run: 4.63% MAPE, evaluated on 18,278 unseen 2026 transactions, from a chronological split trained on data through 2025-12 (Jan 2017 – Dec 2025). valuation.py also carries a 7.22% constant (FALLBACK_MAPE_HDB), but that is a safety fallback used only if metrics.json cannot be read – not a second, independently reported benchmark.
• Temporal Guardrail Coverage: Guardrail assertions in tests/test_valuation.py verify handling across fixtures spanning current-year sales, prior-year transactions (1990–2025), and unparseable date strings, ensuring that elapsed market growth is systematically flagged as non-comparable rather than misreported as a discount.
• Table Integrity: Unit tests assert that rendered Markdown table cells match underlying dictionary values and unrounded feature metrics exactly, since Python – not the LLM – formats every cell.

9. Key Takeaways

Integrating domain-specific machine learning into an autonomous agent pipeline highlighted four core principles for production AI engineering:

1. Lightweight beats heavyweight for read-heavy local agents: Replacing PostGIS and PostgreSQL with an in-memory SQLite table eliminated thousands of lines of Docker orchestration while speeding up query execution times.
2. Respect temporal validity: In real estate and financial time-series data, an ML valuation is anchored to a point in time. Agents must distinguish between price discrepancies and elapsed market growth.
3. Never let an LLM do a database or calculator’s job: Formatting data tables in code and restricting the LLM to qualitative analysis eliminates arithmetic and transcription errors at the source.
4. Embrace constrained autonomy: Real-world autonomy is about reliably achieving an ambiguous user goal end-to-end. Compiling specialized multi-tier graphs into reusable DeepAgents sub-agents delivers both the intelligence of LLMs and the reliability of deterministic code.

The complete codebase, notebooks, and model integrations are open source on GitHub: 👉 https://github.com/seehiong/autonomous-hdb-deepagents