The previous post talked about why we’re shifting the data team from writing SQL to training Agents, and mentioned we’re building a six-layer Context Stack. This post breaks down each layer and how we put it into practice.

The Big Picture

                Signal density ↑  Acquisition cost ↑

        ┌─────────────────────────┐
    L6  │  Runtime Exploration    │  Agent probes live
        ├─────────────────────────┤
    L5  │    Agent Memory         │  Interactive correction → self-evolving
        ├─────────────────────────┤
    L4  │  Domain Knowledge       │  Organizational knowledge → analysis frameworks
        ├─────────────────────────┤
    L3  │ Code-Level Semantics    │  ETL code → real semantics
        ├─────────────────────────┤
    L2  │  Curated Semantics      │  Expert annotations → metrics/relationships
        ├─────────────────────────┤
    L1  │   Schema & Usage        │  Metadata + query patterns
        └─────────────────────────┘

From bottom to top, signal density increases, and so does the cost of acquiring it. Below I walk through each layer: what we did and why we chose it.

L1: Catalog First, Not RAG

We started by dumping all table schemas and query history into embeddings, retrieving via RAG at query time. Once we crossed a thousand tables and a few hundred metrics, pure semantic retrieval wasn’t reliable enough.

Then we added a lightweight structure called Manifest. Each manifest file stores only an entry’s id, name, aliases, and ref. All manifests together fit in a few thousand tokens of context. The Agent loads manifests directly into context at startup, like a “table of contents” page. Once it hits a match, it pulls details by ref.

Precise routing first, fetch details on demand. RAG is the fallback, not the main path.

L2: Metrics Must Be Structured, Natural Language Isn’t Enough

We have metric clusters like “DAU”, “eDAU”, “playback DAU”, same name, different definitions across business lines. If you only rely on hand-written natural language descriptions, the Agent will mix them up.

We wrote metrics, dimensions, and table JOIN graphs as structured yaml. Each metric explicitly spells out source_tables, filter, cross_day_rule, related_metrics. Once structured, the Agent doesn’t pick wrong among similar metrics.

This layer is the most labor-intensive in the stack. But looking back, it carries 80% of daily queries.

L3: A Table’s Meaning Lives in the ETL Code

Schema tells you which columns a table has, but it won’t tell you that this table only includes App-side traffic, excluding in-car and third-party channels. That information lives in the ETL code that produces the table.

We have our own task scheduling system that stores the full SQL, scheduling config, and upstream/downstream dependencies for every output table. We built a tool wrapper on top of it, so when L2 isn’t enough, the Agent can pull a table’s production SQL and reverse-engineer its real meaning and lineage.

ETL code is already describing “how this table is computed.”

After plugging this layer in, the Agent’s ability to distinguish between two similarly named but semantically different tables improved significantly.

L4: Domain Knowledge Must Be Independent of Skills

Early on we went the route of “write one skill per business domain.” One skill for push analysis, one for membership analysis. Each skill stuffed routing, domain knowledge, SQL templates, and execution logic together. After writing a few, things got messy: the same metric was defined differently across two skills, domain knowledge couldn’t be exhaustively enumerated, skills mixed “knowledge” with “execution flow”, any change required touching a whole chunk.

The fix is to extract domain knowledge from skills and maintain it separately as yaml and md.

After extraction, topic-analysis skills degraded into “lightweight routing + special flow orchestration.” For most scenarios, a generic analysis Agent reading domain knowledge can handle it, no need to write a separate skill per business domain.

L5: Memory Has to Expire

Reusing corrections is table stakes, the user fixes it once, don’t get it wrong next time. But memory can’t be write-only, otherwise after half a year it’s just noise.

We added a hotness_score to each memory entry, normalizing hit frequency with a sigmoid and multiplying by a 7-day half-life decay. Memories from 30 days ago carry about 5% weight, no manual cleanup needed, memory expires naturally.

Meanwhile, high-hotness memories enter a review queue. Confirmed valuable ones get promoted back to source files in L2 or L4, becoming “solidified knowledge.” L5 is essentially a temporary buffer. Its value isn’t in long-term storage itself, but in promptly pushing high-frequency corrections down to lower layers.

L6: Probe Results Should Be Persisted

The fallback layer. When L1 through L5 have no answer, the Agent queries the warehouse live, verifying schema, sampling data, tracing lineage.

We added one more step: every valuable finding from L6 probing gets suggested for promotion back to lower layers. Table structure → L1, field meaning → L2 glossary, business understanding → L5 memory. On-the-fly probing isn’t cheap, if it can be persisted, don’t make the Agent probe again.

The whole stack is a feedback loop, ephemeral knowledge from upper layers settles down, ready to be used directly next time.

Why Not Ontology

Banks and financial institutions building data Q&A often go the ontology route, formalizing concepts, relationships, and rules entirely, letting AI reason on a deterministic knowledge graph.

Ontology’s appeal is determinism. Concepts have precise definitions, relationships have clear boundaries, reasoning paths are auditable. It makes sense in finance, the definition of “Return on Equity” hasn’t changed in ten years, regulators demand explainability, error costs are extremely high. Spending two years building a formal knowledge system is worth it.

But ontology has two implicit assumptions: domain knowledge can be exhaustively formalized, and maintenance cost is sustainable.

For fast-iterating business scenarios, neither holds. Today there’s a new “immersive DAU”, tomorrow a new experiment metric, the day after some table’s definition quietly changes. Ontology can’t keep up with the pace of change.

L2 is essentially a lightweight ontology, metrics have id, definition, source_tables, related_metrics, dimensions have value ranges and sql_snippet, tables have JOIN graphs. It just doesn’t pursue exhaustiveness, structured enough to be useful, and what can’t be managed gets handed off to upper layers.

A pure ontology approach tries to compress everything into one layer. The advantage of layering is that you don’t need to formalize all knowledge upfront. Structure the high-frequency 80% into L2 first, let L3 through L6 gradually cover and persist the rest.

Still in Phase 1

I have to be honest, this Stack is only through Phase 1, L1 + L2 manifest routing + detail lookup. L3 is partially connected, L4 is being extracted, L5 and L6 aren’t really activated yet.

Phase 1 has been running noticeably more stably than the ChatBI era of pure RAG retrieval. The “shape” of errors has changed too. Before, the Agent would confidently pick a wrong table. Now, the Agent knows it’s uncertain and will go ask for details or fall back to semantic retrieval.

Once at an internal sharing session, an executive asked: “How many AB experiments does the company have? Which ones have been running for over 6 months, don’t conform to the company’s global-optimum experiment spec, and have negative xx metrics? List them.”

We had never tested this kind of question. I was nervous at the time, thought it wouldn’t run. The Agent ran for 40 minutes and produced a result. The user was very satisfied.

Through that process, the Agent showed strong goal-driven behavior, doing whatever it took to complete the task, no excuses, real execution. Our direction is right.

A Few Takeaways

First, Context must be layered, and the layers must have clear collaboration boundaries. Stuffing everything into one big prompt, one big RAG index, or one big skill, none of it works well.

Second, the layering isn’t arbitrary. The six layers hold up because each has clean boundaries. L1 is physical fact, L2 is human-curated semantics, L3 is truth reverse-engineered from code, L4 is organizational consensus, L5 is the Agent’s own learning, L6 is on-the-fly probing. Clear boundaries enable independent iteration.

Third, the biggest payoff of layering is that skills become lightweight. Originally bloated topic skills degraded into lightweight routers, most domain knowledge moved into L4. Less skill code, higher quality.

Finally, this road is far from finished. L5’s persistence mechanism, L6’s probing protocol, cross-layer ID system, all still iterating. The next thing to really validate is: once the Stack stabilizes, can the Agent reliably deliver auditable, correctable results, and drive more insights and corresponding strategies?