An account-intelligence system that pre-embeds ten million companies into a resident vector index, and one that dispatches agents to research those same companies live on demand, look like the same product to a buyer and behave like opposite workloads on a server. The first is a lookup against something you built ahead of time. The second is a fleet of reasoning steps you pay for at the moment of the question. Both are marketed as “AI-native GTM,” and both find accounts that match an ideal customer profile. Underneath, one is a memory-and-storage problem and the other is a VRAM problem, and confusing them is how a team buys the wrong hardware.
The distinction is not a GTM invention. It comes from retrieval engineering, where index-time and query-time (or “index-time RAG versus real-time RAG”) name a real design fork: precompute and store, or defer the work to request time. Data-infrastructure vendors have started applying the same fork to GTM context. Airbyte frames it as “runtime tool-calling … uses live data at query time” versus “a pre-materialized context layer,” and ZoomInfo argues a GTM intelligence layer should be continuously refreshed rather than built on a stale snapshot that degrades the moment it ships. That framing is useful, and it is still emerging rather than a settled GTM taxonomy, so treat it as a lens you are importing, not a standard you are citing.
The payoff of getting the lens right is concrete. It tells you which OpenMetal line carries each half of an account-discovery stack: TAM modeling, lookalike scoring, and fast filtered retrieval over a stable universe sit on v5 non-GPU nodes, while live intent monitoring and agentic account research sit on single-tenant GPU nodes. It tells you where your commercially sensitive prospect data lives, and it tells you which half of the stack has a cost you can forecast and which half does not.
Key Takeaways
- Both archetypes are memory-bound, not compute-bound. Index-time is capped by index-in-RAM, query-time by KV-cache-in-VRAM.
- Index-time routes to v5. An 8-channel DDR5-6400 node, 1 TB base to a 4 TB ceiling, holds a resident index and queries it in milliseconds.
- Query-time routes to a single-tenant GPU node. VRAM sets the ceiling: 141 GB (H200) or 96 GB (NVIDIA RTX PRO 6000) caps concurrent agents.
- Single-tenant bare metal keeps prospect data and agent reasoning in your boundary, not on a shared API tenancy.
- Fixed capacity absorbs the variance metered billing amplifies, since query-time token volume swings run to run while a monthly price does not.
Requirements beyond the listed configurations
The GPU configurations described here (a single card as-built, with a second discrete card orderable) are the shipping catalog, not the boundary of what OpenMetal will build. The catalog is actively expanding. Requirements past these configurations, different accelerators, higher GPU density per node, different interconnect topologies, or larger multi-node builds, are evaluated against the requirement rather than picked from a fixed menu. If your account-intelligence workload needs a shape that is not listed, that is a design conversation worth starting early.
The lens, and the two archetypes it names
At index time, you run the account and contact universe through an embedding model once, build an approximate-nearest-neighbor index, and store it. At query time your sales query is embedded and matched against that index in milliseconds. The work is front-loaded and amortized. The weakness is staleness: the index reflects the corpus as of the last ingestion run.
At query time, in the account-research sense, you defer almost everything. Agents fetch pages live, reason across multiple steps, re-rank, and synthesize an answer per request. Storage is light and freshness is total, but each request carries a real inference bill and a variable latency that depends on how many steps the agent takes.
| Property | Index-time discovery | Query-time discovery |
|---|---|---|
| When the work happens | Ahead of the query (batch) | At the moment of the query |
| Binding constraint | DRAM capacity, then NVMe IOPS | VRAM (KV cache), then memory bandwidth |
| Freshness | Snapshot between rebuilds | Live |
| Per-query cost | Near zero marginal | High and variable |
| Recompute trigger | Corpus change or model change | None (nothing is stored) |
| OpenMetal line | v5 non-GPU | Single-tenant GPU |
Most real stacks run both: a materialized index for the stable bulk of the market, and live agents for the volatile tail where freshness decides the deal. The design question is which fraction of your knowledge is stable enough to amortize.
Index-time is a memory-and-storage decision
An in-memory index has to stay resident to answer in milliseconds. Graph indexes like HNSW hold the raw vectors plus a navigation graph, and the graph adds roughly 1.5 to 2 times the raw vector bytes. That makes the sizing arithmetic simple and unforgiving:
100,000,000 accounts x 1,024 dims x 4 bytes (float32) = ~419 GB raw vectors
+ HNSW graph overhead (~1.5x) = ~630 GB residentA 100M-account universe at 1,024 dimensions needs roughly 630 GB of RAM before it serves a single query. OpenMetal’s XL v5 node, with 8 memory channels of DDR5-6400 per socket and a 1 TB base configuration, holds that comfortably and answers from memory. Grow the universe, raise the embedding dimension, or keep several indexes warm, and you climb toward the 4 TB ceiling. Two honest notes come with that climb. The published 4 TB maximum uses special-order RDIMMs and runs at DDR5-5200 rather than the DDR5-6400 of the base fill, so the maximum-capacity configuration is not the maximum-bandwidth configuration. And component pricing and lead time for a memory build-out are quoted at the time of order, not fixed in advance. Route that choice on how firm your requirement is: a known index size favors ordering the fill populated, since its cost is already inside the fixed monthly price, while an index you are still sizing favors starting at the base and adding capacity on the same node when the number settles.
When the index outgrows what you want to keep in RAM, the constraint moves rather than disappears. A disk-based index such as DiskANN keeps compressed vectors in memory and pushes the full graph to NVMe, which turns the bottleneck into random-read latency. This is where the drive spec becomes the system spec. The Micron 7500 MAX 6.4TB sustains 1.1 million random-read IOPS and holds sub-1ms latency at the six-nines percentile up to queue depth 128, per the Micron 7500 Tech Prod Spec. That six-nines floor is what keeps DiskANN tail latency predictable under a busy query load, and it is a property of the drive, not of cluster tuning. For a corpus larger than a single node, all-NVMe Ceph gives you a distributed store on the same media discipline, with no split journal tier to become the slow path.
| Index tier | Where it lives | OpenMetal substrate | Trades |
|---|---|---|---|
| Resident HNSW | System RAM | XL v5, DDR5-6400, up to 4 TB | Fastest, capped by RAM |
| DiskANN | NVMe, compressed vectors in RAM | Micron 7500 MAX, 1.1M IOPS | 5-10x more vectors per node, IOPS-bound |
| Distributed | Ceph, all-NVMe OSDs | v5 cluster | Universe beyond one node |
The one recurring compute cost of the index-time mode is embedding. Building the index is a batch job that saturates a GPU well, and steady-state single-query embedding is small enough to run on CPU, where the Intel AMX units in Granite Rapids handle it without a dedicated card. The trap is the model upgrade. Vectors from two different embedding models occupy different spaces, so a query embedded with the new model cannot be compared against a corpus embedded with the old one. As the Drift-Adapter work (arXiv 2509.23471) states, upgrading an embedding model “typically requires re-encoding the entire corpus and rebuilding the Approximate Nearest Neighbor index,” and avoiding that naive full re-encode is worth more than a 100x recompute saving. Budget the embedding model change as a periodic GPU burst against the same corpus, not as a one-time build.
Query-time is a VRAM decision
The live-agent mode moves the ceiling onto the GPU, and specifically onto its memory rather than its math. Autoregressive decoding is dominated by moving weights and the KV cache from VRAM to the compute units, which makes it memory-bandwidth bound. NVIDIA states it plainly: the KV cache requirement grows “linearly with batch size and sequence length,” and token generation “is a memory-bound operation” where transfer speed, not arithmetic, dominates latency. Adding FLOPS does not help a workload that is waiting on memory.
For an agent fleet this has a direct consequence. Every concurrent agent holds a KV cache proportional to its context length, and those caches sit in the same VRAM as the model weights. The rough shape is easy to reproduce:
KV bytes per request ~= 2 (K and V) x layers x kv_heads x head_dim x seq_len x bytes
Total VRAM = model weights + (KV bytes per request x concurrent agents)Concurrency and context length compete for the same capacity, so the size of VRAM sets how many agents you can research with at once. That is why the memory figure is the headline spec. An H200 node carries 141 GB of HBM3e at 4.8 TB/s, and an NVIDIA RTX PRO 6000 Blackwell Server Edition carries 96 GB of GDDR7 at 1597 GB/s. More resident room is more concurrent agents and longer per-account research before the box saturates.
Two facts keep the sizing honest. OpenMetal GPU servers are single-tenant bare metal with full root access, not a shared instance and not a hosted tenancy, so the entire card and its bandwidth are yours with no noisy neighbor. And when a node carries a second card, that is two discrete GPUs, not one pooled accelerator. You scale across them with data or pipeline parallelism, and you size each model to a single card’s memory rather than assuming the two capacities add into one address space. We do not assert a tokens-per-second figure here, because a defensible throughput number for this workload needs an OpenMetal benchmark rather than a datasheet estimate.
The cost shape only the query-time half has
Index-time cost is smooth. You size for the batch window and the steady query rate, and the marginal cost of a sales query is near zero. Query-time cost is the opposite, and its defining trait is variance. One 2026 study from the Stanford Digital Economy Lab found that running the same agent on the same task, costs varied by up to 30x depending on how many tool calls and retries a given run happened to take. That is not a rounding error in a budget, it is the budget.
A metered per-token API converts that variance straight into an unbounded invoice, because you cannot forecast an agent fleet’s monthly bill from its user count the way you can a chat product. A fixed single-tenant node converts the same variance into a utilization question against a cost you already know. You still own the idle capacity between bursts, but you are never surprised by the bill, egress is included rather than metered per call, and there is no per-GPU-hour meter turning a busy research day into a line item. For a workload whose token volume swings this hard, moving the variance off the invoice and onto a utilization curve is the financial argument, and it is available because the capacity is fixed and yours.
What this looks like on OpenMetal
The stack routes to two lines. The index plane and the agent plane are different boxes by design.
| Field | Index plane | Query plane |
|---|---|---|
| Product and link | XL v5 bare metal | OpenMetal GPU server |
| Delivery model | Single-tenant bare metal, dedicated node | Single-tenant bare metal, dedicated node |
| Access | Root, full firmware and driver control | Root, full driver and firmware control, your own hypervisor or none |
| Base configuration | Dual Xeon 6530P, 8-channel DDR5-6400, 1 TB RAM | Dual Xeon 6530P, 1 TB RAM, one GPU as-built |
| What you must add | RAM fill toward 4 TB if the resident index needs it (arithmetic above); pricing and lead time quoted at order | A second discrete GPU if concurrency needs it (two discrete cards, not pooled) |
| Local storage | All-NVMe, Micron 7500 MAX 6.4TB class, boot and data isolated | All-NVMe boot and data on the same media discipline |
| Network | Private and public tiers per node, included egress posture, present-state only | Private and public tiers per node, included egress posture, present-state only |
| Software boundary | You bring the vector engine, embedding model, and index; OpenMetal provides the node and network | You bring the serving stack, model weights, and agent orchestration; OpenMetal provides the node |
| Validation path | Proof of concept against your own corpus and query mix | Proof of concept against your own agent fleet and concurrency target |
| Pricing path | Bare metal pricing is public; GPU node pricing is quoted, contact OpenMetal | GPU node pricing is quoted, contact OpenMetal |
Route by your workload, not by which is newer
If your addressable market is stable enough to embed and you mostly query it, the work is an index, and the index wants RAM and NVMe on a v5 node. If your edge is catching a buying signal the hour it appears, the work is a live agent fleet, and the fleet wants VRAM on a single-tenant GPU node. Most serious account-intelligence stacks run both planes, because the stable bulk of a market and its volatile tail have different half-lives. The reason to separate them onto different lines is not preference, it is that they saturate different memory, and a box sized for one starves the other.
The lens does the routing for you once you know which memory a given feature fills. That is the whole practical use of borrowing it from retrieval engineering: it turns “we need AI GTM infrastructure” into two specific, sizable, orderable questions.
Sources
- Airbyte, “Context Architecture for Production AI Agents”. The pre-materialized context layer versus runtime tool-calling framing this article borrows for the index-time and query-time split.
- ZoomInfo, “GTM AI Intelligence Layer: The Foundation for Agentic Sales”. The case for serving live GTM intelligence rather than a stale snapshot.
- NVIDIA, “Mastering LLM Techniques: Inference Optimization”. Source for the KV cache growing linearly with batch size and sequence length, and for token generation being a memory-bound operation.
- “Drift-Adapter: A Practical Approach to Near Zero-Downtime Embedding Model Upgrades in Vector Databases” (arXiv:2509.23471). Source for the full-corpus re-encode and ANN index rebuild that an embedding-model upgrade forces.
- Stanford Digital Economy Lab, “How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks”. Source for same-task agent runs varying by up to 30x in token consumption.
- Micron, “7500 NVMe SSD Technical Product Specification” (Rev. A, 10/2023). Source for the Micron 7500 MAX random-read IOPS and six-nines tail-latency figures.
Talk to an architect
The fastest way to size either plane is against your own numbers: the account universe and embedding dimension for the index plane, the concurrency target and context length for the agent plane. Bring those, and we will map them to a configuration and a validation plan you can prove on your own workload before you commit.