Vantage Launches Inference Allocation
Allocate hourly-billed GPU and compute costs from Modal, Baseten, Together AI, AWS, Azure, and Google Cloud to the teams, applications, and customers behind the requests, using the same tagged logs as token-billed model spend.

Today, Vantage is launching Inference Allocation, enabling customers to allocate the cost of models they host on cloud infrastructure that are billed by the hour, in proportion to the requests and tokens that they serve. Inference Allocation supports both self-hosted models on dedicated deployments from providers such as Baseten and Together AI and hosted inference on platforms such as Modal and Amazon Bedrock. Teams can now see hourly-billed inference costs by team, application, customer, or any other tag they attach to their requests, right next to their token-billed model costs.
Enterprises are increasingly running open-weight and fine-tuned models alongside the frontier models they call from providers like OpenAI, Anthropic, and SpaceXAI. These models let teams serve specialized models built for their own products, keep sensitive data within the regions and environments they control, and lower unit costs on high-volume workloads. Teams self-host them on dedicated deployments from providers such as Baseten and Together AI, or run them on hosted platforms such as Modal and Amazon Bedrock, where they're billed by the hour rather than per token. Vantage's LLM Enrichment already allocates per-token LLM costs from providers such as OpenAI and Anthropic across the teams, applications, and customers recorded in request logs, since the bill and the logs are both measured in tokens. Previously, customers had no way to apply that same allocation to hourly-billed models. Their request logs recorded every request in tokens, but inference providers bill for the hours their GPUs, CPUs, and memory ran, whether the model served ten requests or ten million. Nothing on the bill shows which teams, applications, or customers those hours went to, so the cost of running these models showed up as one shared line item.
Now, with Inference Allocation, customers can allocate hourly-billed inference costs using the same tagged request logs they already send to Vantage, so the same tag keys allocate spend across both token-billed and hourly-billed models. To get started, customers connect their request logs, whether from an AI gateway such as LiteLLM, Cloudflare AI Gateway, or Bifrost, from native logs such as Amazon Bedrock model invocation logs, or through a custom upload with the Custom LLM integration. They select which connected providers to allocate, such as Modal, Baseten, Together AI, AWS, Azure, or Google Cloud. To convert tokens into a share of the hours billed, Vantage uses a methodology modeled on how API-based LLM providers price their models: it counts every request along with its input, output, cache read, and cache write tokens, and customers set token weights for each token type based on the compute it takes to serve. Vantage then splits each model’s full daily cost, including idle time, across the tags on the requests it served that day, in proportion to each tag’s requests and weighted tokens, so the allocated costs always add up to what the provider billed. Customers can then group and filter these costs by team, application, customer, and any other attribute in their logs in Cost Reports, Virtual Tags, Budgets, and Cost Alerts. For example, a customer support company serving an open-weight model on Baseten Dedicated Inference can charge each of its own customers for their share of GPU spend using the customer tag it already logs on every request, while the same tag continues to split its OpenAI and Anthropic costs.
Inference Allocation is generally available to Vantage customers that are contracted for Enrichment. To get started, open the enrichment integration that carries your request logs on the Integrations page, and check Inference allocation in Configure Cost Integrations for each provider you want to allocate. Then set your inference allocation weights. You can learn more in the Inference Allocation documentation.
Frequently Asked Questions
1. What is being launched today?
Vantage is launching Inference Allocation, which allocates the cost of models that are billed by the hour to the teams, applications, and customers that sent requests to them. It uses the same tagged request logs customers already send to Vantage for token allocation, so hourly-billed and token-billed model costs can be reported with the same tags.
2. Who has access to Inference Allocation?
Inference Allocation is generally available to Vantage customers with contracted Build Hours. It is designed for engineering, AI platform, and FinOps teams that run models billed by the hour, whether self-hosted on dedicated deployments or on hosted inference platforms, and need to charge back or show back that spend to the teams, products, or end customers using it. Customers who don’t have contracted Build Hours yet and want to use Inference Allocation can contact sales@vantage.sh to discuss adding Build Hours.
To use it, you need an active Modal, Baseten, Together AI, AWS, Azure, or Google Cloud cost integration, plus request logs for the models you run there, sent to Vantage through a Custom LLM, LiteLLM, Cloudflare AI Gateway, or Bifrost integration, or, for AWS, through Amazon Bedrock model invocation logs.
3. How much does this cost?
Inference Allocation consumes your contracted Build Hours to allocate and tag your provider cost data. The number of Build Hours consumed depends on the number of log records ingested and cost rows processed each month.
As a baseline, a customer with 30M cost rows and 5M log records per month can expect to consume up to about 40 Build Hours per month with daily refreshes.
4. Which providers and log sources are supported?
At launch, Inference Allocation supports hourly-billed costs from Modal, Baseten, and Together AI, as well as AWS, Azure, and Google Cloud:
- Modal: CPU, memory, and GPU compute for each Modal App, matched to your requests by the App name on the cost row.
- Baseten: Dedicated Inference deployment hours, matched by the deployment’s model name. Baseten Model APIs costs are billed per token and continue to use token allocation.
- Together AI: Dedicated Inference GPU hours, both on-demand and reserved, matched by the endpoint name. Together AI Serverless Inference costs are billed per token and aren’t part of Inference Allocation.
- AWS, Azure, and Google Cloud: cost rows metered in hours, GPU-hours, core-hours, or GiB-hours for the models you run in those accounts, matched to your requests by model. On AWS, EC2 instance hours aren’t included. These integrations can also use token allocation for their per-token model costs, such as Amazon Bedrock per-token usage, at the same time (see question 10).
Request logs can come from the Custom LLM, LiteLLM, Cloudflare AI Gateway, or Bifrost integrations on the Integrations page. For AWS, you can also use the Amazon Bedrock model invocation logs you've connected to Vantage: open the log bucket's Manage tab to add Inference allocation for the connected AWS account, and token allocation stays on for that account.
5. How does Inference Allocation work?
Inference Allocation runs as part of each provider’s regular cost import. Vantage groups your request logs by UTC day, provider, and model, then matches each hourly-billed cost row to the requests for the same model on the same UTC day.
Because the requests are logged in tokens and the cost row is billed in hours, Vantage converts each tag combination’s requests into work units using your weights (see question 6), then splits each cost row by each combination’s share of the work units. Each allocated row carries the tags from its requests, plus Managed AI Tags such as vntg:ai:model. The resulting amounts are derived allocations of provider-billed costs, and they always add up to the original cost row.
The model on the cost row doesn’t need to exactly match the model in your logs. Vantage compares the App, deployment, or endpoint name on the bill with the models in your logs that day and picks the closest match. For the most reliable matching, name your Modal Apps, Baseten deployments, and Together AI endpoints after the model they serve, and log that same model name on each request.
6. How are work units calculated?
GPU time isn't measured per request, so Vantage estimates each request's share of a model's compute in work units, calculated from your request logs and your token weights. Token weights are numbers you set for each token type (input, output, cache read, and cache write) to reflect how much compute that type takes to serve on your hardware. They're unrelated to the model weights in "open-weight models." The resulting amounts are allocations of the provider's billed cost and always add up to the billed total. Each request counts as one work unit, plus one weighted unit for every 1,000 tokens of each type:
For example, a Baseten Dedicated Inference cost row for a gemma-3-27b deployment costs $480 for the day and covers 24 GPU hours. Your token weights are 0.25 for input tokens and 1.0 for output tokens. These requests have no cache read or cache write tokens, so those two weights don't change the result. That day's requests for the model fall into two tag combinations:
The two combinations total 28,000 work units, so the single $480 row becomes two rows:
The total is unchanged at $480 and 24 hours. Because each request counts as a work unit, a team that sends many short requests can carry a larger share than its token volume alone suggests.
To choose token weights, a common starting point is how API-based LLM providers price the same token types. Output tokens are generated one at a time and usually cost several times more to serve than input tokens, and cache reads usually cost less than uncached input. A weight of 0.25 on input and 1.0 on output counts each output token as four input tokens. If you don't have a basis for different weights yet, enter 1 for every token type to weight them equally. Teams with their own benchmarks, such as GPU time per token type from load testing, can use those ratios instead. You can change token weights at any time, and Vantage re-allocates past periods to match (see question 9).
7. How do I set up Inference Allocation?
First, connect the Modal, Baseten, Together AI, AWS, Azure, or Google Cloud account whose costs you want to allocate, and send request logs for the models you host there through a Custom LLM, LiteLLM, Cloudflare AI Gateway, or Bifrost integration, or, for AWS, through Amazon Bedrock model invocation logs.
Next, open that integration from the Integrations page and go to its provider selection table. For Custom LLM and Cloudflare AI Gateway, select Review after a new source's scan finishes, or open the Manage tab of an existing source. For LiteLLM, select Configure Providers on a new collector, or select Edit and then Manage Providers on an existing collector. For Bifrost, open the feed's Manage tab and select Configure Providers. For Amazon Bedrock model invocation logs, open the log bucket's Manage tab. The table has two columns, Token allocation and Inference allocation. Select Inference allocation for each Modal, Baseten, Together AI, AWS, Azure, or Google Cloud cost integration you want to allocate, then confirm or save your selection. Vantage preselects Inference allocation for Modal, Baseten, and Together AI integrations it finds the first time it scans your logs.
Finally, set your weights on the Configure Inference Costs page (see question 8). When you confirm a new Custom LLM or Cloudflare AI Gateway source with Inference allocation selected, Vantage takes you straight to that page. To open it later, or after saving a selection on LiteLLM, Bifrost, Amazon Bedrock model invocation logs, or an existing source, return to the integration's source list, open the ellipses (...) menu next to Add Sources, and select Configure Inference Costs. The menu appears once at least one source has Inference allocation turned on. Vantage allocates costs on each provider's next data refresh.
8. How do I set weights on the Configure Inference Costs page?
Each row on the Configure Inference Costs page is one period. Enter a weight for each of the four token types: Input and Output under Input & Output Tokens, and Cache read and Cache write under Cache Tokens. Each weight must be greater than 0. Fields start blank, so enter 1 in every field to weight each token type equally. Select Save Configuration to apply your changes, and Vantage confirms with Inference weights saved.
To use one set of weights for all billing dates, use a single period and leave its Date range blank. To change weights over time, such as after moving a model to different hardware, select Add period to add a row and set a Date range on each row. New periods copy their weights from the latest period. Date ranges for the same token type can't overlap, and a period without a date range can't be combined with other periods.
Weights apply to the whole Vantage account, so every integration with Inference allocation selected uses the same weights, and the page notes when they apply to more than one type of source. Vantage only allocates a billing date when all four weights cover it. Blank weights don't inherit values from other periods, so fill in every weight and leave no gaps between dated periods. Until weights are saved, Inference Allocation doesn't run and costs pass through unallocated, even if you want every token type weighted equally.
9. What happens if I change my weights, including for past periods?
When you save new weights, Vantage automatically re-allocates every billing month affected by the change, back to the start of your account's data retention, for as long as the matching request logs remain available. Only months whose effective weights changed are re-processed, and changes that apply only to future dates take effect going forward. Saving weights for the first time also allocates past billing periods that have matching request logs. You don't need to trigger a re-import. Totals don't change; only the split between tag combinations does.
10. Can one integration use Token Allocation and Inference Allocation at the same time?
Yes. Select both Token Allocation and Inference Allocation for the same cost integration, and Vantage routes each request to one of them using the inference tag on the request. Requests tagged "inference": "true" are allocated against time-billed inference costs, and all other requests are allocated against token-billed costs. Each request is used once, so usage is never counted twice across the two.
This is useful for providers that bill both ways. For example, a Baseten account can use token allocation for Model APIs costs, which are billed per token, and Inference Allocation for Dedicated Inference costs, which are billed by the hour, from a single stream of request logs. The allocation types available depend on the provider:
- OpenAI and Anthropic: Token Allocation only
- Modal and Together AI: Inference Allocation only
- Baseten, AWS, Azure, and Google Cloud: both
If an integration only has Inference allocation selected, such as a Modal integration, you don't need to add the tag.
11. What format does the inference tag use?
Add an inference key to the tags object of each request record, with the string value "true" (case-insensitive) or the boolean true. Requests without the tag, or with any other value, are treated as token requests when both allocation types are selected. Records follow the Token Cost Allocation Specification: event_id, timestamp (UTC, with a Z suffix), provider, model, and a usage object with at least one positive token count are required. For example, a request served by a Baseten Dedicated Inference deployment looks like this:
Every other key in tags, such as team and customer, becomes a tag on the allocated cost rows. If you send logs through LiteLLM, Cloudflare AI Gateway, or Bifrost, add inference alongside the other request metadata you already use for tags.
12. Are idle costs allocated?
Yes. For each model and UTC day, Vantage splits the model's full compute cost for the day across the requests that model served that day, including hours when the deployment was running but idle. Vantage matches cost rows to requests by UTC day, not by hour, so an hour with no requests isn't tracked separately from the rest of the day. This lets teams fully account for the cost of self-hosted inference when building chargeback or showback models of their workloads.
How the split behaves in common situations:
- Days without requests: no costs are allocated. If no requests for a model match a cost row on a given UTC day, that row passes through unchanged, with its original cost and usage, the provider's existing tags, and no request tags. It still counts toward your provider totals, and it appears as untagged when you group by one of your request tag keys in Cost Reports.
- Few or many requests: a model's full daily cost is always split across the requests it served that day, so on a quiet day a small number of requests carry the whole day's cost, and on a busy day the same cost is spread across many more requests.
- Incomplete logs: if your logs cover only part of a model's traffic on a given day, the logged requests carry that day's full cost, so send logs for all of a model's traffic for the most accurate split.
- Leftover rows: if part of a cost row can't be attributed to matching requests, such as when
resource_account_idon your requests points to a different provider account than the cost row, that part stays on a leftover row. The leftover row keeps thevntg:ai:modeltag and the provider's existing tags, but no request tags.
13. What happens if request logs arrive late?
Vantage reprocesses recent days within a rolling three-day window. If request logs for a day arrive within that window, including a day that initially had no matching requests, that day's costs are allocated using the new logs when it's reprocessed.
14. Are usage quantities split the same way as costs?
Yes. When the provider reports a usage quantity on the cost row, such as Baseten Dedicated Inference hours or Together AI GPU hours, Vantage splits that quantity in the same proportions as the cost. In the example in question 6, the 24 GPU hours become 16.29 hours for team=support and 7.71 hours for team=search, and the total stays at 24 hours. Modal cost rows don't include a usage quantity, so only their cost is split.
15. What can I group and filter allocated costs by?
Allocated inference rows carry the following tags, which you can use in Cost Reports, Virtual Tags, Budgets, and Cost Alerts:
- Your request tags, such as
team,application,customer,environment, orfeature vntg:ai:model, the model from your request logs that the cost row matchedvntg:ai:model_provider, the model family, when Vantage recognizes it- The provider's existing tags, such as
modal:description,baseten:deployment, andtogether:capacity_type
Allocated inference rows don't carry the vntg:ai:token_type tag, because each row covers compute time rather than a single token type. See Managed AI Tags for more on Vantage-managed vntg:ai: tags. Use the same tag keys and values across your request logs so one dimension lines up across providers; team and Team are different keys.
16. How are credits handled?
Credits and other negative charges on time-billed inference costs are allocated in the same proportions as positive charges for the same model and day, and they keep their negative sign. Each tag combination's net cost reflects its share of the credit, and totals still reconcile. If no requests match, the credit passes through unallocated.
17. What role do I need in Vantage to set up Inference Allocation?
You must have the Organization Owner or Integration Owner role to select Inference allocation for a cost integration and to save weights on the Configure Inference Costs page.
Other users can view the resulting allocated costs according to their existing workspace and Cost Report permissions.
18. How fresh is the data, and can I allocate historical costs?
Inference Allocation runs during each provider's regular cost import and follows that provider's data refresh cadence. Modal and Baseten costs refresh once daily. Recent days are reprocessed within a rolling three-day window, so late-arriving request logs are picked up automatically.
Past billing periods are allocated when you first save weights and re-allocated when your weights change, back to the start of your account's data retention, as long as matching request logs remain available (see question 9).
19. What happens if I turn off Inference Allocation or stop sending logs?
If you clear Inference allocation for a cost integration, or stop the source that carries its request logs, Vantage stops allocating new costs for it. Costs that were already allocated stay in place, and a stopped source can be resumed at any time. If you stop sending request logs, costs for days without matching requests pass through unallocated.
20. Will Inference Allocation change my provider cost totals?
No. Splits are additive: the allocated rows and any leftover row always add up to the original cost row, so provider-level totals in your existing Cost Reports don't change. Inference Allocation only adds new dimensions to the underlying rows.
21. How can I check how well my requests matched my costs?
Open the source that carries your request logs and select View enrichment history from its ellipses (...) menu. Each run shows Bill Match, the share of provider cost rows that received matching requests. Select the Tokens Kept percentage on a run to open Token Metrics, where the Inference Join section shows how many requests matched your time-billed inference costs and lets you view sample records for requests that didn't.
22. Can I view Inference Allocation sources and results through the Vantage API?
Yes. The Vantage API provides read access to the sources that carry your request logs and to per-run match statistics. Inference allocation selections and weights are set in the console. Use an API token with the read scope that belongs to a user who can manage integrations, such as an Organization Owner or Integration Owner. The following endpoints are available:
GET /v2/enrichment_sourceslists your sources, with each source'stoken,type,title,integration_token,activestatus, andcreated_atdate.GET /v2/enrichment_sources/{enrichment_source_token}returns a single source.GET /v2/enrichment_sources/{enrichment_source_token}/statisticsreturns match statistics for each run, with optionalprovider,page, andlimitquery parameters.
For example, the following request returns the latest statistics for Modal costs on one source:
An abbreviated response looks like this:
Full details are available in the Vantage API documentation.
23. Can I view Inference Allocation sources through Terraform?
Yes. The Vantage Terraform provider includes read-only vantage_enrichment_sources and vantage_enrichment_source data sources, which return the same source details as the API, such as type, title, integration_token, and active. Inference allocation selections and weights are set in the console. The provider uses the same API token requirements described in the previous question. For example:
24. Does Vantage collect prompts or completions?
No. Vantage reads only the provider, model, timestamp, token counts, and the tags you attach to each request. It does not collect or store prompt or completion content or credentials, and this data is not used to train any models.
Sign up for a free trial.
Get started with tracking your cloud costs.
