Cloud Run GPU or Model API: The Break-Even Math

The spreadsheet said the open model would save seventy percent. The GPU invoice said otherwise.

The calculation had divided monthly API spend by the advertised price of a GPU. It had not included required CPU and memory, warm idle time, failed private attempts, hosted fallbacks, model-loading time, or the fact that a single GPU could not finish the assumed workload inside the purchased hours.

The arithmetic was correct. The workload model was imaginary.

This is part 3 of a four-part series:

1. Hybrid AI Model Routing Is an Operating Boundary
2. A Private GPT-OSS Service on Cloud Run
3. Cloud Run GPU or Model API: The Break-Even Math
4. Where Open Models Fit in DigDeepIQ's Acreage-Reporting Workflow

All prices in this post are public list-price snapshots checked on July 18, 2026. They are inputs to a model, not promises about a future invoice. Region, free-tier credits, discounts, data residency, batch tiers, storage, logging, network traffic, and negotiated contracts can change the result.

Start With the Capacity Floor#

Cloud Run currently offers NVIDIA L4 and NVIDIA RTX PRO 6000 Blackwell GPUs for services. A GPU service uses instance-based billing. The GPU, CPU, and memory are billed for the life of the instance, including idle time while that instance exists. The service can scale to zero, but a configured minimum instance is charged while idle.

Using the default Cloud Run instance-based list rates shown for Tier 1 regions:

Cloud Run requires at least 4 vCPU and 16 GiB of memory for an L4. It requires at least 20 vCPU and 80 GiB for an RTX PRO 6000.

The baseline hourly calculations are:

L4, no zonal redundancy
= (GPU + 4 vCPU + 16 GiB) * 3,600
= (0.0001867 + 4*0.000018 + 16*0.000002) * 3,600
= $1.04652 per active instance-hour
 
L4, zonal redundancy
= $1.42164 per active instance-hour
 
RTX PRO 6000, no zonal redundancy
= $3.186792 per active instance-hour
 
RTX PRO 6000, zonal redundancy
= $3.920868 per active instance-hour

The private GPT-OSS deployment in part 2 starts with 32 GiB of system memory for more headroom. That L4 configuration costs approximately **$1.16172peractivehour\ast \ast ,not$1.04652. The lower figure is useful as a comparable hardware floor.

At 730 hours in an average month:

Those values are gross list-price estimates before any applicable Cloud Run free-tier credits. They also exclude Cloud Storage, Artifact Registry, build minutes, logging, networking, and the CPU routing gateway. The gateway is usually small compared with the GPU, but "small" is not the same as zero.

Compare the Same Request#

Token-priced APIs need a workload shape.

For the examples below, one request has:

• 2,000 uncached input tokens;
• 500 output tokens;
• no web search, code execution, or other priced tool call;
• standard synchronous pricing rather than batch or priority pricing.

The current public rates used here are:

The request formula is:

request cost = input tokens / 1,000,000 * input rate
             + output tokens / 1,000,000 * output rate

At 100,000 requests per month, the simple API totals are:

This table does not prove that one model can replace another. It only prices the same token shape. Quality, latency, context behavior, tool use, and structured-output accuracy still require task-specific evaluation.

The First Break-Even Test#

Ignore fallbacks for one minute. Divide the L4 hourly floor by the API request cost:

This is the first useful surprise.

A private L4 route needs only about 78 of the example requests per active hour to beat Sonnet's token price. It needs about 419 per hour to beat Gemini Flash. If the workload arrives slowly across the whole day and keeps the GPU alive, the cheaper hosted API can win despite having no open weights.

At an always-warm 730-hour floor, the request counts become approximately:

Scale-to-zero and scheduled batches can change the picture because they reduce active hours. They do not remove the throughput requirement.

Throughput Is Part of the Price#

Suppose an L4 delivers an aggregate 30 output tokens per second for the real request mix. The $1.04652hourlyfloorbuys108,000outputtokens,orapproximately$9.69 per million aggregate output tokens.

If batching and concurrency increase aggregate output throughput, capacity gets cheaper:

This is not directly equivalent to a provider's output-token rate because the GPU is also processing input, caching prefixes, and serving whole requests. It is a sensitivity table. It shows why a benchmark at concurrency one can make self-hosting look expensive and why an unrealistic concurrency assumption can make it look free.

For 100,000 example requests completed in 200 active hours, the service must process:

requests per hour: 500
output tokens per second: 69.4
input plus output tokens per second: 347.2

Whether one L4 can satisfy that workload with acceptable p95 latency and quality is an empirical question. The cost model is invalid until the load test answers it.

Fallback Creates Double-Spend#

Assume the private route handles 85 percent of requests successfully and 15 percent fall back to the hosted API. Also assume the L4 is active for 200 hours at the minimum configuration.

Private compute costs:

200 hours * $1.04652 = $209.30

The hybrid monthly comparison becomes:

Against Gemini Flash, the theoretical saving is only $3.20beforestorage,networking,engineering,monitoring,andreview.Withthe32GiBdeploymentconfiguration,theprivatecomputeportionrisestoabout$232.34 and the hybrid route becomes more expensive.

Against Sonnet, the margin is large enough to fund meaningful operating work if GPT-OSS passes the task-quality gate. Against GPT-5.4 mini, the margin exists but can disappear when active hours or fallback rise.

The break-even request rate with a 15 percent fallback is:

break-even requests/hour
= L4 hourly cost / (hosted request cost * private acceptance rate)

Fallback quality is good product behavior. Hidden fallback cost is bad financial analysis.

A Reusable Calculator#

This calculator keeps the assumptions visible:

type Workload = {
  requests: number;
  inputTokensPerRequest: number;
  outputTokensPerRequest: number;
  privateAcceptanceRate: number;
  activeGpuHours: number;
};
 
type ApiPrice = {
  inputPerMillion: number;
  outputPerMillion: number;
};
 
export function compareHybridCost(workload: Workload, api: ApiPrice, gpuHourlyCost = 1.04652) {
  const apiCostPerRequest =
    (workload.inputTokensPerRequest / 1_000_000) * api.inputPerMillion +
    (workload.outputTokensPerRequest / 1_000_000) * api.outputPerMillion;
 
  const allHosted = workload.requests * apiCostPerRequest;
  const fallbackRequests = workload.requests * (1 - workload.privateAcceptanceRate);
  const privateCompute = workload.activeGpuHours * gpuHourlyCost;
  const hybrid = privateCompute + fallbackRequests * apiCostPerRequest;
 
  return {
    apiCostPerRequest,
    allHosted,
    privateCompute,
    fallbackRequests,
    hybrid,
    savingsBeforeOverhead: allHosted - hybrid,
  };
}
 
console.log(
  compareHybridCost(
    {
      requests: 100_000,
      inputTokensPerRequest: 2_000,
      outputTokensPerRequest: 500,
      privateAcceptanceRate: 0.85,
      activeGpuHours: 200,
    },
    { inputPerMillion: 0.75, outputPerMillion: 4.5 },
  ),
);

I would extend it with:

• storage and model transfer;
• gateway compute;
• log ingestion and trace retention;
• build and vulnerability-scanning cost;
• engineering and on-call allocation;
• separate cold and warm latency;
• retry and timeout rates;
• batch API discounts;
• cached-input rates;
• provider contract discounts;
• the cost of human review and correction.

Cost Per Accepted Business Result#

Tokens are not the final denominator for DigDeepIQ.

DigDeepIQ's public launch story describes a business problem whose completion time could range from minutes to days depending on the acreage-reporting case. USDA describes acreage reports as consequential records containing acreage, planting dates, shares, land identifiers, practices, and types. A cheap model call that creates a material correction does not lower the cost of the report.

For a document workflow, I want:

cost per accepted report
= (model compute
  + hosted fallback
  + deterministic processing
  + human review
  + correction and rework)
  / accepted reports

Consider two illustrative routes:

These are illustrative numbers, not DigDeepIQ measurements. They show why optimizing model cost alone can increase business cost. Better provenance can make a more expensive inference route cheaper by reducing review and correction.

The cost ledger should therefore join technical and workflow metrics:

request_id
task_class
private_model_attempt_cost
hosted_fallback_cost
private_acceptance
review_seconds
correction_required
material_error
final_disposition

The finance question becomes: which route produces the lowest cost per accepted, auditable result at the required service level?

Three Operating Patterns#

Bursty background processing#

Reports arrive in a queue, latency is measured in minutes, and work can be batched. Scale the GPU from zero, drain the queue with useful batching, and return to zero. This is the strongest Cloud Run GPU cost shape because active hours align with work.

User-facing intermittent traffic#

Requests arrive throughout the day and users expect a fast first token. The GPU may remain alive between requests or repeatedly cold-start. A low-cost hosted API can beat private capacity even at meaningful monthly volume. Scheduled warm windows can help when business hours are predictable.

Sustained high utilization#

The GPU remains busy and aggregate throughput is strong. Private capacity can beat expensive hosted routes decisively. At this point, compare Cloud Run with other serving options too; a continuously utilized workload may benefit from reservations, managed model endpoints, GKE, or Compute Engine economics.

Cloud Run is attractive because it makes zero-to-one and bursty operation simple. It is not guaranteed to be the cheapest home for a permanently saturated GPU.

The Decision Gate#

I would approve the private route only when all five statements are true:

1. The task-specific evaluation clears the quality and provenance threshold.
2. The load test proves the assumed throughput at the required p95 latency.
3. The observed private acceptance rate keeps fallback below the budget.
4. Active hours and scale-down behavior produce a margin after operating overhead.
5. Cost per accepted business result improves, not only cost per model attempt.

If the calculation works only by assuming perfect private acceptance, maximum batching, zero idle time, and no engineering cost, it does not work.

Run the model in shadow traffic. Measure the workload. Replace every spreadsheet assumption with an observed distribution. Then decide whether the private route earns production traffic.

Next: Where Open Models Fit in DigDeepIQ's Acreage-Reporting Workflow.

Sources#

• Cloud Run pricing
• Cloud Run GPU support
• Cloud Run GPU inference best practices
• OpenAI model pricing
• OpenAI GPT-5.4 mini
• Anthropic Claude API pricing
• Gemini Developer API pricing
• Introducing DigDeepIQ
• USDA RMA: Getting Acreage Reporting Right