Tag: inference

3 entries tagged "inference" — 2 posts, 1 link.

Posts

Mar 12, 2026 — 9 min — Platform & AI

The Faster Transformers Stack Behind GPT-OSS

Why Hugging Face's faster Transformers work matters beyond GPT-OSS, and how kernels, MXFP4, parallelism, KV cache, batching, and model loading change practical LLM runtime decisions.

Outcome: Mapped the GPT-OSS-era Transformers runtime features into concrete decisions about memory, compute, cache behavior, batching, and serving boundaries.

transformers gpt-oss hugging face inference model performance

Oct 27, 2025 — 10 min — Platform & AI

In-Warehouse Inference on Snowflake and BigQuery

A practical runbook for scoring changed rows close to the data using Snowflake Streams and Tasks or BigQuery scheduled queries and remote models.

Outcome: Compared Snowflake and BigQuery patterns for scheduled in-warehouse inference, corrected CDC assumptions, and defined monitoring, grants, and deployment checks.

snowflake bigquery mlops inference data engineering

Links

Articlemedium.comMar 19, 2026Permalink

Scaling Inference To Billions of Users And Agents

Federico Iezzi, Google Cloud

This is useful because it connects agent adoption to inference architecture. Agents do not make one call; they fan out across planning, retrieval, tool use, retries, and evaluation, which changes the serving math quickly.

Worth keeping as a scale reference for Google Cloud AI work. The product question is where inference cost becomes a feature constraint rather than a backend detail.

All tags