Tag: gpt-oss

A production-oriented Cloud Run and vLLM design that keeps GPT-OSS private, separates routing from inference, and makes model loading, readiness, concurrency, and rollback explicit.

Reader outcome: Produced a deployable service contract for private OpenAI-compatible inference with IAM authentication, pinned artifacts, measured concurrency, and a scale-to-zero operating path.

A practical MLX-first recipe for experimenting with openai/gpt-oss-20b on a 64GB Apple Silicon Mac without confusing local LoRA work for CUDA-scale training.

Reader outcome: Defined a local 64GB MacBook Pro fine-tuning path for GPT-OSS 20B that prioritizes Harmony formatting, MLX quantized LoRA, small evals, and a clear fallback to NVIDIA when scale is required.

Why Hugging Face's faster Transformers work matters beyond GPT-OSS, and how kernels, MXFP4, parallelism, KV cache, batching, and model loading change practical LLM runtime decisions.

Reader outcome: Mapped the GPT-OSS-era Transformers runtime features into concrete decisions about memory, compute, cache behavior, batching, and serving boundaries.