Today we're announcing CougarLLM, a globally distributed inference server built on Tigris. It takes any open-weight model, distributes its weights globally, and serves inference from whatever region is closest to your users. Zero egress fees. One endpoint. No configuration. It's so global, that even your fallback region has a fallback region. In our absolutely unbiased internal benchmark, CougarLLM significantly improves inference latency and reduces serving cost. We will be releasing the benchmark code within the week. With CougarLLM, we are stepping onto the same stage as vLLM, TGI, Triton, and SGLang. They are the household names in modern inference serving, the systems every team evaluates before production traffic gets real. CougarLLM belongs in that conversation, but with a dramatic twist: it treats global object storage as part of the runtime itself. If you want the practical version of that story, start with [multi-region bucket placement](/content/blog/multi-region-dual-region-buckets/index.html). _Figure 1: CougarLLM enters the inference group chat with vLLM, Hugging Face_ _TGI, NVIDIA Triton, and SGLang, arriving fashionably late but with very_ _strong opinions about global storage._ ## Why global inference gets painful fast Most teams start in one region, then add a second when latency complaints become unavoidable. At that point, users in Tokyo and users in Frankfurt are still hitting infrastructure that was never designed to feel local everywhere. The hard part is not one inference server. The hard part is keeping giant model weights and hot cache state close to users across regions without paying a small fortune in transfer and operational overhead. This is exactly the kind of data placement problem Tigris already solves for objects. > We almost shipped yet another agent framework, mostly out of peer pressure. > Then we noticed teams were burning money shuffling model weights worldwide, so > we figured an inference server is a safer bet. > > \- Ovais Tariq, CEO & Co-founder, Tigris Data Like vLLM, TGI, Triton, and SGLang, CougarLLM serves inference requests. Unlike most inference servers, it treats globally distributed object storage as part of the runtime for weights, cache, and rollouts. You bring weights in GGUF, SafeTensors, or PyTorch format, upload once to Tigris, and serve globally from there. No region-by-region file choreography. ## A simple architecture diagram The server works seamlessly. Users send requests to one CougarLLM global endpoint. That endpoint picks the nearest healthy region for each request. At the same time, model weights replicate out from a single Tigris bucket so each region can serve locally. The result is straightforward in theory and chaotic in practice: one URL, local-ish latency, and no regional replica babysitting. CougarLLM routes each request to the nearest region while model weights replicate from a single Tigris bucket. User in Tokyo CougarLLM Global Endpoint User in São Paulo User in Frankfurt Weights + KV Cache Weights + KV Cache Weights + KV Cache Tigris Bucket (single source of truth) _Figure 2: CougarLLM request flow. One global endpoint, nearest-region_ _routing, and model weights replicating from a single Tigris bucket like this_ _was always a totally normal thing to do._ **Global weight distribution.** In this CougarLLM design, Tigris replicates model weights to regions where requests originate. If traffic shifts from Europe to Asia, the weights follow. No manual replication, no CDN gymnastics, just [dynamic data placement](/content/docs/concepts/regions/index.html) applied to tensors. **Single global endpoint.** Your app hits one URL, and CougarLLM routes each request to the nearest node with warm weights. No region picker, fewer latency spreadsheets, fewer haunted dashboards. **Zero egress on weight distribution.** Replicating 70B weights across regions usually costs real money. On Tigris, [data transfer is free](/content/docs/account-management/billing/index.html). CougarLLM inherits that, so global replication can be aggressively boring from a billing perspective. > Everyone else is trying to cash in on inference, so this is our extremely > tasteful attempt to do the same. > > \- David Myriel, Head of Trolling at Tigris Data ## Why KV cache becomes chaos at global scale KV cache is why long conversations get expensive fast. It grows with context, it is session-specific, and it usually gets trapped on one machine. In CougarLLM, Tigris holds durable KV state while inference nodes keep the hot working set local for low-latency decoding. That means cross-region recovery can reuse more context instead of replaying everything from scratch. No, we are not streaming token-by-token KV reads from object storage. ## Experiment with models without duplicating everything A 70B checkpoint is large enough that naive copying for every experiment gets expensive immediately. CougarLLM uses [Bucket Forks](/content/docs/forks/index.html) so you clone instantly and pay for deltas, not for 40 near-identical copies and a prayer. For the non-joke deep dive, read [Fork Buckets Like Code](/content/blog/fork-buckets-like-code/index.html) and [Bucket Forking Deep Dive](/content/blog/bucket-forking-deep-dive/index.html). ## Ship model updates without breaking production CougarLLM supports **shadow deployments** using the same [shadow bucket](/content/docs/migration/index.html) mechanism Tigris offers for storage migrations. Route a percentage of inference traffic to the new model, compare outputs, and promote it when you're confident. Rollback is instant. The old weights are still there, you're just changing which fork gets traffic, not performing emergency archaeology at 2 a.m. If you want the production pattern, see [How to Migrate to Tigris with Shadow Buckets](/content/blog/shadow-bucket/index.html). > CougarLLM is not real. We invented it with complete confidence, three coffees, and a suspiciously detailed architecture diagram. But everything it relies on is real. [Global distribution](/content/docs/concepts/regions/index.html), [zero egress fees](/content/docs/account-management/billing/index.html), [bucket forks](/content/docs/forks/index.html), [shadow buckets](/content/docs/migration/index.html), and [S3 compatibility](/content/docs/api/s3/index.html) are all shipping features of Tigris today. If this post made you think "I actually want that," the building blocks are already here, no fake big-cat branding required. Want global storage for your real AI stack? Store model weights, datasets, and artifacts close to your users with globally distributed, S3-compatible object storage and zero egress fees. [Get Started with Tigris](/content/docs/get-started/index.html) - [Why global inference gets painful fast](/content/blog/cougarllm/#why-global-inference-gets-painful-fast/index.html) - [A simple architecture diagram](/content/blog/cougarllm/#a-simple-architecture-diagram/index.html) - [Why KV cache becomes chaos at global scale](/content/blog/cougarllm/#why-kv-cache-becomes-chaos-at-global-scale/index.html) - [Experiment with models without duplicating everything](/content/blog/cougarllm/#experiment-with-models-without-duplicating-everything/index.html) - [Ship model updates without breaking production](/content/blog/cougarllm/#ship-model-updates-without-breaking-production/index.html)