Tigris Regions and x-tigris-regions header

Hey!

I have both a request and a question about Tigris regions.

The regions page for Tigris mentions Tigris is deployed in Singapore. I have a server in Singapore which debug.fly.dev shows hitting SIN:

=== Headers ===
Host: debug.fly.dev
Fly-Request-Id: 01J16MRRX4PZYF27K629PY04A9-sin
…
Fly-Region: sin

=== Headers ===
Host: debug.fly.dev
Fly-Request-Id: 01J16MRRX4PZYF27K629PY04A9-sin
…
Fly-Region: sin

However, in the Grafana Metrics Panel for Tigris I don’t see any requests hitting the sin location. It would be really nice if there was more information exposed in Tigris’s response headers (edge location hit, storage location retrieved from) for debugging. Based on the latency (~250 ms) even after a few requests to a 10 KB object, I assume I’m going somewhere else. I do ~1‑2 ms when ICMP pinging the Tigris endpoint directly.

Secondly, I forced my object to be in all regions via the x-tigris-regions header, and I see some curious information being returned sometimes:

< x-tigris-regions: fra,syd,sjc,hkg,jnb,lhr,mad,iad,ord,gru,nrt

< x-tigris-regions: fra,syd,sjc,hkg,jnb,lhr,mad,iad,ord,gru,nrt

Sometimes without sin?

< x-tigris-regions: fra,gru,jnb,nrt,mad,hkg,iad_mr,ord,lhr,syd,sjc_mr

< x-tigris-regions: fra,gru,jnb,nrt,mad,hkg,iad_mr,ord,lhr,syd,sjc_mr

Sometimes with _mr?

< x-tigris-regions: fra,sjc_mr,sin,gru,syd,iad,ord,jnb,lhr,nrt,mad

< x-tigris-regions: fra,sjc_mr,sin,gru,syd,iad,ord,jnb,lhr,nrt,mad

Sometimes sin comes back?

What’s with that, and what does _mr mean?

Thanks for your help!

Solution

Hey, thanks for reporting this. We will debug what’s going on with the requests to your bucket in the SIN region.

Regarding the *_mr value reported in the x-tigris-regions header, that is something internal and should not have been reported in the header value. We will fix that.

Thanks for fixing the _mr header, those look gone now!

Any response on more information exposed in Tigris’s response headers? Edge location hit, storage location retrieved from, would be really nice for debugging.

Even just the edge location hit would be nice, somewhere in the headers where it’d be parsable (part of the server or a separate custom header like x-tigris-region). It’d be helpful at just understanding a bit of what’s going on and if the latency is reasonable.