# Snapshot and Fork put performance

🍴Bucket Forking

Hi, I was testing snapshots and forking and noticed for each additional layer there was a pretty significant increase in latency for different ops. For example, a 1kb put could go from 7-8 ms to 15 ms on a fork and each additional fork caused an additional increase in latency.

My assumptions with how forks work is it's basically like a CoW filesystem. Out of curiosity is there some compaction step that brings down this latency over time or is this additional latency always present with an increased number of layers?

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/16/26, 11:43 AM

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/16/26, 11:43 AM

This is unexpected

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/16/26, 11:44 AM

Which op you are seeing the latency increase?

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•3/16/26, 12:35 PM

I noticed it across all ops

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•3/16/26, 12:36 PM

I was testing against a single region in iad from a hetzner vm

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•3/16/26, 12:42 PM

I'm going to rerun the benchmark and I'll also share the repo with you

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/16/26, 12:42 PM

That would be great

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•3/16/26, 1:41 PM

[https://github.com/vmg-dev/tigris-storage-bench/tree/main](https://github.com/vmg-dev/tigris-storage-bench/tree/main)

I ran the benchmark with the ts and go sdks just to confirm it wasn't an sdk issue

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•3/16/26, 1:42 PM

[https://github.com/vmg-dev/tigris-storage-bench/blob/main/GO_BENCHMARK_RESULTS.md](https://github.com/vmg-dev/tigris-storage-bench/blob/main/GO_BENCHMARK_RESULTS.md)

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•3/16/26, 1:42 PM

Notice how latency climbs as fork depth increases

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/16/26, 6:01 PM

Thanks for sharing this

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/16/26, 6:01 PM

Let me run it as well

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/16/26, 6:53 PM

So I just tested by running this tool

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/16/26, 6:53 PM

I don't see variance in write operations:

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/16/26, 6:54 PM

I do see variance in get results

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/16/26, 7:11 PM

I will get back to you on this

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/16/26, 7:14 PM

And I have added support for `--location``--location` [https://github.com/vmg-dev/tigris-storage-bench/pull/1](https://github.com/vmg-dev/tigris-storage-bench/pull/1)

GitHub

[feat: make bucket location configurable via --location flag by ovai...](https://github.com/vmg-dev/tigris-storage-bench/pull/1)

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•3/16/26, 8:10 PM

Weird, I definitely didnt make up that result but I realize now for any test over a network I need to start testing from multiple datacenters.

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•3/16/26, 8:10 PM

Ill merge your pr once im back to my laptop

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•3/16/26, 8:11 PM

I dont see why the dc would influence this though. I got this result consistently.

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/17/26, 1:34 AM

I am not saying you made up the result. I don't expect overhead in write requests which my tests show

[ot](https://community.tigrisdata.com/u/938129329953591377)•3/17/26, 1:34 AM

While read overhead is clear in your results and mine

[ot](https://community.tigrisdata.com/u/938129329953591377) [I am not saying you made up the result. I don't expect overhead in write request...](https://community.tigrisdata.com/m/1483337732955701300?focus=1483337732955701300)

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•3/17/26, 8:52 AM

I know, just wanted to emphasize it, thanks for looking into this

[ot](https://community.tigrisdata.com/u/938129329953591377)•4/8/26, 3:11 PM

Hi [@vmg](https://community.tigrisdata.com/u/138497924542758912) we have made improvements to the performance of snapshots. Do you mind running your benchmarks again?

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•4/8/26, 8:45 PM

sure ill try running them

[ot](https://community.tigrisdata.com/u/938129329953591377)•4/8/26, 8:50 PM

Cool

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•4/9/26, 12:44 AM

It seems like the perf hit is still there for deep forks

[vmg](https://community.tigrisdata.com/u/138497924542758912) OP•4/9/26, 12:44 AM

Writes are unaffected by fork depth but reads are

[ot](https://community.tigrisdata.com/u/938129329953591377)•4/9/26, 11:26 AM

I see

[himank](https://community.tigrisdata.com/u/1028933275726651424)•4/13/26, 3:26 PM

[@vmg](https://community.tigrisdata.com/u/138497924542758912) we are looking into it actively, should've an update for you this week.

[himank](https://community.tigrisdata.com/u/1028933275726651424)•4/22/26, 12:48 PM

[@vmg](https://community.tigrisdata.com/u/138497924542758912) do you want to give it a try? We rolled out improvements for deep forks.
