Checksum support on multipart upload S3 compatibility?

Hi Tigris team - does Tigris support the x-amz-checksum methods to validate the uploaded content? I just tried this with CreateMultipartUpload / UploadPart / CompleteMultipartUpload and it seems like the headers are accepted and pushed through, but the data is not rejected with 400 BadDigest if the checksum is mismatched. A regular PutObject with a mismatched Content-MD5 header does return 400 BadDigest, but not with a mismatched crc32.

Is there any way to do checksums on multipart uploads?

Solution

Thanks for flagging this, I do see your point about the current behavior with flexible checksums misleading. Let me relay your message to the team. We are not treating those headers as enforced integrity checks on multipart uploads today and responses may still echo them.

Yes, you could use each part’s ETag from UploadPart and send those back on CompleteMultipartUpload. That ties completion to the bytes the service stored. For a single-shot object, PutObject + Content-MD5 is the header-based digest that this stack explicitly enforces with BadDigest.

Great, thanks - to make sure I fully understand, I need to manually verify the returned etag matches the locally computed MD5, there is no rejection path from UploadPart or CompleteMultipartUpload?

Also, just as a piece of feedback, just blindly accepting x-amz-checksum-* and passing it through even if the checksum mismatches (IMO) violates the principle of least surprise, ideally it would be rejected as unsupported or perhaps the header is just not persisted through. Luckily I was able to just get claude to build a bunch of test cases to validate this is the case so I didn't lose too much time on it, but if I was trusting those checksums to validate my data I could have been in trouble later down the track.

Solution

For reliability, I'd use the ETags returned from UploadPart and CompleteMultipartUpload, that's what common SDKs also do.

Awesome job on the testing and clear write-up btw, very helpful.