Repository navigation
Conversation
Hello nicolas2bert,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files
... and 1 file with indirect coverage changes @@ Coverage Diff @@
## development/9.4 #6378 +/- ##
===================================================
+ Coverage 86.73% 86.74% +0.01%
===================================================
Files 213 213
Lines 14631 14630 -1
===================================================
+ Hits 12690 12691 +1
+ Misses 1941 1939 -2
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Incorrect fix versionThe
Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:
Please check the |
| // We replicate further only when the write carries a | ||
| // microVersionId, because this is what stops a replication loop. | ||
| // A write has no x-scal-micro-version-id header only during an | ||
| // upgrade: backbeat sends this header since 9.5.0, when cascaded | ||
| // replication was added. An older backbeat does not send it. | ||
| // In that case we store the object as a replica and we don't | ||
| // replicate it further (CLDSRV-1078). | ||
| // Example: site A is not upgraded yet. Its backbeat sends a new | ||
| // tag for the object key0, with no header. We store the tag on | ||
| // our key0. Our bucket replicates to site A, but we don't send | ||
| // key0 back there. |
There was a problem hiding this comment.
| // We replicate further only when the write carries a | |
| // microVersionId, because this is what stops a replication loop. | |
| // A write has no x-scal-micro-version-id header only during an | |
| // upgrade: backbeat sends this header since 9.5.0, when cascaded | |
| // replication was added. An older backbeat does not send it. | |
| // In that case we store the object as a replica and we don't | |
| // replicate it further (CLDSRV-1078). | |
| // Example: site A is not upgraded yet. Its backbeat sends a new | |
| // tag for the object key0, with no header. We store the tag on | |
| // our key0. Our bucket replicates to site A, but we don't send | |
| // key0 back there. | |
| // Forward only with x-scal-micro-version-id: it prevents replication loops. | |
| // Backbeat < 9.5 doesn't send it (mid-upgrade): store as replica, don't forward (CLDSRV-1078). |
| // We send the tag t=1 for key0, like backbeat does when a tag is set on | ||
| // the source object. microVersionId is the one of the source object. | ||
| // When it is undefined we send no x-scal-micro-version-id header, like | ||
| // a backbeat older than 9.5.0. |
There was a problem hiding this comment.
| // We send the tag t=1 for key0, like backbeat does when a tag is set on | |
| // the source object. microVersionId is the one of the source object. | |
| // When it is undefined we send no x-scal-micro-version-id header, like | |
| // a backbeat older than 9.5.0. | |
| // Tag update as sent by backbeat. No microVersionId: no header, like backbeat < 9.5. |
|
@bert-e approve |
Incorrect fix versionThe
Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:
Please check the The following options are set: approve |
|
ping |
Integration data createdI have created the integration data for the additional destination branches.
The following branches will NOT be impacted:
You can set option The following options are set: approve |
Build failedThe build for commit did not succeed in branch w/9.6/bugfix/CLDSRV-1078/store-replica-without-microversionid-header The following options are set: approve |
Build failedThe build for commit did not succeed in branch w/9.5/bugfix/CLDSRV-1078/store-replica-without-microversionid-header The following options are set: approve |
A backbeat older than 9.5 sends no x-scal-micro-version-id header. putMetadata answered a 500 to its metadata-only updates when the destination bucket replicates further (bidirectional replication), so tags, ACL, legal hold and retention set on a 9.5 site never reached a 10.1 site (S3C-11679). - store such a write as a plain replica instead of refusing it - don't replicate it further: without a microVersionId nothing can stop a replication loop
0574080 to
d50ac7a
Compare
|
/reset |
History mismatchMerge commit #35305c8a9fbdf13b9d23bf637bc3d4bbe2214dc7 on the integration branch It is likely due to a rebase of the branch Please use the The following options are set: approve |
| // tag for the object key0, with no header. We store the tag on | ||
| // our key0. Our bucket replicates to site A, but we don't send | ||
| // key0 back there. | ||
| const forward = hasNextHop && hasMicroVersionId; |
There was a problem hiding this comment.
When hasNextHop && !hasMicroVersionId, the next hop is now skipped with no log line, so during a mixed-version upgrade nobody can tell why a replica was not sent on. Add a log so this case can be seen.
| const forward = hasNextHop && hasMicroVersionId; | |
| const forward = hasNextHop && hasMicroVersionId; | |
| if (hasNextHop && !hasMicroVersionId) { | |
| log.info('putMetadata: no x-scal-micro-version-id, storing replica without next hop', { | |
| bucketName, | |
| objectKey, | |
| }); | |
| } |
|
ping |
|
/status |
Status
The following options are set: approve |
History mismatchMerge commit #35305c8a9fbdf13b9d23bf637bc3d4bbe2214dc7 on the integration branch It is likely due to a rebase of the branch Please use the The following options are set: approve |
|
@bert-e approve |
|
/reset |
Reset completeI have successfully deleted this pull request's integration branches. The following options are set: approve |
|
I have successfully merged the changeset of this pull request
The following branches have NOT changed:
This pull request did not target the following hotfix branch(es) so they
Please check the status of the associated issue CLDSRV-1078. Goodbye nicolas2bert. The following options are set: approve |
setup
Bidirectional replication: bucket A replicates to bucket B, and bucket B replicates to bucket A. Site B is new, site A is still old. A customer with one-way replication is not affected.
what happens
It happens for every such change, as long as site A is old and site B is new. Nothing special has to go wrong. It is the normal state between the two upgrades.
why the new cloudserver B refuses it
Bucket B replicates to bucket A. So when the new cloudserver B stores a replica, it wants to mark it "to replicate further". To do that safely it needs the x-scal-micro-version-id header: this is what stops a replication loop. An old backbeat doesn't send that header, so the new cloudserver B returns a 500.
With one-way replication, bucket B has no rule. The new cloudserver B has nowhere to send the replica, so it accepts the change.
A cascade (bucket A to bucket B to bucket C) would hit the same 500, because bucket B has a rule there too. But cascade is not supported on the old version, so no customer upgrading has that setup.
how we fix it
The new cloudserver B no longer refuses a write that has no header. It stores it as a plain replica, and it doesn't mark it to replicate further.