archival: Disable cross-term compaction

[Lazin]

Currently, the segment_collector can collect data from segments generated in different terms. The component is used to generate compacted reuploads. This commit fixes this by disabling the cross term segment collection.

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v24.3.x
• v24.2.x
• v24.1.x

Release Notes

• none

[WillemKauf]

Currently, the segment_collector can collect data from segments generated in different terms.

Can you add some context as to why this is a problem/why this behavior needs changing/fixing?

[Lazin]

@WillemKauf it's needed to handle OffsetForLeaderEpoch requests correctly. If we're merging two segments produced in two different terms we're loosing information about the term. In this case we can't reply to the request correctly.

[WillemKauf]

Thanks, the context as to why this is needed might be helpful in either the commit message or the comment above the function declaration, for posterity.

Nice test.

[vbotbuildovich]

Retry command for Build#59728

please wait until all jobs are finished before running the slash command

/ci-repeat 1
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":2,"enable_failures":false,"mixed_versions":false,"with_iceberg":true,"with_tiered_storage":false}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":2,"enable_failures":false,"mixed_versions":false,"with_iceberg":true,"with_tiered_storage":true}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":2,"enable_failures":true,"mixed_versions":false,"with_iceberg":true,"with_tiered_storage":false}
tests/rptest/tests/random_node_operations_test.py::RandomNodeOperationsTest.test_node_operations@{"cloud_storage_type":2,"enable_failures":true,"mixed_versions":false,"with_iceberg":true,"with_tiered_storage":true}

[vbotbuildovich]

CI test results

test results on build#59728

test_id	test_kind	job_url	test_status	passed
rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=False.mixed_versions=False.with_tiered_storage=False.with_iceberg=True.cloud_storage_type=CloudStorageType.ABS	ducktape	https://buildkite.com/redpanda/redpanda/builds/59728#0193c0f2-7195-46b8-9bf9-c04ad747bf08	FAIL	0/1
rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=False.mixed_versions=False.with_tiered_storage=True.with_iceberg=True.cloud_storage_type=CloudStorageType.ABS	ducktape	https://buildkite.com/redpanda/redpanda/builds/59728#0193c0f2-7195-46b8-9bf9-c04ad747bf08	FAIL	0/1
rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=True.mixed_versions=False.with_tiered_storage=False.with_iceberg=True.cloud_storage_type=CloudStorageType.ABS	ducktape	https://buildkite.com/redpanda/redpanda/builds/59728#0193c0f2-7195-46b8-9bf9-c04ad747bf08	FAIL	0/1
rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=True.mixed_versions=False.with_tiered_storage=True.with_iceberg=True.cloud_storage_type=CloudStorageType.ABS	ducktape	https://buildkite.com/redpanda/redpanda/builds/59728#0193c0f2-7195-46b8-9bf9-c04ad747bf08	FAIL	0/1
test_archival_service_rpfixture.test_archival_service_rpfixture	unit	https://buildkite.com/redpanda/redpanda/builds/59728#0193c0ac-8b14-4759-b0b0-7e0ce61fc7d0	FAIL	0/2
test_archival_service_rpfixture.test_archival_service_rpfixture	unit	https://buildkite.com/redpanda/redpanda/builds/59728#0193c0ac-8b15-4542-bf08-c8ce72305901	FAIL	0/2

test results on build#61294

test_id	test_kind	job_url	test_status	passed
rptest.tests.archival_test.ArchivalTest.test_single_partition_leadership_transfer.cloud_storage_type=CloudStorageType.ABS	ducktape	https://buildkite.com/redpanda/redpanda/builds/61294#0194ad3d-4638-4897-ac9a-81fd9ce8af12	FLAKY	1/2
rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=False.mixed_versions=True.with_tiered_storage=True.with_iceberg=False.with_chunked_compaction=True.cloud_storage_type=CloudStorageType.ABS	ducktape	https://buildkite.com/redpanda/redpanda/builds/61294#0194ad3d-4638-4897-ac9a-81fd9ce8af12	FLAKY	1/2
rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=True.mixed_versions=True.with_tiered_storage=True.with_iceberg=False.with_chunked_compaction=True.cloud_storage_type=CloudStorageType.ABS	ducktape	https://buildkite.com/redpanda/redpanda/builds/61294#0194ad3d-4638-4897-ac9a-81fd9ce8af12	FLAKY	1/2
storage_single_thread_rpunit.storage_single_thread_rpunit	unit	https://buildkite.com/redpanda/redpanda/builds/61294#0194acf4-4240-4b3d-8d70-77673fd3262d	FLAKY	1/2
test_archival_service_rpfixture.test_archival_service_rpfixture	unit	https://buildkite.com/redpanda/redpanda/builds/61294#0194acf4-4240-4b3d-8d70-77673fd3262d	FAIL	0/2

[andrwng]

@WillemKauf it's needed to handle OffsetForLeaderEpoch requests correctly. If we're merging two segments produced in two different terms we're loosing information about the term. In this case we can't reply to the request correctly.

What was the application-facing behavior before? This feels like it deserves a release note

[dotnwat]

Currently, the segment_collector can collect data from segments generated in different terms. The component is used to generate compacted reuploads.

The following refers to the former

This commit fixes this by disabling the cross term segment collection.

But I'm not able to extract out the problem that is fixed from the former description. Can you elaborate a little bit on what is being fixed?

EDIT: i see @WillemKauf asked a similar question. can you update the cover letter please?