[DOCFIX] Add warning for distributedCp limitations

Cherry-pick of existing commit. orig-pr: #18608 orig-commit: f3cf054 orig-commit-author: Rico Chiu <[email protected]> pr-link: #18609 change-id: cid-be4807b887956808990a5094b4afee63780bfd84
Alluxio · May 15, 2024 · 28eba92 · 28eba92
1 parent 0546554
commit 28eba92
Showing 1 changed file with 6 additions and 0 deletions.
diff --git a/docs/en/operation/User-CLI.md b/docs/en/operation/User-CLI.md
@@ -847,6 +847,12 @@ Please wait for command submission to finish..
 Submitted migrate job successfully, jobControlId = JOB_CONTROL_ID_2
 ```
 
+Please note below are known limitations for the distributed copy command.
+- Limited Scalability: No more than 1 million total number of files should be moved concurrently. Note that a copy job may stay active for a short period after the last file is copied.
+- Manual Integrity Validation: Verification between source and destination files relies on the response code from the underlying data lake storage. In case the response code is unreliable, we recommend manual verification of source and destination checksums.
+- Manual Cleanup: In certain failure scenarios, a user may need to manually remove partially written contents in destination directories and restart the failed jobs.
+- Limited Observability: Status checks are limited to using the command line for each job individually.
+
 ### du
 
 The `du` command outputs the total size and amount stored in Alluxio of files and folders.