Add k8s membership #18471
Open
Add k8s membership #18471
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
### What changes are proposed in this pull request? - remove redundant info, add redirect link to section on local cache quick start doc - update external links to open in new tab ### Why are the changes needed? better user journey ### Does this PR introduce any user facing changes? n/a pr-link: Alluxio#18083 change-id: cid-062ece308861d9ccb90f85f10e8e43e858b87d5f
Reduce CPU overhead by replacing while loop with await. The previous version waits for finishing sending data by using a **while loop**. In that way, cpu overhead would be pretty high. Thus, instead of waiting for finishing sending data with a **while loop**, we use a **BlockingQueue** to notice it when all data have been sent out successfully. pr-link: Alluxio#18068 change-id: cid-19a79070508e464299ff61645d711e9bc498654f
### What changes are proposed in this pull request? We will make copy existing directory fail the validation, which will skip copy the directory. But in the meanwhile, we shall pass the validation for move, since we want to delete any existing moved directory. ### Why are the changes needed? bugs from e2e tests. ### Does this PR introduce any user facing changes? na pr-link: Alluxio#18085 change-id: cid-7619ee0b6902dd95ae520a70631ddffc31b7670c
### What changes are proposed in this pull request? rearrange nav menu to move: - fuse sdk docs under reference/client apis/posix api - presto sdk under compute/presto - glossary & troubleshooting under reference ### Why are the changes needed? better doc flow ### Does this PR introduce any user facing changes? n/a pr-link: Alluxio#18084 change-id: cid-be60f2eb69f210c0d2044b1e2ecf9e2fa28f062a
New cli for collectInfo does not seem to work, fix the bug to keep the arguments in line with the Java code. pr-link: Alluxio#18080 change-id: cid-2a677899ad0964d79349e287b170afa0715c6cf7
### What changes are proposed in this pull request? Improve pom module name pr-link: Alluxio#18032 change-id: cid-c1ba348d25318ff0581e2bcbc5748bba1602631e
### What changes are proposed in this pull request? Worker checks UFS rename operation result and throws expcetion if failed ### Why are the changes needed? The rename operation result is omitted before. If operation fails, Alluxio client does not know the result. ### Does this PR introduce any user facing changes? N/A pr-link: Alluxio#18089 change-id: cid-d6f93fe33adb42d2f9f5500a170cb2e15ab27aba
### What changes are proposed in this pull request? Remove tiered identity ### Why are the changes needed? This feature is no longer supported ### Does this PR introduce any user facing changes? N/A Entry points have been removed previously. This PR removes the src code pr-link: Alluxio#18033 change-id: cid-92a6d61ad51f12b708721e26ceab063359b5f2e5
### What changes are proposed in this pull request? Fix alluxio-fuse bug with `/tmp` symlink on macOS On macOS, the directory `/tmp/` is a symbolic link to `/private/tmp/`. This caused the `fuse_mounted` function to fail when attempting to mount a UFS to `/tmp/mnt`, as the `mount` command outputs the path as `/private/tmp/mnt`. This commit corrects this issue by resolving the symlink before checking the mount status. ### Why are the changes needed? Fix this bug which fails to detect if alluxio-fuse successfully mounts a remote UFS ### Does this PR introduce any user facing changes? NA pr-link: Alluxio#18093 change-id: cid-60a42e81b3c5f6ffe3f75b733817f4a58ec047cc
### What changes are proposed in this pull request? should use interface factory for membership mgr module for scheduler at time of master startup, otherwise setting STATIC membership mgr will not start master. ### Why are the changes needed? if alluxio.worker.membership.manager.type is set to STATIC, master won't be able to start as it needs etcd related info. ### Does this PR introduce any user facing changes? N/A pr-link: Alluxio#18096 change-id: cid-c052114c1aef013049dcf9fccaded4663bf2dd8a
This PR fixed the shaded client version problem in dora example pom.
Current version in dora/example/pom.xml written as `<version>2.9.0</version>`. As a result, all updates on the shaded client after 2.9.0 won't work (e.g. removal of swift ufs). Now it is written as `<version>${project.version}</version>`
pr-link: Alluxio#18100
change-id: cid-0cb94bb5a3360b3cd347c2c09c7c741517ce8647
Run `bash -c java -version` might be not work on remote machines, reporting the following bug. ``` java.lang.RuntimeException: Failed to successfully complete the job: Task execution failed: Error: error defining alluxio environment --- at /cli/src/alluxio.org/cli/launch/launch.go:49 (Run.func1) --- Caused by: error checking java version compatibility --- at /cli/src/alluxio.org/cli/env/env.go:132 (InitAlluxioEnv) --- Caused by: error finding java version from `/usr/lib/jvm/java/bin/java -version` --- at /cli/src/alluxio.org/cli/env/env.go:246 (checkJavaVersion) --- Caused by: exec: "bash": executable file not found in $PATH ``` Run `java -version` directly can solve this problem. pr-link: Alluxio#18105 change-id: cid-4bd1baf636c25fe4ef6614a4f58d3d27d9485ddc
### What changes are proposed in this pull request? The reverse resolution should follow the same pattern as defined in `DoraCacheFileSystem.convertAlluxioPathToUfsPath()` and be made public. This further allows inheritance and overriding. pr-link: Alluxio#18106 change-id: cid-072732ed2bc3bd3889e173c4cda4000e8860d35b
remove the indentation in sitemap and manage indentation in css pr-link: Alluxio#18087 change-id: cid-40d1e854933ab1502e8976c9335d3e8b6a1b42ec
Handle the reserved character ':' for HTTP server. pr-link: Alluxio#18104 change-id: cid-3134592ee4256cf3ee1396117ae02bbaf55d9c36
### What changes are proposed in this pull request? Please outline the changes and how this PR fixes the issue. ### Why are the changes needed? Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, describe the bug. ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including 1. change in user-facing APIs 2. addition or removal of property keys 3. webui pr-link: Alluxio#18112 change-id: cid-7f60491f8a152bf9d1ba4319d2558b1e0ac7ac4c
### What changes are proposed in this pull request? Please outline the changes and how this PR fixes the issue. ### Why are the changes needed? Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, describe the bug. ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including 1. change in user-facing APIs 2. addition or removal of property keys 3. webui pr-link: Alluxio#18113 change-id: cid-586e7941f2f4b737f0b0ebe7c15cb13d182e6d0c
### What changes are proposed in this pull request?
Return FileNotFound when a file is being create/write. No need to refresh its metadata.
Metadata will be refreshed when file is closed.
### Why are the changes needed?
If a file is being written to, and another getStatus() comes for such file,
this file probably has not been created in UFS. At that time,
the getStatus() thinks that this file does not exists. So the worker tries to remove its metadata and its data from cache.
ERROR logs were generated in such case.
This patch is to handle this correctly and remove such ERROR logs:
```
2023-09-05 17:16:32,802 ERROR LocalCacheManager - Failed to delete page PageId{FileId=f4b0d1f405d7419069129a1262eae9329e9c19ed2584fc3fe27656a698344047, PageIndex=0} (isTemporary: false) from pageStore.
alluxio.exception.PageNotFoundException: /Volumes/ramdisk/LOCAL/1048576/824/f4b0d1f405d7419069129a1262eae9329e9c19ed2584fc3fe27656a698344047/0
at alluxio.client.file.cache.store.LocalPageStore.delete(LocalPageStore.java:135)
at alluxio.client.file.cache.LocalCacheManager.deletePage(LocalCacheManager.java:889)
at alluxio.client.file.cache.LocalCacheManager.delete(LocalCacheManager.java:694)
at alluxio.client.file.cache.LocalCacheManager.delete(LocalCacheManager.java:706)
at alluxio.client.file.cache.LocalCacheManager.lambda$deleteFile$5(LocalCacheManager.java:839)
at java.lang.Iterable.forEach(Iterable.java:75)
at alluxio.client.file.cache.LocalCacheManager.deleteFile(LocalCacheManager.java:839)
at alluxio.client.file.cache.NoExceptionCacheManager.deleteFile(NoExceptionCacheManager.java:209)
at alluxio.worker.dora.DoraMetaManager.invalidateCachedFile(DoraMetaManager.java:337)
at alluxio.worker.dora.DoraMetaManager.removeFromMetaStore(DoraMetaManager.java:178)
at alluxio.worker.dora.DoraMetaManager.loadFromUfs(DoraMetaManager.java:131)
at alluxio.worker.dora.PagedDoraWorker.getGrpcFileInfo(PagedDoraWorker.java:392)
at alluxio.worker.grpc.DoraWorkerClientServiceHandler.getStatus(DoraWorkerClientServiceHandler.java:201)
at alluxio.grpc.BlockWorkerGrpc$MethodHandlers.invoke(BlockWorkerGrpc.java:1589)
at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182)
at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:355)
at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:867)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
```
### Does this PR introduce any user facing changes?
No
pr-link: Alluxio#18108
change-id: cid-47d15fcde921a539a6e3c64e7ac25fc400f26034
### What changes are proposed in this pull request? Add isReadOnly() for FuseFileStream and implement this interface in various Streams. ### Why are the changes needed? This interface is used to determine if the Stream is only. This is needed to determine if a real data flush is needed. ### Does this PR introduce any user facing changes? N/A pr-link: Alluxio#18114 change-id: cid-6a60e9e917bb0f8a439d76b51beaaba2b723dde4
Script `launch-process` was renamed to `launch-process-bash`. Update the name of the script in `entrypoint.sh` pr-link: Alluxio#18115 change-id: cid-7053532e9fac93ef3f987e1525ae7260ac3a8dc0
### What changes are proposed in this pull request? Add a config to turn on/off the sdk cache fallback ### Why are the changes needed? the read fallback should be turn off by default to avoid retry storm to UFS ### Does this PR introduce any user facing changes? Addition of property keys pr-link: Alluxio#18099 change-id: cid-2c893a617ac5e90086db98610434db6d763c1fe9
One more place to count the cache hit requests pr-link: Alluxio#17878 change-id: cid-0cb55b7ca627832ce2b53e80ea58cfd6a2730bfa
### What changes are proposed in this pull request? Fix the array out of bound exception in presto ### Why are the changes needed? Because we saw earlier that the LocalCacheManager swallows all exceptions, if there is an error accessing one of the cached files, the exception will be ignored. However, the offset may have changed during the failed read attempt in LocalCacheManager. When LocalCacheManager returns -1 on error and tries to reread from the lower layer storage, the offset could be out of bounds. Regarding why this issue only appears in versions 2.9.3 and later, it's because the offset became a member variable of our target buffer starting from 2.9.3. In earlier versions it was always a local variable, so there was no compounding of offset errors. ### Does this PR introduce any user facing changes? no pr-link: Alluxio#18098 change-id: cid-c7b949d9146847b14ba672df56e21bc4bc5ad705
Count the hit and miss of ListStatus and GetFileInfo in worker pr-link: Alluxio#17848 change-id: cid-97c8efb81088d7e9b636ef927fcbc62a166fe085
### What changes are proposed in this pull request? Change metric type of LocalCacheState from counter to gauge. ### Why are the changes needed? The type of LocalCacheState metric is not reasonable since the value of this metric is a enum type not an increasable value ### Does this PR introduce any user facing changes? No pr-link: Alluxio#18070 change-id: cid-61037d3580e2f25f782e1c876ced0ecad5b312a4
### What changes are proposed in this pull request? Change name of NoopMembershipManager to MasterMembershipManager. ### Why are the changes needed? N/A ### Does this PR introduce any user facing changes? N/A pr-link: Alluxio#18097 change-id: cid-9f39e3b1a9b8cdad51e49bbb5d12e86d21b07e23
Reverts Alluxio#18070 revert per request on backward compatibility pr-link: Alluxio#18119 change-id: cid-aeecbff3bc9d6bff41eec25405e87a2f2b79079f
### What changes are proposed in this pull request? Refactor the Netty read handler of worker. ### Why are the changes needed? The previous implementation creates a state machine per read request, instead of per channel. This implies that if two read requests are sent over the same channel, the worker would possibly use one channel to send data of different files or regions. This can lead to data corruption. This PR proposes to use a state machine per channel, and handles channel events throughout the whole lifecycle of the channel. Things like a faulty client sending a second request over the same channel before the first request is completed, is handled gracefully with a client error. The state transitions look like the following:  ### Does this PR introduce any user facing changes? No. ### Tests Tests have been done with basic Alluxio CLI tools, as well as automated PrestoDB and TPC-DS tests. pr-link: Alluxio#17479 change-id: cid-bb8b2c70f0bf0bd84e73d9e858bf6e80706427aa
Refactor Netty read handler to allow subclassing. pr-link: Alluxio#18120 change-id: cid-d0d4125459644d8e92c04dd9decdb622f14cbdb6
Introduction Chapter translation pr-link: Alluxio#18176 change-id: cid-c31d1ab3c4365f0bdb0d0b17418c92885a109276
### What changes are proposed in this pull request? Improve page loading performance by loading the page form UFS directly, instead of creating a reader. ### Why are the changes needed? To improve the data loading performance & reduce the memory consumption ### Does this PR introduce any user facing changes? N/A pr-link: Alluxio#18389 change-id: cid-46e89dcfd22dcbbb4090df7f6c8861408ef8cb67
### What changes are proposed in this pull request? As the title states, this PR does a few things: 1. Allow rename() to overwrite an existing path, if specified in the option 2. The method `rename` in `AbstractFileSystem`, now will process the input path and rerun `rename` method when caught `AlluxioException` or `AlluxioRuntimeException`. Instead of log the exception and return end. 3. Also add some path checks during the rename 4. Reenable `FileSystemRenameIntegrationTest` by adding configs to `LocalAlluxioClusterResource` and reenable UT cases. pr-link: Alluxio#18263 change-id: cid-2870bf87fea8a3b2419e5b10a05423ff2dede6a2
### What changes are proposed in this pull request? Now WORKER_FUSE_ENABLED is invalid. ### Why are the changes needed? improve code quality. ### Does this PR introduce any user facing changes? no. pr-link: Alluxio#18349 change-id: cid-8064a8ef19179c31cdce01a833847ff530c1f776
### What changes are proposed in this pull request? Add call points of the new metrics ### Why are the changes needed? The new metrics are already defined in the previous pr, and need to be called in this pr. ### Does this PR introduce any user facing changes? no pr-link: Alluxio#18351 change-id: cid-1b35a87f41a1f836005c368881378329f4c77b25
Fixed a single spelling error. ### What changes are proposed in this pull request? Please outline the changes and how this PR fixes the issue. ### Why are the changes needed? Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, describe the bug. ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including 1. change in user-facing APIs 2. addition or removal of property keys 3. webui pr-link: Alluxio#18310 change-id: cid-77c21de69cf13a6c94be418adf2be95e0f347d42
### What changes are proposed in this pull request? clean up unmount of fuse ### Why are the changes needed? alluxio-fuse may cause AlluxioFuse continues to hang, and also the current flow to unmount is scattered and all over the place. ### Does this PR introduce any user facing changes? no pr-link: Alluxio#18241 change-id: cid-0246bd831466396697efe1977547d365ad63ba3d
### What changes are proposed in this pull request? Please outline the changes and how this PR fixes the issue. ### Why are the changes needed? Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, describe the bug. ### Does this PR introduce any user facing changes? Please list the user-facing changes introduced by your change, including 1. change in user-facing APIs 2. addition or removal of property keys 3. webui pr-link: Alluxio#18412 change-id: cid-fa4b35605d8f97d79c159140613aacf27d3c1836
Fix the bug that `HdfsUfsStatusIterator` returns wrong path (the getName() method of `UfsStatus` should return a relative path, but not just the file name). pr-link: Alluxio#18424 change-id: cid-8b15fbd345f26562a06c5d9f52bdbc90a53b2039
Fix the bug that HDFS iterator doesn't work. pr-link: Alluxio#18429 change-id: cid-dbb5f1a246bce706f7b3b5d179ad158218dccf83
### What changes are proposed in this pull request? Shade all the class under javax ### Why are the changes needed? We need this change to fix the maven build error in presto ### Does this PR introduce any user facing changes? no pr-link: Alluxio#18450 change-id: cid-91aa8f5a00867b0f44165eb486fa168cc710504c
### What changes are proposed in this pull request? Add a `WorkerClusterView` to allow more efficiently indexing and filtering workers by different criteria. ### Why are the changes needed? The current return type `List<WorkerInfo>` of the methods `getAllWorkers` etc. of `MembershipManager` does not allow efficiently indexing and filtering workers, e.g. getting a worker by it ID, finding all lost workers. This is partially motivated by the review comments in Alluxio#18434 (comment) ### Does this PR introduce any user facing changes? No. pr-link: Alluxio#18441 change-id: cid-e28beb96752cc45cdad973b98f2d689ce1581b0f
### What changes are proposed in this pull request? Fix the shading issue for javax security ### Why are the changes needed? you might see class not found in javax's security module ### Does this PR introduce any user facing changes? no pr-link: Alluxio#18455 change-id: cid-d04ba461930b7abba2613810e8d7d2ae77d7e3be
### What changes are proposed in this pull request? Make the buffer size in s3 proxy configurable, which is currently 8KB and unchangeable. pr-link: Alluxio#18459 change-id: cid-7ebb81819b6f13222496a341cbef743cf50bda7b
### What changes are proposed in this pull request? 1. Move `WorkerState` enum from master package to the wire package. 2. Add a new worker state `UNRECOGNIZED` and use it as the default state. 3. Make worker state in `WorkerInfo` an enum. 4. Add copy constructors to `WorkerInfo` and `WorkerNetAddress`. ### Why are the changes needed? 1. Make sure the state of worker can be enumerated. 2. Allow safely copying mutable `WorkerInfo` and `WorkerNetAddress` objects. ### Does this PR introduce any user facing changes? No. pr-link: Alluxio#18460 change-id: cid-8daf9c1e3ebe8e862a9b0dabb669c80918f5b8b0
### What changes are proposed in this pull request? Put back the restriction of reuse worker identity for non-k8s env deployment. ### Why are the changes needed? Worker identity gets reused by wrong deployment behaviors such as copy conf/ over for new worker setup, as opposed to k8s deployment is thru operator / automation, bare metal deployment has no way of prevention, thus putting back the restriction for non-k8s deployment. ### Does this PR introduce any user facing changes? No. pr-link: Alluxio#18454 change-id: cid-295ea352895b16c2a5f0a23fa790c9f42a5e3881
### What changes are proposed in this pull request? Replace `List<BlockWorkerInfo>` with `WorkerClusterView` in APIs. Important APIs that are changed: 1. `FileSystemContext.getCachedWorkers` now returns `WorkerClusterView` 2. `WorkerLocationPolicy.getPreferredWorkers` (as well as all its implementations) now accepts a `WorkerClusterView` as the first argument (but still returns `List<BlockWorkerInfo>` as the returned list must be ordered) APIs that are using `List<BlockWorkerInfo>` (or `List<WorkerInfo>`) but *not* migrated to `WorkerClusterView`: 1. `alluxio.master.scheduler.WorkerProvider.getWorkerInfos` returns `List<WorkerInfo>`. 2. Job service related APIs, e.g. `alluxio.job.plan.PlanDefinition.selectExecutors` Notable behavior change: Now `EtcdMembershipManager` assigns the correct state (`LIVE` or `LOST`) for all workers in its `WorkerInfo` struct. Before this change, this information is not available and the state defaults to `UNRECOGNIZED`. ### Why are the changes needed? Allow more efficient indexing and filtering workers by worker ID. ### Does this PR introduce any user facing changes? No. pr-link: Alluxio#18451 change-id: cid-5052d2faa506f4de6e4b0df7062c5def3e09df1c
### What changes are proposed in this pull request? Fix outdated worker address info returned by consistent hash policy Summary of changes: 1. `ConsistentHashProvider` only concerns about `WorkerIdentity` when building the hash ring. Therefore, the APIs have been limited to accept and return `WorkerIdentity`s. 2. `ConsistentHashProvider.refresh` now accepts a set of worker identities instead of a list, as the order does not matter. 3. Added a test to cover the bug fix. ### Why are the changes needed? Fix a bug where the consistent hash provider caches the `BlockWorkerInfo` of all workers, and when a worker changes its network addresses but its ID stays the same, the hash provider won't update the worker's info. A client will continue to use the outdated network address. The fix is to make the hash provider to only consider `WorkerIdentity`s, and let the client to figure out the worker's address with the ID provided by the consistent hash provider. ### Does this PR introduce any user facing changes? No. pr-link: Alluxio#18434 change-id: cid-93f1601d846385f314c79a556c8705d3983a1199
This PR improves RESTful load api, use JSON format as the response content.
## Examples:
### Submit Job Example:
```
// 20231211174310
// http://localhost:28080/v1/load?path=hdfs://node01:8020/testRoot/testDirectory2&opType=submit&verbose=true
{
"success": true,
"jobId": "0dbc0f47-580b-420f-b50e-d08a170746c8",
"path": "hdfs://node01:8020/testRoot/testDirectory2",
"message": "Load 'hdfs://node01:8020/testRoot/testDirectory2' is successfully submitted. JobId: 0dbc0f47-580b-420f-b50e-d08a170746c8\n"
}
```
### Get Job Progress Example:
```
// 20231211180110
// http://localhost:28080/v1/load?path=hdfs://node01:8020/testRoot/testDirectory2&opType=progress&verbose=true
{
"jobState": "RUNNING",
"path": "hdfs://node01:8020/testRoot/testDirectory2",
"message": "Progress for loading path 'hdfs://node01:8020/testRoot/testDirectory2':\n\tSettings:\tbandwidth: unlimited\tverify: false\tmetadata-only: false\n\tTime Elapsed: 00:00:03\n\tJob State: RUNNING\n\tStage: RETRYING\n\tInodes Scanned: 4\n\tInodes Processed: 4\n\tBytes Loaded: 0B out of 0B\n\tThroughput: 0B/s\n\tFile Failure rate: 0.00%\n\tSubtask Failure rate: 0.00%\n\tFiles Failed: 0\n\tRecent failed subtasks: \n\tRecent retrying subtasks: \n\tSubtask Retry rate: 0.00%\n\tSubtasks on Retry Dead Letter Queue: 0\n",
"respProperties": {
"Files Failed": "0",
"Recent failed subtasks": "",
"Subtask Retry rate": "0.00%",
"Throughput": "0B/s",
"File Failure rate": "0.00%",
"Subtasks on Retry Dead Letter Queue": "0",
"Time Elapsed": "00",
"Bytes Loaded": "0B out of 0B",
"Stage": "RETRYING",
"Inodes Scanned": "4",
"Inodes Processed": "4",
"Recent retrying subtasks": "",
"Subtask Failure rate": "0.00%",
"Settings": "bandwidth",
"Job State": "RUNNING",
"Progress for loading path 'hdfs": "//node01"
}
}
```
```
// 20231211174358
// http://localhost:28080/v1/load?path=hdfs://node01:8020/testRoot/testDirectory2&opType=progress&verbose=true
{
"jobState": "SUCCEEDED",
"path": "hdfs://node01:8020/testRoot/testDirectory2",
"message": "Progress for loading path 'hdfs://node01:8020/testRoot/testDirectory2':\n\tSettings:\tbandwidth: unlimited\tverify: false\tmetadata-only: false\n\tTime Elapsed: 00:00:16\n\tJob State: SUCCEEDED\n\tInodes Scanned: 4\n\tInodes Processed: 4\n\tBytes Loaded: 0B out of 0B\n\tThroughput: 0B/s\n\tFile Failure rate: 0.00%\n\tSubtask Failure rate: 0.00%\n\tFiles Failed: 0\n\tRecent failed subtasks: \n\tRecent retrying subtasks: \n\tSubtask Retry rate: 0.00%\n\tSubtasks on Retry Dead Letter Queue: 0\n",
"respProperties": {
"Files Failed": "0",
"Recent failed subtasks": "",
"Subtask Retry rate": "0.00%",
"Throughput": "0B/s",
"File Failure rate": "0.00%",
"Subtasks on Retry Dead Letter Queue": "0",
"Time Elapsed": "00",
"Bytes Loaded": "0B out of 0B",
"Inodes Scanned": "4",
"Inodes Processed": "4",
"Recent retrying subtasks": "",
"Subtask Failure rate": "0.00%",
"Settings": "bandwidth",
"Job State": "SUCCEEDED",
"Progress for loading path 'hdfs": "//node01"
}
}
```
### Stop Job Example:
```
// 20231211180219
// http://localhost:28080/v1/load?path=hdfs://node01:8020/testRoot/testDirectory2&opType=stop&verbose=true
{
"success": true,
"path": "hdfs://node01:8020/testRoot/testDirectory2",
"message": "Load 'hdfs://node01:8020/testRoot/testDirectory2' is successfully stopped.\n"
}
```
```
// 20231211180153
// http://localhost:28080/v1/load?path=hdfs://node01:8020/testRoot/testDirectory2&opType=stop&verbose=true
{
"success": false,
"path": "hdfs://node01:8020/testRoot/testDirectory2",
"message": "Cannot find load job for path hdfs://node01:8020/testRoot/testDirectory2, it might have already been stopped or finished\n"
}
```
pr-link: Alluxio#18464
change-id: cid-1fff9a23457064ab71534909449c60a6b0123f22
### What changes are proposed in this pull request? Remove additional white space in alluxio-fuse script ### Why are the changes needed? alluxio-fuse unmount <mnt_point> is unable to find the pid of AlluxioFuse process because the grep content isn't correct. ### Does this PR introduce any user facing changes? No pr-link: Alluxio#18465 change-id: cid-3e70b0c8edbaa0ba50d744fd6155b0d494a243f9
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
Please outline the changes and how this PR fixes the issue.
Why are the changes needed?
Please clarify why the changes are needed. For instance,
Does this PR introduce any user facing changes?
Please list the user-facing changes introduced by your change, including