Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add UfsUrlBench #18345

Open
wants to merge 100 commits into
base: use-ufs-path
Choose a base branch
from
Open
Changes from all commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
c5f0a15
an example implementation purely to measure the amount of effort to c…
jiacheliu3 Jun 14, 2023
ef0c2da
add a wrapper
jiacheliu3 Jun 15, 2023
2f2d3da
add new class
jiacheliu3 Jun 15, 2023
781132f
Add refactor of openFile, and other small fix.
YichuanSun Jul 21, 2023
7d7f206
Add the refactor of ls test. Add part of ctor of UfsUrl.java.
YichuanSun Jul 28, 2023
c762773
Add part of UfsUrl.java, add a little part of UfsUrlTest.java.
YichuanSun Jul 28, 2023
6bf1897
Add ls command refactor version.
YichuanSun Jul 29, 2023
5b320a9
fix the double slash dir bug.
YichuanSun Jul 31, 2023
d3dc6dd
a backup for further refactor. Next try to solve the root dir problem.
YichuanSun Jul 31, 2023
d1477b1
Refactor part of UfsUrl, add method "isAncestorOf".
YichuanSun Jul 31, 2023
2a00bd9
Only contain the UfsUrl design.
YichuanSun Aug 1, 2023
8be2a8d
Cherry-pick from the UfsUrl data structure design branch.
YichuanSun Aug 1, 2023
16e64c3
Leave only one todo of Tony Sun in UfsUrl.java, add a field mDepth to…
YichuanSun Aug 1, 2023
90e0d35
Update dora/core/client/fs/src/main/java/alluxio/client/file/BaseFile…
YichuanSun Aug 1, 2023
a6f07bb
Delete useless comments and resolved todo. Delete UfsUrlUtils.java fo…
YichuanSun Aug 1, 2023
ce37d65
Cherry-pick from UfsUrl design branch: Delete useless comments and re…
YichuanSun Aug 1, 2023
bcf4316
Merge branch 'ufs-url-yichuan-UfsUrl' of https://github.com/YichuanSu…
YichuanSun Aug 1, 2023
0966942
Bug is still exist.
YichuanSun Aug 1, 2023
17ecf49
extract the modification of mine from the merge commit.
YichuanSun Aug 2, 2023
a8cc11a
Cherry-pick from today's change.
YichuanSun Aug 2, 2023
c653ffe
A primary version of UfsUrl.java.
YichuanSun Aug 2, 2023
6de92c3
Add static factory method, refactor ctor of UfsUrl.
YichuanSun Aug 3, 2023
3a59a21
Add static factory method, refactor ctor of UfsUrl. A primary full ve…
YichuanSun Aug 3, 2023
d88c816
First review of myself, remove redundant comments and finished todo, …
YichuanSun Aug 3, 2023
6efb017
Update dora/core/client/fs/src/main/java/alluxio/client/file/DoraCach…
YichuanSun Aug 4, 2023
a4a5359
Update dora/core/common/src/main/java/alluxio/util/UfsUrlUtils.java
YichuanSun Aug 4, 2023
96cce95
Update dora/core/common/src/main/java/alluxio/uri/UfsUrl.java
YichuanSun Aug 4, 2023
3eedfa7
Update dora/core/common/src/main/java/alluxio/uri/UfsUrl.java
YichuanSun Aug 4, 2023
2f8fca5
Update dora/core/common/src/main/java/alluxio/uri/UfsUrl.java
YichuanSun Aug 4, 2023
9ffeb33
Update dora/core/common/src/main/java/alluxio/uri/UfsUrl.java
YichuanSun Aug 4, 2023
5d4c877
Update dora/core/common/src/main/java/alluxio/uri/UfsUrl.java
YichuanSun Aug 4, 2023
4bcfe60
Update dora/core/common/src/main/java/alluxio/uri/UfsUrl.java
YichuanSun Aug 4, 2023
f157e6b
Modification based on reviewer's comments. Next commit I will handle …
YichuanSun Aug 4, 2023
eb6a31c
merge the GitHub modification from remote branch.
YichuanSun Aug 4, 2023
4589ee1
merge the GitHub modification from remote branch.
YichuanSun Aug 4, 2023
7e870a2
resolve conflict with remote branch.
YichuanSun Aug 4, 2023
7fb9b7d
Merge branch 'ufs-url-yichuan-UfsUrl' of https://github.com/YichuanSu…
YichuanSun Aug 4, 2023
01fb5d2
Refactor createInstance method in UfsUrl.java
YichuanSun Aug 4, 2023
d3ed7fa
delete BaseFileSystem.java
YichuanSun Aug 4, 2023
eab95a7
Merge branch 'use-ufs-path' into ufs-url-yichuan-UfsUrl
YichuanSun Aug 4, 2023
5105755
Merge UfsUrlUtils.java to PathUtils.java
YichuanSun Aug 4, 2023
005ddb6
Merge branch 'ufs-url-yichuan-UfsUrl' of https://github.com/YichuanSu…
YichuanSun Aug 4, 2023
c09fe1a
Add a javadoc, fix a small bug.
YichuanSun Aug 5, 2023
77f2b99
Add all javadocs.
YichuanSun Aug 7, 2023
f32194c
Remove deprecated prompt in FileSystem.java.
YichuanSun Aug 7, 2023
5a7ac52
rewrite a todo, restore a line format.
YichuanSun Aug 7, 2023
bee90ff
Add some fix, improve UfsUrlTest.java
YichuanSun Aug 7, 2023
8184a86
Delete comments and finished todo.
YichuanSun Aug 7, 2023
7c8a84c
Interesting error, isn't it?
YichuanSun Aug 7, 2023
4816e87
rerun test.
YichuanSun Aug 7, 2023
d9fdc2c
Ready to review in a new round.
YichuanSun Aug 8, 2023
ea90c98
fix checkstyle
YichuanSun Aug 8, 2023
2210f48
remove invalid todo
YichuanSun Aug 8, 2023
f0bf399
Address reviewer's comments.
YichuanSun Aug 9, 2023
31aa4b4
set UfsUrl.mProto to final
YichuanSun Aug 9, 2023
0bb14f4
Refactor UfsUrl createInstance factory method again.
YichuanSun Aug 9, 2023
c7543c4
fix a small bug.
YichuanSun Aug 9, 2023
b26fcf1
rerun test
YichuanSun Aug 9, 2023
c9e7e0a
getFullPath judge elems from the second element(If any)
YichuanSun Aug 10, 2023
7fd0505
fix some bugs of generatePathComponents.
YichuanSun Aug 10, 2023
c6b0586
fix some bugs of generatePathComponents.
YichuanSun Aug 10, 2023
cf2b3a9
fix checkstyle
YichuanSun Aug 10, 2023
2457ec4
fix bugs one more time.
YichuanSun Aug 10, 2023
727cc4d
fix bugs two more time.
YichuanSun Aug 10, 2023
114811a
add UfsUrlTest.hashCodeTest
YichuanSun Aug 10, 2023
889c2da
add a few comments.
YichuanSun Aug 10, 2023
240fdaa
support "."
YichuanSun Aug 11, 2023
c70d371
asString -> toString, remove toProto.
YichuanSun Aug 14, 2023
561a470
refactor another time.
YichuanSun Aug 14, 2023
9d7631f
solve comments.
YichuanSun Aug 14, 2023
c006f8c
rename UfsUrl.getProto -> UfsUrl.toProto
YichuanSun Aug 14, 2023
0cb73a4
remove optional getScheme
YichuanSun Aug 14, 2023
e38d422
extract path concat function as a static method in PathUtils
YichuanSun Aug 15, 2023
844b9cd
fix the empty scheme and alluxio scheme cases in all ctors of UfsUrl
YichuanSun Aug 15, 2023
9be69f2
remove useless mRootUfsUrl field in UfsBaseFileSystem
YichuanSun Aug 15, 2023
26f2bc9
solve the latest comments.
YichuanSun Aug 15, 2023
fc2be91
solve rebase conflict...
YichuanSun Aug 15, 2023
4066457
rebase to add UfsUrl design.
YichuanSun Aug 10, 2023
1a56e01
rebasing with ufs-url-yichuan-UfsUrl
YichuanSun Aug 14, 2023
e33acd8
tmp save.
YichuanSun Aug 15, 2023
bcc8548
add all UT of UfsUrl
YichuanSun Aug 17, 2023
df5b7a3
merge and resolve conflicts with the remote branch
YichuanSun Aug 17, 2023
90a1202
rename some methods
YichuanSun Aug 17, 2023
62ed832
fix checkstyle
YichuanSun Aug 17, 2023
72597b0
performance improvement.
YichuanSun Aug 17, 2023
8aea3d2
pre assign StringBuilder
YichuanSun Aug 17, 2023
3987848
handle some todos in the UfsUrl.java
YichuanSun Aug 22, 2023
d435333
Add all UT of UfsUrl.java, refactor UfsUrl.joinUnsafe
YichuanSun Aug 22, 2023
7f25ab5
refactor path normalization function
YichuanSun Aug 23, 2023
27235d7
refactor removeRedundantSlashes
YichuanSun Aug 23, 2023
ab107b9
Replace java.net.uri with FilenameUtils and add comments
YichuanSun Aug 24, 2023
00c2d3e
replace ExpectedException with assertThrow, and other small fix.
YichuanSun Aug 28, 2023
3bf56a8
remove the suuport of "." and ".." parsing.
YichuanSun Aug 28, 2023
1e12393
fix checkstyle.
YichuanSun Aug 28, 2023
0847301
refactor Parser ctor, add a test case.
YichuanSun Aug 28, 2023
e3f95ef
getParentURL now returns Optional.
YichuanSun Aug 28, 2023
b7d7751
update
YichuanSun Aug 29, 2023
3cf8ee1
fix checkstyle.
YichuanSun Aug 29, 2023
fddec47
remove useless case
YichuanSun Aug 29, 2023
a2a01ad
Isolate the enrichment of UfsUrlBench
YichuanSun Oct 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions dora/microbench/src/main/java/alluxio/UfsUrlBench.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
/*
* The Alluxio Open Foundation licenses this work under the Apache License, version 2.0
* (the "License"). You may not use this work except in compliance with the License, which is
* available at www.apache.org/licenses/LICENSE-2.0
*
* This software is distributed on an "AS IS" basis, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
* either express or implied, as more fully set forth in the License.
*
* See the NOTICE file distributed with this work for information regarding copyright ownership.
*/

package alluxio;

import alluxio.uri.UfsUrl;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;
import org.openjdk.jmh.infra.Blackhole;
import org.openjdk.jmh.results.format.ResultFormatType;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.CommandLineOptionException;
import org.openjdk.jmh.runner.options.CommandLineOptions;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;

import java.util.concurrent.TimeUnit;

/**
* Benchmarks for {@link alluxio.uri.UfsUrl}.
*/

@Fork(value = 1, jvmArgsPrepend = "-server")
@Warmup(iterations = 2, time = 3, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 6, time = 3, timeUnit = TimeUnit.SECONDS)
@BenchmarkMode(Mode.Throughput)

public class UfsUrlBench {

@State(Scope.Benchmark)
public static class BenchParams {
@Param({"abc://localhost:8080/1/2/3/4/5",
"abc://bucket/1/2////3 4/////56 7/"})
public String mPaths;
}

@Benchmark
public void testCtorOfUfsUrl(Blackhole bh, BenchParams benchParams) throws Exception {
bh.consume(UfsUrl.createInstance(benchParams.mPaths));
}

@Benchmark
public void testCtorOfAlluxioURI(Blackhole bh, BenchParams benchParams) throws Exception {
bh.consume(new AlluxioURI(benchParams.mPaths));
}

public static void main(String[] args) throws RunnerException, CommandLineOptionException {
Options argsFromCli = new CommandLineOptions(args);
Options opts = new OptionsBuilder()
.parent(argsFromCli)
.include(UfsUrlBench.class.getName())
.result("result.json")
.resultFormat(ResultFormatType.JSON)
.build();
new Runner(opts).run();
}
}