Use perfect hashing instead of string matching in SyscallHintProcessor::execute_syscall_hint #969

fguthmann · 2023-08-31T06:05:17Z

TITLE

Description

Use perfect hashing instead of string matching in SyscallHintProcessor::execute_syscall_hint
Added benchmark to evaluate the improvement of perfect hashing instead of string matching.
Added regenerate map-hints in Makefile in case of changes in the map-hints

String matching:
execute_hint_ReplaceClass
time: [1.9303 µs 1.9330 µs 1.9362 µs]
change: [-0.1123% +0.1559% +0.4553%] (p = 0.31 > 0.05)
No change in performance detected.
Found 20 outliers among 100 measurements (20.00%)
10 (10.00%) low mild
4 (4.00%) high mild
6 (6.00%) high severe

execute_hint_GetBlockTimestamp
time: [1.9258 µs 1.9287 µs 1.9320 µs]
change: [-0.8323% -0.6463% -0.4463%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

execute_hint_GetCallerAddress
time: [1.9190 µs 1.9229 µs 1.9265 µs]
change: [-0.7294% -0.5602% -0.3896%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

execute_hint_LibraryCall
time: [1.9302 µs 1.9335 µs 1.9369 µs]
change: [-0.1857% -0.0504% +0.0902%] (p = 0.48 > 0.05)
No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
2 (2.00%) high mild

execute_hint_CallContract
time: [1.9534 µs 1.9557 µs 1.9581 µs]
change: [-0.4339% -0.2422% -0.0719%] (p = 0.01 < 0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

execute_hint_DelegateCall
time: [1.9717 µs 1.9729 µs 1.9741 µs]
change: [+0.8818% +1.0116% +1.1426%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low severe
1 (1.00%) low mild
5 (5.00%) high mild
1 (1.00%) high severe

execute_hint_GetSequencerAddress
time: [1.9383 µs 1.9402 µs 1.9424 µs]
change: [+0.7789% +0.9272% +1.0693%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

execute_hint_GetTxSignature
time: [1.9632 µs 1.9660 µs 1.9708 µs]
change: [+0.5390% +0.8505% +1.1208%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe

execute_hint_GetBlockNumber
time: [1.9497 µs 1.9524 µs 1.9553 µs]
change: [+1.0570% +1.2371% +1.4039%] (p = 0.00 < 0.05)
Performance has regressed.

execute_hint_StorageWrite
time: [1.9887 µs 1.9937 µs 1.9989 µs]
change: [+1.2108% +1.5823% +2.0031%] (p = 0.00 < 0.05)
Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe

execute_hint_EmitEvent time: [2.0097 µs 2.0277 µs 2.0476 µs]
change: [+2.7899% +3.4766% +4.1252%] (p = 0.00 < 0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
3 (3.00%) high mild
6 (6.00%) high severe

execute_hint_StorageRead
time: [1.9775 µs 1.9815 µs 1.9856 µs]
change: [+1.7393% +1.9131% +2.0854%] (p = 0.00 < 0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
7 (7.00%) high mild

execute_hint_AddrIs250 time: [1.7458 µs 1.7557 µs 1.7671 µs]
change: [+4.6455% +5.3485% +6.1267%] (p = 0.00 < 0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
7 (7.00%) high mild
2 (2.00%) high severe

execute_hint_DelegateCallL1Handler
time: [1.9872 µs 1.9952 µs 2.0039 µs]
change: [+1.9367% +2.2690% +2.6729%] (p = 0.00 < 0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe

execute_hint_Deploy time: [2.0427 µs 2.0614 µs 2.0811 µs]
change: [-2.3490% +0.5074% +2.7194%] (p = 0.73 > 0.05)
No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe

execute_hint_SendMessageToL1
time: [1.9739 µs 1.9810 µs 1.9885 µs]
change: [+1.8528% +2.1470% +2.4419%] (p = 0.00 < 0.05)
Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

execute_hint_GetContractAddress
time: [1.9902 µs 2.0032 µs 2.0188 µs]
change: [-0.8259% +0.7065% +1.8322%] (p = 0.36 > 0.05)
No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
1 (1.00%) low mild
8 (8.00%) high mild
4 (4.00%) high severe

execute_hint_LibraryCallL1Handler
time: [1.9902 µs 1.9984 µs 2.0078 µs]
change: [+3.7067% +4.6042% +5.5777%] (p = 0.00 < 0.05)
Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe

execute_hint_GetTxInfo time: [1.9813 µs 1.9858 µs 1.9911 µs]
change: [+3.0563% +3.3250% +3.6393%] (p = 0.00 < 0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe

execute_hint_AddrBoundPrime
time: [1.7687 µs 1.7723 µs 1.7767 µs]
change: [+2.7885% +3.1002% +3.4760%] (p = 0.00 < 0.05)
Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

 Running bench/internals.rs (target/release/deps/internals-7674eeb7794bb0e7)

Perfect hashing:
execute_hint_ReplaceClass
time: [1.9510 µs 1.9557 µs 1.9607 µs]
change: [+0.4249% +0.7793% +1.1406%] (p = 0.00 < 0.05)
Change within noise threshold.

execute_hint_GetBlockTimestamp
time: [1.9646 µs 1.9669 µs 1.9690 µs]
change: [+1.2666% +1.4769% +1.6756%] (p = 0.00 < 0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) low mild
1 (1.00%) high mild

execute_hint_GetCallerAddress
time: [1.9420 µs 1.9450 µs 1.9490 µs]
change: [+1.4539% +1.6414% +1.8466%] (p = 0.00 < 0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe

execute_hint_LibraryCall
time: [1.9488 µs 1.9535 µs 1.9596 µs]
change: [+1.0124% +1.2324% +1.4800%] (p = 0.00 < 0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
3 (3.00%) high mild
2 (2.00%) high severe

execute_hint_CallContract
time: [1.9631 µs 1.9637 µs 1.9643 µs]
change: [+0.0122% +0.1358% +0.2652%] (p = 0.03 < 0.05)
Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low mild
6 (6.00%) high mild
3 (3.00%) high severe

execute_hint_DelegateCall
time: [1.9489 µs 1.9593 µs 1.9712 µs]
change: [-1.3873% -1.0866% -0.7777%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild

execute_hint_GetSequencerAddress
time: [1.9470 µs 1.9516 µs 1.9559 µs]
change: [+0.3011% +0.4869% +0.6508%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
7 (7.00%) low severe
2 (2.00%) low mild

execute_hint_GetTxSignature
time: [1.9568 µs 1.9625 µs 1.9677 µs]
change: [-0.5106% -0.2094% +0.0330%] (p = 0.13 > 0.05)
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

execute_hint_GetBlockNumber
time: [1.9668 µs 1.9694 µs 1.9718 µs]
change: [+0.0323% +0.2249% +0.4239%] (p = 0.02 < 0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

execute_hint_StorageWrite
time: [1.9554 µs 1.9578 µs 1.9601 µs]
change: [-1.5227% -1.1430% -0.8231%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
5 (5.00%) low severe
3 (3.00%) low mild
4 (4.00%) high mild
2 (2.00%) high severe

execute_hint_EmitEvent time: [1.9532 µs 1.9570 µs 1.9602 µs]
change: [-3.2677% -2.6163% -2.0280%] (p = 0.00 < 0.05)
Performance has improved.

execute_hint_StorageRead
time: [1.9547 µs 1.9597 µs 1.9640 µs]
change: [-1.3140% -1.0506% -0.8182%] (p = 0.00 < 0.05)
Change within noise threshold.

execute_hint_AddrIs250 time: [1.6846 µs 1.6851 µs 1.6856 µs]
change: [-5.5878% -4.9086% -4.2799%] (p = 0.00 < 0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
4 (4.00%) low mild
3 (3.00%) high mild
3 (3.00%) high severe

execute_hint_DelegateCallL1Handler
time: [1.9610 µs 1.9647 µs 1.9692 µs]
change: [-2.1808% -1.7741% -1.3686%] (p = 0.00 < 0.05)
Performance has improved.
Found 19 outliers among 100 measurements (19.00%)
6 (6.00%) low mild
4 (4.00%) high mild
9 (9.00%) high severe

execute_hint_Deploy time: [1.9600 µs 1.9634 µs 1.9674 µs]
change: [-3.5317% -2.8818% -2.2645%] (p = 0.00 < 0.05)
Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
4 (4.00%) high mild
8 (8.00%) high severe

execute_hint_SendMessageToL1
time: [1.9498 µs 1.9540 µs 1.9598 µs]
change: [-1.6289% -1.3253% -1.0119%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low mild
4 (4.00%) high mild
3 (3.00%) high severe

execute_hint_GetContractAddress
time: [1.9525 µs 1.9557 µs 1.9594 µs]
change: [-2.0391% -1.6601% -1.3250%] (p = 0.00 < 0.05)
Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
1 (1.00%) high mild
14 (14.00%) high severe

execute_hint_LibraryCallL1Handler
time: [1.9685 µs 1.9703 µs 1.9719 µs]
change: [-4.5724% -3.6412% -2.8250%] (p = 0.00 < 0.05)
Performance has improved.

execute_hint_GetTxInfo time: [1.9449 µs 1.9482 µs 1.9514 µs]
change: [-2.5722% -2.2669% -1.9856%] (p = 0.00 < 0.05)
Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
9 (9.00%) low severe
4 (4.00%) high mild
2 (2.00%) high severe

execute_hint_AddrBoundPrime
time: [1.6946 µs 1.6953 µs 1.6959 µs]
change: [-5.5571% -5.2259% -4.9106%] (p = 0.00 < 0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe

 Running bench/internals.rs (target/release/deps/internals-7674eeb7794bb0e7)

Checklist

…ng-in-syscallhintprocessorexecute_syscall_hint

src/syscalls/deprecated_syscall_handler.rs

…ng-in-syscallhintprocessorexecute_syscall_hint

codecov-commenter · 2023-09-07T12:05:33Z

Codecov Report

Merging #969 (836e94f) into main (4539844) will decrease coverage by 0.01%.
The diff coverage is 95.65%.

@@            Coverage Diff             @@
##             main     #969      +/-   ##
==========================================
- Coverage   89.98%   89.97%   -0.01%     
==========================================
  Files          49       49              
  Lines       13036    13055      +19     
==========================================
+ Hits        11730    11746      +16     
- Misses       1306     1309       +3

Files	Coverage Δ
src/lib.rs	`98.76% <ø> (ø)`
src/syscalls/deprecated_syscall_handler.rs	`98.10% <95.65%> (-0.29%)`	⬇️

src/syscalls/deprecated_syscall_handler.rs

Cargo.toml

…ng-in-syscallhintprocessorexecute_syscall_hint

Cargo.toml

Oppen

Looks great. There are two details with the benchmark and we would need a little pipeline to check what's in the repo is up to date (essentially, it needs to rebuild the codegen and fail if the generated code changed).

bench/hashing_running_time.rs

…ng-in-syscallhintprocessorexecute_syscall_hint

fguthmann linked an issue Aug 31, 2023 that may be closed by this pull request

Use perfect hashing instead of string matching in SyscallHintProcessor::execute_syscall_hint #101

Closed

fannyguthmann added 3 commits August 31, 2023 11:48

add phf_map problem with the HintProcessorData string

94d1dc6

add phf_map problem with the HintProcessorData string

342b8f4

wip

ee28c82

fguthmann force-pushed the use-perfect-hashing-instead-of-string-matching-in-syscallhintprocessorexecute_syscall_hint branch from 98ef602 to ee28c82 Compare August 31, 2023 09:20

fannyguthmann added 3 commits September 5, 2023 15:37

wip

73cffb2

wip

5fabd70

Merge branch 'main' into use-perfect-hashing-instead-of-string-matchi…

55db822

…ng-in-syscallhintprocessorexecute_syscall_hint

Oppen reviewed Sep 5, 2023

View reviewed changes

src/syscalls/deprecated_syscall_handler.rs Outdated Show resolved Hide resolved

fannyguthmann added 4 commits September 7, 2023 13:32

created perfect hashing mapping intead of strings

c4bfe78

Merge branch 'main' into use-perfect-hashing-instead-of-string-matchi…

1550b82

…ng-in-syscallhintprocessorexecute_syscall_hint

Removed hint_code.rs

efee61e

remove hint_code from mod.rs

9bc956a

Oppen reviewed Sep 7, 2023

View reviewed changes

src/syscalls/deprecated_syscall_handler.rs Show resolved Hide resolved

fannyguthmann added 4 commits September 11, 2023 13:58

wip

5b7cfe9

wip

e04efca

merge main

6b7d9ed

wip

3c9bdbe

juanbono reviewed Sep 14, 2023

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

fannyguthmann added 8 commits September 15, 2023 14:29

added benchmarking

39e1e8c

format

6171235

wip

69676ae

Merge branch 'main' into use-perfect-hashing-instead-of-string-matchi…

ed9d731

…ng-in-syscallhintprocessorexecute_syscall_hint

wip

8ff4c3f

wip

a361785

fix format

2c0427d

Merge branch 'main' into use-perfect-hashing-instead-of-string-matchi…

bfc8106

…ng-in-syscallhintprocessorexecute_syscall_hint

juanbono reviewed Sep 22, 2023

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

use hint_code in hint_code_map

8d7b23a

fannyguthmann added 4 commits September 28, 2023 13:34

use hint_code in hint_code_map

b611e4e

corrected duplicated Cargo.toml

3035bf4

Added Makefile line to regerate hint_map if necessary

12b29ef

Modifile Makefile line to be regenerate map-hints

b5a6d4a

fguthmann marked this pull request as ready for review September 28, 2023 16:17

Oppen reviewed Sep 28, 2023

View reviewed changes

bench/hashing_running_time.rs Outdated Show resolved Hide resolved

bench/hashing_running_time.rs Outdated Show resolved Hide resolved

fannyguthmann added 2 commits September 29, 2023 15:55

test without the #[allow(dead_code)]

0b1a9fa

resolve dead code

7f08a2c

Oppen approved these changes Sep 29, 2023

View reviewed changes

fannyguthmann added 5 commits October 2, 2023 12:53

wip

40813da

wip

e54d5e8

remove unsued dependencies awc , actix-web and clap

66e1a82

Merge branch 'main' into use-perfect-hashing-instead-of-string-matchi…

836e94f

…ng-in-syscallhintprocessorexecute_syscall_hint

resolve merge conflict

5862f4b

fguthmann closed this Nov 30, 2023

fguthmann deleted the use-perfect-hashing-instead-of-string-matching-in-syscallhintprocessorexecute_syscall_hint branch November 30, 2023 12:54

fguthmann removed a link to an issue Nov 30, 2023

Use perfect hashing instead of string matching in SyscallHintProcessor::execute_syscall_hint #101

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use perfect hashing instead of string matching in SyscallHintProcessor::execute_syscall_hint #969

Use perfect hashing instead of string matching in SyscallHintProcessor::execute_syscall_hint #969

fguthmann commented Aug 31, 2023 •

edited

Loading

codecov-commenter commented Sep 7, 2023 •

edited

Loading

Oppen left a comment

Use perfect hashing instead of string matching in SyscallHintProcessor::execute_syscall_hint #969

Use perfect hashing instead of string matching in SyscallHintProcessor::execute_syscall_hint #969

Conversation

fguthmann commented Aug 31, 2023 • edited Loading

TITLE

Description

Checklist

codecov-commenter commented Sep 7, 2023 • edited Loading

Codecov Report

Oppen left a comment

Choose a reason for hiding this comment

fguthmann commented Aug 31, 2023 •

edited

Loading

codecov-commenter commented Sep 7, 2023 •

edited

Loading