Code

Feature/increase timeouts#8

Code

Closed

cre4ture wants to merge 101 commits intoHexilee:mainHexilee/tifs:mainfrom cre4ture:feature/increase_timeoutscre4ture/tifs-hashfs:feature/increase_timeoutsCopy head branch name to clipboard

+17,518-2,860

Commits

Commits on May 14, 2024

• update rust toolchain an tikv-client version

  cre4ture

  committed
  fecbb03

  Copy full SHA for fecbb03

• increase timeouts and retries for spinning of transaction

  cre4ture

  committed
  9f4fe99

  Copy full SHA for 9f4fe99

• dependency update ++

  cre4ture

  committed
  11fc8ea

  Copy full SHA for 11fc8ea

Commits on May 18, 2024

• retry on other errors

  cre4ture

  committed
  7c6f4f7

  Copy full SHA for 7c6f4f7

• speedup: consider noatime option, store file handlers only locally

  cre4ture

  committed
  80cb782

  Copy full SHA for 80cb782

• add key-prefix feature that allows to use multiple filesystems on same DB

  cre4ture

  committed
  96f04e1

  Copy full SHA for 96f04e1

• hashedblocks: first step

  cre4ture

  committed
  ba0cc14

  Copy full SHA for ba0cc14

Commits on May 19, 2024

• first hashblock writing implementation

  cre4ture

  committed
  3a0421c

  Copy full SHA for 3a0421c

• implement read + adding unittests for write+read

  cre4ture

  committed
  f387858

  Copy full SHA for f387858

• fix two issues + logging

  cre4ture

  committed
  2d0fd27

  Copy full SHA for 2d0fd27

• cache for TiKV reads

  cre4ture

  committed
  8569d24

  Copy full SHA for 8569d24

• cache writes, fix bug in read, free fh-ids at release.

  cre4ture

  committed
  658ecde

  Copy full SHA for 658ecde

Commits on May 20, 2024

• improve logging

  cre4ture

  committed
  14fc4b0

  Copy full SHA for 14fc4b0

• avoid write to inode for opening inode by tagging inode with seperate key

  Show description for 69ddd43
  cre4ture

  committed
  69ddd43

  Copy full SHA for 69ddd43

• cleanup

  cre4ture

  committed
  d55f066

  Copy full SHA for d55f066

• cache inodes, multiple parallel transactions, StopWatch

  cre4ture

  committed
  9185b61

  Copy full SHA for 9185b61

Commits on May 22, 2024

• update mtime only when changes written. option to disable completely

  cre4ture

  committed
  f5b3f7b

  Copy full SHA for f5b3f7b

Commits on May 25, 2024

• validation of writes and reads

  cre4ture

  committed
  26aa242

  Copy full SHA for 26aa242

Commits on May 26, 2024

• first refactoring to support raw_read. BUT doesn't work

  cre4ture

  committed
  272c533

  Copy full SHA for 272c533

• working, but slow raw hash block storage

  cre4ture

  committed
  94db20f

  Copy full SHA for 94db20f

• async execution of write raw blocks

  cre4ture

  committed
  b3e74dc

  Copy full SHA for b3e74dc

• pure raw mode working

  cre4ture

  committed
  cf9cd16

  Copy full SHA for cf9cd16

• async put without wait

  cre4ture

  committed
  9706ca1

  Copy full SHA for 9706ca1

• make smarter file handler

  cre4ture

  committed
  2ebb2cc

  Copy full SHA for 2ebb2cc

Commits on May 28, 2024

• optionally disable hashed block existence check

  cre4ture

  committed
  01a45a8

  Copy full SHA for 01a45a8

• allow parallel accesses to TiKV cluster

  cre4ture

  committed
  2234b9e

  Copy full SHA for 2234b9e

• configurable parallelism

  cre4ture

  committed
  fe90d33

  Copy full SHA for fe90d33

• write cache for speedup

  cre4ture

  committed
  56ee8f9

  Copy full SHA for 56ee8f9

Commits on May 29, 2024

• simple read ahead implementation

  cre4ture

  committed
  9d1c7af

  Copy full SHA for 9d1c7af

Commits on Jun 1, 2024

• working logical inodes

  cre4ture

  committed
  730ce3b

  Copy full SHA for 730ce3b

• working read of hashes of file

  cre4ture

  committed
  4ea76d0

  Copy full SHA for 4ea76d0

• file level API to read out internal file hashes

  cre4ture

  committed
  7f6a51a

  Copy full SHA for 7f6a51a

• split async_fs.rs for better overview

  cre4ture

  committed
  a050b8a

  Copy full SHA for a050b8a

• split tikv_fs.rs

  cre4ture

  committed
  186a5c3

  Copy full SHA for 186a5c3

• flush write cache at "fsync" and "flush" system calls

  cre4ture

  committed
  46fa236

  Copy full SHA for 46fa236

Commits on Jun 4, 2024

• sha254/512 as aternative hashers; split of Inode; cleanup

  cre4ture

  committed
  13c82b0

  Copy full SHA for 13c82b0

• add and fix tests for key generation, hash_len computation

  cre4ture

  committed
  38260a7

  Copy full SHA for 38260a7

• small transaction feature

  cre4ture

  committed
  98092d8

  Copy full SHA for 98092d8

• split into own file: txn_data_cache

  cre4ture

  committed
  a2596fb

  Copy full SHA for a2596fb

Commits on Jun 5, 2024

• split metadata into dynamic and static, adding hash algo to check

  cre4ture

  committed
  6b72a11

  Copy full SHA for 6b72a11

Commits on Jun 6, 2024

• intermediate untested state: critical section

  cre4ture

  committed
  2aa8fbf

  Copy full SHA for 2aa8fbf

Commits on Jun 8, 2024

• dir content is stored only as keys; critical section for clear.

  cre4ture

  committed
  cce3800

  Copy full SHA for cce3800

• 1s heartbeat for cleanup and status logging

  cre4ture

  committed
  50c7f58

  Copy full SHA for 50c7f58

Commits on Jun 9, 2024

• flexible transaction refactoring

  cre4ture

  committed
  6790246

  Copy full SHA for 6790246

Commits on Jun 11, 2024

• tokio console

  cre4ture

  committed
  20c986b

  Copy full SHA for 20c986b

• readahead improvement

  cre4ture

  committed
  62ebaf1

  Copy full SHA for 62ebaf1

Commits on Jun 15, 2024

• mix

  cre4ture

  committed
  5b1bda0

  Copy full SHA for 5b1bda0

• try with named uses

  cre4ture

  committed
  31709be

  Copy full SHA for 31709be

• refcounting for writes

  cre4ture

  committed
  c2f75af

  Copy full SHA for c2f75af

• fix issue with recursion of critical section drop implementation

  cre4ture

  committed
  62f3a05

  Copy full SHA for 62f3a05

• implement reference counting for blocks - write+delete

  cre4ture

  committed
  c043ed8

  Copy full SHA for c043ed8

Commits on Jun 16, 2024

• add more logs

  cre4ture

  committed
  5647474

  Copy full SHA for 5647474

• spin for each mini transaction, also the ones with only one statement

  cre4ture

  committed
  dd71fff

  Copy full SHA for dd71fff

• faster detection of dead client connection

  cre4ture

  committed
  65bb400

  Copy full SHA for 65bb400

• disable printing og backtrace for unknown errors

  cre4ture

  committed
  28c7722

  Copy full SHA for 28c7722

Commits on Jun 18, 2024

• improve robustness of the transaction retry

  cre4ture

  committed
  2115a8b

  Copy full SHA for 2115a8b

Commits on Jun 24, 2024

• intermediate commit

  cre4ture

  committed
  53ff34e

  Copy full SHA for 53ff34e

• intermediate ++

  cre4ture

  committed
  a3538f2

  Copy full SHA for a3538f2

• intermediate state ++

  cre4ture

  committed
  6f88883

  Copy full SHA for 6f88883

Commits on Jun 25, 2024

• compiles finally -> split by HashFsInterface

  cre4ture

  committed
  0001dcb

  Copy full SHA for 0001dcb

• fixes to make it work

  cre4ture

  committed
  19bef71

  Copy full SHA for 19bef71

Commits on Jun 27, 2024

• adaptions for grpc-hashfs

  cre4ture

  committed
  dd7d94e

  Copy full SHA for dd7d94e

• add readily implemented grpc client and server for hash-fs

  cre4ture

  committed
  75ff073

  Copy full SHA for 75ff073

• last fixes to have it really working

  cre4ture

  committed
  a6f33d6

  Copy full SHA for a6f33d6

Commits on Jun 28, 2024

• avoid conflicts in size update splitting transaction and using local lock

  cre4ture

  committed
  6657497

  Copy full SHA for 6657497

• use relative path for tikv dependency

  cre4ture

  committed
  1054357

  Copy full SHA for 1054357

• ensure output dir exists to avoid error on fresh checkouts

  cre4ture

  committed
  c0788f9

  Copy full SHA for c0788f9

• client considers endpoint paramter, server listens on all interfaces

  cre4ture

  committed
  066d0b0

  Copy full SHA for 066d0b0

• cleanup (fix unneded connection of hash-fs client to tikv)

  cre4ture

  committed
  2690c8c

  Copy full SHA for 2690c8c

Commits on Jun 29, 2024

• allow parallel grpc calls by using multiple clients

  cre4ture

  committed
  6ce418c

  Copy full SHA for 6ce418c

• move binarie source code to typical location

  cre4ture

  committed
  cd95cc0

  Copy full SHA for cd95cc0

• cleanup: split client code

  cre4ture

  committed
  6d98845

  Copy full SHA for 6d98845

• cleanup: split server code

  cre4ture

  committed
  103dc99

  Copy full SHA for 103dc99

• cleanup: renamings

  cre4ture

  committed
  7934f4c

  Copy full SHA for 7934f4c

• small tuning to avoid one un-needed grpc interaction during write

  cre4ture

  committed
  a6a9b84

  Copy full SHA for a6a9b84

• add logs

  cre4ture

  committed
  dad3bc0

  Copy full SHA for dad3bc0

• extract Object Pool in own file. add logs

  cre4ture

  committed
  f81392a

  Copy full SHA for f81392a

• remove unneeded lock

  cre4ture

  committed
  6f3f7b3

  Copy full SHA for 6f3f7b3

• split InoSize to improve write performance

  cre4ture

  committed
  31ae813

  Copy full SHA for 31ae813

• split transactions

  cre4ture

  committed
  ec2add5

  Copy full SHA for ec2add5

• add more logging information

  cre4ture

  committed
  50b113c

  Copy full SHA for 50b113c

• add more information to log

  cre4ture

  committed
  129c455

  Copy full SHA for 129c455

• try to avoid commit failures by using 1 phase commit

  cre4ture

  committed
  fddbdaa

  Copy full SHA for fddbdaa

Commits on Jun 30, 2024

• add more information to logs

  cre4ture

  committed
  f251319

  Copy full SHA for f251319

• try to avoid conflicts by using local locks

  cre4ture

  committed
  d0bcb75

  Copy full SHA for d0bcb75

• try with local lock with reduced scope

  cre4ture

  committed
  e27bfd0

  Copy full SHA for e27bfd0

• array style hb_increment_reference_count, key-parser refactoring

  cre4ture

  committed
  7b8f321

  Copy full SHA for 7b8f321

• add+fix unittests

  cre4ture

  committed
  a91e546

  Copy full SHA for a91e546

• fix

  cre4ture

  committed
  fe2ea1b

  Copy full SHA for fe2ea1b

• renaming transaction to fuse_to_hashfs

  cre4ture

  committed
  77e3503

  Copy full SHA for 77e3503

• vectorized transactions on hashfs interface

  cre4ture

  committed
  2c6d8ee

  Copy full SHA for 2c6d8ee

Commits on Jul 4, 2024

• vectorization of write "cache"

  cre4ture

  committed
  5565860

  Copy full SHA for 5565860

Commits on Jul 5, 2024

• prevent GRPC call when input is zero length

  cre4ture

  committed
  c59e717

  Copy full SHA for c59e717

• introduce .@opened_inodes and .@snapshots

  cre4ture

  committed
  75138ec

  Copy full SHA for 75138ec

• prepare snapshot creation

  cre4ture

  committed
  7c6ba5a

  Copy full SHA for 7c6ba5a

Commits on Jul 6, 2024

• initial code for snapshot creation + refactoring

  cre4ture

  committed
  c44c31b

  Copy full SHA for c44c31b

• enable the creation of snapshots by with file-explorer + fixes

  cre4ture

  committed
  14108e0

  Copy full SHA for 14108e0

• fix missing reference cnt increment for snapshot data

  cre4ture

  committed
  13a81ae

  Copy full SHA for 13a81ae

• fix grpc interface

  cre4ture

  committed
  547d1c4

  Copy full SHA for 547d1c4

Commits on Jul 10, 2024

• write some information to the README

  cre4ture

  committed
  9c0da27

  Copy full SHA for 9c0da27

• Merge remote-tracking branch 'origin/main' into feature/increase_timeouts

  Show description for 3bbae8e
  cre4ture

  committed
  3bbae8e

  Copy full SHA for 3bbae8e