Code

Aggregator support batch serialize#9777

Code

Open

guo-shaoge wants to merge 101 commits intopingcap:masterpingcap/tiflash:masterfrom guo-shaoge:hashagg_batch_serializeguo-shaoge/tiflash:hashagg_batch_serializeCopy head branch name to clipboard

+3,488-1,222

Commits

Commits on Nov 25, 2024

• new hash

  Show description for b869b2f
  guo-shaoge

  committed
  b869b2f

  Copy full SHA for b869b2f

Commits on Nov 26, 2024

• prefetch done

  Show description for e8a2df8
  guo-shaoge

  committed
  e8a2df8

  Copy full SHA for e8a2df8

Commits on Nov 27, 2024

• executeImplBatchStringHashMap done

  Show description for b314166
  guo-shaoge

  committed
  b314166

  Copy full SHA for b314166

• handle resize exception done

  Show description for ec6e892
  guo-shaoge

  committed
  ec6e892

  Copy full SHA for ec6e892

Commits on Nov 28, 2024

• tmp save

  Show description for 9dc702d
  guo-shaoge

  committed
  9dc702d

  Copy full SHA for 9dc702d

• revert Serialized Key changes

  Show description for ce1f767
  guo-shaoge

  committed
  ce1f767

  Copy full SHA for ce1f767

• refine

  Show description for 8ac8beb
  guo-shaoge

  committed
  8ac8beb

  Copy full SHA for 8ac8beb

• refine

  Show description for 0053ce8
  guo-shaoge

  committed
  0053ce8

  Copy full SHA for 0053ce8

Commits on Nov 29, 2024

• fix unit test

  Show description for fcf8ed2
  guo-shaoge

  committed
  fcf8ed2

  Copy full SHA for fcf8ed2

Commits on Dec 2, 2024

• refine

  Show description for ae7b969
  guo-shaoge

  committed
  ae7b969

  Copy full SHA for ae7b969

• unit test

  Show description for 3a86617
  guo-shaoge

  committed
  3a86617

  Copy full SHA for 3a86617

• prefetch

  Show description for 623fef5
  guo-shaoge

  committed
  623fef5

  Copy full SHA for 623fef5

• fix

  Show description for 19f320d
  guo-shaoge

  committed
  19f320d

  Copy full SHA for 19f320d

Commits on Dec 3, 2024

• refine

  Show description for 3a226df
  guo-shaoge

  committed
  3a226df

  Copy full SHA for 3a226df

• refine

  Show description for c44ace7
  guo-shaoge

  committed
  c44ace7

  Copy full SHA for c44ace7

• revert new hasher

  Show description for 3e30f95
  guo-shaoge

  committed
  3e30f95

  Copy full SHA for 3e30f95

• debug low distinct value

  Show description for ea85d19
  guo-shaoge

  committed
  ea85d19

  Copy full SHA for ea85d19

• Revert "revert new hasher"

  Show description for 16937ff
  guo-shaoge

  committed
  16937ff

  Copy full SHA for 16937ff

• refine original code path

  Show description for d2fba57
  guo-shaoge

  committed
  d2fba57

  Copy full SHA for d2fba57

• Reapply "revert new hasher"

  Show description for 71b6ecd
  guo-shaoge

  committed
  71b6ecd

  Copy full SHA for 71b6ecd

Commits on Dec 4, 2024

• one level old hash; two level new hash

  Show description for 40ceb08
  guo-shaoge

  committed
  40ceb08

  Copy full SHA for 40ceb08

• Revert "one level old hash; two level new hash"

  Show description for 4cb24c2
  guo-shaoge

  committed
  4cb24c2

  Copy full SHA for 4cb24c2

• revert new hasher; refine original code path

  Show description for c02cf71
  guo-shaoge

  committed
  c02cf71

  Copy full SHA for c02cf71

Commits on Dec 5, 2024

• fix case

  Show description for 352b710
  guo-shaoge

  committed
  352b710

  Copy full SHA for 352b710

Commits on Dec 13, 2024

• refine

  Show description for e5ab87c
  guo-shaoge

  committed
  e5ab87c

  Copy full SHA for e5ab87c

Commits on Dec 15, 2024

• is_serialized_key

  Show description for 30d99be
  guo-shaoge

  committed
  30d99be

  Copy full SHA for 30d99be

• fix

  Show description for 01cae80
  guo-shaoge

  committed
  01cae80

  Copy full SHA for 01cae80

• fix start_row

  Show description for 95c597d
  guo-shaoge

  committed
  95c597d

  Copy full SHA for 95c597d

Commits on Dec 16, 2024

• Merge branch 'master' into hashagg_prefetch

  guo-shaoge

  authored
  83fb879

  Copy full SHA for 83fb879

Commits on Dec 17, 2024

• revert prefetch for StringHashTable

  Show description for e0dea79
  guo-shaoge

  committed
  e0dea79

  Copy full SHA for e0dea79

• comment

  Show description for 143fee3
  guo-shaoge

  committed
  143fee3

  Copy full SHA for 143fee3

Commits on Dec 18, 2024

• fmt

  Show description for 6cea464
  guo-shaoge

  committed
  6cea464

  Copy full SHA for 6cea464

Commits on Dec 19, 2024

• comment

  Show description for 6750737
  guo-shaoge

  committed
  6750737

  Copy full SHA for 6750737

Commits on Dec 20, 2024

• prefetch agg data

  Show description for 2e94829
  guo-shaoge

  committed
  2e94829

  Copy full SHA for 2e94829

Commits on Dec 22, 2024

• try fix ci

  Show description for dc0973a
  guo-shaoge

  committed
  dc0973a

  Copy full SHA for dc0973a

Commits on Dec 24, 2024

• prefetch threshold(2MB); mini batch; prefetch exists key; prefetch insertResultInto

  Show description for 9623368
  guo-shaoge

  committed
  9623368

  Copy full SHA for 9623368

• clean code

  Show description for 0ef6ba6
  guo-shaoge

  committed
  0ef6ba6

  Copy full SHA for 0ef6ba6

• fix prefetch agg func is 0

  Show description for 49d4188
  guo-shaoge

  committed
  49d4188

  Copy full SHA for 49d4188

Commits on Dec 25, 2024

• fix case

  Show description for b4a1bde
  guo-shaoge

  committed
  b4a1bde

  Copy full SHA for b4a1bde

• refine agg func is zero

  Show description for 738ab2a
  guo-shaoge

  committed
  738ab2a

  Copy full SHA for 738ab2a

• refine

  Show description for 361f988
  guo-shaoge

  committed
  361f988

  Copy full SHA for 361f988

• refine comment

  Show description for 5158885
  guo-shaoge

  committed
  5158885

  Copy full SHA for 5158885

Commits on Dec 26, 2024

• getHash full

  Show description for 80b7e22
  guo-shaoge

  committed
  80b7e22

  Copy full SHA for 80b7e22

• minor refine

  Show description for c41dba4
  guo-shaoge

  committed
  c41dba4

  Copy full SHA for c41dba4

• hashvals sizeof 16

  Show description for ab31beb
  guo-shaoge

  committed
  ab31beb

  Copy full SHA for ab31beb

Commits on Dec 27, 2024

• fix case

  Show description for 00cc20d
  guo-shaoge

  committed
  00cc20d

  Copy full SHA for 00cc20d

• remove macro

  Show description for 4aec90b
  guo-shaoge

  committed
  4aec90b

  Copy full SHA for 4aec90b

• remove macro 2

  Show description for 606e013
  guo-shaoge

  committed
  606e013

  Copy full SHA for 606e013

• fix

  Show description for a7361e1
  guo-shaoge

  committed
  a7361e1

  Copy full SHA for a7361e1

• refine comment

  Show description for bdb21d3
  guo-shaoge

  committed
  bdb21d3

  Copy full SHA for bdb21d3

• refine

  Show description for 4153592
  guo-shaoge

  committed
  4153592

  Copy full SHA for 4153592

Commits on Dec 29, 2024

• vec 16

  Show description for bbd5c45
  guo-shaoge

  committed
  bbd5c45

  Copy full SHA for bbd5c45

Commits on Dec 31, 2024

• basically done

  Show description for e4b57c8
  guo-shaoge

  committed
  e4b57c8

  Copy full SHA for e4b57c8

• fix compilation

  Show description for 1574825
  guo-shaoge

  committed
  1574825

  Copy full SHA for 1574825

• fmt

  Show description for 4203af0
  guo-shaoge

  committed
  4203af0

  Copy full SHA for 4203af0

• compile && nt_optimization

  Show description for 29021b2
  guo-shaoge

  committed
  29021b2

  Copy full SHA for 29021b2

Commits on Jan 1, 2025

• unit test

  Show description for a3cd638
  guo-shaoge

  committed
  a3cd638

  Copy full SHA for a3cd638

• refine

  Show description for d155847
  guo-shaoge

  committed
  d155847

  Copy full SHA for d155847

Commits on Jan 2, 2025

• fix

  Show description for d3b0300
  guo-shaoge

  committed
  d3b0300

  Copy full SHA for d3b0300

• Merge branch 'master' into batch_serialize

  guo-shaoge

  authored
  e8564aa

  Copy full SHA for e8564aa

Commits on Jan 3, 2025

• test new impl

  Show description for abd55ac
  guo-shaoge

  committed
  abd55ac

  Copy full SHA for abd55ac

• Merge branch 'batch_serialize' of github.com:guo-shaoge/tiflash into batch_serialize

  guo-shaoge

  committed
  4cac26a

  Copy full SHA for 4cac26a

• test ci impl

  Show description for c07d13a
  guo-shaoge

  committed
  c07d13a

  Copy full SHA for c07d13a

Commits on Jan 6, 2025

• Revert "test ci impl"

  Show description for 086b630
  guo-shaoge

  committed
  086b630

  Copy full SHA for 086b630

• Revert "test new impl"

  Show description for db8d490
  guo-shaoge

  committed
  db8d490

  Copy full SHA for db8d490

• change name

  Show description for 84ee65b
  guo-shaoge

  committed
  84ee65b

  Copy full SHA for 84ee65b

• is_fast -> ensure_unique

  Show description for 3800d0f
  guo-shaoge

  committed
  3800d0f

  Copy full SHA for 3800d0f

• batchSerializeImpl -> serializeToPosImpl

  Show description for a6fac1f
  guo-shaoge

  committed
  a6fac1f

  Copy full SHA for a6fac1f

• ci

  Show description for 19982d3
  guo-shaoge

  committed
  19982d3

  Copy full SHA for 19982d3

Commits on Jan 7, 2025

• refine

  Show description for 47cdf91
  guo-shaoge

  committed
  47cdf91

  Copy full SHA for 47cdf91

• refine

  Show description for 6627558
  guo-shaoge

  committed
  6627558

  Copy full SHA for 6627558

• compute hashvals before each mini batch

  Show description for 530bb27
  guo-shaoge

  committed
  530bb27

  Copy full SHA for 530bb27

• handleMiniBatchImpl

  Show description for 358e851
  guo-shaoge

  committed
  358e851

  Copy full SHA for 358e851

• fmt

  Show description for 3d79a20
  guo-shaoge

  committed
  3d79a20

  Copy full SHA for 3d79a20

• remove useless code

  Show description for 51a90b5
  guo-shaoge

  committed
  51a90b5

  Copy full SHA for 51a90b5

• minor refine

  Show description for e108584
  guo-shaoge

  committed
  e108584

  Copy full SHA for e108584

• fmt

  Show description for 761b34d
  guo-shaoge

  committed
  761b34d

  Copy full SHA for 761b34d

Commits on Jan 8, 2025

• Merge branch 'master' of github.com:pingcap/tiflash into batch_serialize

  guo-shaoge

  committed
  1342f6a

  Copy full SHA for 1342f6a

Commits on Jan 9, 2025

• only enable mini batch when prefetch

  Show description for 538bad4
  guo-shaoge

  committed
  538bad4

  Copy full SHA for 538bad4

Commits on Jan 10, 2025

• refine

  Show description for 2a6a5f5
  guo-shaoge

  committed
  2a6a5f5

  Copy full SHA for 2a6a5f5

• refine

  Show description for 7d910e5
  guo-shaoge

  committed
  7d910e5

  Copy full SHA for 7d910e5

• refine

  Show description for 6a2b333
  guo-shaoge

  committed
  6a2b333

  Copy full SHA for 6a2b333

Commits on Jan 12, 2025

• fix

  Show description for 5343014
  guo-shaoge

  committed
  5343014

  Copy full SHA for 5343014

Commits on Jan 13, 2025

• tmp save

  Show description for 82059d6
  guo-shaoge

  committed
  82059d6

  Copy full SHA for 82059d6

• refine convertToBlocksImplFinal

  Show description for 9c9ff5e
  guo-shaoge

  committed
  9c9ff5e

  Copy full SHA for 9c9ff5e

• sortKey no virtual function call

  Show description for cff4dc2
  guo-shaoge

  committed
  cff4dc2

  Copy full SHA for cff4dc2

• avoid sortKey virtual function call

  Show description for 2b32e79
  guo-shaoge

  committed
  2b32e79

  Copy full SHA for 2b32e79

• fmt

  Show description for 25406de
  guo-shaoge

  committed
  25406de

  Copy full SHA for 25406de

• refine

  Show description for a9f86b0
  guo-shaoge

  committed
  a9f86b0

  Copy full SHA for a9f86b0

Commits on Jan 14, 2025

• refine

  Show description for 4c6f931
  guo-shaoge

  committed
  4c6f931

  Copy full SHA for 4c6f931

• Merge branch 'batch_serialize' of github.com:guo-shaoge/tiflash into hashagg_batch_serialize

  guo-shaoge

  committed
  1dd0fe7

  Copy full SHA for 1dd0fe7

• batch serialize

  Show description for a2a6116
  guo-shaoge

  committed
  a2a6116

  Copy full SHA for a2a6116

• fix

  Show description for 774e905
  guo-shaoge

  committed
  774e905

  Copy full SHA for 774e905

• fix batch serialize

  Show description for 83d092e
  guo-shaoge

  committed
  83d092e

  Copy full SHA for 83d092e

• fix assert

  Show description for e4f6307
  guo-shaoge

  committed
  e4f6307

  Copy full SHA for e4f6307

• fix wrong result

  Show description for e500a81
  guo-shaoge

  committed
  e500a81

  Copy full SHA for e500a81

• fix string size

  Show description for 8bbb062
  guo-shaoge

  committed
  8bbb062

  Copy full SHA for 8bbb062

Commits on Jan 15, 2025

• remove batch_size from template paramenters

  Show description for 2e7bdfe
  guo-shaoge

  committed
  2e7bdfe

  Copy full SHA for 2e7bdfe

• fix string size for HashMethodFastPathTwoKeysSerialized

  Show description for cd3d115
  guo-shaoge

  committed
  cd3d115

  Copy full SHA for cd3d115

• fmt

  Show description for 1bb38e3
  guo-shaoge

  committed
  1bb38e3

  Copy full SHA for 1bb38e3

• fix string size

  Show description for 59f6578
  guo-shaoge

  committed
  59f6578

  Copy full SHA for 59f6578