Code

Diff from Huggingface main#16

Code

Draft

jlamypoirier wants to merge 34 commits intohuggingface_mainbigcode-project/transformers:huggingface_mainfrom mainbigcode-project/transformers:mainCopy head branch name to clipboard

+1,735-5

Commits

Commits on Feb 7, 2023

• Add files

  jlamypoirier

  committed
  48efb8d

  Copy full SHA for 48efb8d

• Changes

  jlamypoirier

  committed
  5145458

  Copy full SHA for 5145458

• tests

  jlamypoirier

  committed
  d5dd307

  Copy full SHA for d5dd307

• fix

  jlamypoirier

  committed
  4fdf8c1

  Copy full SHA for 4fdf8c1

• fix

  jlamypoirier

  committed
  c25f221

  Copy full SHA for c25f221

• Megatron conversion

  jlamypoirier

  committed
  2134a34

  Copy full SHA for 2134a34

• Add todos

  jlamypoirier

  committed
  33bb045

  Copy full SHA for 33bb045

• fix

  jlamypoirier

  committed
  e92b63f

  Copy full SHA for e92b63f

• format

  jlamypoirier

  committed
  2f32703

  Copy full SHA for 2f32703

• fix

  jlamypoirier

  committed
  035c73c

  Copy full SHA for 035c73c

Commits on Feb 8, 2023

• Merge pull request #6 from bigcode-project/gpt2_bigcode

  Show description for 3ba6b3d
  jlamypoirier

  authored
  3ba6b3d

  Copy full SHA for 3ba6b3d

• Multi-query attention (#4)

  jlamypoirier

  authored
  9ab47d9

  Copy full SHA for 9ab47d9

Commits on Feb 11, 2023

• add test to ensure mqa and mha have the same behaviour

  minimario

  committed
  878025e

  Copy full SHA for 878025e

Commits on Feb 14, 2023

• Megatron conversion script (#8)

  jlamypoirier

  authored
  92d98c1

  Copy full SHA for 92d98c1

Commits on Feb 15, 2023

• Upcasting, scaling, masking and fused kernels to match Megatron-LM (#10)

  jlamypoirier

  authored
  03716fa

  Copy full SHA for 03716fa

Commits on Feb 20, 2023

• change test to use santacoder, add seed for the random inputs, increase tolerance

  minimario

  committed
  97f734c

  Copy full SHA for 97f734c

• add train mode test

  minimario

  committed
  8ea03f4

  Copy full SHA for 8ea03f4

Commits on Feb 21, 2023

• add attention parameters to initialization, parameterize test

  minimario

  committed
  400adc3

  Copy full SHA for 400adc3

Commits on Feb 24, 2023

• use parameterized test

  minimario

  committed
  edd1622

  Copy full SHA for edd1622

Commits on Feb 26, 2023

• Merge pull request #11 from minimario/main

  Show description for f1e84c0
  mayank31398

  authored
  f1e84c0

  Copy full SHA for f1e84c0

Commits on Feb 28, 2023

• Add santacoder (#9)

  jlamypoirier

  authored
  7f00e1c

  Copy full SHA for 7f00e1c

Commits on Mar 2, 2023

• More optimizations (#13)

  jlamypoirier

  authored
  b989169

  Copy full SHA for b989169

• Fast inference (#7)

  jlamypoirier

  authored
  d4451b4

  Copy full SHA for d4451b4

• Add gpu optimizations to base model (#14)

  jlamypoirier

  authored
  9c3c548

  Copy full SHA for 9c3c548

Commits on Apr 13, 2023

• Merge commit '7dcd8703ef904adc3ac19b47f769879221c33849' into main

  jlamypoirier

  committed
  e0e4921

  Copy full SHA for e0e4921

• Merge commit '349e1242d9551c3996f21b491d416d033e6a36fa' into main

  jlamypoirier

  committed
  9c2a8b9

  Copy full SHA for 9c2a8b9

• Merge commit '656e869a4523f6a0ce90b3aacbb05cc8fb5794bb' into main

  jlamypoirier

  committed
  9471327

  Copy full SHA for 9471327

• Merge commit 'e0921c6b53310a47b10f01633809b2b9f785a465' into main

  jlamypoirier

  committed
  5f00363

  Copy full SHA for 5f00363

• Merge remote-tracking branch 'huggingface/main' into main

  jlamypoirier

  committed
  459b5b8

  Copy full SHA for 459b5b8

• Fix merge

  jlamypoirier

  committed
  7671e1e

  Copy full SHA for 7671e1e

• Update conversion script

  jlamypoirier

  committed
  2fe9ae3

  Copy full SHA for 2fe9ae3

• Fix and style

  jlamypoirier

  committed
  7c45f0c

  Copy full SHA for 7c45f0c

• Reduce diff

  jlamypoirier

  committed
  05a6225

  Copy full SHA for 05a6225

Commits on Apr 24, 2023

• Add back experimental features (#17)

  jlamypoirier

  authored
  8b0cb2c

  Copy full SHA for 8b0cb2c