BERT Layer Built-In Backward Pass

[MaximilianSchreff]

This PR adds the backward pass for previously introduced full BERT layer from the BERT transformer architecture to SystemDS as a built-in operation.

This PR is part of a series of PRs to support the BERT architecture in SystemDS. The BERT layer is the component in the BERT architecture.

Includes

• Backward pass

Testing:

Added 2 test cases comparing the backward pass results against HuggingFace Transformer Library's implementation, which uses PyTorch Autograd module for backward pass computation, for correctness.

• The tests validate:
  • Backward pass against HuggingFace Transformer Library's transformers.models.bert.modeling_bert.BertLayer
  • For every single gradient from the whole layer

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.38%. Comparing base (e022eaf) to head (91ec288).
Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #2213   +/-   ##
=========================================
  Coverage     72.38%   72.38%           
  Complexity    45325    45325           
=========================================
  Files          1467     1467           
  Lines        170629   170629           
  Branches      33260    33260           
=========================================
+ Hits         123512   123513    +1     
+ Misses        37728    37725    -3     
- Partials       9389     9391    +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Baunsgaard]

LGTM