You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[kazu/steps/ner/opsin.py extendString() method reworks entity matches to account for Transformer model matches that tend to identify only a part of entities with longer names - which perhaps indicates need for more generic logic around handling transformer model matches]
@RichJackson my gut says maybe this logic should be wrapped up in the TransformersModelForToeknClassificationNerStep like we have the NonContiguousEntitySplitter, so we don't get the match wrong to start with, and need to then fix it here much later in the pipeline. It would also mean either having opsin used to some extent by all users, or having it be pretty flexibly configurable, or writing our own logic to decide when extending is reasonable.
What do you think?
The text was updated successfully, but these errors were encountered:
Original comment from @EFord36
[kazu/steps/ner/opsin.py extendString() method reworks entity matches to account for Transformer model matches that tend to identify only a part of entities with longer names - which perhaps indicates need for more generic logic around handling transformer model matches]
@RichJackson my gut says maybe this logic should be wrapped up in the TransformersModelForToeknClassificationNerStep like we have the NonContiguousEntitySplitter, so we don't get the match wrong to start with, and need to then fix it here much later in the pipeline. It would also mean either having opsin used to some extent by all users, or having it be pretty flexibly configurable, or writing our own logic to decide when extending is reasonable.
What do you think?
The text was updated successfully, but these errors were encountered: