Skip to content

Commit

Permalink
Set extensions when write_conllu() is called in UD train script (expl…
Browse files Browse the repository at this point in the history
…osion#4618)

* Set extensions when write_conllu() is called

`run_eval.py` uses the `write_conllu()` function from `ud_train.py` by
itself, so it needs to set the token extensions if necessary.

* Switch from try to if
  • Loading branch information
adrianeboyd authored and ines committed Nov 11, 2019
1 parent 0b9a5f4 commit f415e9b
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions bin/ud/ud_train.py
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,13 @@ def evaluate(nlp, text_loc, gold_loc, sys_loc, limit=None):


def write_conllu(docs, file_):
if not Token.has_extension("get_conllu_lines"):
Token.set_extension("get_conllu_lines", method=get_token_conllu)
if not Token.has_extension("begins_fused"):
Token.set_extension("begins_fused", default=False)
if not Token.has_extension("inside_fused"):
Token.set_extension("inside_fused", default=False)

merger = Matcher(docs[0].vocab)
merger.add("SUBTOK", None, [{"DEP": "subtok", "op": "+"}])
for i, doc in enumerate(docs):
Expand Down

0 comments on commit f415e9b

Please sign in to comment.