Skip to content

Commit

Permalink
updating informal algorithm description
Browse files Browse the repository at this point in the history
  • Loading branch information
anwala committed Jul 16, 2020
1 parent 172d177 commit fa404a2
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 3 deletions.
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ __pycache__
.gitignore

Dockerfile
informal_alg.txt
LICENSE
README.md
setup.py
6 changes: 3 additions & 3 deletions informal_alg.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Highly informal algorithm description of sumgram
Highly informal algorithm description of Sumgram (2020-07-15)

Step 1
1. Add plain_text into doc_lst
Expand All @@ -7,7 +7,7 @@ Step 1

Step 2
1. extract_top_ngrams(): Extract top n-grams from text, top is defined by top DF (for multiple documents) or top TF (for single documents)
2. pos_glue_split_ngrams(): If Stanford CoreNLP server (POS tagger) is active: replace children subset ngrams (e.g., "national hurricane") with superset parent multi-word proper noun (e.g., "national hurricane center") extracted by multi_word_proper_nouns(). Subset means overlap is 1.0 and match order is preserved (e.g., "hurricane national" is NOT subset of "national hurricane center" since even though overlap is 1.0, but match is out of order)
2. pos_glue_split_ngrams(): If Stanford CoreNLP server (POS tagger) is active: replace children subset ngrams (e.g., "national hurricane") with superset. parent multi-word proper noun (e.g., "national hurricane center") extracted by extract_proper_nouns(). Subset means overlap is 1.0 and match order is preserved (e.g., "hurricane national" is NOT subset of "national hurricane center" since even though overlap is 1.0, but match is out of order)
3. mvg_window_glue_split_ngrams()
4. rm_subset_top_ngrams()

Expand Down Expand Up @@ -41,7 +41,7 @@ for ngram in top_ngrams[:x]
both = 'federal emergency management agency'

if k = 2,
left = 'the federal emergency management',#comma counts as word
left = 'the federal emergency management',#commas counts as words
right = 'emergency management agency,'
both = 'the federal emergency management agency,'
'''
Expand Down

0 comments on commit fa404a2

Please sign in to comment.