Skip to content

Commit

Permalink
Merge pull request #29 from mcwdsi/update-2023-12
Browse files Browse the repository at this point in the history 
Update DrOn from RxNorm_full_12042023. Adds missing SBDs, updates labels to RxNorm's latest labels, implements dose form and units mapping to the extent available from the prior build process.
  • Loading branch information
@hoganwr
hoganwr authored Jul 23, 2024
2 parents 5742023 + 5a6e204 commit 5787d54
Show file tree
Hide file tree
Showing 26 changed files with 62,915 additions and 231 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/qc.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,15 @@ jobs:
ontology_qc:
# The type of runner that the job will run on
runs-on: ubuntu-latest
container: obolibrary/odkfull:v1.4
container: obolibrary/odkfull:v1.5.1

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- uses: actions/checkout@v3

- name: Run ontology QC checks
env:
DEFAULT_BRANCH: main
run: cd src/ontology && echo "DRON in its current form is too big for GitHub Actions"
run: echo "DRON in its current form is too big for GitHub Actions"

1 change: 1 addition & 0 deletions docs/odk-workflows/ManageDocumentation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,4 @@ The documentation is _not_ automatically updated from the Markdown, and needs to
Just make sure you give GitHub 2-5 minutes to build the pages!



2 changes: 1 addition & 1 deletion docs/odk-workflows/RepoManagement.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Updating your ODK repository

Your ODK repositories configuration is managed in `src/ontology/dron-odk.yaml`. Once you have made your changes, you can run the following to apply your changes to the repository:
Your ODK repositories configuration is managed in `src/ontology/dron-odk.yaml`. The [ODK Project Configuration Schema](https://github.com/INCATools/ontology-development-kit/blob/master/docs/project-schema.md) defines all possible parameters that can be used in this config YAML. Once you have made your changes, you can run the following to apply your changes to the repository:


```
Expand Down
171 changes: 120 additions & 51 deletions src/ontology/Makefile
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ----------------------------------------
# Makefile for dron
# Generated using ontology-development-kit
# ODK Version: v1.4
# ODK Version: v1.5.1
# ----------------------------------------
# IMPORTANT: DO NOT EDIT THIS FILE. To override default make goals, use dron.Makefile instead

Expand All @@ -15,6 +15,9 @@
# 2. [Refresh imports](all_imports)
# 3. [Update repo to latest ODK](update_repo)

# Fingerprint of the configuration file when this Makefile was last generated
CONFIG_HASH= c15ccab5eeab92844591b6c33e398db0bdeaa3b472273506368bae37fe44271e


# ----------------------------------------
# Standard Constants
Expand Down Expand Up @@ -47,9 +50,9 @@ REPORT_FAIL_ON = None
REPORT_LABEL = -l true
REPORT_PROFILE_OPTS =
OBO_FORMAT_OPTIONS =
SPARQL_VALIDATION_CHECKS = owldef-self-reference iri-range label-with-iri multiple-replaced_by
SPARQL_VALIDATION_CHECKS = owldef-self-reference iri-range label-with-iri multiple-replaced_by dc-properties
SPARQL_EXPORTS = basic-report class-count-by-prefix edges xrefs obsoletes synonyms
ODK_VERSION_MAKEFILE = v1.4
ODK_VERSION_MAKEFILE = v1.5.1

TODAY ?= $(shell date +%Y-%m-%d)
OBODATE ?= $(shell date +'%d:%m:%Y %H:%M')
Expand All @@ -64,6 +67,11 @@ FORMATS = $(sort owl obo json owl)
FORMATS_INCL_TSV = $(sort $(FORMATS) tsv)
RELEASE_ARTEFACTS = $(sort $(ONT)-base $(ONT)-full dron-lite )

ifeq ($(ODK_DEBUG),yes)
ODK_DEBUG_FILE = debug.log
SHELL = $(SCRIPTSDIR)/run-command.sh
endif

# ----------------------------------------
# Top-level targets
# ----------------------------------------
Expand All @@ -74,12 +82,16 @@ RELEASE_ARTEFACTS = $(sort $(ONT)-base $(ONT)-full dron-lite )
all: all_odk

.PHONY: all_odk
all_odk: odkversion test all_assets
all_odk: odkversion config_check test custom_reports all_assets

.PHONY: test
test: odkversion reason_test sparql_test robot_reports $(REPORTDIR)/validate_profile_owl2dl_$(ONT).owl.txt
echo "Finished running all tests successfully."

.PHONY: test
test_fast:
$(MAKE_FAST) test

.PHONY: release_diff
release_diff: $(REPORTDIR)/release-diff.md

Expand All @@ -90,13 +102,52 @@ reason_test: $(EDIT_PREPROCESSED)

.PHONY: odkversion
odkversion:
echo "ODK Makefile version: $(ODK_VERSION_MAKEFILE) (this is the version of the ODK with which this Makefile was generated, \
not the version of the ODK you are running)" &&\
echo "ROBOT version (ODK): " && $(ROBOT) --version
@echo "ODK Makefile $(ODK_VERSION_MAKEFILE)"
@odk-info --tools
.PHONY: config_check
config_check:
@if [ "$$(sha256sum $(ONT)-odk.yaml | cut -c1-64)" = "$(CONFIG_HASH)" ]; then \
echo "Repository is up-to-date." ; else \
echo "Your ODK configuration has changed since this Makefile was generated. You may need to run 'make update_repo'." ; fi


$(TMPDIR) $(REPORTDIR) $(MIRRORDIR) $(IMPORTDIR) $(COMPONENTSDIR) $(SUBSETDIR):
mkdir -p $@

# ----------------------------------------
# ODK-managed ROBOT plugins
# ----------------------------------------

# Make sure ROBOT knows where to find plugins
export ROBOT_PLUGINS_DIRECTORY=$(TMPDIR)/plugins

# Override this rule in dron.Makefile to install custom plugins
.PHONY: custom_robot_plugins
custom_robot_plugins:


.PHONY: extra_robot_plugins
extra_robot_plugins:


# Install all ROBOT plugins to the runtime plugins directory
.PHONY: all_robot_plugins
all_robot_plugins: $(foreach plugin,$(notdir $(wildcard /tools/robot-plugins/*.jar)),$(ROBOT_PLUGINS_DIRECTORY)/$(plugin)) \
$(foreach plugin,$(notdir $(wildcard ../../plugins/*.jar)),$(ROBOT_PLUGINS_DIRECTORY)/$(plugin)) \
custom_robot_plugins extra_robot_plugins \

# Default rule to install plugins
$(ROBOT_PLUGINS_DIRECTORY)/%.jar:
@mkdir -p $(ROBOT_PLUGINS_DIRECTORY)
@if [ -f ../../plugins/$*.jar ]; then \
ln ../../plugins/$*.jar $@ ; \
elif [ -f /tools/robot-plugins/$*.jar ]; then \
cp /tools/robot-plugins/$*.jar $@ ; \
fi

# Specific rules for supplementary plugins defined in configuration


# ----------------------------------------
# Release assets
# ----------------------------------------
Expand Down Expand Up @@ -186,10 +237,10 @@ validate_profile_%: $(REPORTDIR)/validate_profile_owl2dl_%.txt

SPARQL_VALIDATION_QUERIES = $(foreach V,$(SPARQL_VALIDATION_CHECKS),$(SPARQLDIR)/$(V)-violation.sparql)

sparql_test: $(EDIT_PREPROCESSED) catalog-v001.xml | $(REPORTDIR)
sparql_test: $(EDIT_PREPROCESSED) | $(REPORTDIR)
ifneq ($(SPARQL_VALIDATION_QUERIES),)

$(ROBOT) verify --catalog catalog-v001.xml -i $(EDIT_PREPROCESSED) --queries $(SPARQL_VALIDATION_QUERIES) -O $(REPORTDIR)
$(ROBOT) verify -i $(EDIT_PREPROCESSED) --queries $(SPARQL_VALIDATION_QUERIES) -O $(REPORTDIR)
endif

# ----------------------------------------
Expand All @@ -202,6 +253,10 @@ $(REPORTDIR)/$(SRC)-obo-report.tsv: $(SRCMERGED) | $(REPORTDIR)
$(REPORTDIR)/%-obo-report.tsv: % | $(REPORTDIR)
$(ROBOT) report -i $< $(REPORT_LABEL) $(REPORT_PROFILE_OPTS) --fail-on $(REPORT_FAIL_ON) --base-iri $(URIBASE)/DRON_ --base-iri $(URIBASE)/dron --print 5 -o $@

check_for_robot_updates:
echo "You are not using a custom profile, so you are getting the joy of the latest ROBOT report!"


# ----------------------------------------
# Release assets
# ----------------------------------------
Expand All @@ -218,15 +273,15 @@ RELEASE_ASSETS = \
$(SUBSET_FILES)

.PHONY: all_assets
all_assets: $(ASSETS)
all_assets: $(ASSETS) check_rdfxml_assets

.PHONY: show_assets
show_assets:
echo $(ASSETS)
du -sh $(ASSETS)

check_rdfxml_%: %
@check-rdfxml $<
@check-rdfxml $<

.PHONY: check_rdfxml_assets
check_rdfxml_assets: $(foreach product,$(MAIN_PRODUCTS),check_rdfxml_$(product).owl)
Expand Down Expand Up @@ -414,9 +469,9 @@ recreate-%:
no-mirror-recreate-%:
$(MAKE) COMP=true IMP=false IMP_LARGE=false MIR=false PAT=true $(COMPONENTSDIR)/$*.owl -B

$(COMPONENTSDIR)/%: | $(COMPONENTSDIR)
touch $@
.PRECIOUS: $(COMPONENTSDIR)/%
$(COMPONENTSDIR)/%.owl: | $(COMPONENTSDIR)
test -f $@ || touch $@
.PRECIOUS: $(COMPONENTSDIR)/%.owl



Expand All @@ -429,93 +484,99 @@ IMP=true # Global parameter to bypass import generation
MIR=true # Global parameter to bypass mirror generation
IMP_LARGE=true # Global parameter to bypass handling of large imports

ifeq ($(strip $(MIR)),true)


## ONTOLOGY: ro
.PHONY: mirror-ro
.PRECIOUS: $(MIRRORDIR)/ro.owl
mirror-ro: | $(TMPDIR)
if [ $(MIR) = true ] && [ $(IMP) = true ]; then curl -L $(OBOBASE)/ro.owl --create-dirs -o $(MIRRORDIR)/ro.owl --retry 4 --max-time 200 &&\
$(ROBOT) convert -i $(MIRRORDIR)/ro.owl -o $@.tmp.owl &&\
mv $@.tmp.owl $(TMPDIR)/$@.owl; fi
curl -L $(OBOBASE)/ro.owl --create-dirs -o $(TMPDIR)/ro-download.owl --retry 4 --max-time 200 && \
$(ROBOT) convert -i $(TMPDIR)/ro-download.owl -o $(TMPDIR)/$@.owl


## ONTOLOGY: apollo_sv
.PHONY: mirror-apollo_sv
.PRECIOUS: $(MIRRORDIR)/apollo_sv.owl
mirror-apollo_sv: | $(TMPDIR)
if [ $(MIR) = true ] && [ $(IMP) = true ]; then curl -L $(OBOBASE)/apollo_sv.owl --create-dirs -o $(MIRRORDIR)/apollo_sv.owl --retry 4 --max-time 200 &&\
$(ROBOT) convert -i $(MIRRORDIR)/apollo_sv.owl -o $@.tmp.owl &&\
mv $@.tmp.owl $(TMPDIR)/$@.owl; fi
curl -L $(OBOBASE)/apollo_sv.owl --create-dirs -o $(TMPDIR)/apollo_sv-download.owl --retry 4 --max-time 200 && \
$(ROBOT) convert -i $(TMPDIR)/apollo_sv-download.owl -o $(TMPDIR)/$@.owl


## ONTOLOGY: go
.PHONY: mirror-go
.PRECIOUS: $(MIRRORDIR)/go.owl
mirror-go: | $(TMPDIR)
if [ $(MIR) = true ] && [ $(IMP) = true ]; then curl -L $(OBOBASE)/go.owl --create-dirs -o $(MIRRORDIR)/go.owl --retry 4 --max-time 200 &&\
$(ROBOT) convert -i $(MIRRORDIR)/go.owl -o $@.tmp.owl &&\
mv $@.tmp.owl $(TMPDIR)/$@.owl; fi
curl -L $(OBOBASE)/go.owl --create-dirs -o $(TMPDIR)/go-download.owl --retry 4 --max-time 200 && \
$(ROBOT) convert -i $(TMPDIR)/go-download.owl -o $(TMPDIR)/$@.owl


## ONTOLOGY: obi
.PHONY: mirror-obi
.PRECIOUS: $(MIRRORDIR)/obi.owl
mirror-obi: | $(TMPDIR)
if [ $(MIR) = true ] && [ $(IMP) = true ]; then curl -L $(OBOBASE)/obi.owl --create-dirs -o $(MIRRORDIR)/obi.owl --retry 4 --max-time 200 &&\
$(ROBOT) convert -i $(MIRRORDIR)/obi.owl -o $@.tmp.owl &&\
mv $@.tmp.owl $(TMPDIR)/$@.owl; fi
curl -L $(OBOBASE)/obi.owl --create-dirs -o $(TMPDIR)/obi-download.owl --retry 4 --max-time 200 && \
$(ROBOT) convert -i $(TMPDIR)/obi-download.owl -o $(TMPDIR)/$@.owl


## ONTOLOGY: ogms
.PHONY: mirror-ogms
.PRECIOUS: $(MIRRORDIR)/ogms.owl
mirror-ogms: | $(TMPDIR)
if [ $(MIR) = true ] && [ $(IMP) = true ]; then curl -L $(OBOBASE)/ogms.owl --create-dirs -o $(MIRRORDIR)/ogms.owl --retry 4 --max-time 200 &&\
$(ROBOT) convert -i $(MIRRORDIR)/ogms.owl -o $@.tmp.owl &&\
mv $@.tmp.owl $(TMPDIR)/$@.owl; fi
curl -L $(OBOBASE)/ogms.owl --create-dirs -o $(TMPDIR)/ogms-download.owl --retry 4 --max-time 200 && \
$(ROBOT) convert -i $(TMPDIR)/ogms-download.owl -o $(TMPDIR)/$@.owl


## ONTOLOGY: bfo
.PHONY: mirror-bfo
.PRECIOUS: $(MIRRORDIR)/bfo.owl
mirror-bfo: | $(TMPDIR)
if [ $(MIR) = true ] && [ $(IMP) = true ]; then curl -L $(OBOBASE)/bfo.owl --create-dirs -o $(MIRRORDIR)/bfo.owl --retry 4 --max-time 200 &&\
$(ROBOT) convert -i $(MIRRORDIR)/bfo.owl -o $@.tmp.owl &&\
mv $@.tmp.owl $(TMPDIR)/$@.owl; fi
curl -L $(OBOBASE)/bfo.owl --create-dirs -o $(TMPDIR)/bfo-download.owl --retry 4 --max-time 200 && \
$(ROBOT) convert -i $(TMPDIR)/bfo-download.owl -o $(TMPDIR)/$@.owl


## ONTOLOGY: omo
.PHONY: mirror-omo
.PRECIOUS: $(MIRRORDIR)/omo.owl
mirror-omo: | $(TMPDIR)
if [ $(MIR) = true ] && [ $(IMP) = true ]; then curl -L $(OBOBASE)/omo.owl --create-dirs -o $(MIRRORDIR)/omo.owl --retry 4 --max-time 200 &&\
$(ROBOT) convert -i $(MIRRORDIR)/omo.owl -o $@.tmp.owl &&\
mv $@.tmp.owl $(TMPDIR)/$@.owl; fi
curl -L $(OBOBASE)/omo.owl --create-dirs -o $(TMPDIR)/omo-download.owl --retry 4 --max-time 200 && \
$(ROBOT) convert -i $(TMPDIR)/omo-download.owl -o $(TMPDIR)/$@.owl


## ONTOLOGY: chebi
.PHONY: mirror-chebi
.PRECIOUS: $(MIRRORDIR)/chebi.owl
ifeq ($(strip $(IMP_LARGE)),true)
mirror-chebi: | $(TMPDIR)
if [ $(MIR) = true ] && [ $(IMP) = true ] && [ $(IMP_LARGE) = true ]; then curl -L $(OBOBASE)/chebi.owl.gz --create-dirs -o $(MIRRORDIR)/chebi.owl.gz --retry 4 --max-time 200 &&\
$(ROBOT) convert -i $(MIRRORDIR)/chebi.owl.gz -o $@.tmp.owl &&\
mv $@.tmp.owl $(TMPDIR)/$@.owl; fi
curl -L $(OBOBASE)/chebi.owl.gz --create-dirs -o $(MIRRORDIR)/chebi.owl.gz --retry 4 --max-time 200 && \
$(ROBOT) convert -i $(MIRRORDIR)/chebi.owl.gz -o $(TMPDIR)/$@.owl
else
mirror-chebi:
@echo "Not refreshing chebi because refreshing large imports is disabled (IMP_LARGE=$(IMP_LARGE))."
endif


## ONTOLOGY: pr
.PHONY: mirror-pr
.PRECIOUS: $(MIRRORDIR)/pr.owl
ifeq ($(strip $(IMP_LARGE)),true)
mirror-pr: | $(TMPDIR)
if [ $(MIR) = true ] && [ $(IMP) = true ] && [ $(IMP_LARGE) = true ]; then curl -L $(OBOBASE)/pr.owl.gz --create-dirs -o $(MIRRORDIR)/pr.owl.gz --retry 4 --max-time 200 &&\
$(ROBOT) convert -i $(MIRRORDIR)/pr.owl.gz -o $@.tmp.owl &&\
mv $@.tmp.owl $(TMPDIR)/$@.owl; fi
curl -L $(OBOBASE)/pr.owl.gz --create-dirs -o $(MIRRORDIR)/pr.owl.gz --retry 4 --max-time 200 && \
$(ROBOT) convert -i $(MIRRORDIR)/pr.owl.gz -o $(TMPDIR)/$@.owl
else
mirror-pr:
@echo "Not refreshing pr because refreshing large imports is disabled (IMP_LARGE=$(IMP_LARGE))."
endif


$(MIRRORDIR)/%.owl: mirror-% | $(MIRRORDIR)
if [ $(IMP) = true ] && [ $(MIR) = true ] && [ -f $(TMPDIR)/mirror-$*.owl ]; then if cmp -s $(TMPDIR)/mirror-$*.owl $@ ; then echo "Mirror identical, ignoring."; else echo "Mirrors different, updating." &&\
if [ -f $(TMPDIR)/mirror-$*.owl ]; then if cmp -s $(TMPDIR)/mirror-$*.owl $@ ; then echo "Mirror identical, ignoring."; else echo "Mirrors different, updating." &&\
cp $(TMPDIR)/mirror-$*.owl $@; fi; fi

else # MIR=false
$(MIRRORDIR)/%.owl:
@echo "Not refreshing $@ because the mirrorring pipeline is disabled (MIR=$(MIR))."
endif



Expand Down Expand Up @@ -604,9 +665,13 @@ ROBOT_RELEASE_IMPORT_MODE=$(ROBOT) merge --input $<
# ROBOT pipeline that removes imports, then merges components. This is for release artefacts that start from "base"
ROBOT_RELEASE_IMPORT_MODE_BASE=$(ROBOT) remove --input $< --select imports --trim false merge $(patsubst %, -i %, $(OTHER_SRC))

# base: All the axioms as they are editted by the editors, excluding reasoning
$(ONT)-base.owl: $(EDIT_PREPROCESSED) $(OTHER_SRC)
$(ROBOT_RELEASE_IMPORT_MODE_BASE) \
# base: A version of the ontology that does not include any externally imported axioms.
$(ONT)-base.owl: $(EDIT_PREPROCESSED) $(OTHER_SRC) $(IMPORT_FILES)
$(ROBOT_RELEASE_IMPORT_MODE) \
reason --reasoner ELK --equivalent-classes-allowed asserted-only --exclude-tautologies structural --annotate-inferred-axioms False \
relax \
reduce -r ELK \
remove --base-iri $(URIBASE)/DRON --axioms external --preserve-structure false --trim false \
$(SHARED_ROBOT_COMMANDS) \
annotate --link-annotation http://purl.org/dc/elements/1.1/type http://purl.obolibrary.org/obo/IAO_8000001 \
--ontology-iri $(ONTBASE)/$@ $(ANNOTATE_ONTOLOGY_VERSION) \
Expand Down Expand Up @@ -676,13 +741,16 @@ update_repo:
update_docs:
mkdocs gh-deploy --config-file ../../mkdocs.yaml

# Note to future generations: prepending ./ is a safety measure to ensure that
# the environment does not malicously set `CLEANFILES` to `\`.
# Note to future generations: computing the real path relative to the
# current directory is a way to ensure we only clean up directories that
# are located below the current directory, regardless of the contents of
# the *DIR variables.
.PHONY: clean
clean:
[ -n "$(MIRRORDIR)" ] && [ $(MIRRORDIR) != "." ] && [ $(MIRRORDIR) != "/" ] && [ $(MIRRORDIR) != ".." ] && [ -d ./$(MIRRORDIR) ] && rm -rf ./$(MIRRORDIR)/*
[ -n "$(TMPDIR)" ] && [ $(TMPDIR) != "." ] && [ $(TMPDIR) != "/" ] && [ $(TMPDIR) != ".." ] && [ -d ./$(TMPDIR) ] && rm -rf ./$(TMPDIR)/*
[ -n "$(UPDATEREPODIR)" ] && [ $(UPDATEREPODIR) != "." ] && [ $(UPDATEREPODIR) != "/" ] && [ $(UPDATEREPODIR) != ".." ] && [ -d ./$(UPDATEREPODIR) ] && rm -rf ./$(UPDATEREPODIR)/*
for dir in $(MIRRORDIR) $(TMPDIR) $(UPDATEREPODIR) ; do \
reldir=$$(realpath --relative-to=$$(pwd) $$dir) ; \
case $$reldir in .*|"") ;; *) rm -rf $$reldir/* ;; esac \
done
rm -f $(CLEANFILES)

.PHONY: help
Expand All @@ -701,6 +769,7 @@ Core commands:
* prepare_release_fast: Run the entire release pipeline without refreshing imports, recreating components or recompiling patterns.
* update_repo: Update the ODK repository setup using the config file dron-odk.yaml
* test: Running all validation tests
* test_fast: Runs the test suite, but without updating imports or components
* odkversion: Show the current version of the ODK Makefile and ROBOT.
* clean: Delete all temporary files
* help: Print ODK Usage information
Expand All @@ -718,7 +787,7 @@ Imports management:

Editor utilities:
* validate_idranges: Make sure your ID ranges file is formatted correctly
* normalize_src: Load and safe your dron-edit file after you to make sure its serialised correctly
* normalize_src: Load and save your dron-edit file after you to make sure its serialised correctly
* explain_unsat: If you have unsatisfiable classes, this command will create a markdown file (tmp/explain_unsat.md) which will explain all your unsatisfiable classes
* validate-all-tsv: Check all your tsv files for possible problems in syntax. Use ALL_TSV_FILES variable to list files
* validate-tsv: Check a tsv file for syntactic problems with tsvalid. Use TSV variable to pass filepath, e.g. make TSV=../my.tsv validate-tsv.
Expand Down
Loading

0 comments on commit 5787d54

Please sign in to comment.