AKC updates to v1.5

[schristley]

Consistency issues with the schema file as reported by the AKC project.

[javh]

From the call:

• Do we need do a v1.5 release if we're going to release v2.0?
• Are the changes to TimePoint and RepertoireGroup backwards incompatibility with v1.5? IIRC, the TimePoint changes weren't intended for v1.5 (I could be wrong).

[schristley]

From the call:

• Do we need do a v1.5 release if we're going to release v2.0?

Yes, the ADC is still using the v1.x release and probably will for a while yet, even when v2.0 is released.

• Are the changes to TimePoint and RepertoireGroup backwards incompatibility with v1.5? IIRC, the TimePoint changes weren't intended for v1.5 (I could be wrong).

yeah I'm pretty sure some of the changes are backwards incompatible, which is also why need v1.x for awhile as it requires converting data and code for the v2 schema.

[schristley]

@javh there's no rush on a 1.5.x release as we are working from the release-1.5 branch, but at some point when we get everything working, we should put out a release.

[javh]

Got it. Doesn't that mean we need to back out those TimePoint and RepertoireGroup changes from this PR, as they aren't compatible with v1.5? (But leave the RepertoireGroup changes in the v2.0 PR.)

[bcorrie]

Yes, I was going to mention that. If there are changes that are not backwards compatible, then it is a 1.6 release, not 1.5

[bcorrie]

From the call:

• Do we need do a v1.5 release if we're going to release v2.0?

Yes, the ADC is still using the v1.x release and probably will for a while yet, even when v2.0 is released.

@schristley why do you feel these fixes are needed in 1.5. None of them are critical (they aren't bug fixes per se), and some of them are breaking backwards compatibility to change the name of a field (e.g. unit -> time_point_unit) or change the type in a way that isn't particularly critical (number => integer). Both break backwards compatibility, and require repository and/or API implementation changes. This would also make 1.5 "more" incompatible with 1.4, which would make things worse, not better, at least in my opinion. If I am not mistaken most of the ADC repositories are v1.4 not v1.5 - I know all of ours are running v1.4.

The RepertoireGroupDetail perhaps provides better structure, but doesn't change anything other than the JSON/YAML. So this isn't really necessary is it as it isn't fixing a bug.

Why can't these wait for 2.0?

The one exception I could see is the Taxonomy additions, as those changes should have been there in the first place.

[bcorrie]

If I am not mistaken most of the ADC repositories are v1.4 not v1.5 - I know all of ours are running v1.4.

Indeed, the v1.2 ADC API spec, which is the latest is coupled to the v1.4.1 specification. We did not release a version of the ADC API spec that supported v1.5. It is confusing I admit, as the API version numbering and AIRR Standard version numbering are too tightly coupled, but we never resolved that.

#760

So from an AKC perspective, I think we should be looking at either extracting data from repositories using the ADC v1.2/AIRR v1.4 specs or the ADC v2/AIRR v2 specs.

Any AKC changes should probably target the v2.0 spec, no?

[schristley]

From the call:

• Do we need do a v1.5 release if we're going to release v2.0?

Yes, the ADC is still using the v1.x release and probably will for a while yet, even when v2.0 is released.

@schristley why do you feel these fixes are needed in 1.5.

I'm being loose with versioning, if the schema isn't backward compatible then yes it's 1.6. Call them bugs, call it schema cleaning or harmonization, but these are issues brought up in #813 for AKC integration. I don't think the ADC has to move to v1.6, might just move to v2.0 if that's ready.

[bcorrie]

I'm being loose with versioning, if the schema isn't backward compatible then yes it's 1.6. Call them bugs, call it schema cleaning or harmonization, but these are issues brought up in #813 for AKC integration. I don't think the ADC has to move to v1.6, might just move to v2.0 if that's ready.

"Ready" is an interesting term, but I think the AKC would be best served by having fixes we have identified now integrated into v2.0, and not make any changes to the 1.x versions. There might be some minor fixes we could do, but anything that breaks backwards compatibility I would suggest is not a good way to go.

[bcorrie]

So can we close this PR and focus on the v2.0 pull request #815

[schristley]

I'm being loose with versioning, if the schema isn't backward compatible then yes it's 1.6. Call them bugs, call it schema cleaning or harmonization, but these are issues brought up in #813 for AKC integration. I don't think the ADC has to move to v1.6, might just move to v2.0 if that's ready.

"Ready" is an interesting term, but I think the AKC would be best served by having fixes we have identified now integrated into v2.0, and not make any changes to the 1.x versions. There might be some minor fixes we could do, but anything that breaks backwards compatibility I would suggest is not a good way to go.

Fixes are going into v2, but the ADC and the AKC are not converting to V2 yet, and I don't think we even have a timeline for that. So for now, we need to work with 1.x

[bcorrie]

So for now, we need to work with 1.x

Yes, but I am suggesting that the AKC work with the existing ADC v1.2/AIRR v1.4 and any changes driven by the AKC would go into v2.0.

Currently VDJServer is running ADC API v1.2, just like the iReceptor repositories. Are you intending to upgrade VDJServer to AIRR 1.5.x or AIRR v1.6?

I am not intending on upgrading any of the iReceptor repositories beyond the current ADC API v1.2/AIRR v1.4 until we upgrade to v2.0. So even if there is a new version of the AIRR Standard, the data that is pulled out of the ADC is unlikely to have these new features that you are suggesting unless the repositories maintainers upgrade. The v2.0 upgrade is going to be hugely time consuming by itself...

[schristley]

Currently VDJServer is running ADC API v1.2, just like the iReceptor repositories. Are you intending to upgrade VDJServer to AIRR 1.5.x or AIRR v1.6?

From a source code perspective, VDJServer is already using AIRR 1.5 release-branch for the airr-js code that I added. Will I convert the data to v1.6 before going to v2.0. Probably not. I don't know, it might depend upon the timing, but I don't think it is necessary. At the same time, I don't think the changes are that major either. I have to review all of the changes completely before I make any decision there.

The AKC is using AIRR 1.x in a couple different ways. One is your conversion script, but that's tied to the version of the data itself and essentially independent from the AIRR standards version. The other is the schema conversion, and that's what would be affected, but right now we don't have anything using that linkml schema

[bcorrie]

The AKC is using AIRR 1.x in a couple different ways. One is your conversion script, but that's tied to the version of the data itself and essentially independent from the AIRR standards version.

The converter works on the data that comes out of the ADC, but it does need to know which version of the standard the data is being produced with. It would need a separate configuration file to do the conversion correctly if the ADC API version was different.