Reference and Attachment properties - add a clear way to reference and attach data into credentials

[Drabiv]

I would like to propose to add to the spec the properties Reference and Attachment and remove the notion of Profile, and properties Evidence and Term of Use.

The addition of property Reference will allow to have a universal way to reference anything that is relevant to the credential (e.g. credential's human readable presentation, other credentials, terms of use, evidences etc.) References can be external (pointing outside of credential) or internal (pointing to a node in credential (for example, to Attachment)).

The addition of property Attachment will allow to add into credential anything that might be useful for understanding claim and needs to be available inside credential for security reasons.

Attachment property will allow to drop the confusing notion of Profile making the idea that stands behind Profile (a bundle of credentials) clearer to understand and implement. It is important for many use cases when Profile has multiple subjects or subject is not clear.

The Profile will be replaced by a credential with attached credentials and Evidence and Terms of Use will be replaced by attachments with type "Evidence" and "Terms of Use".

Please, see below the example of a credential with proposed Reference and Attachment properties.

{
  "id": "9f26450a-1efb-11e8-b467-0ed5f89f718b",
  "type": [ "Credential", "AccessToResource"  ],
  "issuer": "did:example:123xxx789",
  "issued": "2018-03-03",
  "claim": {
    "id": "did:example:123xxx789",
    "resourceID": "http://example.com/example123",
    "access": "allowed"
  },
  "reference": [ {
      "id": "14cb57e2-2862-402e-8dd7-f2123a464456 //reference to human readable presentation of this credential",
      "type": [ "Image" ],
      "purpose": [ "Presentation" ],
      "hash": "8fe784bfeaf12ddb27ffcb2e515162af9218e95d5ef430bf13",
      "location": [ "http://example.com/example.img" ]
    },
    {
      "id": "http://example.com/example789 //reference to another statement, located externaly",
      "type": [ "Credential" ],
      "purpose": [ "Other" ],
      "hash": "18EE24150DCB1D96752A4D6DD0F20DFD8BA8C38527E40AA8509B7ADECF78F9C6",
      "location": [ "http://example.com/example789" ]
    },
    {
      "id": "http://example.com/example789 //reference to another statement, located in the attachment property of this credential",
      "type": [ "Credential" ],
      "purpose": [ "Other" ],
      "hash": "18EE24150DCB1D96752A4D6DD0F20DFD8BA8C38527E40AA8509B7ADECF78F9C6",
      "location": [ "$.attachment[:].id=ff227bae-dacf-4ef0-94be-b2550ab2ee0q" ]
    },
    {
      "id": "https://dmv.example.gov/evidence/f2aeec97-fc0d-42bf-8ca7-0548192d4231 //reference to evidence",
      "type": [ "DocumentVerification" ],
      "purpose": [ "Evidence" ],
      "hash": "1feasdfbfesdgff12ddsdb27ffcb2e515162ge95d5ef430bf1u",
      "location": "$.attachment[:].id=https://dmv.example.gov/evidence/f2aeec97-fc0d-42bf-8ca7-0548192d4231"
    },
    {
      "id": "18EE24150DCB1D96752A4D6DD0F20DFD8BA8C38527E40AA8509B7ADECF78F9C6 //reference to some binary file",
      "type": [ "BinaryFile" ],
      "purpose": [ "Other" ],
      "hash": "18EE24150DCB1D96752A4D6DD0F20DFD8BA8C38527E40AA8509B7ADECF78F9C6",
      "location": [ "http://example.com/file.zip.123" ]
    }
  ],
  "attachment": [
    {
      "type": [ "Evidence" ],
      "content": {
        "id": "https://dmv.example.gov/evidence/f2aeec97-fc0d-42bf-8ca7-0548192d4231",
        "type": "DocumentVerification",
        "verifier": "https://dmv.example.gov/issuers/14",
        "evidenceDocument": "DriversLicense",
        "subjectPresence": "Physical",
        "documentPresence": "Physical"
      }
    },
    {
      "type": [ "Credential", "ProofOfAgeCredential" ],
      "content": {
        "id": "ff227bae-dacf-4ef0-94be-b2550ab2ee0q",
        "type": [ "Credential" ],
        "issuer": "did:example:example.gov",
        "issued": "2018-03-03",
        "claim": {
          "id": "did:example:123xxx789",
          "ageOver": 21
        },
        "proof": [{
            "type": "RSA...",
            "created": "2018-03-03T21:10:38Z",
            "creator": "did:example:example.gov",
            "nonce": "ea227bae-dacf-4ef0-94be-b2550ab2ee0f",
            "signatureValue": "g4j9UrpHM4/uu32NlTw0HDaSaYF2sykskfuByD7UbuqEc"
          }
        ]
      }
    }
  ],
  "proof": [ {
      "type": "EdDsaSAPublicKeySecp256k1",
      "created": "2018-03-03T21:10:38Z",
      "creator": "did:example:123xxx789",
      "nonce": "ea227bae-dacf-4ef0-94be-b2550ab2ee0f",
      "signatureValue": "g4j9UrpHM4/uu32NlTw0HDaSaYF2sykskfuByD7UbuqEc"
    }
  ]
}

[dlongley]

I'm a general -1 to the "catch-all bucket" pattern. It usually an anti-pattern to create predicates that have no clear semantics -- and then to embed the actual semantics within objects in the bucket. It does not adhere to the graph model. This isn't strictly always the case -- and there can be fairly generic categories of relationships that work out, but I think the above is too generic and wouldn't be a step in the right direction. That doesn't mean it isn't a good attempt at trying to tease out how to remedy some issues we have in the data model.

To elaborate on the issue with the "bucket" model take a look at, for example, the "purpose" field above. This field is actually defining the relationship between the credential and the referenced object but appears within the object instead of as a link from the credential to the object. From a graph modeling perspective, it should really be a predicate/property/key that relates it to the credential rather than being a one-off property on the reference object itself.

What if you want to use a reference object for a different purpose? Who defines what that purpose is? The answer is the entity that establishes the relationship (i.e. the credential here). In a graph model, this is through predicates (the properties/keys in the JSON syntax). Therefore that predicate should define how X has chosen to relate itself to Y. Having this as a general model that one can assume without special contextual rules is powerful. To be more clear, we have this:

A credential C with a "reference" to some document D. Why is it referenced? We don't know. We have to look at D to figure that out ... because D defines how it can be related to anything else and it does it with a special property of its own -- which is an inversion of how relationships typically work.

Instead, if C has an "evidence" relationship with document D, then we know right away that D is something that was used as evidence to create C. We don't even have to know anything about D yet to understand that. The semantics are clear -- the relationship between C and D is clear. Now -- if we're looking for evidence, we know exactly where to look ... at objects that are related to C via the "evidence" field. This is what an application that would use evidence would want to do and would be looking for -- and it wouldn't need to waste its time with anything else.

Another way of looking at this is to take the above modeling change to the extreme. Suppose we just created a single field "data" in a credential (the ultimate "catch-all bucket"). Then we put every object that C is related to into that field. From there, each object could define what its possible relationships are, potentially each in its own special way. This, of course, is a mess in a graph model -- and the reason why the "bucket" approach is usually an anti-pattern.

It's better to define strong semantics for the relationships between objects and use those as the keys for referencing things. Usually, when you create a term like "reference" as a predicate, what you're saying is that you don't know the semantics. That means someone who is trying to use your data won't know either. That's a problem. Or, maybe the semantics are just hiding behind some special rules... now they have to know those special rules to unravel the mystery -- and code up every kind of special rule they want to use.

That approach creates unnecessary work for developers -- and is a major reason why everyone keeps rewriting the same applications -- just in slightly different ways. To avoid that, one thing you need is an extensible, open data model and some rules that govern it. It takes a little extra work upfront to get it right, which is one reason people don't bother. Another reason some people don't bother is because they don't need wide interoperability, which is fine. But if you do need it, and I argue that we do per our charter, doing that extra bit of work can save endless inefficiency in the longer term.

Lastly, I can't emphasize enough that we've taken the approach of modeling our data as a graph. One reason for this is that, if you are careful about doing so, then your data can work with both graph-based systems and tree-based systems. If you only model things as a naive tree, you end up implicitly embedding context at every level of the tree and relationships are not explicitly linked using a general form. This means you cut out the possibility for your data model to be used as-is by graph-based systems (and there are a lot of these... including "the Web"). We can support both here -- so we should be careful not to cut either out.

[Drabiv]

Thanks for reply @dlongley

Let me provide background for this proposal and my understanding of VC standard. Hopefully it will allow to come to a common understanding and solution.

My understanding of VC standard:
What is VC standard? Is it a universal way to digitally sign data OR a universal way to create "small size" credentials (passports, driving licenses, diploma certificates etc).

I understand VC standard as a universal standard to sign data.
Although I realise, (from the name of the standard and from the examples provided in the spec) that currently it's main use cases are in the area of the "small size" credentials.

Background of the proposal:
I am trying to implement VC standard to sign various "big size" documents.

For example:

• business contracts
  (often business contracts contain attached copies of identity documents, references to different parties, appendixes with a lot of data that explain the contract)
• court decisions
  (often court decisions includes references to (attachments of) witness statements, expert statements, evidences)
• complex certificates (educational, occupational, medical)
  (Complex certificates (diploma supplements, CVs, licenses, permissions, prescriptions) may have many data points that need to reference to evidences or credentials that prove values in the data points. For security and convenience it would be perfect to have referenced data in the credentials itself.)

Requirements:

1. reference to human readable presentation of the credential (have temper-proofed evidence of what user saw when he signed the credential)
2. reference to evidence that proves that credential's claim is true
3. reference to a credential issued by trusted authority stating that they verified the evidence
4. reference to other credentials that are related to this credential (relationship between credential may be easy to define and may be not easy to define)
5. reference to other data that is related to this credential (relationship between credential and data may or may not be clear)
6. referenced data should be tamper-proofed
7. credential needs to be portable and off-line verifiable OR referenced data needs to be not deletable while adding that data into DLT is not possible
8. credential needs to be easily readable by developers

Solutions:

• To satisfy requirements 1,2,3,4,5 add to specification a universal way to reference. Use properties Reference, Purpose(of reference), Hash (of referenced data) to do so.
• To satisfy requirements 6, 7 add property Attachment - allow to add serialized data into Attachment property.
• To satisfy requirement 8 decouple notion of Reference and Attachment. (We could have only a property of Reference and add data straight into reference when we need portability, but that credential would be difficult to read for developers, hence proposal to have Attachment property at the bottom of the credential.)

Comments:
@dlongley I did not understand the issue of graph-based vs tree-based approach and catch-all bucket pattern. As per my understanding adding properties Reference, Purpose(of reference), Hash (of referenced data) continue graph-based approach of VC standard. I am quite new to this topic so I am probably missing something.

My main point/request to the spec is to provide a clear way to reference and attach data into a credential (taking into the account that relationship between credential and referenced data may or may not be clearly defined).

IMO, the proposed properties add an universal, standard way to reference, and attach data in credentials. Coincidently they also provide a way to bundle credentials and explain what that bundle means - this allows to drop from the spec, IMO quite confusing and hard to implement notion of Profile.

To enhance interoperability we will probably need to agree on properties of attachments referenced as "Evidence", "Terms of Use" etc.

[dlongley]

@Drabiv,

What is VC standard? Is it a universal way to digitally sign data OR a universal way to create "small size" credentials (passports, driving licenses, diploma certificates etc).

I understand VC standard as a universal standard to sign data.
Although I realise, (from the name of the standard and from the examples provided in the spec) that currently it's main use cases are in the area of the "small size" credentials.

I don't think it is either of these. It is not a universal way to digitally sign data (this is, for example, left to the Linked Data Signatures work -- or an existing standard such as JOSE if it suits your use case). This work is also not restricted to "small size" credentials, but the scope is restricted to credentials and data directly related to them.

I may have more to say about the rest of your comments later -- just getting this quick opinion out there.

[dlongley]

@Drabiv,

I think each of the requirements you mentioned may need a separate issue (or already has one or something for it in the data model). I'm not convinced there's a general solution to this problem.

reference to human readable presentation of the credential (have temper-proofed evidence of what user saw when he signed the credential)

#42

reference to evidence that proves that credential's claim is true

I believe we have "evidence" in the data model.

reference to a credential issued by trusted authority stating that they verified the evidence
reference to other credentials that are related to this credential (relationship between credential may be easy to define and may be not easy to define)

Can you open a separate issue to address these?

reference to other data that is related to this credential (relationship between credential and data may or may not be clear)

As argued above, I think the actual kind of relationship is important, so I don't think saying "this is related" without saying how is a good thing.

referenced data should be tamper-proofed credential needs to be portable and off-line verifiable OR referenced data needs to be not deletable while adding that data into DLT is not possible

This seems to need its own issue (and some of it may be out of scope for this group).
Related: #132

credential needs to be easily readable by developers

This seems to be a general requirement ... we're using JSON and JSON-LD which are intended to make the credential readable by developers.

[Drabiv]

@dlongley

I think each of the requirements you mentioned may need a separate issue (or already has one or something for it in the data model)

Those issues are closely interwined and I think should be addressed together. I will open separate issues and reference this one to have them understood as being a part of a solution to this issue.

I'm not convinced there's a general solution to this problem.

Why not to use proposed "reference", "reference.purpose" and "attachment" properties? Because it mixes graph-based approach with tree-based? It is simple and effective as per my reckoning.

reference to human readable presentation of the credential (have temper-proofed evidence of what user saw when he signed the credential)

#42

Issue #42 is about how to render Credential, what I am proposing is to have a standard way to hash presentation that user saw during signing and reference it in the credential, or serialize presentation and attach into credential itself. The later is not elegant, but it is bulletproof against loss of data.

credential needs to be easily readable by developers

This seems to be a general requirement ... we're using JSON and JSON-LD which are intended to make >the credential readable by developers.

This requirement should be understood in the context of all other requirements. Meaning - make possible to reference and attach data in credentials and make it simple to read and understand.

[Drabiv]

Created issues #135 and #136 to separate issues with referencing credentials vs. referencing generic data
The below requirements should be satisifed in both issues.

• relation of referenced/attached data to the credential may or may not be clear at the moment of creating credential
• referenced data should be tamper-proofed (in rare cases may be allowed to be not tamper-proofed)
• credential may need to be portable and off-line verifiable
• referencing/attaching data needs to be easily understood by developers

[burnburn]

Hi @Drabiv , we'd like to have you join the Working Group so we can more fully consider your suggestions. Please see #135 (comment) for details.

[Drabiv]

@burnburn I have applied to become IE to join the group. Thank you!

[msporny]

This is a duplicate of #135 and #136, closing.