OpenConceptLab/ocl_issues#147: Users can view collections that are marked private if the user query is specified

[karuhanga]

Address OpenConceptLab/ocl_issues#147

[karuhanga]

Still adding permissions tests.

[rkorytkowski]

For permission tests please see https://github.com/OpenConceptLab/oclapi/tree/master/integration-tests

[karuhanga]

Hey @rkorytkowski,
I've encountered a issue on this I'd like your help with.

[karuhanga]

@rkorytkowski
This is a problem I have been able to trace down to mongodb-engine by comparing the queryset query and what is actually executed on mongo. I'd really appreciate help with this. I have also added the issue to the project's issue repo(django-nonrel/mongodb-engine#238).

original qs

SELECT collection_collection.id, collection_collection.public_access, collection_collection.created_at, collection_collection.updated_at, collection_collection.created_by, collection_collection.updated_by, collection_collection.is_active, collection_collection.extras, collection_collection.uri, collection_collection.mnemonic, collection_collection.parent_type_id, collection_collection.parent_id, collection_collection.name, collection_collection.full_name, collection_collection.default_locale, collection_collection.supported_locales, collection_collection.website, collection_collection.description, collection_collection.external_id, collection_collection.custom_validation_schema, collection_collection.references, collection_collection.collection_type, collection_collection.preferred_source, collection_collection.repository_type, collection_collection.custom_resources_linked_source FROM collection_collection WHERE (collection_collection.is_active = True  AND collection_collection.parent_id = 5d721432394d010a2a73c0e3  AND collection_collection.parent_type_id = 5d664271394d01003b4a8133 ) [<Collection: 86285>, <Collection: 86772>]
{
        "op" : "query",
        "ns" : "ocl.collection_collection",
        "query" : {
                "find" : "collection_collection",
                "filter" : {
                        "parent_id" : "5d721432394d010a2a73c0e3",
                        "is_active" : true,
                        "parent_type_id" : ObjectId("5d664271394d01003b4a8133")
                },
                "ntoreturn" : 21
        },
        "ts" : ISODate("2019-09-06T08:09:23.162Z")
}


qs from public access 
SELECT collection_collection.id, collection_collection.public_access, collection_collection.created_at, collection_collection.updated_at, collection_collection.created_by, collection_collection.updated_by, collection_collection.is_active, collection_collection.extras, collection_collection.uri, collection_collection.mnemonic, collection_collection.parent_type_id, collection_collection.parent_id, collection_collection.name, collection_collection.full_name, collection_collection.default_locale, collection_collection.supported_locales, collection_collection.website, collection_collection.description, collection_collection.external_id, collection_collection.custom_validation_schema, collection_collection.references, collection_collection.collection_type, collection_collection.preferred_source, collection_collection.repository_type, collection_collection.custom_resources_linked_source FROM collection_collection WHERE (collection_collection.is_active = True  AND collection_collection.parent_id = 5d721432394d010a2a73c0e3  AND collection_collection.parent_type_id = 5d664271394d01003b4a8133  AND NOT (collection_collection.public_access = None )) [<Collection: 86772>]
{
        "op" : "query",
        "ns" : "ocl.collection_collection",
        "query" : {
                "find" : "collection_collection",
                "filter" : {
                        "parent_id" : "5d721432394d010a2a73c0e3",
                        "public_access" : {
                                "$ne" : "None"
                        },
                        "is_active" : true,
                        "parent_type_id" : ObjectId("5d664271394d01003b4a8133")
                },
                "ntoreturn" : 21
        },
        "ts" : ISODate("2019-09-06T08:09:23.166Z")
}


qs from permissions
SELECT collection_collection.id, collection_collection.public_access, collection_collection.created_at, collection_collection.updated_at, collection_collection.created_by, collection_collection.updated_by, collection_collection.is_active, collection_collection.extras, collection_collection.uri, collection_collection.mnemonic, collection_collection.parent_type_id, collection_collection.parent_id, collection_collection.name, collection_collection.full_name, collection_collection.default_locale, collection_collection.supported_locales, collection_collection.website, collection_collection.description, collection_collection.external_id, collection_collection.custom_validation_schema, collection_collection.references, collection_collection.collection_type, collection_collection.preferred_source, collection_collection.repository_type, collection_collection.custom_resources_linked_source FROM collection_collection WHERE (collection_collection.is_active = True  AND collection_collection.parent_id = 5d721432394d010a2a73c0e3  AND collection_collection.parent_type_id = 5d664271394d01003b4a8133  AND ((collection_collection.parent_id = 5d721432394d010a2a73c0e3  AND collection_collection.parent_type_id = 5d664271394d01003b4a8133 ) OR (collection_collection.parent_type_id = 5d664271394d01003b4a8131  AND collection_collection.parent_id IN (5d721432394d010a2a73c0e8)))) [<Collection: 86285>, <Collection: 86772>]
{
        "op" : "query",
        "ns" : "ocl.collection_collection",
        "query" : {
                "find" : "collection_collection",
                "filter" : {
                        "$or" : [
                                {
                                        "parent_id" : "5d721432394d010a2a73c0e3",
                                        "parent_type_id" : ObjectId("5d664271394d01003b4a8133")
                                },
                                {
                                        "parent_id" : {
                                                "$in" : [
                                                        "5d721432394d010a2a73c0e8"
                                                ]
                                        },
                                        "parent_type_id" : ObjectId("5d664271394d01003b4a8131")
                                }
                        ],
                        "parent_id" : "5d721432394d010a2a73c0e3",
                        "is_active" : true,
                        "parent_type_id" : ObjectId("5d664271394d01003b4a8133")
                },
                "ntoreturn" : 21
        },
        "ts" : ISODate("2019-09-06T08:09:23.173Z")
}



qs from both
SELECT collection_collection.id, collection_collection.public_access, collection_collection.created_at, collection_collection.updated_at, collection_collection.created_by, collection_collection.updated_by, collection_collection.is_active, collection_collection.extras, collection_collection.uri, collection_collection.mnemonic, collection_collection.parent_type_id, collection_collection.parent_id, collection_collection.name, collection_collection.full_name, collection_collection.default_locale, collection_collection.supported_locales, collection_collection.website, collection_collection.description, collection_collection.external_id, collection_collection.custom_validation_schema, collection_collection.references, collection_collection.collection_type, collection_collection.preferred_source, collection_collection.repository_type, collection_collection.custom_resources_linked_source FROM collection_collection WHERE ((collection_collection.is_active = True  AND collection_collection.parent_id = 5d721432394d010a2a73c0e3  AND collection_collection.parent_type_id = 5d664271394d01003b4a8133  AND NOT (collection_collection.public_access = None )) OR (collection_collection.is_active = True  AND collection_collection.parent_id = 5d721432394d010a2a73c0e3  AND collection_collection.parent_type_id = 5d664271394d01003b4a8133  AND ((collection_collection.parent_id = 5d721432394d010a2a73c0e3  AND collection_collection.parent_type_id = 5d664271394d01003b4a8133 ) OR (collection_collection.parent_type_id = 5d664271394d01003b4a8131  AND collection_collection.parent_id IN (5d721432394d010a2a73c0e8))))) [<Collection: 86285>, <Collection: 3870>, <Collection: 86772>, <Collection: 53710>]
{
        "op" : "query",
        "ns" : "ocl.collection_collection",
        "query" : {
                "find" : "collection_collection",
                "filter" : {
                        "$or" : [
                                {
                                        "parent_id" : "5d721432394d010a2a73c0e3",
                                        "public_access" : {
                                                "$ne" : "None"
                                        },
                                        "is_active" : true,
                                        "parent_type_id" : ObjectId("5d664271394d01003b4a8133")
                                },
                                {
                                        "parent_id" : "5d721432394d010a2a73c0e3",
                                        "parent_type_id" : ObjectId("5d664271394d01003b4a8133")
                                },
                                {
                                        "parent_id" : {
                                                "$in" : [
                                                        "5d721432394d010a2a73c0e8"
                                                ]
                                        },
                                        "parent_type_id" : ObjectId("5d664271394d01003b4a8131")
                                },
                                {
                                        "parent_id" : "5d721432394d010a2a73c0e3",
                                        "is_active" : true,
                                        "parent_type_id" : ObjectId("5d664271394d01003b4a8133")
                                }
                        ],
                },
                "ntoreturn" : 21
        },
        "ts" : ISODate("2019-09-06T08:09:23.178Z")
}

[rkorytkowski]

Thanks @karuhanga for further investigation. I'll be looking into that on Monday.

[karuhanga]

@rkorytkowski so I managed to find a work around for now.
I have forked mongo-db-engine and added tests to replicate this(karuhanga/mongodb-engine@7755014).
Feel free to check these out if you ever get the bandwidth to fix it. I'll try to look into it, but I am prioritising issues on the repo.

[karuhanga]

Awaiting resolution of OpenConceptLab/ocl_issues#151.

[rkorytkowski]

I think I might have coincidentally fixed the issue in commits I've just pushed. Could you please verify with the latest master?

[karuhanga]

I think I might have coincidentally fixed the issue in commits I've just pushed. Could you please verify with the latest master?

It appears so. Still looking through a few failing tests from the ones I'd written to be sure. However it seems like a new one has cropped up.
cf:
https://api.qa.openconceptlab.org/collections/?q=Roya*&limit=10&page=1&verbose=true
and
https://api.qa.openconceptlab.org/collections/?q=Roya*&limit=10&page=1&verbose=true&user=admin.
The one with the user query does not respect the q param.
Also, I am worried about the duplication in the source and collection tests going forward.

[rkorytkowski]

@karuhanga, thanks for testing. I'll look into the bug you found. I refactored tests as time allowed to eliminate duplication. At some point one has to simply stop refactoring ;) You are always welcome to refactor further.

[rkorytkowski]

Fixed user param query on collections in ec1734c