merge to update

.github/workflows/integration_tests.yaml

@@ -0,0 +1,27 @@
+name: Integration Tests
+
+on: pull_request
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  integration_tests:
+    name: Integration tests
+    uses: uc-cdis/.github/.github/workflows/integration_tests.yaml@master
+    with:
+      SERVICE_TO_TEST: fence
+    secrets:
+      CI_AWS_ACCESS_KEY_ID: ${{ secrets.CI_AWS_ACCESS_KEY_ID }}
+      CI_AWS_SECRET_ACCESS_KEY: ${{ secrets.CI_AWS_SECRET_ACCESS_KEY }}
+      JENKINS_API_TOKEN: ${{ secrets.JENKINS_API_TOKEN }}
+      QA_DASHBOARD_S3_PATH: ${{ secrets.QA_DASHBOARD_S3_PATH }}
+      CI_TEST_ORCID_USERID: ${{ secrets.CI_TEST_ORCID_USERID }}
+      CI_TEST_ORCID_PASSWORD: ${{ secrets.CI_TEST_ORCID_PASSWORD }}
+      CI_TEST_RAS_USERID: ${{ secrets.CI_TEST_RAS_USERID }}
+      CI_TEST_RAS_PASSWORD: ${{ secrets.CI_TEST_RAS_PASSWORD }}
+      CI_TEST_RAS_2_USERID: ${{ secrets.CI_TEST_RAS_2_USERID }}
+      CI_TEST_RAS_2_PASSWORD: ${{ secrets.CI_TEST_RAS_2_PASSWORD }}
+      CI_SLACK_BOT_TOKEN: ${{ secrets.CI_SLACK_BOT_TOKEN }}
+      CI_SLACK_CHANNEL_ID: ${{ secrets.CI_SLACK_CHANNEL_ID }}

.secrets.baseline

@@ -314,14 +314,14 @@
         "filename": "tests/data/test_indexed_file.py",
         "hashed_secret": "a62f2225bf70bfaccbc7f1ef2a397836717377de",
         "is_verified": false,
-        "line_number": 411
+        "line_number": 449
       },
       {
         "type": "Secret Keyword",
         "filename": "tests/data/test_indexed_file.py",
         "hashed_secret": "c258a8d1264cc59de81f8b1975ac06732b1cf182",
         "is_verified": false,
-        "line_number": 432
+        "line_number": 470
       }
     ],
     "tests/keys/2018-05-01T21:29:02Z/jwt_private_key.pem": [
@@ -422,5 +422,5 @@
       }
     ]
   },
-  "generated_at": "2024-08-22T19:43:39Z"
+  "generated_at": "2024-11-04T09:20:13Z"
 }

README.md

@@ -137,6 +137,7 @@ See detailed explanation [here](docs/additional_documentation/setup.md)
 
 1. [Terminologies](docs/additional_documentation/terminology.md)
 2. [Accessing Data](docs/additional_documentation/data_access.md#accessing-data)
-3. [Token management](docs/additional_documentation/token_management.md)
-4. [fence-create](docs/additional_documentation/fence_create.md)
-5. [Default expiration times](docs/additional_documentation/default_expiration_times.md)
+3. [user.yaml guide](docs/additional_documentation/user.yaml_guide.md)
+4. [Token management](docs/additional_documentation/token_management.md)
+5. [fence-create](docs/additional_documentation/fence_create.md)
+6. [Default expiration times](docs/additional_documentation/default_expiration_times.md)

docs/additional_documentation/fence_create.md

@@ -53,7 +53,7 @@ curl --request POST https://FENCE_URL/oauth2/token?grant_type=client_credentials
 
 The optional `--expires-in` parameter allows specifying the number of *days* until this client expires. The recommendation is to rotate credentials with the `client_credentials` grant at least once a year (see [Rotate client credentials](#rotate-client-credentials) section).
 
-NOTE: In Gen3, you can grant specific access to a client the same way you would to a user. See the [user.yaml guide](https://github.com/uc-cdis/fence/blob/master/docs/user.yaml_guide.md) for more details.
+NOTE: In Gen3, you can grant specific access to a client the same way you would to a user. See the [user.yaml guide](https://github.com/uc-cdis/fence/blob/master/docs/additional_documentation/user.yaml_guide.md) for more details.
 
 NOTE: Client credentials tokens are not linked to a user (the claims contain no `sub` or `context.user.name` like other tokens). Some Gen3 endpoints that assume the token is linked to a user, or whose logic require there being a user, do not support them. For an example of how to adapt an endpoint to support client credentials tokens, see [here](https://github.com/uc-cdis/requestor/commit/a5078fae27fa258ac78045cf2bb89cb2104f53cf). For an example of how to explicitly reject client credentials tokens, see [here](https://github.com/uc-cdis/requestor/commit/0f4974c25343d2185c7cdb48dcdeb58f97800672).
 

docs/additional_documentation/setup.md

@@ -154,4 +154,4 @@ saved by the OAuth client to use with
 ## Quickstart with Helm
 
 You can now deploy individual services via Helm!
-Please refer to the Helm quickstart guide HERE (https://github.com/uc-cdis/fence/blob/master/docs/quickstart_helm.md)
+Please refer to the Helm quickstart guide HERE (https://github.com/uc-cdis/fence/blob/master/docs/additional_documentation/quickstart_helm.md)

docs/additional_documentation/user.yaml_guide.md

@@ -16,7 +16,7 @@
 
 The `user.yaml` file is one way to get authorization information into Gen3. It is ingested via [Fence's `usersync` script](usersync.md). The format of this file is tightly coupled with the notions of resource, role and policy as defined by Gen3's policy engine, [Arborist](https://github.com/uc-cdis/arborist#arborist).
 
-For Gen3 Data Commons that do not use Arborist or that use the Google Data Access method of [Google Service Account Registration](https://github.com/uc-cdis/fence/blob/master/docs/google_architecture.md#google-account-linking-and-service-account-registration), refer to the [Deprecated format](#deprecated-format) section.
+For Gen3 Data Commons that do not use Arborist or that use the Google Data Access method of [Google Service Account Registration](https://github.com/uc-cdis/fence/blob/master/docs/additional_documentation/google_architecture.md#google-account-linking-and-service-account-registration), refer to the [Deprecated format](#deprecated-format) section.
 
 In a fully deployed Gen3 Commons using [Cloud Automation](https://github.com/uc-cdis/cloud-automation), the `user.yaml` file is usually hosted in S3 and configured via the `global.useryaml_s3path` setting of the Gen3 Data Commons manifest:
 ```

fence/auth.py

@@ -19,6 +19,7 @@
 from fence.user import get_current_user
 from fence.utils import clear_cookies
 from fence.config import config
+from fence.authz.auth import check_arborist_auth
 
 logger = get_logger(__name__)
 
@@ -100,6 +101,12 @@ def set_flask_session_values(user):
 
     user = query_for_user(session=current_app.scoped_session(), username=username)
     if user:
+        if user.active == False:
+            # Abort login if user.active == False:
+            raise Unauthorized(
+                "User is known but not authorized/activated in the system"
+            )
+
         _update_users_email(user, email)
         _update_users_id_from_idp(user, id_from_idp)
         _update_users_last_auth(user)
@@ -111,7 +118,11 @@ def set_flask_session_values(user):
             set_flask_session_values(user)
             return
     else:
-        # we need a new user
+        if not config["ALLOW_NEW_USER_ON_LOGIN"]:
+            # do not create new active users automatically
+            raise Unauthorized("New user is not yet authorized/activated in the system")
+
+        # add the new user
         user = User(username=username)
 
         if email:
@@ -264,25 +275,9 @@ def get_user_from_claims(claims):
     )
 
 
-def admin_required(f):
-    """
-    Require user to be an admin user.
-    """
-
-    @wraps(f)
-    def wrapper(*args, **kwargs):
-        if not flask.g.user:
-            raise Unauthorized("Require login")
-        if flask.g.user.is_admin is not True:
-            raise Unauthorized("Require admin user")
-        return f(*args, **kwargs)
-
-    return wrapper
-
-
 def admin_login_required(function):
-    """Compose the login required and admin required decorators."""
-    return login_required({"admin"})(admin_required(function))
+    """Use the check_arborist_auth decorator checking on admin authorization."""
+    return check_arborist_auth(["/services/fence/admin"], "*")(function)
 
 
 def _update_users_email(user, email):