Add pluggable authentication in the grid

[sihil]

What does this change?

The grid is currently hardwired into the Guardian's https://github.com/guardian/pan-domain-authentication library. This is specific to the guardian and it makes sense that other adopters would want to blaze their own trail.

This PR:

• implements the ability to plug in your own authentication provider by specifying user and machine authentication provider classes implementing appropriate traits in the configuration.
• implements pan domain authentication and api key implementations of the user and machine authentication providers to replicate existing behaviour.
• implements a generic provider loader mechanism used to dynamically load image processors and authentication providers (and likely more providers in the future).

Consumers need to implement a UserAuthenticationProvider trait. They can also implement a MachineAuthenticationProvider trait to override the way machine to machine calls can be done. These are defined as shown below.

sealed trait AuthenticationProvider {
  def initialise(): Unit = {}
  def shutdown(): Future[Unit] = Future.successful(())

  /**
    * A function that allows downstream API calls to be made using the credentials of the inflight request
    * @param request The request header of the inflight call
    * @return A function that adds appropriate authentication headers to a WSRequest or returns a runtime error
    */
  def onBehalfOf(request: Principal): Either[String, WSRequest => WSRequest]
}

object AuthenticationProvider {
  type RedirectUri = String
}

trait UserAuthenticationProvider extends AuthenticationProvider {
  /**
    * Establish the authentication status of the given request header. This can return an authenticated user or a number
    * of reasons why a user is not authenticated.
    * @param request The request header containing cookies and other request headers that can be used to establish the
    *           authentication status of a request.
    * @return An authentication status expressing whether the
    */
  def authenticateRequest(request: RequestHeader): AuthenticationStatus

  /**
    * If this provider supports sending a user that is not authorised to a federated auth provider then it should
    * provide a function here to redirect the user. The function signature takes the the request and returns a result
    * which is likely a redirect to an external authentication system.
    */
  def sendForAuthentication: Option[RequestHeader => Future[Result]]

  /**
    * If this provider supports sending a user that is not authorised to a federated auth provider then it should
    * provide a function here that deals with the return of a user from a federated provider. This should be
    * used to set a cookie or similar to ensure that a subsequent call to authenticateRequest will succeed. If
    * authentication failed then this should return an appropriate 4xx result.
    * The function should take the Play request header and the redirect URI that the user should be
    * sent to on successful completion of the authentication.
    */
  def sendForAuthenticationCallback: Option[(RequestHeader, Option[RedirectUri]) => Future[Result]]

  /**
    * If this provider is able to clear user tokens (i.e. by clearing cookies) then it should provide a function to
    * do that here which will be used to log users out and also if the token is invalid.
    * This function takes the request header and a result to modify and returns the modified result.
    */
  def flushToken: Option[(RequestHeader, Result) => Result]
}

trait MachineAuthenticationProvider extends AuthenticationProvider {
  /**
    * Establish the authentication status of the given request header. This can return an authenticated user or a number
    * of reasons why a user is not authenticated.
    * @param request The request header containing cookies and other request headers that can be used to establish the
    *           authentication status of a request.
    * @return An authentication status expressing whether the
    */
  def authenticateRequest(request: RequestHeader): ApiAuthenticationStatus
}

Much like the pluggable image processor pipeline, these authentication providers are loaded dynamically at runtime. The loader supports companion objects and also classes with constructors that can optionally have arguments (a play.api.Configuration instance and/or a AuthenticationProviderResources instance - shown below).

/**
  * Case class containing useful resources for authentication providers to allow concurrent processing and external
  * API calls to be conducted.
  * @param commonConfig the Grid common config object
  * @param context an execution context
  * @param actorSystem an actor system
  * @param wsClient a play WSClient for making API calls
  */
case class AuthenticationProviderResources(commonConfig: CommonConfig,
                                           actorSystem: ActorSystem,
                                           wsClient: WSClient,
                                           controllerComponents: ControllerComponents)

TODO

• Integrate the new Authentication class into the application (and remove the existing version)
• Write an class loader to load and instantiate providers (or perhaps make the image processor one generic)
• Review the AuthenticationProviderResources that can be passed into a provider
• Make PandaUser more generic (possibly including a TypedMap for extensions?)
• Write tests to confirm and assert the behaviour of the various components
  • Authentication.authenticateRequest
  • Authentication.onBehalfOf
  • PandaAuthenticationProvider (fairly hard to do and only used by the Guardian)
  • ApiKeyAuthenticationProvider
• Add required configuration changes to PROD before merging

Follow on PRs

• Write documentation on implementing extension points (including authentication) Write documentation for extending the Grid #3103
• Implementing a default provider that simply lets users in with a hardcoded user ID for simpler spinning up Implement a default provider for testing the Grid #3104
• Implementing bundled provider to support OIDC Implement a generic OIDC authentication provider #3105

How can success be measured?

Other organisations are able to implement their own idea of authentication without having to fight with the Guardian's pan domain authentication library.

Who should look at this?

It would be great to have @mbarton and @akash1810 review this PR.

Tested?

• locally
• on TEST

[mbarton]

I think the level of abstraction makes sense. Probably worth us using this to connect to auth0 or another easy to set up provider to shake out any issues though.

[prout-bot]

Seen on auth, usage, collections, image-loader, image-loader-projection, metadata-editor (merged by @sihil 9 minutes and 49 seconds ago) Please check your changes!

[prout-bot]

Seen on kahuna, leases (merged by @sihil 9 minutes and 54 seconds ago) Please check your changes!

[prout-bot]

Seen on media-api (merged by @sihil 9 minutes and 59 seconds ago) Please check your changes!

[prout-bot]

Seen on cropper (merged by @sihil 10 minutes and 53 seconds ago) Please check your changes!