error validating CSRF token After Migrate from AWS ECS to AWS EKS

[hugolesta]

Hello community, I hope you are doing well.

I'm trying to migrate SFTPgo from AWS ECS to AWS EKS, I'm using the same secrets and environment variables, I verified that the SFTPgo solution is connecting to the database, the only different thing is traefik as an ingress controller which passes all the traffic directly to the service HTTP, the UI is working + health however at the moment to authenticate as an admin I'm getting the following issue

{"level":"debug","time":"2024-04-23T16:50:53.617","sender":"httpd","message":"error validating CSRF token \"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOlsiQ1NSRiIsIjEwLjIuMTIuMTM1Il0sImV4cCI6MTcxMzkxMjYzOCwianRpIjoiY29qdWRuaDM5dmwzZmJsdHMw.........\": token is unauthorized"}

I did a forward port to the pod by using port 8080 and I can do the authentication with no issues.

Domain is the same that I have on ECS since I'm switching DNS to the Application Load Balancer behind Traefik

Steps to reproduce

1. Migration from AWS ECS to AWS EKS
2. Switch DNS for the same domain to the ALB behind AWS EKS + Traefik
3. Do a sign-in from the SFTPgo WebClient

Expected behavior

1.UI error: unable to verify form token: token is unauthorized
2. Logs as I mentioned before

SFTPGo version

WebClient - 2.5.6

Data provider

Mysql

Installation method

SFTPgo Helmchart: https://github.com/sagikazarmark/helm-charts

Configuration

I used the default values from SFTPgo Helm chart except the following envvars and service config

envVars:

• name: SFTGO_LOG_VERBOSE
  value: "1"
• name: SFTPGO_COMMON_DEFENDER__BAN_TIME
  value: "15"
• name: SFTPGO_COMMON_DEFENDER__BAN_TIME_INCREMENT
  value: "100"
• name: SFTPGO_COMMON_DEFENDER__ENABLED
  value: "true"
• name: SFTPGO_COMMON_DEFENDER__OBSERVATION_TIME
  value: "15"
• name: SFTPGO_COMMON_DEFENDER__THRESHOLD
  value: "5"
• name: SFTPGO_DATA_PROVIDER__DRIVER
  value: "mysql"
• name: SFTPGO_DATA_PROVIDER__HOST
  value: "db-sftpgo.acc.domain.io"
• name: SFTPGO_DATA_PROVIDER__NAME
  value: "sftpgo"
• name: SFTPGO_DATA_PROVIDER__PASSWORD
  valueFrom:
  secretKeyRef:
  name: sftp-creds
  key: SFTPGO_DATA_PROVIDER__PASSWORD
• name: SFTPGO_DATA_PROVIDER__PORT
  value: "3306"
• name: SFTPGO_DATA_PROVIDER__USERNAME
  value: "sftpgo_mgr_acc"
• name: SFTPGO_HTTP_TIMEOUT
  value: "15000"

 service:
  annotations: 
    service.beta.kubernetes.io/aws-load-balancer-type: external
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    service.beta.kubernetes.io/aws-load-balancer-security-groups: "secgp-elb-default"
    external-dns.alpha.kubernetes.io/hostname: "sftpgok8s-acc-nlb.acc.domain.io"

  type: LoadBalancer

  loadBalancerIP:

  loadBalancerSourceRanges: []

  ports:
    sftp:
      port: 22
      nodePort:

    ftp:
      port: 21

      nodePort:

    webdav:
      port: 81
      nodePort:

    http:
      port: 80

      nodePort:

  externalTrafficPolicy:
  sessionAffinity:

Relevant log output

{"level":"debug","time":"2024-04-23T16:50:53.617","sender":"httpd","message":"error validating CSRF token \"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOlsiQ1NSRiIsIjEwLjIuMTIuMTM1Il0sImV4cCI6MTcxMzkxMjYzOCwianRpIjoiY29qdWRuaDM5dmwzZmJsdHMw.........\": token is unauthorized"}

Additional info

We are managing our clusters with ArgoCD + CDK8s however, Traefik is being installed through its HelmChart.

Traefik additional Arguments:

additionalArguments:
- --api.insecure
- --entryPoints.web.proxyProtocol.insecure
- --entryPoints.web.forwardedHeaders.insecure

Additional Questions

1. Is there any documentation that refers to CSRF validation?

I'd appreciate any kind of help you can provide!

[hugolesta]

I figured out that the issue wasn't in Traefik actually, it was located on AWS ALB which didn't have Sticky sessions activated, After activating it it started working, however, I found it difficult to connect when the number of replicas was superior to 1, so I got some errors from the UI in between.

Other than that, we are doing configuration of SFTPgo with terraform, however, after switching the DNS and applying a new terraform plan the API is throwing 401 errors with the message Your token is not valid which means to me is also related to CSRF token.

Error: Could not read SFTPGo Event Action pimc-bi2-json-to-pim-transfer: status: 401, body: {"message":"Your token is not valid"}

I'll bring more updates in case I find the root cause of it.

[drakkan]

I want to share some general considerations for everyone. “You” does not specifically refer to you here.

You can keep experimenting and trying and maybe you will be able to find some workarounds or proper solutions to your issues.
You will spend your time on this, and therefore your company will incur costs, to use SFTPGo probably less than optimally and this will repeat every time you will have a question or a real issue. And what if you find a real bug?

Now suppose everyone does the same, the project will not be financially sustainable and at some point will no longer be maintained: we can't keep working for free forever, do you work for free?

As time passes, new security issues will be disclosed or new features will be required to interact with modern clients. Nobody will work on these things and after a while you will be forced to move to a different product and your company will have to incur other costs for the migration and probably to pay for the license.

[hugolesta]

@drakkan thanks for your message, the point that you have raised is understandable, I agree with the thoughts in general.

When it comes to this discussion to provide value to the community, this means, I'm providing feedback for a migration from AWS ECS to AWS EKS, I can imagine I'm not the only one, and it will probably be other people in the future that will do the same, so I'll be happy just with at least bring feedback and also happy to contribute in case I find a bug.

If SFTPgo supports this kind of migration it will be visible the reliability of this product/project, and it will lead to new investments and contributions.

This discussion comes with more insights based on the current analysis, is well known that SFTPgo supports container strategy, however, it's not really portable at this point due to this kind of token issue.

Feedback:

• Any time the container spins up the token is valid and terraform works, however, after a while starts failing again

even the SSH connection is possible however it comes with permission denied errors due to the directory is not created in the container.

sftp>
sftp> dir
remote readdir("/"): Permission denied

I just tested this behavior by creating a new user in this migration.

Terraform resource:

locals {
sftpgo_home_dir_prefix  = "/srv/sftpgo/data"
}
resource "sftpgo_user" "ims_sftp_acc_test" {
  username = "ims_sftp_${local.env}_test"
  status   = 1
  home_dir = "${local.sftpgo_home_dir_prefix}/ims_sftp_${local.env}_test"
  password = data.aws_secretsmanager_secret_version.pimc_filer_ims_sftp_user.secret_string
  permissions = {
    "/" = "*",
  }
  filesystem = {
    provider = 0 # Provider. 0 = local filesystem, 1 = S3 Compatible, 2 = Google Cloud, 3 = Azure Blob, 4 = Local encrypted, 5 = SFTP, 6 = HTTP
  }
  virtual_folders = [{
    name         = "pimc"
    quota_files  = "-1"
    quota_size   = "-1"
    virtual_path = "/"
  }]
}

sftpgo@ip-10-2-13-57:~$ cd /srv/sftpgo/data/
sftpgo@ip-10-2-13-57:/srv/sftpgo/data$ ls
<empty>

In any case, I have more feedback I'll provide more details over here.