⚠️[NOTE] BREAKING CHANGE since v1.31.0

[BlackDex]

Hello,

We intend to remove the custom Websocket implementation currently included in Vaultwarden.
This implementation uses a separate port (3012) which you needed to configure correctly into your reverse proxy for Websocket notifications to work.

Thanks to the work of @SergioBenitez and @Mai-Lapyst to add Websocket support to Rocket we can now provide this without custom ports and complex reverse proxy configurations.

Also, the new Login with device feature only works via this new Rocket Websocket implementation, and not via the old implementation.
So, to make full use of Vaultwarden we strongly encourage you to adjust your reverse proxy configuration to work with the new setup, since the old way will be removed from the next stable version.

Please check the wiki's Proxy Examples for the updated versions.
Also, if you see any outdated example and know how to update it, please be so kind to help us and update the wiki example.

If you have any questions regarding this, please place your questions here.

For the PR which includes these changes see: #4001

[redpixy]

Is there any example for Nginx Proxy Manager?

[jpdsc]

Just enable websocket support and that's all, you can remove any other mapping from custom locations.
Example:

[BlackDex]

@pheiduck That is an invalid config since it still seems to use port 3012, which isn't needed anymore right now already.
And that will break in the next minor version.

[pheiduck]

@BlackDex It's working by removing the last 2 blocks.

[dkornahrens]

When you say working by removing the last two blocks, what do you mean? The last two lines? My NPM is working, but I'm getting something showing it going online/offline every few minutes.

[PrivatePuffin]

Can you guys please comply with semver and not push breaking changes in minor version increases?

[BlackDex]

semver is nice for libraries and crates. Not specifically for applications in our opinion.

[PrivatePuffin]

With the whole industry moving towards either calver or semver, even for applications, choosing "crapver" is a very odd choice. Mostly due to easier support of automation.

Which means all users and downstreams using automated (or semi-automated) updating utilities need to specifically account for the opinion of vaultwarden maintainers on versioning. That's not user or downstream friendly imho.

[BlackDex]

Currently we only have either latest, x.xx.x or testing as tagged images. So any reason you gave is not applicable to Vaultwarden currently. Also, how do you make a calver for a breaking change?? You can't really up the month or year following the rules.

[PrivatePuffin]

So any reason you gave is not applicable to Vaultwarden currently

I don't see how latest, version, or testing, can cause any automated maintenace tooling to detect potentially breaking changes. Whereas they can with semver. so yes for sure my reasoning 100% applies to Vaultwarden.
If it didn't why would I complain about it? Running one of the bigger vaultwarden downstreams, this is kinda a big issue for us.

Mostly with Calver, you know right-away that each version is potentially unsafe. Hence it is actively adviced against for anything that might end up in an automation pipeline. But at least you know from the versioning style that it is unsafe.

[zacknewman]

I agree that SemVer adherence would be nice and is applicable to all crates including binary crates. The Cargo Book recommends SemVer adherence even for binary crates:

"The potential breaking and compatible changes to an application are too numerous to list, so you are encouraged to use the spirit of the SemVer spec to guide your decisions on how to apply versioning to your application, or at least document what your commitments are."

Obviously one is always allowed to do anything within the laws of mathematics/physics, but I only see positives with SemVer adherence. Other binary crates (e.g., ripgrep) seem to follow SemVer.

[masterwishx]

Using docker in Unraid + Nginx Proxy Manager , just wanted to be sure what we need to do:

1. we need to remove env :
   WEBSOCKET_ENABLED=true,WebSocket Port=3012

2. in NPM to remove custom locations:
   /notifications/hub/negotiate 4743
   /notifications/hub 3012

[BlackDex]

That seems to be valid yes.

[masterwishx]

Thanks, websocket working fine after changes

[Gerardv514]

After checking, am I just removing the port 3012 from the setup? I don’t see anything else referencing web sockets.

(This comment and the below are for Synology setup using Synology’s built in reverse proxy).

[AJV20]

Could you take a screenshot of your environment variables? Of course delete any personal info, I'm trying to set it up as well but don't know how to add the push api. Also did you keep the port settings? Or do I need to remove 3012?

[Gerardv514]

Could you take a screenshot of your environment variables? Of course delete any personal info, I'm trying to set it up as well but don't know how to add the push api. Also did you keep the port settings? Or do I need to remove 3012?

[RainerZufall00]

@Gerardv514 so did you remove the port 3012 from the container settings? Trying to enable the push notification but so far it does not work. When I‘m removing something on my mobile app it is directly removed in the webapp. But with the mobile apps the push does not work…the logs don‘t have errors anymore (only 200 - OK) , so pretty strange.

[Gerardv514]

@RainerZufall00 Yes I removed port 3012. If mobile push isn’t working I’d first start with did you generate a push install ID & key, and were they added to as environment variables along with push enabled = true variable?

I’m also using synology reverse proxy. In my reverse proxy entry under custom header tab, I also have header upgrade and value $http_upgrade and header connection and value $connection_upgrade added as well.

[RainerZufall00]

@Gerardv514 hm thanks for the answer...well yeah I have the environment variables set and also use the synology reverse proxy...I also tried to reinstall the mobile apps on Android and iOS....Probably need to set the log level to info and check them again....

[ecesarini]

Hi,
I'm using vaultwarden 1.30.0 and web-vault v2023.10.0 in production.
Unfortunately the containers are running on a CentOS 7 host and Apache httpd as reverse proxy: so the web server version is stuck at Apache/2.4.6 and i can't use the httpd wiki example shown here by fbartels.
My current httpd configuration is:

SSLProxyEngine On
RewriteEngine On

RewriteCond %{HTTP:Upgrade} =websocket [NC]
RewriteRule /notifications/hub(.*) ws://localhost:3012/$1 [P,L]

ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/

ProxyRequests Off
ProxyPreserveHost On
RequestHeader set X-Real-IP %{REMOTE_ADDR}s

As this thread suggests, I'm thinking it would be a good idea update my proxy configuration about new websocket beahviour.
According to this post I guess i could set docker env variable WEBSOCKET_ENABLED=false and adjusting httpd conf with something like this:

SSLProxyEngine On
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/

RewriteEngine   on
RewriteCond     %{HTTP:Upgrade}       websocket                   [NC]
RewriteCond     %{HTTP:Connection}    upgrade                     [NC]
RewriteRule     ^/?(.*)               "ws://localhost:8080/$1"    [P,L]

ProxyRequests Off
ProxyPreserveHost On
RequestHeader set X-Real-IP %{REMOTE_ADDR}s

Do you think it could work? Is there a way to test if all the vaultwarden functionalities will work fine?
My testing environment is very limited compared with the production one and i'm not so sure to know very well what are all the websocket tasks.

Thank you very much

[BlackDex]

See: #3598 (comment)
That should probably be helpful.

[ecesarini]

Thank you @BlackDex,
in my 2nd httpd snippet i was referring to that kind of solution.
Is there a way to be sure that the websocket is working fine (e.g some tests via bitwarden clients) ? I would test it in my small testing environment before updating production server.

[BlackDex]

Easiest way would be, login via two different browsers, and edit a cipher, like the login name or cipher name in one browser, and it should be updated almost instantly on the other.

[ecesarini]

Very helpful,
and thanks again

[Lunchtime0614]

Hi - I'm curious if the proxy example for Cloudflare is applicable to those using Cloudflare tunnel (which used to be called Cloudflare Access and is now Zero Trust).

Do I still create the rule as shown in the screen print - https://github.com/dani-garcia/vaultwarden/wiki/Proxy-examples and if so am I copying the two values for the URI values to start/does not start with (these are hard to read but it looks like /notifications/hub and /notifications/hub/negotiate), rewrite to 3012?

[masterwishx]

No, latest versions don't use 3012 port anymore for websocket

[euDominic]

is there any useful configuration for HAproxy inside pfsense?

[BlackDex]

HAProxy in it's basics sends all headers 1:1 to the backend.
So, basically, just remove everything linked to port 3012 or /notifications

Also see the HAproxy - v1.29.0+ (by @BlackDex) example, which just is a 1:1 forwarding and adds the correct IP headers.

[rama31244]

I can't see any obvious changes in the updated traefik config. Is that because it hasn't been updated or does traefik not require changes for vaultwarden 1.31.0? TIA

[BlackDex]

The examples are updated. Not sure since even you are running Vaultwarden. But in the previous examples there were mentions of port 3012 for example. Which are now gone.

[Spunkie]

I've updated to 1.30.1 and made the changes to websockets in preparation for this.

Is there a way for me to verify websockets communication is actually working on the new port?

[masterwishx]

Using it for some time, working fine

[Gerardv514]

Yes this is what I have set up and is working.

[jradwan]

Ok, thanks. I added the PUSH settings to my config and then logged out of the iOS app and re-connected. No push notifications yet but I'll keep tinkering. What e-mail address should I be using to generate the keys? If I have multiple users in my VW instance, does it matter which e-mail is used?

[BlackDex]

The key is per server. Bitwarden also never sees anything in regards to a user mail address or something during those pushes.

[jradwan]

Got it working, thanks!

[jeffshead]

I updated to v1.30.1 and broke websockets. I followed the info posted in this thread but none of the clients are updating unless I manually run a sync on them. Everything was working before I updated and edited the Stack in Portainer and the config in Nginx Proxy Manager.

I see the following in the log:

[2023-12-22 16:41:41.598][start][INFO] Rocket has launched from http://0.0.0.0:80
[2023-12-22 16:41:45.743][request][INFO] GET /notifications/hub?access_token=eyJ0sfeH763jfdiL
[2023-12-22 16:41:45.743][vaultwarden::api::notifications::_][WARN] Request guard `rocket_ws :: WebSocket` is forwarding.
[2023-12-22 16:41:45.743][rocket::response::responder::_][WARN] Response was `None`.
[2023-12-22 16:41:45.743][rocket::server::_][WARN] Responding with registered (not_found) 404 catcher.
[2023-12-22 16:41:45.743][response][INFO] (web_files) GET /<p..> [10] => 404 Not Found
[2023-12-22 16:42:41.573][vaultwarden::api::core::two_factor][DEBUG] Sending notifications for incomplete 2FA logins
[2023-12-22 16:42:41.574][vaultwarden::api::core::accounts][DEBUG] Purging auth requests
[2023-12-22 16:42:41.615][request][INFO] GET /alive
[2023-12-22 16:42:41.616][response][INFO] (alive) GET /alive => 200 OK
[2023-12-22 16:43:24.711][request][INFO] GET /notifications/hub?access_token=eyJ0sfeH763jfdiL

The only change that I made to the Portainer stack was to switch from WEBSOCKET_ENABLED=true to WEBSOCKET_ENABLED=false. I never included WebSocket Port=3012 in the stack so that did not have to be removed.

In Nginx Proxy Manager (NPM), on the advanced tab of the Vaultwarden proxy host, I switched from:

# Vault / API
  location / {
    proxy_pass http://vaultwarden:80;
    real_ip_header CF-Connecting-IP; # Cloudflare
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
 }

  # WebSocket (Notification System)
  location /notifications/hub {
    proxy_pass http://vaultwarden:3012;
    real_ip_header CF-Connecting-IP; # Cloudflare
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header X-Real-IP       $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }

  # Override to support negotiation endpoint for above
  location /notifications/hub/negotiate {
    proxy_pass http://vaultwarden:80;
  }

To:

location / {
      #proxy_http_version 1.1;
      #proxy_set_header Upgrade $http_upgrade;
      #proxy_set_header Connection $connection_upgrade;

      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;

      proxy_pass http://vaultwarden:80;
    }

The first three lines in the location above are commented out because they already exist in the default NPM config.

So what am I doing wrong?

---

UPDATE:
I removed everything from the advanced tab of the Vaultwarden proxy host in NPM. That fixed the issue.

[n3roGit]

Is there a way to test that the websocket actually works via 443?

"https://vault.example.net/notifications/hub/" no longer seems to be implemented.

[yelch]

check here: https://github.com/dani-garcia/vaultwarden/wiki/Enabling-WebSocket-notifications
via developper tools (F12)

[deanacollins]

I'm sorry if this isn't the right place for this, but I am pulling my hair out! I've upgraded to 1.30.5 recently and I've noticed that external access is broken. I've tried to following the points in this post through and I can't get it to work. All I get when accessing outside of my network is ERR_SSL_PROTOCOL_ERROR when using the WEBUI and Exception message: Connection Failure in the Android/iOS apps. I host my vaultwarden instance on a Synology NAS on DSM 7.

I don't know where to start look for this problem, any help will be gratefully received.

Couple of notes;

• I've completely redeployed the container for the latest image removing all configuration previously used.
• I've removed port 3012 from the container
• I've removed the custom headers from the synology proxy

This feels like a proxy issue to me, but I can't find it.

[deanacollins]

I've managed to resolve this by moving away from the synology reverse proxy. Switched over to cloudflare tunnels (which I should have done anyway) and now it is working perfectly.

[Gerardv514]

Your original problem is due to not setting the certificate. I was able to replicate the exact error message you had. Under control panel, security, certificate….you need to set where your BW subdomain is, to your BW reverse proxy.

Additionally, you will want to keep the custom headers under reverse proxy; this is so that vault sync still occurs between the browser extension and I think desktop app.

[yelch]

Hi. Thank you for this great port.
Just to make sure: I use vaultwarden on a synology server in a docker container with a custom subdomain and reverse proxy. Everything is working perfectly since years with the browser-plugins and also the official android app.

Valutwarden is on 1.30.5, Web is on 2024.1.2b

The reverse proxy just forwards my custom domain 443 to the localhost 8123 (no proxy for 8124)

My containers port settings are:

and the env-vars are:

--> So I don't need to adjust anything, right?

[BlackDex]

The wiki is maintained by the community.

[Gerardv514]

I see an edit button at the top of the page, I suppose that is how to edit.

Once one saves, does it go thru a review process and then published; or is it published upon Saving?

[daggettbiber]

Hi. Thank you for this great port.
Just to make sure: I use vaultwarden on a synology server in a docker container with a custom subdomain and reverse proxy. Everything is working perfectly since years with the browser-plugins and also the official android app.

Valutwarden is on 1.30.5, Web is on 2024.1.2b

The reverse proxy just forwards my custom domain 443 to the localhost 8123 (no proxy for 8124)

My containers port settings are:

and the env-vars are:

--> So I don't need to adjust anything, right?

Hi Yelch,

Can you please provide a screenshot of the "custom header" tab? Thank you.

[Gerardv514]

Should look something like this

[yelch]

Should look something like this

Yes, it is the same as on your screenshot

[terrorbyte704]

Hello, I am a new user I am trying to configure this in haproxy in pfsense but I can't seem to get it to work here is my configs

[xqdoo00o]

Just found the ws connected but not working with v1.30.5.
For example, I open 2 browser window, logined the same account and check in network console all ws are connected, but if I add a folder or change name in 1 window, the other window won't changed anything.
So the websocket test method on wiki is not working..

[BlackDex]

Within the same browser using the exact same account will not work. The client will think it executed the change it self and ignores the message.

The wiki stats to use two different browser, not two different browser windows. Or use one private/incognito browser window

[durd]

I seem to have browser websockets working. But not mobile push notifications, neither changes nor device logins.
I'm not sure if this is a known issue? Afaik it's never worked for me :(

I'm using Caddy according to the wiki page and have enabled PUSH_ENABLED=true in my docker-compose file along with id and key and URLs.

I'm using latest stable of Caddy, Vaultwarden and Bitwarden clients :/

[W1BTR]

A bit confused here, does a new reverse proxy setup need any ports other than port 443 going to port 80?

[Gerardv514]

No

[xOzryelx]

I can't get this new setup running no matter what I tried. Watchtower upgraded the container a few weeks ago, since then my instance doesn't want to run. I followed the config example and tried quite a lot of variants, but I need some help.

Using docker compose:

services:
  vaultwarden:
      container_name: vaultwarden
      image: vaultwarden/server:latest
      restart: unless-stopped
      ports:
        - 8080:80/tcp
        #- 3012:3012/tcp
      volumes:
      - /home/vaultwarden/vw-data:/data:rw
      environment:
        - COMPOSE_HTTP_TIMEOUT=120
        - LOG_FILE=/data/vaultwarden.log
        - WEBSOCKET_ENABLED=false

with Apache as a reverse proxy to a sublocation /bitwarden/ following the examples in the docs

<Location /bitwarden/>
  RewriteEngine On
  RewriteCond %{HTTP:Upgrade} =websocket [NC]
  RewriteRule /notifications/hub(.*) ws://127.0.0.1:8080/bitwarden/notifications/hub/$1 [P,L]
  RewriteRule /notifications/anonymous-hub(.*) ws://127.0.0.1:8080/fjbitwarden/notifications/anonymous-hub/$1 [P,L]

  ProxyPass http://127.0.0.1:8080/bitwarden timeout=60 connectiontimeout=5

  ProxyPreserveHost Off
  RequestHeader set X-Real-IP %{REMOTE_ADDR}s
  RequestHeader setifempty Connection "Upgrade"
  RequestHeader setifempty Upgrade "websocket"
</Location>

I have the rest of my options in a config.json where I also set the domain

"domain": "https://somedomain.de/bitwarden/",

In my browser I only get a 502

Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request

Reason: Error reading from remote server

Apache Logs give me this:

[Fri Nov 08 23:36:17.740075 2024] [proxy_http:error] [pid 144932:tid 144932] (70007)The timeout specified has expired: [client IPv6] AH01102: error reading status line from remote server 127.0.0.1:8080
[Fri Nov 08 23:36:17.740173 2024] [proxy:error] [pid 144932:tid 144932] [client IPv6] AH00898: Error reading from remote server returned by /bitwarden/

Anyone got an idea what might be wrong?

[xOzryelx]

Upped the logging to debug and ditched docker-compose for the moment

Getting this error in the log

[09.11.2024 13:24:20.962][response][INFO] (alive) GET /bitwarden/alive => 200 OK
[09.11.2024 13:24:33.561][request][INFO] GET /bitwarden
[09.11.2024 13:24:33.561][response][INFO] (web_index) GET /bitwarden/ => 200 OK
[09.11.2024 13:24:33.587][request][INFO] GET /bitwardentheme_head.4cb181fc19f2a308ba73.js
[09.11.2024 13:24:33.587][rocket::server::_][ERROR] No matching routes for GET /bitwardentheme_head.4cb181fc19f2a308ba73.js.
[09.11.2024 13:24:33.587][rocket::server::_][WARN] No 404 catcher registered. Using Rocket default.
[09.11.2024 13:24:33.587][response][INFO] 404 Not Found
[09.11.2024 13:24:33.600][request][INFO] GET /bitwardenapp/main.62a346fdfb5b10f77c32.css
[09.11.2024 13:24:33.601][rocket::server::_][ERROR] No matching routes for GET /bitwardenapp/main.62a346fdfb5b10f77c32.css text/css.
[09.11.2024 13:24:33.601][rocket::server::_][WARN] No 404 catcher registered. Using Rocket default.
[09.11.2024 13:24:33.601][response][INFO] 404 Not Found
[09.11.2024 13:24:33.614][request][INFO] GET /bitwardenapp/polyfills.575bcaf3121104d1c55d.js
[09.11.2024 13:24:33.614][rocket::server::_][ERROR] No matching routes for GET /bitwardenapp/polyfills.575bcaf3121104d1c55d.js.
[09.11.2024 13:24:33.614][rocket::server::_][WARN] No 404 catcher registered. Using Rocket default.
[09.11.2024 13:24:33.614][response][INFO] 404 Not Found
[09.11.2024 13:24:33.627][request][INFO] GET /bitwardenapp/vendor.0d3c724de0f2ab138712.js
[09.11.2024 13:24:33.627][rocket::server::_][ERROR] No matching routes for GET /bitwardenapp/vendor.0d3c724de0f2ab138712.js.
[09.11.2024 13:24:33.628][rocket::server::_][WARN] No 404 catcher registered. Using Rocket default.
[09.11.2024 13:24:33.628][response][INFO] 404 Not Found
[09.11.2024 13:24:33.633][request][INFO] GET /bitwardenapp/main.7c1287a291f3bb48f2e0.js
[09.11.2024 13:24:33.633][rocket::server::_][ERROR] No matching routes for GET /bitwardenapp/main.7c1287a291f3bb48f2e0.js.
[09.11.2024 13:24:33.633][rocket::server::_][WARN] No 404 catcher registered. Using Rocket default.
[09.11.2024 13:24:33.633][response][INFO] 404 Not Found
[09.11.2024 13:24:33.701][request][INFO] GET /bitwardencca56971e438d22818d6.json
[09.11.2024 13:24:33.701][rocket::server::_][ERROR] No matching routes for GET /bitwardencca56971e438d22818d6.json.
[09.11.2024 13:24:33.701][rocket::server::_][WARN] No 404 catcher registered. Using Rocket default.
[09.11.2024 13:24:33.701][response][INFO] 404 Not Found

[BlackDex]

Try adding a trailing / to your proxy pass

[xOzryelx]

Sadly does not make a difference

I copied the example from the wiki which doesn't have the trailing /

[xOzryelx]

Ok, I switched from my distro provided docker to the official docker repo, so I got a way newer version running now.

Still no change in behaviour

[xOzryelx]

OMFG, I think I'll go back to primary school to give myself a second chance

For some reason fail2ban blocked all the traffic

Gonna go hide and be ashamed for a while now

Looks like my postfix rule caught bitwarden trying to send mails, which fails locally for some reason. Gotta investigate that

Sorry for the trouble