Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker Deployment and WebSocket Questions for Beginners #2708

Open
manjieqi opened this issue Aug 25, 2024 · 16 comments
Open

Docker Deployment and WebSocket Questions for Beginners #2708

manjieqi opened this issue Aug 25, 2024 · 16 comments

Comments

@manjieqi
Copy link

manjieqi commented Aug 25, 2024

GoAccess is very beautiful, thank you to the contributors for their efforts, but I encountered some problems during deployment, although I have consulted a lot of materials, but due to my limited level, I still feel confused, if you can answer, I would be very grateful.

It can currently be accessed via https://goaccess.xxxxx.com:8443/, but there are still some issues.
image

First, I noticed that the browser was constantly requesting ws, but it seemed to have failed.
image

Next, I'm confused about this port 7890, can it be accessed directly through the IP:7890 range? But my browser seems unable to open it.

Additionally, I configured real-time-html to true, but it doesn't seem to be updating automatically.

Here is my deployed configuration file:

/home/ubuntu/goaccess/compose.yml

services:
  goaccess:
    image: allinurl/goaccess
    container_name: goaccess
    ports:
      - 0.0.0.0:7890:7890
    volumes:
      - ./config:/srv/config
      - /home/ubuntu/nginx/html/goaccess:/srv/report
      - /home/ubuntu/nginx/logs:/srv/logs
    command: ["--no-global-config", "--config-file=/srv/config/goaccess.conf", -e TZ="Asia/Shanghai"]
    restart: unless-stopped
    networks:
      - nginx

networks:
  nginx:
    external: true

/home/ubuntu/goaccess/config/goaccess.conf

time-format %H:%M:%S
date-format %Y-%m-%d
log-format COMBINED
log-file /srv/logs/access.log
output /srv/report/index.html
#ws-url goaccess.xxxxx.com:8443
real-time-html true

/home/ubuntu/nginx/compose.yml

services:
  nginx:
    image: nginx:1.26
    container_name: nginx
    ports:
      - 0.0.0.0:8080:80
      - 0.0.0.0:8443:443
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./conf.d:/etc/nginx/conf.d
      - ./certs:/etc/nginx/certs
      - ./html:/usr/share/nginx/html
      - ./logs:/var/log/nginx
    restart: unless-stopped
    networks:
      - nginx

networks:
  nginx:
    external: true

/home/ubuntu/nginx/conf.d/goaccess.conf

# HTTPS 服务器配置
server {
    listen 443 ssl;
    server_name goaccess.xxxxx.com;

    ssl_certificate /etc/nginx/certs/xxxxx.com.crt;
    ssl_certificate_key /etc/nginx/certs/xxxxx.com.key;

    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off;
    ssl_protocols TLSv1.3;

    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Content-Type-Options nosniff;
    add_header X-Frame-Options DENY;
    add_header X-XSS-Protection "1; mode=block";

    # 代理配置 
    location / {

        root /usr/share/nginx/html/goaccess;
        index index.html;

        # proxy_pass http://goaccess:7890;
        # proxy_set_header Host $host;
        # proxy_set_header X-Real-IP $remote_addr;
        # proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        # proxy_set_header X-Forwarded-Proto $scheme;
    }
}
@manjieqi
Copy link
Author

It seems like the issue has been resolved through AI, but there are two problems. When accessing through a browser, it first loads the old static files and then updates to the latest files, is this not due to my configuration error? Additionally, the Docker image seems to automatically generate anonymous volumes, which are not being used. If there's anything that needs improvement in my configuration, please let me know.

/home/ubuntu/goaccess/compose.yml

services:
  goaccess:
    image: allinurl/goaccess
    container_name: goaccess
    ports:
      - 0.0.0.0:7890:7890
    volumes:
      - ./config:/srv/config
      - /home/ubuntu/nginx/logs:/srv/logs
      - /home/ubuntu/nginx/certs:/srv/certs
      - /home/ubuntu/nginx/html/goaccess:/srv/report
    command: ["--no-global-config", "--config-file=/srv/config/goaccess.conf"]
    restart: unless-stopped
    networks:
      - nginx

networks:
  nginx:
    external: true

/home/ubuntu/goaccess/config/goaccess.conf

tz Asia/Shanghai
time-format %H:%M:%S
date-format %Y-%m-%d
log-format COMBINED
log-file /srv/logs/access.log
output /srv/report/index.html
real-time-html true
ws-url wss://goaccess.xxxxx.com:7890
ssl-cert /srv/certs/xxxxx.com.crt
ssl-key /srv/certs/xxxxx.com.key

/home/ubuntu/nginx/compose.yml

services:
  nginx:
    image: nginx:1.26
    container_name: nginx
    ports:
      - 0.0.0.0:8080:8080
      - 0.0.0.0:8443:8443
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./conf.d:/etc/nginx/conf.d
      - ./certs:/etc/nginx/certs
      - ./html:/usr/share/nginx/html
      - ./logs:/var/log/nginx
    restart: unless-stopped
    networks:
      - nginx

networks:
  nginx:
    external: true

/home/ubuntu/nginx/conf.d/goaccess.conf

# HTTPS 服务器配置
server {
    listen 8443 ssl;
    server_name goaccess.xxxxx.com;

    ssl_certificate /etc/nginx/certs/xxxxx.com.crt;
    ssl_certificate_key /etc/nginx/certs/xxxxx.com.key;

    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off;
    ssl_protocols TLSv1.3;

    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Content-Type-Options nosniff;
    add_header X-Frame-Options DENY;
    add_header X-XSS-Protection "1; mode=block";

    # 代理配置 
    location / {
        root /usr/share/nginx/html/goaccess;
        index index.html;
    }

    location /ws {
        proxy_pass http://goaccess:7890;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
    }
}

@allinurl
Copy link
Owner

I'm glad to hear you resolved the original issue. When it comes to loading the report, it's normal for GoAccess to bootstrap the initial data during the process.

Honestly, unless I'm missing something, I don’t see anything in the yaml or nginx config that would cause unused anonymous volumes to be created automatically. The volumes seem to be explicitly defined and mapped to specific paths on both the host and the containers.

Maybe it's something outside the config? Check what docker volume ls and docker inspect <container_id> show, and make sure to look at the path ids too.

Let me know if that helps.

@manjieqi
Copy link
Author

$ docker inspect goaccess
{
    "Type": "volume",
    "Name": "218de37f2ea6788172d825361a548a7a0ffaa5acd2ddae2767714349a21011b6",
    "Source": "/var/lib/docker/volumes/218de37f2ea6788172d825361a548a7a0ffaa5acd2ddae2767714349a21011b6/_data",
    "Destination": "/var/www/goaccess",
    "Driver": "local",
    "Mode": "",
    "RW": true,
    "Propagation": ""
}

I saw the Dockerfile in the root directory of the repository defining this volume, which should be the origin of the anonymous volume. I didn't mount the file in this directory because the documentation didn't mention it. thx.

@allinurl
Copy link
Owner

Thanks for sharing that. You are right, since there's nothing explicitly mounted to that directory when running the container, Docker creates an anonymous volume automatically. The VOLUME /var/www/goaccess line in the Dockerfile creates this anonymous volume.

@ineiti
Copy link
Contributor

ineiti commented Jan 14, 2025

Thanks a lot for your docker-compose.yaml - it let me set up mine in no time :) The only thing I changed was to change the output back to something like report.html so it can live in parallel with my static site, and to use the traefik-configuration from my server. So I get this docker-compose.yaml file:

services:
  # This is my existing static site
  web:
    image: nginx
    volumes:
      - ./nginx/log:/var/log/nginx
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./public:/usr/share/nginx/html
    labels:
      - "traefik.enable=true"
      - "fqdn=ineiti.ch"
    networks:
      - traefik

  # And the goaccess which I piggyback on it
  goaccess:
    image: allinurl/goaccess
    container_name: goaccess
    ports:
      - 0.0.0.0:7890:7890
    volumes:
      - ./goaccess:/srv/config
      - ./nginx/log:/srv/logs
      - ./public:/srv/report
    command: ["--no-global-config", "--config-file=/srv/config/goaccess.conf"]
    labels:
      - "traefik.enable=true"
      - "fqdn=xxx.ineiti.ch"
    restart: unless-stopped
    networks:
      - traefik

networks:
  traefik:
    external:
      name: traefik_traefik

I'm using the traefik-configuration explained here: https://ineiti.ch/posts/traefik-101/traefik-101/

@allinurl would you be interested in a PR for a docker-compose.yaml file and a configuration?

@allinurl
Copy link
Owner

@ineiti yeah that PR would be great! Thanks!

@ineiti
Copy link
Contributor

ineiti commented Jan 15, 2025

Here you go - #2786

I did not look too much at the rest of the repository, so I might be off to how you want to have things arranged. Happy to follow any feedback you give!

@allinurl
Copy link
Owner

@ineiti Looks great to me, thanks for the PR!

@lordraiden
Copy link

lordraiden commented Jan 22, 2025

@ineiti @manjieqi

There is something I don't understand about this.
I have traefik as my proxy and behind it homeassinstant among other things.

If I'm already using traefik do I need to run the nginx container?

this is my compose, I'm not using nginx, am I right?

  goaccess:
    container_name: ${goaccess_name}
    image: allinurl/goaccess
    <<: [*config, *dns, *sec]
    networks:
      eth1:
        ipv4_address: ${goaccess_ip}
    ports:
      - 7890:7890
    volumes:
      - ${dockerdir}/GoAccess/configs/goaccess.traefik.conf:/srv/config/goaccess.conf
      - ${dockerdir}/WebProxyDMZ/Traefik/logs:/srv/logs:ro
      - ${dockerdir}/GoAccess/public:/srv/report
    command: ["--no-global-config", "--config-file=/srv/config/goaccess.conf"]
    labels:
      fqdn: goaccess.mydomain.com    #do I need this label? what for?
      traefik.enable: true
      traefik.docker.network: eth1
      traefik.http.routers.goaccess.entrypoints: https443
      traefik.http.routers.goaccess.service: goaccess
      traefik.http.routers.goaccess.rule: "Host(`${goaccess_name}.${mydomain}`)"
      traefik.http.routers.goaccess.tls: true
      traefik.http.routers.goaccess.middlewares: local-ipwhitelist@file, rate-limit@file
      traefik.http.services.goaccess.loadbalancer.server.port: 7890

But I get this in the logs

GoAccess  | ==1== GoAccess - version 1.9.3 - Jan 15 2025 14:25:27
GoAccess  | ==1== Config file: /srv/config/goaccess.conf
GoAccess  | ==1== https://goaccess.io - 
GoAccess  | ==1== Released under the MIT License.
GoAccess  | ==1==
GoAccess  | ==1== FILE: /srv/logs/traefik.log
GoAccess  | ==1== Parsed 10 lines producing the following errors:
GoAccess  | ==1==
GoAccess  | ==1== Token '2024-05-31T01:01:31+02:00' doesn't match specifier '%h'
GoAccess  | ==1== Token '2024-05-31T01:01:31+02:00' doesn't match specifier '%h'
GoAccess  | ==1== Token '2024-05-31T01:04:34+02:00' doesn't match specifier '%h'
GoAccess  | ==1== Token '2024-05-31T03:32:45+02:00' doesn't match specifier '%h'
GoAccess  | ==1== Token '2024-05-31T03:45:10+02:00' doesn't match specifier '%h'
GoAccess  | ==1== Token '2024-05-31T06:33:58+02:00' doesn't match specifier '%h'
GoAccess  | ==1== Token '2024-05-31T08:22:33+02:00' doesn't match specifier '%h'
GoAccess  | ==1== Token '2024-05-31T10:24:33+02:00' doesn't match specifier '%h'
GoAccess  | ==1== Token '2024-05-31T10:50:49+02:00' doesn't match specifier '%h'
GoAccess  | ==1== Token '2024-05-31T12:13:37+02:00' doesn't match specifier '%h'
GoAccess  | ==1==
GoAccess  | ==1== Format Errors - Verify your log/date/time format

And this is my conf file.

goaccess.traefik.conf.txt

Can someone helpme to configure this properly so I can see the stats of my traefik proxy?

@ineiti
Copy link
Contributor

ineiti commented Jan 22, 2025

I used the nginx logs because I needed to serve a static page anyway. But it seems that you can also make it work with treafik logs:

https://www.fuzzygrim.com/posts/goaccess-traefik

@lordraiden
Copy link

Ok
I manage to parse the logs correctly and get the report in html, I have updated my config file.

goaccess.conf.txt

The problem now is, if I go to http://10.10.40.37:7890/ I get a blank page, not error just blank. What is wrong?
How can I pass through traefik proxy the visualization of the HTML report "/GoAccess/public:/srv/report" -> GoAccess/public/index.html

@ineiti
Copy link
Contributor

ineiti commented Jan 22, 2025

Oh, I forgot about that part :) The 7890 port is for the websocket access only. goaccess writes a static file to the output directory which must be served.

So perhaps my docker-compose.yaml file with the nginx example is the easiest to use. I remember having seen a traefik plug-in to serve static webpages, but cannot find it anymore.

@lordraiden
Copy link

lordraiden commented Jan 22, 2025

Now it's working

  goaccess:
    container_name: ${goaccess_name}
    image: allinurl/goaccess
    <<: [*config, *sec]

    volumes:
      - ${dockerdir}/GoAccess/configs/goaccess.conf:/srv/config/goaccess.conf
      - ${dockerdir}/WebProxyDMZ/Traefik/logs:/srv/logs
      - ${dockerdir}/GoAccess/public:/srv/report
      - ${dockerdir}/GoAccess/tmp:/tmp
    command: ["--no-global-config", "--config-file=/srv/config/goaccess.conf"]


  static-web-server:
    image: joseluisq/static-web-server:latest
    container_name: ${static_web_server_name}
    <<: [*config, *dns, *sec]
    networks:
      eth1:
        ipv4_address: ${static_web_server_ip}
    environment:
      # Note: those envs are customizable but also optional
      - SERVER_PORT=8080
      - SERVER_ROOT=/public
      - SERVER_LOG_LEVEL=info
    volumes:
      - ${dockerdir}/GoAccess/public:/public
    labels:
      <<: *labels
      traefik.enable: true
      traefik.docker.network: eth1
      traefik.http.routers.staticws.entrypoints: https443
      traefik.http.routers.staticws.service: staticws
      traefik.http.routers.staticws.rule: "Host(`${static_web_server_name}.${mydomain}`)"
      traefik.http.routers.staticws.tls: true
      traefik.http.routers.staticws.middlewares: local-ipwhitelist@file, rate-limit@file
      traefik.http.services.staticws.loadbalancer.server.port: 8080

The problem I have now is that the output file is only generated with I start the container is not being updated when new logs are generated, I have real time enabled so, what I'm missing?
Real time means real time or every X time? can the every X time parameter be changed?

######################################
# Server Options
######################################

# Specify IP address to bind server to.
#
#addr 10.10.40.37

# Run GoAccess as daemon (if --real-time-html enabled).
#
#daemonize false

# Ensure clients send the specified origin header upon the WebSocket
# handshake.
#
#origin http://example.org

# The port to which the connection is being attempted to connect.
# By default GoAccess' WebSocket server listens on port 7890
# See man page or http://gwsocket.io for details.
#
#port 7890

# Write the PID to a file when used along the daemonize option.
#
#pid-file /var/run/goaccess.pid

# Enable real-time HTML output.
#
real-time-html true

@ineiti
Copy link
Contributor

ineiti commented Jan 22, 2025

I don't see what you have in your config, sec configuration. But if you open your developer console on your browser, it will probably tell you that it cannot access the ws endpoint because it's an insecure protocol. I did the following in my docker-compose:

https://github.com/allinurl/goaccess/blob/master/docker-compose/docker-compose.traefik.yaml#L23

This works because I told goaccess to use port 443, which is then handled by traefik to create a secure route. This part was a bit finnicky: getting the port correct in goaccess did not work as I expected. Have a good look at my proposed goaccess.conf file.

@lordraiden
Copy link

I don't see what you have in your config, sec configuration. But if you open your developer console on your browser, it will probably tell you that it cannot access the ws endpoint because it's an insecure protocol. I did the following in my docker-compose:

https://github.com/allinurl/goaccess/blob/master/docker-compose/docker-compose.traefik.yaml#L23

This works because I told goaccess to use port 443, which is then handled by traefik to create a secure route. This part was a bit finnicky: getting the port correct in goaccess did not work as I expected. Have a good look at my proposed goaccess.conf file.

Sorry I deleted the previous post, I think you are answering to that. In my last post the solution is already working, traefik can't host static content so I used a webserver but not nginx. Now works, as you can see in my previous post, but the index.html file is not updating despite configuring it to do it in real time real time

config and sec has no impact, this are general things like restart unless stop dns, etc....

@ineiti
Copy link
Contributor

ineiti commented Jan 23, 2025

@lordraiden what I wrote is still of relevance :) I'm pretty sure the bug you describe indicates that the websocket connection fails. Did you look at the developer console in your browser? Chrome puts some messages in the javascript tab, else look in the network tab to see whether the connection is successfully set up.

The problem with the port is also relevant. On my system I had troubles getting goaccess to contact the correct port. The port in the ws-url configuration was ignored, and only the port configuration was used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants