Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Can't scan any ROMs #1571

Open
crispybegs opened this issue Feb 2, 2025 · 8 comments
Open

[Bug] Can't scan any ROMs #1571

crispybegs opened this issue Feb 2, 2025 · 8 comments
Assignees

Comments

@crispybegs
Copy link

crispybegs commented Feb 2, 2025

RomM version
3.7.3

Describe the bug
Unable to find or match any roms

Expected behavior
roms to be matched.

I used romm when @zurdi15 very first announced it and it used to work great. Then some changes were made and my instance broke and I never bothered fixing it and haven't looked at it for over a year. I wanted to start using it again and I see a lot of stuff has been updated, so I've just tried to get it running again and I'm having a hell of time trying to get it to see any of my roms.

Firstly, this is my compose:

version: "3"

volumes:
  mysql_data:
  romm_resources:
  romm_redis_data:

services:
  romm:
    image: rommapp/romm:latest
    container_name: romm
    restart: unless-stopped
    environment:
      #- PUID=1000
      #- PGID=1000
      - DB_HOST=romm-db
      - DB_NAME=romm 
      - DB_USER=[redacted] 
      - DB_PASSWD=[redacted] 
      - ROMM_AUTH_SECRET_KEY=[redacted] 
      #- IGDB_CLIENT_ID=[redacted] 
      #- IGDB_CLIENT_SECRET=[redacted] 
      #- MOBYGAMES_API_KEY= # https://www.mobygames.com/info/api/
      - STEAMGRIDDB_API_KEY=[redacted] 
      - ROMM_AUTH_USERNAME=[redacted]  
      - ROMM_AUTH_PASSWORD=[redacted]  
    volumes:
      - /media/crispy/NAS_4TB_1/media/games/romm/romm_resources:/romm/resources 
      - /home/crispy/docker/romm/config/romm_redis_data:/redis-data 
      - /media/crispy/NAS_4TB_1/media/games/batocera/roms:/romm/library 
      - /media/crispy/NAS_4TB_1/media/games/romm/assets:/romm/assets 
      - /home/crispy/docker/romm/config/config:/romm/config 
    ports:
      - 3334:8080
    depends_on:
      romm-db:
        condition: service_healthy
        restart: true

  romm-db:
    image: mariadb:latest
    container_name: romm-db
    restart: unless-stopped
    environment:
      - MARIADB_ROOT_PASSWORD=[redacted]  
      - MARIADB_DATABASE=[redacted] 
      - MARIADB_USER=[redacted] 
      - MARIADB_PASSWORD=[redacted] 
    volumes:
      - /media/crispy/NAS_4TB_1/media/games/romm/mysql_data:/var/lib/mysql
    healthcheck:
      test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
      start_period: 30s
      start_interval: 10s
      interval: 10s
      timeout: 5s
      retries: 5

My rom library is used by batocera and follows their folder structure. I followed the batocera example and made a config.yaml and that lives at - /home/crispy/docker/romm/config/config:/romm/config as per my compose.

Not sure if I've messed something up with that config, but you can take a look here: https://pastebin.com/UjUjQ2j4

Sidenote: I had my previous IGDB credentials in the compose and thought maybe they had expired and that was causing any lookups to fail, but visiting the twitch dev applications page it seems you can't save or create anything in your applications. The 'Save' button is greyed out:

Image

Seems to be an ongoing issue, actually - twitchdev/issues#634

.. so I signed up to steamgrid and used their api key instead (i have no idea what steamgrid is or if it's any good, but it's in the romm docs, so..)

Romm starts up as normal, but when trying a scan i get this for every single possible console:

Image

Checking the container logs i see this:

WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47] · adam
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47]   Identified as Coleco Adam 🎮
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47] · vitaquake2
WARNING:  [RomM][scan_handler][2025-02-02 20:20:47]  vitaquake2 not identified ❌
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47] · pico8
WARNING:  [RomM][scan_handler][2025-02-02 20:20:47]  pico8 not identified ❌
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47] · dos
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47]   Identified as DOS 🎮
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47] · wiiu
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47]   Identified as Wii U 🎮
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47] · lutro
WARNING:  [RomM][scan_handler][2025-02-02 20:20:47]  lutro not identified ❌
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47] · vsmile
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47]   Identified as V.Smile 🎮
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47] · tyrian
WARNING:  [RomM][scan_handler][2025-02-02 20:20:47]  tyrian not identified ❌
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47] · coco
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47]   Identified as TRS-80 Color Computer 🎮
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47] · cannonball
WARNING:  [RomM][scan_handler][2025-02-02 20:20:47]  cannonball not identified ❌
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47] · amigacdtv
INFO:	  [RomM][scan_handler][2025-02-02 20:20:47]   Identified as Amiga CD32 🎮
WARNING:  [RomM][scan][2025-02-02 20:20:47]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:48]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:48] · atari800
WARNING:  [RomM][scan_handler][2025-02-02 20:20:48]  atari not identified ❌
WARNING:  [RomM][scan][2025-02-02 20:20:48]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:48]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:48] · gp32
INFO:	  [RomM][scan_handler][2025-02-02 20:20:48]   Identified as GP32 🎮
WARNING:  [RomM][scan][2025-02-02 20:20:48]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:48]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:48] · sufami
WARNING:  [RomM][scan_handler][2025-02-02 20:20:48]  sufami not identified ❌
WARNING:  [RomM][scan][2025-02-02 20:20:48]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:48]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:48] · amiga500
INFO:	  [RomM][scan_handler][2025-02-02 20:20:48]   Identified as Amiga 🎮
WARNING:  [RomM][scan][2025-02-02 20:20:48]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:48]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan_handler][2025-02-02 20:20:48] · hurrican
WARNING:  [RomM][scan_handler][2025-02-02 20:20:48]  hurrican not identified ❌
WARNING:  [RomM][scan][2025-02-02 20:20:48]   ⚠️ No firmware found, skipping firmware scan for this platform
WARNING:  [RomM][scan][2025-02-02 20:20:48]   ⚠️ No roms found, verify that the folder structure is correct
INFO:	  [RomM][scan][2025-02-02 20:20:48] ✔️ Scan completed 
20:20:48 Successfully completed endpoints.sockets.scan.scan_platforms([], <ScanType.QUICK: 'quick'>, [], []) job in 0:00:11.694554s on worker 9fff04fc00054f3aa479748009b5f34d
20:20:48 high: Job OK (b0afa5ca-1ec5-481a-9ac7-c77f05175acf)
20:20:48 Result is kept for 500 seconds
14:M 02 Feb 2025 20:20:48.362 * 100 changes in 300 seconds. Saving...
14:M 02 Feb 2025 20:20:48.363 * Background saving started by pid 1400
1400:C 02 Feb 2025 20:20:48.511 * DB saved on disk
1400:C 02 Feb 2025 20:20:48.511 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
14:M 02 Feb 2025 20:20:48.564 * Background saving terminated with success

Checking the mariadb container i can also see this, which is probably less than ideal I guess:

Image

So i'm kind of stuck now. I see the message about verifying that the folder structure is correct, and i've checked it repeatedly.

/media/crispy/NAS_4TB_1/media/games/batocera/roms does indeed lead to my roms:

Image

Any ideas? I'd love to make romm work again, but this is verging on making me give up.

@crispybegs crispybegs changed the title [Bug] Bug title [Bug] Can't scan any ROMs Feb 2, 2025
@Moonraka
Copy link

Moonraka commented Feb 2, 2025

Delete your entire container and start over. It only makes the MySQL/mariadb user on first build. In your last screenshot it looks like its info was changed since first built

You need to prune any existing or left over containers from romm

Also feel free to come to the discord and make a support thread

@crispybegs
Copy link
Author

crispybegs commented Feb 2, 2025

yes i've nuked the entire thing, containers / volumes / all directories a couple of times but still nothing improving.

db container

Image

romm container

Image

A scan produces the same results

Image

Also i just noticed that, although the setup screens had a green tick next to steamgrid for metadata, the dropdown at the top of the scan page does not include it.

Does that mean nothing is being used for the search, or..?

@Moonraka
Copy link

Moonraka commented Feb 2, 2025

I wonder if you just have your folders linked one too deep for the library in your compose

media/crispy/NAS_4TB_1/media/games/batocera

Try that without the /roms after

Sorry on my phone

Also you will definitely need an IGDB for auto meta data matching. It's free to get

@zodac
Copy link

zodac commented Feb 3, 2025

I wonder if you just have your folders linked one too deep for the library in your compose

media/crispy/NAS_4TB_1/media/games/batocera

Try that without the /roms after

Sorry on my phone

Also you will definitely need an IGDB for auto meta data matching. It's free to get

Agreed, I think it's this line:

/media/crispy/NAS_4TB_1/media/games/batocera/roms:/romm/library

I think the directory you mount needs to include the roms folder, so try:

/media/crispy/NAS_4TB_1/media/games/batocera:/romm/library

@crispybegs
Copy link
Author

ok, changing the path resulted in some action, thank you!

Image

Have to say that's very counterintuitive and should probably be more accurately explained in the readme. The way it's expressed at the moment suggests my original path would work fine, but obviously not.

My next issue is exclusions and the config.yml. As you can see from the screenshot above, I have a load of consoles that I'm not using (for example, 'Arcadia', which has a single 'rom' that is actually an '_info.txt')

Image

..and also a console called 'Stfolder', which is just a syncthing folder

Image

I've tried to add as many exclusions to my config.yml as possible, both through the UI and directly editing it, but it seems hard to get some things to 'stick and, despite doing a complete scan, all the unwanted files and folders are still appearing in the UI

This is my config.yml. Can you suggest where I'm going wrong?

exclude:
  platforms:
  - _info.txt
  - '@eaDir'
  - video
  - .Stfolder
  - images
  - videos
  - '*.Stfolder'
  - '*.txt'
  - '*syncthing*'
  roms:
    multi_file:
      names:
      - _hidden
      - images
      - '@eaDir'
      - '!images'
      - '!orphaned'
      - '!downloading'
      - txt
      - manuals
      - cfg
      - stignore
      - '*videos'
      - '*manuals'
      - .Stfolder
      - '*syncthing*'
      parts:
        extensions:
        - txt
        - xml
        - jpg
        - jpeg
        - images
        - cfg
        - stignore
        - '*videos'
        - .Stfolder
        - '*syncthing*'
        - syncthing
        names:
        - data.xml
        - ._*
        - multidisc
        - manuals
        - cfg
        - '*manuals'
        - syncthing
        - '*syncthing*'
    single_file:
      extensions:
      - xml
      - txt
      - jpg
      - jpeg
      - pdf
      - images
      - manuals
      - '*.cfg'
      - stignore
      - '*.Stfolder'
      - '*syncthing*'
      - syncthing
      names:
      - info.txt
      - ._*
      - '*.nfo'
      - '@eaDir'
      - '*.xml'
      - '!info.txt'
      - mp4
      - '*.mp4'
      - '*.cfg'
      - cfg
      - stignore
      - '*manuals'
      - '*.Stfolder'
      - syncthing
filesystem:
  roms_folder: roms
system:
  platforms:
    bbc: bbcmicro
    dreamcast: dc
    gamecube: ngc
    gamegear: gamegear
    gb: gb
    gba: gba
    gbc: gbc
    lynx: lynx
    mame: arcade
    megadrive: genesis-slash-megadrive
    n64: n64
    neogeo: neogeoaes
    nes: nes
    pcengine: turbografx16--1
    ps2: ps2
    ps3: ps3
    psp: psp
    psx: ps
    saturn: saturn
    sega32x: sega-32x
    segacd: segacd
    snes: snes
    wii: wii
    wiiu: wiiu
    wswan: wonderswan
    wswanc: wonderswan-color
    xbox: xbox
    zxspectrum: zxs
  versions: {}

This is the UI exclusions

Image

@Moonraka
Copy link

Moonraka commented Feb 3, 2025

I'm glad you got your roms are now scanning. It was definitely how the folder was mounted

I haven't really messed with the exclusions myself I'll ask if another tech support from the discord can check this for you.

@crispybegs
Copy link
Author

I'm glad you got your roms are now scanning. It was definitely how the folder was mounted

I haven't really messed with the exclusions myself I'll ask if another tech support from the discord can check this for you.

thank you!

@danblu3
Copy link
Collaborator

danblu3 commented Feb 3, 2025

The config.yml can be powerful but also confusing.

First of all, do you still want the the console to appear? If the answer is no then you will add it here:

exclude: Exclude platforms to be scanned platforms: [] # ['my_excluded_platform_1', 'my_excluded_platform_2']

You would have to manually remove the platform from RomM but when you next scan it will not be seen.

The next exclusion rules are single files FOUND within a platform, so for example if you had a file named manual.pdf

roms: # Single file games section. # Will not apply to files that are in sub-folders (multi-disc roms, games with updates, DLC, patches, etc.) single_file: # Exclude all files with certain extensions to be scanned extensions: [] # ['xml', 'txt']

You would add the .pdf extension in there, so if I scanned and found files within SNES platform, and that platform had gamename.pdf, it will not include those in the scans.

The final exclusion rule is the multi-file which is applied to subfolders.

# Multi files games section # Will apply to files that are in sub-folders (multi-disc roms, games with updates, DLC, patches, etc.) multi_file: # Exclude matched 'folder' names to be scanned (RomM identifies folders as multi file games) names: [] # ['my_multi_file_game', 'DLC']

Using my example again ,if I have a subfolder in there named manuals, I would add manuals to the exclude matched folder names location.

Hope this clear things up for you.

The UI and config.yml are the exact same thing, they talk directly to each other.

EDIT: Cursed github formatting.

@gantoine gantoine added bug Something isn't working and removed bug Something isn't working labels Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants