Typescript, Vitetest, and Autoload

[Bugs5382]

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Fastify version

4.26.1

Plugin version

5.8.0

Node.js version

20.11.1

Operating system

macOS

Operating system version (i.e. 20.04, 11.3, 10)

14.2.1 (23C71)

Description

Greetings Fellow Fastify Community!

As a long-time user, I use fastify-cli as my primary method for running my applications. The app.ts files have the typical structure for doing this with the client (after Typescript compile) with no issue, and autoload works. I had in the past to skip doing unit testing, but this new app I am working on, unit testing is key to making sure everything is working correctly, plus if I get it working here, then others could use it as well.

In short, the unit test has this code (view full source here):

import fastify, {FastifyInstance } from "fastify";
import {fileURLToPath} from "node:url";
import {dirname, join} from "path";
import {describe, test, beforeEach, afterEach, expect } from 'vitest';
import buildApp from '../src/app'

const fileName = fileURLToPath(import.meta.url)
const dirName = dirname(fileName)

let server: FastifyInstance

beforeEach(async () => {
  server = await buildApp(fastify())

  await server.ready()
})
afterEach(async () => {
  // called once after all tests run
  await server.close()
})

However, no matter what I did, looked at, searched, etc. I keep getting this error:

TypeError: Unknown file extension ".ts" for /Users/hel/Documents/removedGIT.nosync/jams/node-microservice-itil/src/plugins/graphql.ts
    at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:160:9)
    at defaultGetFormat (node:internal/modules/esm/get_format:203:36)
    at defaultLoad (node:internal/modules/esm/load:143:22)
    at ModuleLoader.load (node:internal/modules/esm/loader:409:7)
    at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:291:45)
    at link (node:internal/modules/esm/module_job:76:21)

Now if I take out:

  server = await buildApp(fastify())

replace it with

  server = await fastify()
server.register(customHealthCheck)

... as an example... it works. However, the autoload, either directly in the unit test or loading from the function, does not work.

Many thanks in advance, as I am not sure what is going on.

Steps to Reproduce

I have the updated code saved here with this error: https://github.com/JAMS-Project/node-microservice-itil/tree/1-test-build-basic-unit-test-structure

Updated Unit Test Here: https://github.com/JAMS-Project/node-microservice-itil/blob/1-test-build-basic-unit-test-structure/__tests__/basic.test.ts

Expected Behavior

Unit tests should run :)

[Bugs5382]

@mcollina ping png :)

[mcollina]

Essentially vitest + autoload + ts are incompatible with each other and nobody found a way to make it work. It seems vitest is doing something interesting to support its own magic which is incompatible with our magic. PRs are absolutely welcome.

[Bugs5382]

Challenge accepted.

[Bugs5382]

@mcollina So... this was pretty simple..

➜  fastify-autoload git:(master) npx npm-check-updates -u --enginesNode
Upgrading /Users/user/Code/fastify-autoload/package.json
[====================] 25/25 100%

 @fastify/pre-commit       ^2.0.2  →     ^2.1.0
 @fastify/url-data         ^5.0.0  →     ^5.4.0
 @swc-node/register        ^1.5.1  →     ^1.8.0
 @swc/core               ^1.2.129  →     ^1.4.2
 @types/jest              ^29.0.0  →   ^29.5.12
 @types/node              ^20.1.0  →  ^20.11.24
 @types/tap               ^15.0.5  →   ^15.0.11
 esbuild                  ^0.19.2  →    ^0.20.1
 esbuild-register          ^3.4.1  →     ^3.5.0
 fastify              ^4.0.0-rc.2  →    ^4.26.2
 fastify-plugin            ^4.0.0  →     ^4.5.1
 jest                     ^28.1.3  →    ^29.7.0
 standard                 ^17.0.0  →    ^17.1.0
 tap                      ^16.0.0  →    ^18.7.0
 ts-jest                  ^28.0.7  →    ^29.1.2
 ts-node                  ^10.4.0  →    ^10.9.2
 tsd                      ^0.29.0  →    ^0.30.7
 tsm                       ^2.2.1  →     ^2.3.0
 tsx                       ^3.7.1  →     ^4.7.1
 typescript                ^5.0.2  →     ^5.3.3
 vite                      ^5.0.0  →     ^5.1.4
 vitest                   ^0.34.1  →     ^1.3.1

I then attached my code locally, and it ran:

So, do you want to accept this many changes, or do you want me to pair it down to the minimum needed to pass both unit testing here and in my script?

[mcollina]

Please send the PR :)

[Bugs5382]

Roger. Let me get the unit tests working. Some of the unit test functionality has "warnings" due to the impending version 5 of Fastify.

[melroy89]

Essentially vitest + autoload + ts are incompatible with each other and nobody found a way to make it work. It seems vitest is doing something interesting to support its own magic which is incompatible with our magic. PRs are absolutely welcome.

Well. Try to use VITEST=true together with FASTIFY_AUTOLOAD_TYPESCRIPT=1, see also: fastify/help#1014 (comment). My repository wasn't using Vitetest framework, but I still needed to use this environment variable as a workaround for my "normal" ESM typescript project with ts-node. See also my demo repo: https://github.com/melroy89/fastify-node-ts-issue

Looks a bit related to each other.

[zt-9]

I am having the same issue.

[zt-9]

run tests with VITEST=true FASTIFY_AUTOLOAD_TYPESCRIPT=1 does not solve the issue
my command
"test:vitest": "FASTIFY_AUTOLOAD_TYPESCRIPT=1 VITEST=true vitest run"

[Uzlopak]

Can you provide a simple repo as a mvce?

This actually should make it easier to fix, if you say that using env vars make it work. It means that the underlying code is working but the detection for vitest is not.

If you provide a simple repo it becomes super easy to debug and patch it.

[climba03003]

I am closing in favor of #372
Please provide a minimal reproducible example if you think the issue still exist.

[fox1t]

Essentially vitest + autoload + ts are incompatible with each other and nobody found a way to make it work. It seems vitest is doing something interesting to support its own magic which is incompatible with our magic. PRs are absolutely welcome.

@mcollina, this issue still persists but I have a fix for it. The two magics are compatible at last!

The issue

By default, vite-node transform, which Vitest uses to do its magic, is lazy and doesn't run for the code in the node_modules folder. The problem arises when you want to test import/require packages with the TS extension imported from the node_modules folder like @fastify/autoload does.
node itself will throw this error:

TypeError: Unknown file extension ".ts" for /autoload-vitest/src/plugins/my-plugin.ts
{ code: 'ERR_UNKNOWN_FILE_EXTENSION' }

The solution

The way to fix this is to ensure the Node.js process that runs @fastify/autoload can handle TypeScript files.
There are two options to do this:

1. Pass tsx as an import/register option to Vitest (and therefore to vite-node and node itself) like this:

test": "NODE_OPTIONS='--import tsx' vitest run"

2. Ensure vite-node transformations run for the @fastify/autoload package by adding this to your vite.config.mts:

  test: {
		server: {
			deps: { inline: ["@fastify/autoload"] },
		},
	},

I think this is worth mentioning in the @fastify/autoload docs.

You can find here the repro with the instructions to fix it.
https://github.com/fox1t/autoload-vitest

[climba03003]

1. Using --import tsx means it is using another transpiler. I don't think it should be included for vitest.
2. It is fine to add the step for vitest. For me, it is more likes the problem or requirement of vitest itself.

[fox1t]

1. Using --import tsx means it is using another transpiler. I don't think it should be included for vitest.
2. It is fine to add the step for vitest. For me, it is more likes the problem or requirement of vitest itself.

I'm afraid I have to disagree it is not related to vitest. I prefer option one since it is more consistent and works in monorepos. Option 2 doesn't work in monorepos since vitest is not able to parse dependencies of dependencies. If you have appA importing package 1 that is importing @fastify/autoload, the inline option will not have any effect.

[jean-michelet]

It seems there are often issues with typescript support more broadly, do you think there is work to do on this area @climba03003? Maybe improving the documentation?

At the moment, I'm not familiar with this part and it seems rather tricky... But I'm not opposed to work on it.

[climba03003]

I'm afraid I have to disagree it is not related to vitest. I prefer option one since it is more consistent and works in monorepos. Option 2 doesn't work in monorepos since vitest is not able to parse dependencies of dependencies. If you have appA importing package 1 that is importing @fastify/autoload, the inline option will not have any effect.

It should not be included for vitest because you can use any loader for the same effect.
From your statement, I would parse as if you want to use vitest please combine with any TypeScript transpiler.

From this case, I would remove vitest support from the list because we don't need it.
Maybe we can keep it for ESM only, and remove from TypeScript.
So, users will be force using TypeScript loader in combine with vitest.

fastify-autoload/runtime.js

Lines 101 to 116 in 178979f

	supportTypeScript: {
	get () {
	cache.supportTypeScript ??= (
	checkEnvVariable('FASTIFY_AUTOLOAD_TYPESCRIPT') ||
	runtime.tsNode ||
	runtime.vitest ||
	runtime.babelNode ||
	runtime.jest ||
	runtime.swc ||
	runtime.tsm ||
	runtime.tsx ||
	runtime.esbuild
	)
	return cache.supportTypeScript
	}
	},

[fox1t]

Oh sure, tsx was just an example. We can remove the heuristics around the test and add an explicit indication to the docs to use NODE_OPTIONS to inject the same transformer the users already have in their project. The only downside of this approach is that we will force the users to always pass FASTIFY_AUTOLOAD_TYPESCRIPT just for testing but not for the dev (since it will be enabled by supportTypeScript).

@climba03003, we might detect vite-node somehow and print an informative message asking the user to add an explicit import node option. What do you think?

[fox1t]

It seems there are often issues with typescript support more broadly, do you think there is work to do on this area @climba03003? Maybe improving the documentation?

At the moment, I'm not familiar with this part and it seems rather tricky... But I'm not opposed to work on it.

Yes, ts support in autoload was historically a nightmare, but I think things are way better now. I see no direct area of improvement at the moment.

[climba03003]

The only downside of this approach is that we will force the users to always pass FASTIFY_AUTOLOAD_TYPESCRIPT just for testing

I think it is not needed, we can detect various of loader already.

[fox1t]

The only downside of this approach is that we will force the users to always pass FASTIFY_AUTOLOAD_TYPESCRIPT just for testing

I think it is not needed, we can detect various of loader already.

Yes, you are absolutely right!

[fox1t]

@climba03003 unfortunately, I have an update. Using option one breaks vitest: mocks are not working anymore. I suspect it is because another transformer retransforms what the vite-node already mocked. So the only viable option is 2, and it will not work 100% in monorepos when you have sub-dependencies.