[Inference snippets] Templated snippets for inference snippet generation #1255

Wauplin · 2025-03-06T17:54:11Z

What does this PR do?

add handlebars as a dependency for @huggingface/inference
heavily refactor ./packages/inference/src/snippets/python.ts to use templates instead of inline strings
add some snippet templates under packages/inference/src/snippets/templates/{language}/{client}/{templateName}.hbs
- language is Python
- client is e.g. huggingface_hub, requests, etc.
- templateName is related to task name e.g. textToImage, conversationalStream, etc.

=> overall goal is to make it rather easy to update the snippets for a given task + client without entering the JS code
=> if conclusive, we can extend to cURL / JS snippets in a follow-up PR

This PR also includes a lot of small fixes (usually whitespace<>tabs inconsistencies) + added some tests. I did not change anything major in the generated snippets.

What to review?

IMO, no need to review all templates. Better to check the generated snippets + the main JS file directly (https://github.com/huggingface/huggingface.js/blob/ddd62b60d0a858dc808b6dd7b3589977681f5a8b/packages/inference/src/snippets/python.ts). Sorry for the huge PR but I do believe it is a necessary move moving forward 🤗

julien-c · 2025-03-06T18:18:56Z

packages/inference/package.json

@@ -52,9 +52,11 @@
 		"check": "tsc"
 	},
 	"dependencies": {
-		"@huggingface/tasks": "workspace:^"
+		"@huggingface/tasks": "workspace:^",
+		"handlebars": "^4.7.8"


or we could use https://www.npmjs.com/package/@huggingface/jinja maybe 🤯

That would be awesome! 🤩 Although the library was originally designed for ChatML templates, the set of available features should be large enough for these templates.

Maybe @Wauplin can explain what set of features would be required? 👀

basically just ifs and variable replacement from what i've seen

handlebars has pretty much a feature set of 0.00

yea I'm really wary of adding more deps, if we can use jinja it would be great (and we could use jinja for more things)

Will update my PR tomorrow in that direction. As Julien said, I'm not using anything fancy at all so jinja will be more than enough for the job

Sounds good! Let me know if I can help in any way 🫡

I replaced the handlebars dependency by huggingface/jinja and it works like a charm! 86e787a Thanks for the package @xenova! 🤗

well I now have error Couldn't find package "@huggingface/jinja@^0.3.3" required by "@huggingface/inference@*" on the "npm" registry. in the CI though jinja 0.3.3 is available on https://www.npmjs.com/package/@huggingface/jinja 🤔

.vscode/settings.json

SBrandeis · 2025-03-07T10:26:22Z

packages/inference/src/snippets/python.ts

+import { Template } from "@huggingface/jinja";
+import fs from "fs";
+import path from "path";
+import { existsSync as pathExists } from "node:fs";


node packages are not available in contexts other than NodeJS (eg, when using the package in the web browser)

@coyotte508: is this an issue here?

also, I haven't done anything special to be sure the jinja files are exported in the js package. In Python one would need to explicitly set them as package_data in the setup.py but in JS I don't know what should be done

Sure you don't want to make @huggingface/snippets a separate package that works in node-only context and then we don't care about the browser ^^ ? Or move all the snippets code to moon (just export the necessary function from inference)?

With Vite bundler we can do ?raw, with general syntax we can import JSON: https://nodejs.org/api/esm.html#data-imports

But for a general case, you have two solutions IMO:

Instead of jinja files, have TS files with export default "<jinja template">. Then you can do await import(...) inside the code. This may break in some bundlers but we only care about moon use?

Have a gen package (like tasks-gens) and then put all the templates in a TS or JSON file

(anyway at this point it's probably worth it to move the code to moon/separate package IMO, will also make the libs lighter for those that download it)

@coyotte508 can't it be the build script that bundles the files together at build time? (just for browsers? or as an optional dependency?)

Then I think I'll go with a new package 😕
I prefer to have these things open-source instead of in moon-landing to make it easier for external contributions

Or another (dumb?) idea:
@julien-c @coyotte508 what do you think of keeping this code in @huggingface/inference as done in this PR and add a check that raises an error if the code is not executed using nodejs (i.e. if templates not found). No-one will use it anyway, we won't document it and at least things will be kept open-source. No need for an extra package this way.

(keeping things OSS is good for external contributions but also easier to generate these pages: https://huggingface.co/docs/api-inference/tasks/audio-classification#using-the-api with public code here).

#1259 🙈 🙈 🙈

Is there any issue with just ignoring these modules during bundling for the web? 👀 That's what I do for transformers.js, and it seems to work well enough.

packages/inference/src/snippets/templates/python/requests/textToAudio.jinja

Co-authored-by: Simon Brandeis <[email protected]>

packages/inference/src/lib/makeRequestOptions.ts

packages/inference/src/snippets/templates/python/fal_client/textToImage.jinja

…ingface/huggingface.js into templated-inference-python-snippets

coyotte508 · 2025-03-07T14:47:31Z

packages/inference/src/snippets/python.ts

+// Helpers to find + load templates
+
+const rootDirFinder = (): string => {
+	let currentPath = path.normalize(import.meta.url).replace("file:", "");


I'm curious if this will work with moon (import.meta.url), or if it's finally time to move from CJS to ESM cc @Pierrci

are you simply curious or actually wishing for it? x)

(I think it will be fine, but I may be wrong!)

Wauplin added 19 commits March 5, 2025 17:29

tmp commit

ea68541

nope

0006042

draft

8ae6d93

fixing some tab -> whitespaces issues

b86faf5

first move towards templating

f121009

always more

c2231ad

all python snippets as templates + some examples

7090adc

base url for openai snippets

134fa2d

start refacto

d07032e

closer to a good version

2b95376

minimal code + trim all snippets

0766e08

some comments

29ebae9

sort

95a2a71

sort

f529ba6

delete before coming back

b7b6869

moving generated snippets

f8ecf33

remove tmp script

ddd62b6

tabs -> whitespaces

7a498c6

tabs -> whitespaces

e23a2a3

Wauplin requested review from julien-c, hanouticelina, SBrandeis and coyotte508 as code owners March 6, 2025 17:54

Merge branch 'main' into templated-inference-python-snippets

e9a2c13

julien-c reviewed Mar 6, 2025

View reviewed changes

switch from handlebars to jinja

86e787a

SBrandeis reviewed Mar 7, 2025

View reviewed changes

Update .vscode/settings.json

30bbfad

Co-authored-by: Simon Brandeis <[email protected]>

julien-c reviewed Mar 7, 2025

View reviewed changes

packages/inference/src/lib/makeRequestOptions.ts Outdated Show resolved Hide resolved

julien-c reviewed Mar 7, 2025

View reviewed changes

packages/inference/src/snippets/templates/python/fal_client/textToImage.jinja Show resolved Hide resolved

Wauplin and others added 3 commits March 7, 2025 13:04

not private

a280d60

Merge branch 'templated-inference-python-snippets' of github.com:hugg…

274dfe2

…ingface/huggingface.js into templated-inference-python-snippets

Merge branch 'main' into templated-inference-python-snippets

acb99d1

Wauplin mentioned this pull request Mar 7, 2025

Raise if not node #1259

Open

coyotte508 reviewed Mar 7, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Inference snippets] Templated snippets for inference snippet generation #1255

[Inference snippets] Templated snippets for inference snippet generation #1255

Wauplin commented Mar 6, 2025

julien-c Mar 6, 2025 •

edited

Loading

julien-c Mar 6, 2025

xenova Mar 6, 2025

julien-c Mar 6, 2025

julien-c Mar 6, 2025

coyotte508 Mar 6, 2025

Wauplin Mar 6, 2025

xenova Mar 6, 2025

Wauplin Mar 7, 2025

Wauplin Mar 7, 2025

SBrandeis Mar 7, 2025

Wauplin Mar 7, 2025

coyotte508 Mar 7, 2025 •

edited

Loading

julien-c Mar 7, 2025

Wauplin Mar 7, 2025

Wauplin Mar 7, 2025

Wauplin Mar 7, 2025

xenova Mar 7, 2025

coyotte508 Mar 7, 2025

Pierrci Mar 7, 2025

Pierrci Mar 7, 2025

[Inference snippets] Templated snippets for inference snippet generation #1255

Are you sure you want to change the base?

[Inference snippets] Templated snippets for inference snippet generation #1255

Conversation

Wauplin commented Mar 6, 2025

What does this PR do?

What to review?

julien-c Mar 6, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coyotte508 Mar 7, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

julien-c Mar 6, 2025 •

edited

Loading

coyotte508 Mar 7, 2025 •

edited

Loading