[Feature] Accuracy test of VLM

[zhaochenyang20]

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Motivation

In sglang, LLMs have accuracy tests with Hugging Face models:

https://github.com/sgl-project/sglang/blob/main/test/srt/models/test_generation_models.py

https://github.com/sgl-project/sglang/blob/main/test/srt/test_nightly_math_eval.py

We need similar one for VLM also.

Related resources

No response

[adarshxs]

Happy to take this up @zhaochenyang20

[zhaochenyang20]

Happy to take this up @zhaochenyang20

Hey Adarsh, @adarshxs thanks! Welcome to contribute!

[zhaochenyang20]

#3141

Duplicated ones as.