Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transform from video_description modality to video_global_description modality. #18

Open
kdu4108 opened this issue Jul 19, 2024 · 0 comments
Assignees

Comments

@kdu4108
Copy link
Collaborator

kdu4108 commented Jul 19, 2024

Given data from:

video_description modality

root/video_description/shard-00000.tar
 |     ├── 00000.jsonl # this corresponds to one video. each line within it corresponds to one subsequence of frames.
 |     ├── 00001.jsonl
 |     └── ...

and the video_metadata modality

root/metadata/shard-00000.tar
 |     ├── 00000.json # this corresponds to one video.
 |     ├── 00001.json
 |     └── ...

and maybe video_transcript modality

root/video_transcript/shard-00000.tar
 |     ├── 00000.jsonl # this corresponds to one video. each line within it corresponds to one subsequence of frames.
 |     ├── 00001.jsonl
 |     └── ...

generate a global description (one string) for each video. The output format should look like

root/video_transcript/shard-00000.tar
 |     ├── 00000.txt # this corresponds to the global description for one video.
 |     ├── 00001.txt
 |     └── ...

And xxxxx.txt just looks contains a string, e.g.

This is a global description of a video!
@markus583 markus583 self-assigned this Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants