work with numpy

[sugizo]

objective
work with numpy to give filter in this example below is to make video in grayscale

steps

!wget -c https://cdn.creatomate.com/demo/mountains.mp4
!wget -c http://www.jsayles.com/music/sheep.mp3
pip install -U ffmpegio

code

import ffmpegio
import os
import subprocess
import numpy as np
with ffmpegio.open('mountains.mp4', 'rv') as fin, ffmpegio.open('temp.mp4', 'wv', rate_in = fin.rate, overwrite = True) as fout:
    for frame in fin:
        grayscale= np.dot(frame[0,...], [0.299, 0.587, 0.144] )
        fout.write(grayscale)
subprocess.check_call(
    [
        "ffmpeg",
        "-y",
        "-i",
        'temp.mp4',
        "-i",
        'sheep.mp3',
        "-r",
        '10',
        "-vcodec",
        "libx264",
        "-acodec",
        "copy", # aac (smallest), libmp3lame, libtwolame (biggest), copy (faster because not reencoding)
        "-shortest",
        'grayscale.mp4',
    ]
)
os.remove(temp)

result

'<f8'
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/ffmpegio/utils/__init__.py](https://localhost:8080/#) in guess_video_format(shape, dtype)
    426     try:
--> 427         pix_fmt = {
    428             "|u1": {1: "gray", 2: "ya8", 3: "rgb24", 4: "rgba"},

KeyError: '<f8'

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
4 frames
[/usr/local/lib/python3.10/dist-packages/ffmpegio/utils/__init__.py](https://localhost:8080/#) in guess_video_format(shape, dtype)
    432     except Exception as e:
    433         print(e)
--> 434         raise ValueError(
    435             f"dtype ({dtype}) and guessed number of components ({ncomp}) do not yield a pix_fmt."
    436         )

ValueError: dtype (<f8) and guessed number of components (1) do not yield a pix_fmt.

best regards

[tikuma-lsuhsc]

Short answer: grayscale= np.dot(frame[0,...], [0.299, 0.587, 0.144] ).astype(np.uint8)

You're responsible to put the input Numpy array in the correct image pixel format. ffmpegio simply passes the numpy data bytes straight to FFmpeg. Assuming that the video is RGB24 (8-bit per color channel) your input data dtype must be either "<u1" or np.uint8.

Now, if you're performing these fixed function (the same transform on every frame) read multiple frames at a time (as many as your system can handle). Looping one frame at a time significantly slows down the process.

Finally, as I alluded in the other post of yours, I'm a big proponent of using the FFmpeg filters whenever possible. In this case, colorchennelmixer does the job with ffmpegio.transpose. Something along the line of

ffmpegio.transpose('mountains.mp4', 'temp.mp4', overwrite = True, vf='colorchannelmixer=0.299:0.587:0.144:00.299:0.587:0.144:00.299:0.587:0.144:0')

I have not tested this command, but you get the idea. If you want to apply multiple effects (to grayscale + flip) then you can chain multiple filters as needed.