Q/FR: how to do mix-down with an array of panning signals, instead of a spread? (ChannelPanner to generic mixer)

[balintlaczko]

Hi!

I am trying to wrap my head around the awesome multichannel support in signalflow. Thanks to that, I realize I don't have to make many changes to an existing synth patch, passing lists as params generally works very smoothly.

I am now wondering what is the best method to mix N input channels to M output channels, given a panning signal that is also N channels (it could either go between 0...M-1, or -1...1).

I hacked together something that seems to work:

multisine = sf.SineOscillator([220, 300, 402])
multipansig = sf.SineOscillator([0.1, 0.11, 0.12])

n = multisine.num_output_channels # here assuming that multipansig has the same amount
panner = [sf.StereoPanner(multisine[i] / n, multipansig[i]) for i in range(n)]

bus = sf.Bus(2)

for p in panner:
    bus.add_input(p)

graph.play(bus)

This can be generalized by using ChannelPanner instead of StereoPanner, and scaling multipansig to fall between 0 and M-1 (where M is the ChannelPanner's output channels).

m = 16 # number of output channels in the mixer
multisine = sf.SineOscillator([220, 300, 402])
multipansig = (sf.SineOscillator([0.1, 0.11, 0.12]) + 1) / 2 * (m-1) # scale [-1, 1] to [0, m-1]

n = multisine.num_output_channels
panner = [sf.ChannelPanner(m, multisine[i] / n, multipansig[i]) for i in range(n)]

bus = sf.Bus(m)

for p in panner:
    bus.add_input(p)

# mix down to stereo for listening
out_stereo = sf.ChannelMixer(2, bus)

graph.play(out_stereo)

To me it feels like generating the list of panner objects and summing them in a bus is somewhat against the mindset of multichannel audio in signalflow. Is there currently a different way of achieving this? I am also wondering if perhaps ChannelPanner could be upgraded to support multichannel signals as input and pan?

UPDATE: for convenience, here is the same thing as a patch:

class Mixer(sf.Patch):
    def __init__(self, input_sig, pan_sig, num_channels=2):
        super().__init__()
        assert input_sig.num_output_channels == pan_sig.num_output_channels
        n = input_sig.num_output_channels
        panner = [sf.ChannelPanner(num_channels, input_sig[i] / n, pan_sig[i]) for i in range(n)]
        bus = sf.Bus(num_channels)
        for p in panner:
            bus.add_input(p)
        self.set_output(bus)

# test the mixer
m = 16
multisine = sf.SineOscillator([220, 300, 402])
multipansig = (sf.SineOscillator([0.1, 0.11, 0.12]) + 1) / 2 * (m-1) # scale [-1, 1] to [0, m-1]
mix_multi = Mixer(multisine, multipansig, num_channels=m)
mix_stereo = sf.ChannelMixer(2, mix_multi)

graph.play(mix_stereo)

[ideoforms]

Hi @balintlaczko,

Nice use of the multichannel functionality!
There are a couple of things you could do to make it more concise (untested code):

• Instead of using a Bus and adding inputs one by one, you could use Sum: sum = sf.Sum(panner)
• For multipansig: SineLFO([0.1, 0.11, 0.12], 0, m - 1)

Other than that, I think that the code is nice and concise, not sure if it needs much further improvement...

Upgrading ChannelPanner (and indeed other panners) to allow for some multichannel inputs is a good idea, although in some cases there would be some ambiguity: what if it was passed a stereo input, but asked to pan over 5 channels? But perhaps the pan and width properties could allow for multichannel input, and continue to enforce a mono input signal.

This also makes me notice that, in general, SignalFlow needs better documentation of which inputs can (or must) be mono, multichannel, or scalar...

[balintlaczko]

Thanks, sum works! Now that that loop is out, it looks much nicer!

class Mixer(sf.Patch):
    def __init__(self, input_sig, pan_sig, num_channels=2):
        super().__init__()
        assert input_sig.num_output_channels == pan_sig.num_output_channels
        n = input_sig.num_output_channels
        panner = [sf.ChannelPanner(num_channels, input_sig[i] / n, pan_sig[i]) for i in range(n)]
        _sum = sf.Sum(panner)
        self.set_output(_sum)

This is more than enough for my purpose now.
Regarding the

ambiguity: what if it was passed a stereo input, but asked to pan over 5 channels?

I was now working on an "upmixer" that would linearly interpolate the input channels to the target number of channels. So if the input is a 2-ch signal (let's say with the constants [0, 1]), then the samples would be linearly interpolated in a 5-channel output (e.g. [0, 0.25, 0.5, 0.75, 1]). But I must be doing something wrong, because it crashes the ipython kernel every time I'm trying to use it.

class UpMixer(sf.Patch):
    def __init__(self, input_sig, num_channels=5):
        super().__init__()
        n = input_sig.num_output_channels # e.g. 2
        output_x = np.linspace(0, n-1, num_channels) # e.g. [0, 0.25, 0.5, 0.75, 1]
        upmixed_list = []
        for i in range(num_channels):
            output_i = output_x[i]
            a = input_sig[int(output_i)]
            b = input_sig[int(output_i) + 1]
            frac = float(output_i - int(output_i))
            interp = sf.WetDry(a, b, sf.Constant(frac)) # linear interpolation
            upmixed_list.append(interp)
        out = sf.ChannelArray(upmixed_list)
        self.set_output(out)

Whoops, update, I found the error, went too far with the index. Here is the correct one:

class UpMixer(sf.Patch):
    def __init__(self, input_sig, num_channels=5):
        super().__init__()
        n = input_sig.num_output_channels # e.g. 2
        output_x = np.linspace(0, n-1, num_channels) # e.g. [0, 0.25, 0.5, 0.75, 1]
        upmixed_list = []
        for i in range(num_channels - 1):
            output_i = output_x[i]
            a = input_sig[int(output_i)]
            b = input_sig[int(output_i) + 1]
            frac = float(output_i - int(output_i))
            interp = sf.WetDry(a, b, sf.Constant(frac))
            upmixed_list.append(interp)
        # add the last channel
        upmixed_list.append(input_sig[n-1])
        out = sf.ChannelArray(upmixed_list)
        self.set_output(out)

Update again, it only seems to work if I sum together a list of ChannelPanners instead. Not sure why.

class UpMixer(sf.Patch):
    def __init__(self, input_sig, out_channels=5):
        super().__init__()
        n = input_sig.num_output_channels # e.g. 2
        output_x = np.linspace(0, n-1, out_channels) # e.g. [0, 0.25, 0.5, 0.75, 1]
        output_y = output_x * (out_channels - 1) # e.g. [0, 1, 2, 3, 4]
        upmixed_list = [sf.WetDry(input_sig[int(output_i)], input_sig[int(output_i) + 1], float(output_i - int(output_i))) for output_i in output_x[:-1]]
        upmixed_list.append(input_sig[n-1])
        expanded_list = [sf.ChannelPanner(out_channels, upmixed_list[i], float(output_y[i])) for i in range(out_channels)]
        _out = sf.Sum(expanded_list)
        self.set_output(_out)

[ideoforms]

Diagnosing crashes is annoyingly difficult right now :-( It is possible by making a debug build, but painful...
From a quick glance, I suspect this might be on the b = assignment line - trying to index into index_sig with a value that is 1 beyond its input count?

[ideoforms]

Although it's not well-documented (and possibly not fully tested), ChannelMixer actually implements an N-channel to M-channel interpolator which might do something similar to this, which I think should work for both up- and down-mixing. Take a look and see how you get on!

[balintlaczko]

Thanks for the tip! I checked it now, it seems to fill the new in-between channels with zeros, but it correctly places the input values across the new channel-dim:

b = sf.Buffer(3, 1)
b.data[:, :] = np.ones_like(b.data) * np.linspace(0, 1, 3).reshape(3, 1) # [0, 0.5, 1]
b_player = sf.BufferPlayer(b, loop=True)
# (Q: could this instead just be [sf.Constant(num) for num in np.linspace(0, 1, 3)] ?)
b_upmixed = sf.ChannelMixer(5, b_player, True)
mixdown = sf.ChannelMixer(2, b_upmixed)
graph.play(mixdown)
print(b_upmixed.output_buffer[:, -1]) # array([0. , 0. , 0.5, 0. , 1. ], dtype=float32)