Add DAW Controller support for various Arturia controllers

[soyersoyer]

Some Arturia keyboards have a Display and this can be used to display the menu.
I also added support for relative encoders, which sends 61 62 63 for decrement and 66, 67, 68 for increment.

With this config the MiniDexed can be controlled well with MiniLab3's Main Knob and Display in DAW mode:

DAWControllerEnabled=1

The Home is the shift+Main click

Keylab Essential, Keylab Essential 3, KeyLab mkII work also

[github-actions]

Build for testing:
MiniDexed_2024-11-05-1ad2394
Use at your own risk.

[probonopd]

Thank you very much for this contribution @soyersoyer.

Unfortunately I don't have the matching hardware to test this with, but I am sure many users will find this extremely useful.

Semi off topic: Do you know any good/straightforward documentation how the display and encoder on the Keystep communicates with the host over USB? It just so happens that recently I was thinking about building something the other way around (unrelated to MiniDexed), a simple microcontroller + display + encoder combination that does what a Keystep can do (e.g., to browse presets of the free Analog Lab Play), but without needing an actual Keystep.

[diyelectromusic]

Unfortunately I don't have the matching hardware to test this with, but I am sure many users will find this extremely useful.

Well users of this specific MIDI controller maybe, but I'm really not sure about hardcoding support for just one vendor's device in MiniDexed... In that SysEx message, 0x00, 0x20, 0x6B is the Arturia SysEx Vendor ID - right? It's a little hard to tell as I didn't seem to be able to find any mention of the MIDI implementation of Arturia devices from the manufacturer themself. Do you have a link?

If we were to go down this route, then I think we'd have to have some way of querying a connected device and mapping MiniDexed features to the features of the device itself rather than hard-code in specific SysEx messages in my view.

Do we know what other controllers support displays and how they do it?

@probonopd you weren't keen on me adding bespoke SysEx to MiniDexed to support this kind of thing between MiniDexeds, so I'm a little surprised you seem quite happy to be considering adding bespoke vendor MIDI messages other than Yamaha...?

Kevin

[soyersoyer]

how the display and encoder on the Keystep communicates with the host over USB?

Do you think KeyLab? KeyStep doesn't have a display. In the MiniLab 3 there is an 'Arturia' program for Analog Lab, but I don't know how it communicates. Probably after an initial model query, the main Encoder sends relative CC-s, the other encoders send absolute CCs, the Analog lab sets the encoder values on preset change and updates the display with sysex. If you think, I can capture it.

For the DAW mode, I found this:
https://github.com/PrzemekBarski/arturia-keylab-essential-mk3-programming-guide

hardcoding support for just one vendor's device in MiniDexed...

This is a very popular controller, so I think it might be worth adding.

Additional vendors may be added later.

In that SysEx message, 0x00, 0x20, 0x6B is the Arturia SysEx Vendor ID - right?

Yes

Do you have a link?

From Arturia, No.
I found the link above, and some scripts like these:
https://github.com/gluon/AbletonLive11_MIDIRemoteScripts/blob/main/MiniLab_3/midi.py
https://github.com/bitwig/bitwig-extensions/blob/api-18/src/main/java/com/bitwig/extensions/controllers/arturia/minilab3/SysExHandler.java

We can send generic device inquiry with
F0 7E 7F 06 01 F7
So in theory we can match the model, but we don't have a good spec or access to devices, so we'll probably limit the users by only enabling features for certain models without them being able to control the features.

You are right, the DAWDisplayEnabled config option is too broad and should enable some auto-detection mode.
So I think for this we should use the DAWDisplayArturiaEnabled=1 or DAWDisplay=arturia option.

[github-actions]

Build for testing:
MiniDexed_2024-11-07-361a5ce
Use at your own risk.

[omersiar]

@soyersoyer You beat me to it while planning to do the same. It works with my MiniLab3 (firmware 1.1.1). The only issue is the Dumping MIDI to console which seems not reliable.

Also you don't configure the any of the available sliders, knobs to actual functions like Pan, Volume etc. Let me take a look.

[soyersoyer]

I refined it a bit. Now it does a sysex inquiry and only sends sysex if it is MiniLab3 or KeyLab 3 Essential. Does this fix your MIDI issue?
Do you know what is the preferred layout for the sliders and knobs? And how to set them up?

[github-actions]

Build for testing:
MiniDexed_2024-11-08-2fa8b3a
Use at your own risk.

[github-actions]

Build for testing:
MiniDexed_2024-11-08-5eb53ac
Use at your own risk.

[github-actions]

Build for testing:
MiniDexed_2024-11-08-74ef13c
Use at your own risk.

[github-actions]

Build for testing:
MiniDexed_2024-11-08-8601f20
Use at your own risk.

[omersiar]

I currently use like this

And how to set them up?
Pad RGB can be controlled also you can assign what knobs do and how to represent them.
https://github.com/mhugo/sysex
https://gist.github.com/Janiczek/04a87c2534b9d1435a1d8159c742d260

[soyersoyer]

How should they work? Should they send CC-s to channel 1?

[github-actions]

Build for testing:
MiniDexed_2024-11-11-cff7fd1
Use at your own risk.

[soyersoyer]

Now, it maps the encoders. Also updates the encoders values at performance load from TG0.
The order is not the same as you suggested yet.
Isn't the volume better on fader?
I'm a bit unsure about which TG it should set and where to get the values ​​from.

Unfortunately the pads cannot be configured to send CC-s in DAW mode, they send notes only on channel 10.

What I want also:

• If a new value appears from MIDI CC, it would be good to display it for a little time.
• Remap the pad notes to MIDI CC-s.
• Map the MIDI CC buttons automatically.

[github-actions]

Build for testing:
MiniDexed_2024-11-11-9c2a1bb
Use at your own risk.

[soyersoyer]

I think the encoders should set the parameters based on the menu, so if the display is on the TG1, it should set the TG1 parameters, if on the performance, it should set all TG-s parameters, if it is on the effects it should set the effect parameters. I will try to make this later on a different branch.

[github-actions]

Build for testing:
MiniDexed_2024-11-12-453f37e
Use at your own risk.

[Spacepadrille]

Hi there ! Today I am unable to download The MiniDexed_2025... I have this error message : "Failed to generate URL to download artifact." I tried on chrome and also on Brave, I was connected to my GitHub account... Any idea ?

[soyersoyer]

I think it was deleted because of retention-days:3

e9751e6

There is a 500 MB limit on the free Github tier.

It would be good to keep the most recent artifacts on a PR.

This is a hidden extra motivation to keep polishing the code, haha :)

[github-actions]

Build for testing:
MiniDexed_2025-01-17-9bb1774
Use at your own risk.

[Spacepadrille]

Hello, I am very impressed with this developpement for the Arturia's keyboards. It is going fast, improvements every week or less, bravo !
I just want there to report my experience with the MiniDexed/Soyer_mod and my Minilab3 ; I am a little bit lost with all the different pannels we can now display. Surely it is because I don't master MiniDexed's sub menues, and with some practice it will be fine.
But I must admit I experienced a lot of glitches and delay in the display when I turn the knobs to kraft the sound. Sometimes the MiniDexed freeze and I have to unplug/replug it. Maybe my microSD card is not good enough, or my Rapsberry Pi 3 is to slow, I don't know. But it was not like that before the in depth menu implementation.
An other thing : I use a lcd display on my MiniDexed, and this display is way bigger than the Minilab3's one. So I look to this screen instead of the minilab one while performing. And all the stuff with the menues and sub-menues is not displayed on this MiniDexed screen...
I wish you all a very good day.

[probonopd]

@Spacepadrille thanks for testing. Can you give an example (click path) for a menu that is only available on one but not the other display? Thanks.

[Spacepadrille]

> @Spacepadrille thanks for testing. Can you give an example (click path) for a menu that is only available on one but not the other display? Thanks."
I don't know what is a click path :-), but I try to explain ; The two screens displays the same thing for general navigation, performance banks and preset names. But when you activate by a long press on shift the sub-menus, they are not displayed on the MiniDexed screen, only on the arturia one. From there, if I tweak a little bit to fast some knobs, the display freeze, but the actions are done, just the display have a delay after freezing. This make the whole thing difficult to manipulate, and I wonder if it is because of my hardware or not.

[probonopd]

What is "long press on shift the sub-menu"? @soyersoyer do you understand why the two displays might not behave identically in some situations?

[Spacepadrille]

@probonopd : lol my sentence was badly formulated. I mean activating the display of the sub menu by a long press on shift.

[github-actions]

Build for testing:
MiniDexed_2025-01-19-78141bb
Use at your own risk.

[soyersoyer]

But I must admit I experienced a lot of glitches and delay in the display when I turn the knobs to kraft the sound.

Sorry, I didn't test a change, and it updated the color of the pads too many times. That's what caused the glitches. If you experience this still, please let me know.

What is "long press on shift the sub-menu"?

Holding down the Shift key (CC27) will bring up an overlay menu where you can select control layouts for what the 2x4 encoders change.

The available overlay menus change depending on where you are in the menu. The Main, TG, Effect, Voice, Op menus have different overlay layers. But all voice and operator settings are routed to Main / TG / Voice also, so that on one page, for example, you can set OP_EG_R1 of the 6 operators (encoder 1-6) and all of them with encoder 8.
You can see the control values ​​by briefly pressing the shift.

I think it's a good idea to have the overlay menu appear on the default screen as well. However, it would have to be integrated into the MiniDexed menu and I don't know if others would think it's a good idea.

[Spacepadrille]

Thank you for your hard work !

[Spacepadrille]

I think it's a good idea to have the overlay menu appear on the default screen as well. However, it would have to be integrated into the MiniDexed menu and I don't know if others would think it's a good idea.

Maybe this can be "transparent", I mean not activated on minidexed without Arturia mod ? If you select "DAWControllerEnabled=1" you have the display of the sub menus on the MiniDexed screen, otherwise not. I'm not a coder, so maybe I tell sh** ?

[soyersoyer]

I was thinking that it would be possible to set a MIDI CC in the config for the shift button and 8 for the encoders, and it could work with any configurable controller that has 1 button and 8 encoders.

[soyersoyer]

This functionality will be available here:
soyersoyer#2

[GertSchepens]

Oh. What does this closing mean for moving the controler code into the main MiniDexed?

[soyersoyer]

#767 couldn't make it to MiniDexed in 2 months. Maybe probonopd doesn't want to see any vendor specific stuff in it? I don't know.

#767 is a small change compared to this, so if this PR could be merged, I think it would take about 1 year to get there.

Since these output zips are only available for 3 days, I'd have to do something with the code every 3 days for the next year so people can try it out here.

This makes no sense.

There's also no point in asking people to try the change #800, but that's not available (for most people).

I don't see the reason for changing the retention days to 3 anymore, because it seems like actions from public github projects wouldn't count towards storage. e9751e6

The linked PR doesn't have a 3-day limit. soyersoyer#2

I'll always try to rebase on the main from here.

[GertSchepens]

Thanks for the insight :)
I haven't been able to test for a while, but I'll check-in on the other one asap then ^^

[diyelectromusic]

#767 couldn't make it to MiniDexed in 2 months. Maybe probonopd doesn't want to see any vendor specific stuff in it? I don't know.

Well, to be fair - just /one week/ ago there will still some updates required for this "simple" change.

The hazard of having a larger user-base now for code like this is that changing code and testing new functionality is relatively easy, but reviewing those changes and testing for no side effects for existing platforms (remember this supports Pi V1,2,3,4,5,Zero, Zero 2, 400) takes time. People always come forward to test new functionality, especially when it meets an unmet need they had. But very few people come forward and confirm that everything is still fine without side effects on existing platforms.

And you may recall it was I who was concerned about vendor specific code (#743 (comment)) not @probonopd who has only been very supportive of these more experimental changes.

But my main concern is about longevity of support for these large changes, hence wanting to understand the implications if asked for my opinions on them.

As I say, initial implementation is one thing, but it took quite a while to build up knowledge of the code base after the initial creators left, so when faced with a large body of changes (#743, #791, #790, #778, #782, #779, #777, #747, #783, #755, #750, #758), some of which didn't seem to be solving an obvious known issue, some of which were updating core libraries, and some of which were adding a significant amount of code, then we must collectively forgive the fact that volunteer maintainers (who also do this for fun) might take quite a while to catch up with what is going on.

Kevin

[probonopd]

I am very sorry for the 3 days limit. I have been running out of space that GitHub grants. Still would very much like to see these contributions to be merged!

[soyersoyer]

What do public github projects count towards?

[probonopd]

Good question, I need to check. I just got a warning from GitHub toward end of December that I was over 90%.