Feature discussion (OSC and VST)

General Mixxx discussion.... discuss

Moderator: garth

Feature discussion (OSC and VST)

Postby Per Lindgren » Thu Sep 05, 2019 10:50 am

Hi Folks

I'm using Ableton and Reaper (with loads of VSTs), Traktor 3, MrsPinky and now MIXXX. I use a NI Audio DJ interface, 64 byte sample buffer and the lowest latency setting, without dropouts in MIXXX, under Win10 on a modern HP-Omen 15" gaming laptop.

Use case.

Recording, editing and reproduction (synthesis) of scratch movement (timecode positions, fader positions).
Ideally MIXXX could work as a VST plugin, allowing timecode to be recorded in the DAW, and as a playback device (taking timecode data from the DAW as input, along with midi data for fader position (and other parameters controllable in MIXXX).
In LIVE mode the plugin should take input from the timecode vinyl (audio in), while in PLAYBACK mode it would be controlled by the DAW. One could also think about splitting concerns having a VST plugin just for the vinyl input and separate VST for the playback part.

Edting: The timecode data should be possible to represented as 14 bit midi controller data (and visualized in Ableton/Reaper as automation waveforms).

Midi control: (fader position) could be pretty much straightforward and should not need to affect the MIXXX VST that much (there is already some MIDI support).

Meta data: There needs to be some meta data stored with the VST (like a MIXXX project today), to allow for storing associated samples, and such.

Impact/outcome. A very powerful tool for DJ based music production not only intended for the turntablist, but also usable for the mainstream producer. Automation tracks (recorded scratch patterns and not merely audio/sounds) could be re-used in different settings, edited, copied pasted etc. By changing or processing the associated sample(s), would open up for completely new soundscapes to be created. As automation data can be edited, one could create new patterns that is humanly impossible to replicate. So it would become a scratch synthesizer (among other things). (Well I'm aware there are some similar plugins for FL studio, WaveTraveller, ... but the timecode based input is where MIXXX-VST would be unique, also I believe MIXXX to sound quite good. I haven't done any real analysis, measuring distortion, aliasing etc. but its pleasing to the ear at least.)

I have experimented a bit with Mrs Pinky. MrsPinky is available as a VST and the non-VST version supports OSC (admittedly I have not tried that). However I feel that MIXXX have a better sound reproduction engine and vinyl control feels better (the dynamic reproduction of sound seems off with Mrs Pinky).

---
I write here as MIXXX is open source and I have some questions/ideas regarding MIXXX OSC/VST support.

As I understand there is currently no plan to make MIXXX a VST, and that would likely require a lot of work.

Perhaps many of the things mentioned above could be done through OSC (but I fear that latency might become an issue, in comparison to directly operating within the DAW (as there sound buffers are directly passed to/from the DAW)). But as I understand from browsing the forums, OSC support is still not really integrated (I found some git repos, but not sure how much has been merged yet). At least I successfully compiled MIXXX from source under Linux (cloned from master), and should be able to look into the coding from there.

So I can see some different routes from here.

1 Gather forces (if more devs are interested) to revive the idea of MIXXX-VST.
2 Break out the timecode extraction part into a separate code base, and make that into a VST, to start out with. (Maybe some existing VST could be used to reproduce the sound to start out with).
3 OSC support for timecode position (send/receive) in current MIXXX. Then use or write a separate VST plugin that record/playback OSC as controller data in the DAW. Something like https://www.youtube.com/watch?v=GF4H4Tnj2hE.

I have looked a bit into VST coding in JUCE, I'm no C++ expert but fairly experienced program developer (now mainly Rust). I have 0 experience with the MIXXX code base (besides actually getting it to compile which was successful at first attempt).

I would assume 3 is the easiest (lowest hanging fruit), while 1 being the hardest.

Not sure this is the right forum for this type of discussions, if not please direct me to the right "channel".

Best regards
Per
Per Lindgren
 
Posts: 4
Joined: Thu Sep 05, 2019 9:27 am

Re: Feature discussion (OSC and VST)

Postby Be. » Thu Sep 05, 2019 11:33 am

This is a cool idea, but yeah I think it would be a ton of work and repurposing Mixxx for this may not be the easiest way to accomplish what you want. I think the user experience would end up being pretty clunky when repurposing an application not designed to be used this way; Mixxx has lots of features that would be useless distractions for this purpose. For timecode, Mixxx uses the xwax code. I think it would be easier to make this plugin by creating a new codebase that also uses xwax's code instead of hacking Mixxx to behave in ways that are quite different from its main purpose.

xwax is written in C but the VST SDK is in C++, so if you made a VST plugin it would need to be in C++. If you made an LV2 plugin, you could use Rust, but neither Ableton nor REAPER support LV2 plugins.

Note that xwax (like Mixxx) is licensed under the GNU GPL, so if you use it, you'll have to license the code you write under the same license.
Mixxx is free because it's yours!

I heard FLAC and I haven't gone back.
Protect your hearing with earplugs!

Hear my mixes
User avatar
Be.
Mixxx Developer
 
Posts: 2573
Joined: Tue Jan 06, 2015 1:00 am
Location: Chicago, USA

Re: Feature discussion (OSC and VST)

Postby Be. » Thu Sep 05, 2019 12:07 pm

Per Lindgren wrote:In LIVE mode the plugin should take input from the timecode vinyl (audio in), while in PLAYBACK mode it would be controlled by the DAW. One could also think about splitting concerns having a VST plugin just for the vinyl input and separate VST for the playback part.


I don't think you would need separate plugins nor modes for this. You could record the timecode into a DAW and use it as a sidechain input for the plugin.
Mixxx is free because it's yours!

I heard FLAC and I haven't gone back.
Protect your hearing with earplugs!

Hear my mixes
User avatar
Be.
Mixxx Developer
 
Posts: 2573
Joined: Tue Jan 06, 2015 1:00 am
Location: Chicago, USA

Re: Feature discussion (OSC and VST)

Postby Per Lindgren » Fri Sep 06, 2019 7:20 am

Thanks for your input. I think the DAW storing the timecode could work (a bit of latency to expect bouncing it to MIXXX and back to the DAW, but it could likely be compensated for in the DAW).

Big drawback though is that editing scratches would be in the "blind".

I will look into this lib for reading time code and try figure out if its possible to integrate into a VST (I have looked at JUCE and made some simple automation VST just to try out and it seems fairly simple).

Best
Per
Per Lindgren
 
Posts: 4
Joined: Thu Sep 05, 2019 9:27 am

Re: Feature discussion (OSC and VST)

Postby Be. » Fri Sep 06, 2019 10:39 am

Per Lindgren wrote:Big drawback though is that editing scratches would be in the "blind".


On the other hand, it could be really interesting to apply audio effects to timecode to manipulate how it scratches. The fader automation could be done on the track the scratching effect is being applied to.
Mixxx is free because it's yours!

I heard FLAC and I haven't gone back.
Protect your hearing with earplugs!

Hear my mixes
User avatar
Be.
Mixxx Developer
 
Posts: 2573
Joined: Tue Jan 06, 2015 1:00 am
Location: Chicago, USA

Re: Feature discussion (OSC and VST)

Postby Per Lindgren » Sat Oct 12, 2019 11:52 pm

I found a VST pluging doing almost what I want, its called Scratch Track. Its commercial but there is a 30 day trial, so I tested it. Unfortunately I could not get it working as intended, and as I found little to no documentation I tried to contact the developer (twice) to no avail so far. In any case I'm more in favor of open source solutions and I think MIXXX is doing a very good job as a DVS. I really like the simplicity, and the gui is very intuitive. Props to all the devs.

In the meantime I have played around building a low latency crossfader and it works well with MIXXX. I use a bluepill (2$ devboard) and a slighly modified focus fader (non contact optical fader). All code is written in Rust, implementing a USB HID class. I intend to make it an open hardware/software project, with DIY instructions eventually.

For comaprison I got myself a Numark Scratch. Its comes with a inno fader, USB 24 bit build in sound card DVS control and Serato DJ Pro. (Its in the same price range as NI Z2, both targeting the semi-pro market, and not costing you an arm and a leg.)

I also wanted to get rid of my time code vinyls and went for the Phase. I worked out the box with both Serato and MIXXX using the Numark Scratch as the interface. Maybe somebody made a proper mapping for the Numark Scratch for MIXXX (there is quite a lot of automation possible), I just mapped the crossfader. By doing so, I found that the Numark Scratch seems to output erroneous fader position data. I tried to calibrate it but still the fader "jumps around". It can be audiable at times, however I use a very steep curve for scratching so its only if the faulty data is around the cut-in/cut out point it is audiable. I'm just wondering if it could be a software issue on the MIXXX side. Has anyone experienced similar "jumping" behavior? (As I want low latency for scratching I run with the lowest sized buffer, Numark Scratch provides a buffer of 32, MIXXX setting is for 1.33ms at 48kHz. MIXXX says the system reports 5 ms. So I assume MIXXX will handle midi each such frame (1.33ms). Is that correctly assumed?

Looking at the learned midi I find:
''''
<?xml version='1.0' encoding='utf-8'?>
<MixxxControllerPreset schemaVersion="1" mixxxVersion="">
<info/>
<controller id="Numark">
<scriptfiles/>
<controls>
<control>
<group>[Master]</group>
<key>crossfader</key>
<description>MIDI Learned from 222 messages.</description>
<status>0xBF</status>
<midino>0x08</midino>
<options>
<normal/>
</options>
</control>
</controls>
<outputs/>
</controller>
</MixxxControllerPreset>
''''

This looks OK right? (I had a similar setup for my USB, but using the controller 0x05 instead).

I'll start by contacting Thomann (from which I bought the mixer and as for a fader replacement, there is likely something wrong with it.) I'll try MidiOX, to see if I can replicate the strange "jumping behavior" first).

In any case, I'd like to record the MIXXX output into Ableton. Any suggestions. You cannot send the MIXXX signal to several destinations directly, so maybe Jack is my best bet. I tried a bit, but no success. There is this Rewrire and Rearout thingies as well, but they seem not to appropriate to what I want.) Any suggestions? So what I want to do is having Ableton playing the track, I want to record scratches in Ableton. In this setup I need low latency montoring (5ms if possible).

Any suggestions?

/Per
Per Lindgren
 
Posts: 4
Joined: Thu Sep 05, 2019 9:27 am

Re: Feature discussion (OSC and VST)

Postby daschuer » Wed Oct 16, 2019 6:00 am

Interesting ideas.
Currently Mixxx is a bit behind when it comes to interfacing other applications. But we have different demands to change that. Being a VST Plugin Sounds interesting. I can imagine to use VST as an alternative sound API.
This way you can pipe in you stream as aux and pipe it out as master. Will something like this solve your needs?

When it comes to Linux I think Jack can do it as well.
I have a stalled branch that makes Mixxx a native Jack client, maybe we can revive it.

Piping just the transport info of a track out of Mixxx can be probably straight forward be done via OSC.
We have some foundation done, but unfortunately gets also stalled.

I would be very happy if you have interest to continue some of this work.

While the whole project outlined in you initial post is very much work, it would be grate to do some easy steps in the direction. This way you will become familiar with the code base and your work will help other users with the DAW use case as well.

This is a good start to read:
https://www.mixxx.org/wiki/doku.php/bugfix_workflow

I would also be happy to welcome you at https://mixxx.zulipchat.com
This is the place where the developers are around.

A good first step would be to break down your project into small packages and check if there arge edges in the Mixxx code base we need to iron and how itt fits to Mixxx.
daschuer
Mixxx Developer
 
Posts: 1269
Joined: Sun Mar 10, 2013 11:26 am

Re: Feature discussion (OSC and VST)

Postby Per Lindgren » Wed Oct 16, 2019 3:20 pm

Hi Folks

Thanks for your input @dashuer, I will have a look the workflow. I was able to compile MIXXX from source under Lunix (which is my major dev environment). However, for audio Windows (or even OSX) would be the preferred platform.

I'll also check into the zulip chat, may we hook up there (assuming you are daschuer there as well).

Best regards,
Per
Per Lindgren
 
Posts: 4
Joined: Thu Sep 05, 2019 9:27 am

Re: Feature discussion (OSC and VST)

Postby daschuer » Thu Oct 17, 2019 8:27 am

The nice thing is that you can develop with Linux, issue a Pull Request and download the windows binary from AppVeyor.

See you on Zulip
daschuer
Mixxx Developer
 
Posts: 1269
Joined: Sun Mar 10, 2013 11:26 am


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 17 guests