AZSlow

AZ Controller plug-in for Cakewalk SONAR => Discussions => Topic started by: norfolkmastering on March 04, 2022, 05:20:13 PM

Title: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on March 04, 2022, 05:20:13 PM
Hello Alexey

This question relates to my use of AZ Controller to allow remote control of my external analogue mixer (which is complete and fully working - so a big 'thank you' for all your help to date)

At the moment I have to use a total of 9 sends per track to allow:
Track routing to Groups 1 -8 (8 Cakewalk buses)
Track routing to a hardware output

I'm finding on large projects that Cakewalk Screen update response time is very slow and this method also fills the mixer control screen with large number of send level and pan controls which I never use.  This means I'm constantly having to scroll up and down the screen to get to controls I do want to use.

I don't ever need to route a track to more than one bus at the same time, so rather than use sends for this track to bus routing, does the Cakewalk API allow access to the track to output routing function?

If this is possible, I would programme a software state group which could provide MIDI feedback values to tell me which group each track is routed to.

This would be a more elegant way control the track to bus routing in the hardware mixer, rather than the current  method of having to read the send enable states for a very large number of sends.  It would hugely reduce the number of elements in the AZ preset which in turn would speed up the response time of Cakewalk.

Best regards
Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: azslow3 on March 04, 2022, 09:36:25 PM
Do you mean "Output" parameter? It is directly in Strip Action.
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on March 05, 2022, 11:03:58 AM
Hi Alexey

Yes I meant the "Output" parameter.
I previously thought that the "Output" parameter was only a way to access sends.
I did not realise you could access the track output directly using this parameter.  My mistake!

So I made a quick test preset, creating a Software State including the buses which I named 'Group 1' through to 'Group 8'.
I was able to derive feedback MIDI values which tell me which bus each track is routed to.
Fantastic!

So now I will be able to greatly simplify my track to bus routing control.

Many thanks
Best regards
Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on March 08, 2022, 06:21:18 PM
Hello Alexey

I started to implement the revised method of controlling my analogue audio mixer, track to bus routing, using the Cakewalk track output parameter as feedback MIDI values to my PIC.
My development preset is attached. It runs in the 8th instance of AZ controller.

There are 24 AZ controls used to derived the 24 tracks to output status.
T1MONOUT is for track 1 etc.

I have found that these Monitor actions are slowing down the response time and smoothness of the Cakewalk user interface, especially for faders which is an issue.
I have tried setting all the Track Output 'Monitor parameter value' actions to 'Slow' but I'm not sure if that is helping improve the general response of the user interface.
The user interface gets worse as you add more tracks into the project.  With 24 tracks it is not very usable.
Could you have a look at the attached preset please and let me know if there are any ways to adjust the Track Output monitoring to improve the Cakewalk user interface response please.
Or any other ideas to improve the responsiveness of the system due to this issue?

I have a similar but less critical issue.  24 AZ controls are used to set the output of one send in each track to a specified output.  This is driven from my PIC.
The controls are 'T1DAW' to 'T24DAW' in the preset.
When I send the 24 MIDI messages from my PIC, it is taking round 20 seconds for the 24 track sends to set their outputs.  This is a very long time!  It has never failed to work but I am concerned.

Your help would be very much appreciated on these two track to output selection issues.

Regards
Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: azslow3 on March 09, 2022, 09:03:32 PM
Can you do simple test... Leave just one AZ Controller with this preset loaded, but set AZ Controller MIDI in and our to "None". So it works without your hardware.
Check the influence of such instance on performance (in terms of UI responsiveness and CPU use in the Task Manager).

In my test I loaded your preset and changed all 24 monitors to Ultra. The project just has "empty" tracks (32), and some buses (16). On some tracks I put one send, to check T1DAW like controls.
The result: my i9 has CPU load from Cakewalk under 3% (without GUI operations and the transport stopped under 2%). Cakewalk GUI has no perceived response change when your preset is loaded. Singe T1DAW definitively takes less then a second, but I can't test how long it will take to change all 24.

It can be my i9 is so powerful it is not possible to notice any performance penalties. AZ Controller was originally developed on Core2 Celeron, and Centrino based notebook. That time any performance problems was instantly reproducible. But I don't have such slow equipment at the moment (desktop is i9 and notebook is i7).

But it can be something else is degrading the performance on your particular system and/or project and AZ Controller is just "the last drop" which render the degradation visible (once Cakewalk GUI can't do what it wants within fixed time slots, the whole interface becomes almost unusable, but till that happens, independent how far it is from the limit, everything is smooth).

We have to understand which case you observe. What is "idle CPU use" of the project? I mean without AZ Controller(s) and transport stopped? Do GUI operations (f.e. moving sliders) significantly increase CPU load?
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on March 10, 2022, 01:32:45 PM
Hello Alexey

I did some tests as you requested.

For information my PC has a i7-8700K processor running at 3.7GHz and it has 32GB of RAM.

1.
Opened Cakewalk by Bandlab
Loaded the REPLAY_080322-4.spp preset
(all 24 x Track to Output Monitor set to Slow)
AZ Controller set to MIDI in and MIDI out "None"
Refresh Frequency set to 25ms
Loaded Test Project, 48kHz, 24 audio tracks and 9 buses, into Cakewalk by Bandlab.
CPU loading 12%
Fader response poor
Pressing play and moving a group of two faders makes little difference to CPU loading, saw a peak of 13.2%.

2.
Changed all Track to Output Monitors to Ultra
Project will not load.

3.
Delete all feedback events from the Track to Output Monitor controls (but did not delete the Track to Monitor controls, which are still set to Ultra)
CPU is under 1%
Fader response is very smooth
Playback is smooth

4.
My current 'live' system uses no Track to Output Monitor actions but instead uses 8 sends per track as the means to route tracks to buses.  The full system uses two AZ presets with a total of 466 controls (a much higher number of controls than the test preset I sent you).

I loaded both 'live' presets and connected to my usual MIDI ports, including MIDI routing via MIDI-OX.
Set Refresh time to 25ms (which I use in my current system)
Loaded test project to Cakewalk by Bandlab, running at 48kHz sample rate.
24Tracks, each track with 9 sends, 10 buses
All 24 tracks loaded with test audio.
Project loads okay
CPU loading is under 5%
Playback is very smooth
Fader response is very smooth

Reduce number of track sends from 9 to 1, CPU loading drops to under 2%

Add 6 Track to Output Monitors, set to ultra, add one MIDI feedback action to each monitor.
Project loads very slowly.
CPU loading rises from 2% to 13%
Fader response very intermittent
Playback is very jumpy

Remove MIDI feedback action for all 6 Track to Output Monitors (but leave the Output Monitors in preset)
CPU loading drops from 13% to 2%
Playback is very smooth
Fader response is very smooth

SUMMARY
Adding even a small number of Track to Output Monitors with feedback events has a dramatic impact on CPU loading and makes the system almost unusable.
The CPU load is only impacted when each Track to Output Monitor has a feedback event
My current 'live' system has a huge number of monitor actions all with MIDI feedback outputs BUT my current 'live' system has no Track to Output monitors.
My current 'live' system uses 24 Track Input Monitors with feedback actions.  These have no big impact on CPU loading
I am running Cakewalk by Bandlab.  I did one test with an old version of Sonar Platinum and the results were similar.
I did one test raising the refresh frequency from 25ms to 75ms.  It made no big difference to results.

So my tests suggest a problem with the Track to Output Monitor action, when it is linked to a MIDI feedback output.  It makes no big difference to CPU loading if I connect AZ to MIDI ports.

I hope this helpful to your analysis of the issue.

Regards
Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: azslow3 on March 10, 2022, 08:34:16 PM
I think the CWP file you are working with can help. It is unclear why MIDI feedback by itself slow down things, till you continuously (try to) send something in parallel.
I mean output monitors are triggered when output is changed, so effectively when nothing is changing simple "Undefined" Action in Feedback should produce the
same impact as any MIDI output. I mean there should be no MIDI output from that monitors till you change some track output.
Or something is broken (in AZ Controller, Cakewalk internal logic, etc.).

In case I somehow manage to reproduce the issue, I can try to find the reason. I hope CWP file and all SPP for all instances you are running with can clarify.
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on March 11, 2022, 11:54:01 AM
Hi Alexey

I made a simple test CWP with 24 tracks and 9 buses.  I could not upload the CWP to this forum so I have uploaded it to WeTransfer and you can download it using the link https://we.tl/t-hPkMVkhHjx

The test CWP was created on the latest version of Cakewalk by Bandlab.
I also made two AZ presets, one with feedback MIDI outputs and one with no MIDI feedback outputs.  These are both attached and run in the first AZ instance.
I tested both presets with Refresh Frequency at 25ms and then at 75ms.  The results were identical.

With the preset 'AZnoFB', the CWP runs smoothly and Cakewalk CPU stays under 1%
Then I switched to the preset 'AZwithFB' and Cakewalk becomes  unresponsive and CPU rises to over 13%.  In fact, if I try to open the test CWP with preset 'AZwithFB' already loaded, then the Cakewalk CWP will not load.

I have attached images of my PC Task Manager showing the Cakewalk CPU readings.

Let me know if you are able to reproduce the problem on your PC.

Regards
Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: azslow3 on March 12, 2022, 01:09:25 PM
Hi Robert,

I have understood from where it comes... Cakewalk is ridiculously slow in returning information about outputs once the number of available hardware outputs increase. So if I have
just 2 (my standard interface) everything is smooth with AZwithFB under 25ms. But once I enable all outputs of my another interface (10), I notice significant performance degradation.
I guess you have more and so in your case the situation is even worse.

That also explain why changing track outputs takes so long in your case.

I propose you report the observation to Cakewalk. I think they can improve the behavior. You can point to this thread (Cakewalk people know what AZ Controller is...).
I can't do much within AZ Controller, I mean I can try to cache names and see if that improve the performance, but that will yet another dirty workaround. And that will not
solve the problem of slow outputs changes.

Alexey.
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on March 12, 2022, 02:42:26 PM
Hi Alexey

Thanks for gaining an understanding of where the problem comes from.

Today I loaded the test CWP and the AZ preset 'AZwithFB'.
When I allow Cakewalk to see just one pair of hardware outputs, then system response is okay although the CPU load is still high at around 7.8%
As I add more hardware outputs, then system response quickly worsens until it becomes unusable.

I use a total of 40 hardware outputs:
24 hardware outputs to feed my 24T tape machine
16 outputs to feed loudspeakers and artist cue feeds

So I will contact Cakewalk to explain the problem and ask if they are willing to investigate.

For me the bigger issue is the impact which the feedback outputs have on the GUI response.
The slow updating of outputs (by external control) is less of an issue as I only update these outputs when I get ready to bounce tracks from Cakewalk to Tape.

I'll let you know what Cakewalk say and yes I will refer them to this thread.

Again thanks for your help on this.
regards
Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on March 16, 2022, 10:20:41 AM
Hi Alexey

I have contacted Cakewalk and sent them a short screen grab video of the issue.
They have asked if we can tell them 'the exact surface API Command ID that is causing the issue'.
Can you help with that information please.

If there are multiple API commands (e.g. for monitoring and for external control of track to output) could you detail whichever commands are relevant please.

Best regards
Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: azslow3 on March 16, 2022, 12:23:44 PM
Hi Robert,

So they want I profile there code. Well, it can take a while.

Which API commands are used is obvious (for me and for them...), which from these commands produce the effect (and why) is harder to find for me then for them.
I guess you are able to understand: there are calls to get output as a number, the number of possible outputs (I always normalize parameters) and
the name of the output. There is nothing else related to outputs. So one of these calls in not optimized in Cakewalk, causing severe performance penalty
when the number of hardware outputs increase (it seems like the number of buses is not influencing the performance). Monitoring in Cakewalk is not
event driven, so if you want check 24 outputs every 25ms I have to ask Cakewalk for current value (+ value range + value text in case of IO) every time,
for each output.

Theoretically I can find problematic calls and try to avoid them, f.e. by caching (output ranges and texts, at least within one cycle), probably that will help
with monitoring.  AZ Controller already has tons of workarounds for Cakewalk bugs and inefficiency of API... But since several years I ask myself the question
"why should I? there is better coded DAW"...

Cheers,
Alexey.
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on March 16, 2022, 04:32:48 PM
Hi Alexey

I think I understand what you are saying.
I will contact Cakewalk again and encourage them to do some diagnostic work on this.
If you are able to give me any more specifics for your own investigations then of course I will send on to Cakewalk.
I'll let you know how progress goes from their side.
Regards
Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on March 28, 2022, 12:52:43 PM
Hi Alexey

Mark McLeod has given me a test custom build of Cakewalk with some caching enabled in the main control surface handler.  The cache is reset whenever routing is changed, or tracks/buses are re-ordered.

The custom build solves the monitoring issue.  Here is a copy of my notes to Mark:

'Test set up:

38 Hardware outputs.

Track/bus to output monitoring of 24 tracks and 9 buses using AZ Controller.

48kHz project with 24 tracks of audio playing, peak CPU is under 4%

I then linked all 24 track faders and fader response remains very smooth.  CPU loading rises to a peak of around 11% when moving all 24 faders, however playback remains smooth with no stuttering.  The GUI interface remains responsive and the waveform display and time cursor update smoothly.

Then I added an instance of LP EQ to 12 of the tracks.  When playing and moving all 24 track faders, peak CPU loading rose to 15%, however playback remains smooth with no stuttering.  The GUI interface remains responsive and the waveform display and time cursor update smoothly.

Updating of track or bus to output is slow with the AZ controller active.  I did a test using the external controller to update all 24 track to output routing using 24 consecutive MIDI messages.  This took 58 seconds.  This is pretty well what I measured using the GUI to update a single track to output routing which took between 2 and 3 seconds between selecting the new output to seeing the new output label appear on the GUI.  Removing the AZ Controller app reduced the track/bus to output update time to around 0.7seconds on my test project.  I have included these timings for information only.  It does not really impact on a session.'

So hopefully this will get incorporated into the next Cakewalk release and I'll be able to greatly simplify my current Cakewalk to Hardware Mixer track to routing interface.

Thanks for your help in getting this done.

Regards

Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: azslow3 on March 28, 2022, 09:06:24 PM
I have really hoped they will address the issue... I have checked what I could theoretically cache without consequences and all approached can have questionable side effects. To assume the list is the same for several tracks means I am sure track types are the same. But from what I remember in some situation that is just a guess. IO assignments when dealing from Surface API are traditionally buggy in Cakewalk (and sometimes not only with API... remember changing sends output...).
AZ Controller, GUI and I guess re-routing are all working in the same thread. If some operation takes a tick more than it should be, other parts are affected. I think you have tested that AZ Controller dealing with thousands of monitors produce no significant impact on performance. And the code for IO parameters is just 2 API calls longer then for other parameters (since other parameters are normalized on Cakewalk side), so 100-200 IO monitors can not be a problem in AZ Controller side.
Also as you could check in case the number of hardware outputs is small everything worked well. And from all logical thoughts there should be zero performance difference, but we have observed opposite.
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on April 04, 2022, 02:35:13 PM
Hi Alexey

I have spent some time testing the custom build of Cakewalk with added caching in the main control surface handler.
It works well except for one unforeseen issue which I have discussed with Mark McLeod, as follows:

The track Record Arm function is part of what is called the topology graph, which is basically the audio routing graph.  So pressing a track Record Arm button causes the cache to be reset.

The effect is a very slow reaction time from pressing a track Record Arm button to that button being fully illuminated.  (about 1 second).  The impact on the global 'Arm (UnArm) all tracks' button/indicator is even worse (about 2 seconds).  It is not a disaster but you end up double checking to see if a record arm action has actually happened.

I tried, temporarily, removing the track to output feedback actions in my AZ preset and the Record Arm response is fast again, so it is the requirement to monitor track to output status which slows down the cache refresh.  I also tried changing the controller refresh frequency from 25ms to 75ms.  It does not improve the cache refresh time.

There is nothing that Mark can do to fix this problem as the cache is across the whole audio routing graph.

The issue does not impact on any other important track functions, e.g. SOLO, CUT, Fader, Pan.  These are not part of the audio routing graph.

But I wanted to ask if there was any way we might deal with this issue in the AZ app?  Let me explain my small hope!

I use two instances of AZ Controller with Cakewalk, each with one preset.

'REPLAY' I use when I am mixing, so it includes the track to output monitoring.  It does not include the track Record Arm buttons.

'RECORD' I use when recording.  It does include the Record Arm buttons but does not include any requirement to monitor track to output routing status.

I do switch between these two operational modes often (using two screen based buttons), so I want to keep both instances enabled.

So, is there anyway to either:

1. Make the monitoring of track/bus to output feedback actions conditional on another control state (so I mean a supplied MIDI code).  It would have to be a way which temporarily removes the feedback actions from being 'seen' by Cakewalk.

2. Within one instance of AZ Controller, switch off the preset 'on the fly' so I mean again by the application of an incoming MIDI command.

3. Turn on and off an instance of AZ Controller 'on the fly' (same idea as point 2).

Any of these methods would provide a way to switch off the track/bus to output monitoring when it is not required,  This in turn would speed up the Record ARM button response.

Or any other ideas to fix the issue?

Best regards

Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: azslow3 on April 04, 2022, 05:19:12 PM
Hi Robert,

I don't really understand which slow operation Cakewalk does with hardware outputs to get the information nor why that information can't be cached permanently (till hardware configuration is changed)... But well, Mark probably can't change that.

AZ Controller in general is constructed to work with one preset for one (or many) device(s). Several instances for one device make no sense.
Monitoring execute all actions before and including Monitor, so it will not call Cakewalk to check changes in case you prevent it with conditions.
I mean you can define Software Set "Mode" with "REPLAY" and "RECORD" states and put "Mode: REPLAY" condition to output monitors. Or put
"Mode: RECORD - Undefined * (final)" before Monitor Action.

My own presets always react automatically on mode changes, so toggling the mode (in the Overview Tab or with Set State from a button) always
does complete trick. But that requires careful monitoring setup, in other words all actions required when the mode is changed should be
in monitors which are (directly or indirectly) triggered by the switch.


 
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on April 04, 2022, 06:09:30 PM
Hi Alexey

I also don't understand why the hardware outputs cause such a big issue and I agree that semi-permanent caching would seem sensible ... but I guess we need to work with what we have,

I am going to experiment with a Software set as you suggested.  Right now I have one question:

Should I make the Logic Control (where the Parameter Value Monitor action is listed) conditional on the Software state?
OR
Should I make Feedback MIDI action conditional on the Software state?

Remember I need to 'hide' the track to output monitor 'asks' from Cakewalk so it does not overload the system when I am in record mode of working.

Regards
Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: azslow3 on April 04, 2022, 07:40:16 PM
To avoid checks, put condition on Monitor Action in the Logic list. Feedback is executed after check, so conditions will not work (for the purpose) there (in some of your tests
you had no Actions in feedback list... in this case AZ Controller logically "think" there is no reason to monitor... but once you have Actions there, even in case they will not be executed because of conditions, Monitor is still checked).
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on April 05, 2022, 05:42:35 PM
Hi Alexey

Using a Software state as an action condition on the Monitor Actions worked!

So now when I switch to RECORD mode of operation, then the Software State MODE gets changed to RECORD and all of the track to output monitor actions are disabled.
And the Record Arm button response times all work at normal fast speed.

Then when I switch to REPLAY mode of operation, I don't need to use the Record Arm functions, so the Software State gets changed to REPLAY and all of the track to output monitor actions are enabled.

Thanks very much for all your advice around this track to output monitoring saga.

Best regards
Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: azslow3 on April 05, 2022, 08:36:02 PM
Hi Robert,
I know you have fun with Cakewalk. But I have recently looked at some DAW <-> Mixer solution: https://sites.google.com/site/patrickmaillot/x32#h.p_DUz7OP4Wmg0-
May be you can get some information/inspiration from that project (it is open source, so you can check how that is done).
And while AZ Controller still works with Cakewalk only, I have sufficient experience with coding for REAPER to say... REAPER is way better prepared for such projects (strips are more flexible and so can represent almost any logic), it is better coded and Justin put all possible effort to fix any kind of bugs (f.e. such slowness will definitively be considered as a bug). REAPER is almost one man project, but there is one other person who support some parts (John) and it is possible to communicate with both of them.
Also REAPER works on RPI, so good for "embedding" (mixer + RPI + monitor).
Cakewalk's original author has left Cakewalk more then 10 years ago. They had and still have good people, but from strait use observations it is clear that most modifications are like "hacks" on top of each other, instead of rework of core engine. I am surprised they still can keep the thing stable, it must be titanic job changing anything without breaking something else (in practice control surface API is usually a part of that "something else"...).
Title: Re: Accessing the Cakewalk Track Output router
Post by: norfolkmastering on April 06, 2022, 10:53:32 AM
Hi Alexey

I had a look at the X32Reaper interface and it looks to have a lot of possibilities.  I also had a look through a few of the Reaper intro videos to get a feel for Reaper as a control surface.  The GUI seems to be more flexible in layout than Cakewalk (I love the ability to make the faders longer!) but not as developed from a visual perspective.  It also had MTC generation so it should be easy enough to get my tape machine to slave to it.

At some point I will have a look to see if I can get some basic control working between Reaper and my mixer ... would be fine to try!

Best regards
Robert
Title: Re: Accessing the Cakewalk Track Output router
Post by: azslow3 on April 06, 2022, 09:10:59 PM
Visually REAPER by default is... let say "conservative"  ;)
But it is quite tunable. Not only each strip can have different representation (quickly changeable), it can look f.e. like: http://www.houseofwhitetie.com/reaper/imperial/wt_imperial.html