Alternative method of track to group routing

[norfolkmastering]

Hi Alexey

I am making good progress with the test preset.
Putting aside the SysEx blocking function for a moment please, I have another question:

I have a need to ensure that three types of feedback are sent out from AZ in a particular order.
I am trying to use priorities to do this but the results are not consistent.
Let me explain.

With my set up, when a buss name is changed in Cakewalk, it triggers the following feedback outputs:
The new buss name (SysEx)
The buss output name (SysEx)
All of the CCs associated with that bus (input, gain, fader setting etc)

I want the new buss name always to be output first, so I tried setting it to priority 0
The bus output name to be output next, so I set this it to priority 1
Those two work okay.

Next I set the CCs to priority 2
However the CCs are always output first (before the two SysEx messages)
I tried CCs set to priority 10.
Same result, the CCs are always output first.

Do you have an idea why the priorities are not being followed?

It is possible that changing a bus name changes (what Mark McLeod called) 'the audio routing graph' which is part of the cache they introduced to solve an earlier issue I had.  Could this be delaying the sending of the bus name into AZ?

regards
Robert

[azslow3]

If you use my example with delayed SysEx, but do not delay CCs, you will get CCs "in time" but names delays.

Note that changes in bus names are "detected" by AZ Controller logic, it is slowly checking own cache of strip names. Doing that for all strips every cycle is not going to work.
Monitored outputs are checked in each cycle (assuming Ultra monitor), using Cakewalk caching logic.
Most other parameters are checked "in time". Monitors for them are triggered if the value is changed or AZ Controller think the strip (container) is changed.

I suggest you modify your code to accept information in arbitrary order. It is in general not possible to predict when some parameter is updated, when this update is detectable
and when it is detected. DAWs by definition have many parallel operations which at some point are synchronized, but at any particular moment the disposition is not deterministic (like a live of Schrödinger's cat).
Audio is processed in "blocks", all parameters are fixed before the block processing starts and can't be changed till the next block. That processing happens in a separate thread(s). Controlling part accept changes at any time, in parallel to that processing. The changes can be from many places, including GUI and Control Surfaces. I repeat, all that is working in parallel (even physically, modern computers have more then one thread running). DAW has to "orchestrate"  the process, broadcasting changes from one logical part to another, but it is impossible to know in which order all that happens.

One thing is a fact, nothing happens instantly. F.e. when you "Mute" a Track, if you ask Cakewalk immediately what is current status, it returns "Note muted". So it is a bad idea to do "ask to mute, check the result, update LED" in one go. For some sequences (f.e. stop->undo->record) I had to use really huge delays (a second or more), the result was "unpredictable" otherwise.
So just accepting everything in arbitrary order is the only safe option.
 

[norfolkmastering]

Hi Alexey

Thanks for your advice that I need to programme assuming everything is in arbitrary order.  I will move forward with this rule in mind.
The reason I started to look at using priorities is a problem with track output bus names.  Let me explain with a simple example.

Bus 1 is named 'Test1'
One or more tracks is already routed to Bus 1 'Test1'
I change the Bus 1 name to 'Test2'
The tracks already routed to Bus 1 correctly change the Cakewalk screen labels to 'Test 2' but the track output names reported from AZ are remaining at 'Test1'
This is causing big problems with my PIC and mixer routing system.

Are you able to check for this problem please and let me know if you understand what is happening.

Regards
Robert

[azslow3]

He Robert,

Well.. that is definitively logical bug in AZ Controller
Please give me some days to fix.

Cheers,
Alexey.
PS. As you know, getting names of strips and outputs in Cakewalk is "slow". For strips I slowly "scanning" all strips for changes. But for outputs not.
I have to revise the code. I remember there was some "tricks" because brute force way was suffering from the slowness, and probably I have "optimized" it too far.
In that particular case the "value" of output is not changed (it is numerically the same), the name check is optimized away and so it does not work.
Reverting to check the name all the time is not a good option, I have to implement something else (f.e. trigger output name check if some bus name is changed,
that will not trigger on hardware outputs changes and side-chain changes, but the last is buggy in any case and the second is not usual operation...).
 

[norfolkmastering]

Hi Alexey

Thanks for your explanation and a really big thank you for agreeing to look for a fix.

The move to use labels for routing of my audio mixer is a big change but it offers some great advantages for my sessions.  I will be able to:

-  Change bus names (the old routing system required fixed bus names).
-  Rearrange the order of buses in Cakewalk and my audio mixer will follow those positional changes.
-  Be able to work with any number of buses up to eight 'sub' buses and one 'master' bus.  (the old routing system required always eight 'sub' buses and one 'master' bus)

So a lot more flexibility.

Thanks for helping me so much with this.

Regards
Robert

[norfolkmastering]

Hi Alexey

From your message below, could I clarify two points please:

1. You said, Text = "" is a correction for original example.
The test preset shows Text = "
Which is correct?

2. Is this additional Text entry required for 'Parameter value' feedback or just for 'Parameter name' feedback?

Regards
Robert

Hi Robert,

I attach proposed approach. Note that "Timer Once" should have priority 0, so higher then any SysEx sending Monitor. It will work also with other priority,
but can introduce more then one cycle delay (depends in which order monitors are checked).

You may ask why I explicitly reset Timer Monitor, instead of just defining State Monitor which should be triggered automatically.
The answer is... it seems like State Monitor has a bug under such use

So, before sending any SysEx I forbid triggering any other SysEx sending Monitor and schedule resetting of that block for the next cycle.

Text = "" is a correction for original example. If there is no specific Bus/Track, Text = "Parameter name" has no effect, effectively the last value of Text
will be preserved. That can produce "side effects" in complex presets (sending SysEx with wrong name for not existing output/bus).

"Display" actions are just for me, to test how all that works (since I don't have PIC to receive SysExes). Initially I put "~1sec" in Reset options to test, and Display
was updating one by one. With "at next" it should update with cycle delay.

Cheers,
Alexey.

[azslow3]

Text="" is a good idea before any Text = Name/Value. That ensure in case corresponding parameter/strip is not there, the Text sent will be "". It will stay whatever it
was otherwise (by default also "", but if you change is as "engine state" anywhere, the value will be persistent... and such logical mistake is very hard to catch).

[norfolkmastering]

Hi Alexey

I have spent some time optimising my PIC software to work with name based routing and the system seems to work reliably but with two issues to discuss.

1. You are already aware of the issue where sometimes the track output name is reported as the 'old' name after a bus has been renamed to a 'new' name.  I am testing this by routing all 24 tracks to the same bus and then renaming that bus.  The PIC software detects if the track output name does not match any current bus name.  If this is detected then the PIC instructs (via AZ) that track to route to 'none'.  Through repeated testing, I am finding that at least one track output name is reported incorrectly in about 20% of the tests.

2. The other issue relates to the time gap between the first and second 'name' SysEx messages when I rearrange the order of the buses in Cakewalk.  With the refresh rate set to 25ms (which I need for fader response time), the gap between first and second SysEx messages is sometimes well below the minimum expected time of 25ms.  I have measured as low as 9ms.  The gap between the second and all subsequent SysEx messages is always above 25ms.

So I did some experimentation and found that the issue is connected to the number of hardware outputs in the project.
I have 24 hardware outputs (D/A) in my standard project set up.  These are configured as one send per track.

If I remove 12 of these hardware outputs from the project, then the minimum time gap between first and second 'name' SysEx messages is 20ms.  If I remove all 24 hardware outputs from the project, then the minimum gap between first and second SysEx messages is always above 25ms.

There is a software state and logic to set (from the PIC) which hardware output is routed from which track.  I have tried removing this software state and all the logic but it does not fix the issue.

Even though the PIC is running in C, the computation required to recover and then compare string variable names within the PIC is quite slow.  I am concerned that a 9ms gap between the start of the first message and the start of the second message (which means only a few ms between the PIC receiving the final byte of the first message and the arrival of the first byte of the second SysEx) risks overflow of the (2 byte) USART RX buffer and so the second message is lost.

Could you please consider why this might be happening and whether there is any possible solution?

Regards
Robert

[azslow3]

I have never measured how Cakewalk calculate cycles, so is 25ms means between last cycle is finished and new started or just between beginning of cycles. The later case can explain your observation, if cycle processing takes 16ms with big number of tracks, and we know Cakewalk is slow reporting sends and outputs names, it can happened the first SysEx is triggered at the end of processing, so just before 9ms of the next cycle.
There is no feel of "real time" in AZ Controller at the moment... but we can organize "one SysEx per 2 cycles". That will give "normal" speed 50ms, x24 = 1.2sec to update 24 tracks. But I guess that is acceptable in case of such change in the project.

[norfolkmastering]

Hi Alexey

I agree that one SysEx per 2 cycles would be worth trying.  Is it an easy change to the preset?

Regards
Robert

[norfolkmastering]

I have never measured how Cakewalk calculate cycles, so is 25ms means between last cycle is finished and new started or just between beginning of cycles. The later case can explain your observation, if cycle processing takes 16ms with big number of tracks, and we know Cakewalk is slow reporting sends and outputs names, it can happened the first SysEx is triggered at the end of processing, so just before 9ms of the next cycle.
There is no feel of "real time" in AZ Controller at the moment... but we can organize "one SysEx per 2 cycles". That will give "normal" speed 50ms, x24 = 1.2sec to update 24 tracks. But I guess that is acceptable in case of such change in the project.

Hi Alexey
I changed the Monitor reset for the 'CanSendSysEx : Timer' to trigger in 2 cycles and the minimum time between first and second SysEx messages is now 31ms which is fine.  Thanks for your advice to fix that issue.

Please let me know when you have a solution for the reporting of Track Outputs Names as this is still causing routing failures on a regular basis.
Best regards
Robert

[norfolkmastering]

Hi Alexey

A further update on my progress.

I experimented and found a mechanism to trigger a Cakewalk refresh of the track output names.

I found that any instruction received by Cakewalk to set a routing (even if the instruction sets it to its existing destination) causes Cakewalk to refresh the track output names, so I mean the names which sit in the AZ 'buffer'.  So this means that the 'old' track output names are updated with the correct 'new' names.

I added a routine in my PIC which detects when a bus name is changed.  When this is detected, I send a MIDI instruction from the PIC to AZ/Cakewalk to set my Track 1 Send 1 to its already set destination (which is always set to the hardware output 'DAW1'.  This causes no change to my setup but the track output names issue is resolved.

I will continue testing this work-a-round but so far it is 100% reliable.

Of course it would be great to get the issue fixed the correct way but I wanted to let you know I found this way around the issue.

Best regards

Robert

[azslow3]

Hi Robert,

Just to keep you informed. I have not forgotten about the problem. It just happened to be more severe then I have thought at the beginning, so I have to rewrite
the whole sends/output monitoring logic and that takes time.

I must admit current implementation is logically buggy at many places. It is "too fast", I mean it works faster then proper implementation. But can return wrong names/values
for sends, project dependent.

In addition, Cakewalk informs surfaces "there was topology change" in case of most related operations, including changing any send. But it does not do this in case
the output is changed (at least not in Sonar X2). While with strip names I can understand that, at the end changing in names do not change topology, changing output has
the same effect on topology as changing send output. I am going to check later with CbB, and if that is still the case there I probably write feature request.
I mean I can cache sends values but can't cache outputs values, and as we know for some reason I don't understand (even after some explanation from Cakewalk...) that
API calls are "pricey".

[norfolkmastering]

Hi Alexey

Thanks very much for keeping me informed.  I understand your explanation.
For now, the work-a-round which I have implemented is working 100% reliably so I have a functioning system.

To keep you up to date with my PIC programming progress:

My hardware mixer has 24 channels (tracks) which mix down to eight stereo buses (which I call 'groups'), and these groups mix down to one Master bus.

With the name based routing system, I always have one bus in Cakewalk named 'Master'.
The other eight buses can be freely named.

Today, I finished a PIC update which allows the 'Master' bus (in Cakewalk) to be placed in any slot position within the nine buses.
This was an interesting programming challenge because in the hardware mixer, the Master bus pcb is special and, of course, its output is physically connected to the monitor loudspeakers and master stereo tape machine.  So far my update is working but lots of testing still to do.

Next step is to allow the number of 'groups' to be varied from one to eight.  This means I will be able to add extra groups as I need them during a project, name them as I like to, and the hardware mixer will continue to be mapped correctly from Cakewalk/AZ.  So now I will be implementing how to deal with AZ SysEx name outputs for buses which don't exist within the Cakewalk project.  You already gave me very good advice on how to deal with this using the Feedback action 'Text =''.

As always, I could do none of this without AZ Controller and your continued support.  A very  big thank you from me to you!

Best regards
Robert

[azslow3]

Hi Robert,

I have uploaded "pre-release" as test version (https://www.azslow.com/index.php?action=downloads;sa=view;down=28)

Everything related to Input/Output(Strip and Send) is reworked. I try to cache what is possible (according to Sonar X2 behavior, I have not really tested with CbB).
But not cache if that can break things.

So note that "Output" (unlike "Send output") is asking Cakewalk all the time. It make sense to assign "Normal" speed for corresponding monitors, if you notice Cakewalk
is laggy again.

Strip (including buses) name changes are noticed "slowly" (can take seconds), independent from monitors speed (that is done is slowly continuous looping over all strips).
Other changes should be spotted almost instantly. As I wrote, Cakewalk informs about most structural changes (except Input/Output), with a kind of flag "something in structure
is changed". For Input/Output I just follow preset specified speed, it seems like the operation is not extremely heavy. But getting names (for everything) is heavy. So I traverse
strips slowly. Names for Inputs/Outputs are stay the same as long the value is not changed, there is no structural changes and no renames. So I can cache them till one of these
conditions is triggered.

Please let me know if something is broken or odd. The modification is deep, I could easily break something.