AZSlow

Hi Alexey

I am working to best stabilise my Cakewalk/AZ/PIC system as much as possible.

My current focus is trying to improve the reliability of the control system when I load a Cakewalk Sonar project and then (via a screen based button) bring one of my two PIC based controllers 'on-line', so to speak.

There has been a long standing issue that the PIC does not always communicate with AZ Controller at the first attempt.  The work-a-round has been to load that PICs preset from the AZ Controller 'Presets' drop down.  I only use one preset per AZ instance but loading the preset seems to unblock the communication problem.

Can you tell me please whether the AZ preset is automatically loaded, and if 'Yes' then at what point in the sequence of opening Cakewalk and then a Cakewalk project, does this happen?

I am wondering if my PIC is not seeing the automatic preset load and maybe this is the cause of the issue.  All MIDI messages from AZ to my PICs have to go via MIDI-OX due to the requirement to combine MIDI CC/SysEx and MTC to send to the PIC.

Regards
Robert

It is up to Cakewalk, it controls preset loading. It is better not assume any particular sequence/timing. Also project can be opened/closed/switched at any time.

There is "Context" Set with "NoProject" possible state. It can be used to detect some project is loaded.

Note AZ Controller can't detect MIDI Out is "working". So I propose you introduce some "handshake", as surfaces normally do (original surfaces was using MIDI hardware connection, so they couldn't detect other end is present/powered/ready). Both ends should initiate in "Offline" state and one side (normally computer) periodically send "Hello" to other end.

Also you can keep silence from AZ Controller as long as "Context" is "NoProject".

Hi Alexey

Thanks for the explanation.

I do want to introduce some handshaking.

Within my PICs I can add any necessary code but could you give me some advice about the sequence of initiation please.

Specifically:
1. How do I hold Cakewalk/AZ Controller in "Offine" state with regard to external controllers?


2. When you say that one side (normally computer) initiates the "Hello", do you mean Cakewalk/AZ or do you mean the external controller (the PICs in my case)?

3. Could you tell me where I can find the "Context" Set with "NoProject" possible state.  Is this within AZ Controller?  And how might this help with the start-up scheme?

Regards
Robert

You can say "Hello" from both sides, but normally that is done from the host (cakewalk).
Even more, you can send "Hello" only when project is loaded (in "Offline" state)

An example is in Mackie preset, but if you send your current preset I can try to put corresponding code there.

'Device' Set ('Offline' (default), 'Online')

"Hello" control, assigned to response from PIC:
  'Context':'NoProject' Undefined * (final)
  'Device':'Offline' Timer (Slow)
      In timer: send "Hello"
   Set state 'Device' = 'Online'
  <do whatever you want initially, normally triggering something, in 'good style cases' just resetting all
  monitor>

Note that any sending to PIC is better protect with "Device":"Online" condition, so AZ Controller will not send anything till handshake happens.  In Mackie I do this by "Device":"Online" conditions to all monitors which send things (and I send from monitors only).

How that works:
 * if there is no project, nothing will happened
 * if there is project and we are 'Offline' (initially, but can be set later), AZ Controller will periodically send "Hello"
 * in case AZ Controller receives reply and project is loaded, it change the state to 'Online'.

Note in Mackie preset I don't care if project is open.
 

Hi Alexey

I have read the notes from your last post and the scheme should work well with my PICs.

I asked Mark McLeod to provide a summary of the handshake protocol from a Sonar perspective.  You probably know all this already but I thought it would be useful to copy his email text to you.  My only point of concern was his first point which says:

'Sonar calls the Connect method on the DLL, and the DLL should return almost immediately with a success code.'  My question is whether my PIC would have to respond immediately (not preferred) or whether AZ Controller can take care of the response (preferred).

I will send a separate post with a copy of one of my existing presets and some additional information from my side.

Here follows is Mark's reply to me:

'Hi Robert,

Sonar calls the Connect method on the DLL, and the DLL should return almost immediately with a success code. 

If you've got some sort of a handshake, the connect method typically sends a short MIDI message to your device, but should always return immediately - otherwise Sonar will wait for the Connect method. to complete. You definitely don't want this.

Your device may then respond, but this should be handled by the control surface DLL as it would handle any message coming in on MIDI IN.  The connect method should never wait for a reply.

Typically handshakes are SYSEX messages, but they could be any MIDI message you're not using for anything else (e.g. a particular CC message).

In practice, there's normally no need for a traditional handshake message where you send something out and wait for a response. It's actually strongly discouraged. Mackie used to have one, but have since removed this requirement - you'll see that there's a "disable handshake" option in the Mackie Control dialog (which was actually implemented by Alexey!), as most modern Mackie compatible devices don't bother with a handshake.

The Connect method is really only there to validate the software part of the equation - i.e. has it loaded all its required libraries and is working. If you've got more than one of the same control surface running, it'll typically assign a unique device ID at this point. You definitely want to avoid any hardware validation or two-way handshaking in this method, save a simple one-way wake-up message sent to your control surface hardware.

You'll need to check with Alexey as to what AZController actually does in its connect method. I'd be surprised however, if it does anything but just return immediately - with the possible exception of sending out some user-defined MIDI message.

Control surface DLL's should be completely reactive. They should respond to MIDI messages they receive from the hardware, but never expect them as part of a "conversation".

A better alternative to a traditional "request / wait for a reply" handshake, is a simple wake-up message.  In other words, dedicate some MIDI CC to mean "system started".  It could work as follows:

1. The connect method sends a "system started" CC to your hardware. It returns immediately and doesn't care about any reply.
2. Whenever your hardware receives a "system started" CC,  it resets its internal state and sends a "system started" message back to the control surface DLL via MIDI.
3. Whenever the control surface DLL receives a "system started" CC, it resets its internal state, requests the state of everything from Sonar, and sends the usual CC's out to the control surface for fader positions etc.
4. When the control surface hardware is first switched on, it sends out a "system started" CC to the control surface DLL via MIDI.

Using this method, it will cover the following scenarios:

1. Hardware is on, and you then start Sonar:   When the hardware started it up, it sent a "system started" CC.  There was nothing running to receive it, so it was lost in the ether.
When Sonar is started, the control surface DLL sends out its startup CC, the hardware responds with a startup CC, and the control surface DLL responds by requesting Sonar's state and updates all of your faders etc.

2. Hardware is off, you start Sonar, then turn on the hardware:  The control surface DLL sends out its startup CC - there's nothing to receive it, but as the connect method returns immediately, Sonar just carries on.  When you turn on the hardware, it sends a "system started" CC to the control surface DLL via MIDI, the control surface DLL responds by requesting Sonar's state and updates all of your faders etc.

3. You reboot your hardware while Sonar is running: This basically follows scenario 1, but then rebooting your hardware triggers scenario 2.

This should cover all scenarios.

A word of warning - it's fine for your hardware to respond to the Control Surface DLL with a "system started" message when your hardware receives one... but don't do it the other way around.  In other words, the Control Surface DLL should only ever send a "system started" message in the connect method, and never in response to anything else.  Otherwise they'll be sending "system started" messages back and forth forever!

I hope this helps,

Mark.'

Regards
Robert

Hi Alexey

My second post today about the proposed Sonar/AZ/PIC hand shake protocol.

I have attached the REPLAY preset which is used with one of the two instances of AZ Controller I run on Sonar.  This communicates with the 'REPLAY' PIC in my audio mixer.  It is the 8th instance in the list of AZ instances.

The other AZ preset is called RECORD and is used with the other instance of AZ controller.  This RECORD instance communicates with the 'RECORD' PIC in my audio mixer.

My audio mixer can be configured to use the same hardware for two modes of operation:

REPLAY is for mixdown from 24 tracks to 8 groups to 1 master bus.

RECORD is for recording up to 12 sources simultaneously to any of 24 recording tracks (Sonar and Tape Machines)

The audio mixer can only be in one of the two modes at any time.
Selection between the two modes is done with two 'on-screen' buttons which are software interlocked and send MIDI CC messages to both PICs (you can see the buttons in the right side of the Sonar GUI screen image attached).  The buttons were created using the Chameleon custom MIDI controller interface software.

There are three possible states:
1. REPLAY off, RECORD off
2. REPLAY on, RECORD off
3. REPLAY off, RECORD on

I suggest we concentrate on the REPLAY AZ preset and REPLAY PIC to begin with.
Here is my suggested process (based on your notes):

1. PC and PIC are powered up.
2. Sonar is loaded.
3. Default state (no project loaded) : AZ sends out no MIDI messages and does not respond to incoming MIDI messages.
4. Project is loaded : AZ sends out "Hello" message at regular interval and can now respond to incoming MIDI messages.  "Hello" message can be MIDI CH15, CC6, Value 127.
5. Screen GUI 'REPLAY' button is pressed : CC message sent to REPLAY PIC.
6. REPLAY PIC waits for next "Hello" message from AZ, and when received, sends MIDI CH16, CC28, Value 127 message to AZ REPLAY instance.  This triggers a selective Reset action and is the start of the REPLAY mode start-up sequence.

When the REPLAY mode is deselected (either deselect the REPLAY soft button or select the RECORD soft button), there is a PIC procedure to make the REPLAY mode inactive.  At present, this does two things:

1. It closes the REPLAY PIC access to the audio mixer I2C control bus.
2. The REPLAY PIC reads but does not action MIDI messages (CC and SysEx) received from AZ REPLAY instance preset.

An additional option (when REPLAY button is deselected) would be to put the AZ REPLAY instance into 'offline' state but restart the "Hello" message again at regular interval.  This would mean no other MIDI messages from the AZ REPLAY instance to the REPLAY PIC until the REPLAY PIC responded with the start-up MIDI CH16, CC28, Value 127 message to AZ.  I  could send a MIDI message to the AZ REPLAY instance to flag this.  CH15, CC7 (Value not important) would be possible.

It would also make sense, that if a Sonar project project was closed, then the AZ REPLAY instance would go into 'offline' state but if possible, after sending one 'close down' message to the REPLAY PIC.  This could be MIDI CH15, CC6, Value 0.

Let me know please if this suggested scheme makes sense.  I will wait until you have considered the scheme before making any changes to the REPLAY PIC code.

Regards
Robert

Sorry, I am a bit busy at the moment to dive into that.

Theoretically I can check if I can add some logic to "Connect". But I don't think that is required in our scenario, if AZ Controller can't send "Hello", PIC will not receive it and so everything stay "Offline".
Also I don't remember cases when AZ Controller was already sending something useful, while Cakewalk MIDI was not ready yet. I guess they prepare assigned to surface MIDI port before loading surface (sure, some "hot-plug" logic was foreseen, but Sonar was buggy with that since decades and I am not sure all these problems are solved in new Sonar). At least at some continuous time, when assigned MIDI port was not available, surface was not loaded at all.

Handshakes are good and useful for hardware, almost "a must" for complicated devices. Yes, "disable handshake" has allowed to use some simple "Mackie (in)compatible" devices with corresponding plug-in, but that doesn't mean that is good. F.e. NI controllers require quite initialization from "deep integration" solutions, the DAW has to inform the device it is "aware" and that is not possible without handshakes.

Hi Alexey

No problem.  When you have some time to have a look at my proposed scheme, let me know and we can hopefully sort out a practical solution.

I am sure that Sonar is sending some MIDI messages (in my case via AZ Controller) during the Sonar load period (so I mean before a project is loaded).  I don't know whether AZ Controller is active sufficiently early to prevent these messages?  Hopefully it will be.

Regards
Robert

Just to say I was able to set up the 'Device' Set ('Offline' (default), 'Online').

And I was able to set up Logic, Action, Monitor, Timer, Slow, to output a 'Hello' message (every 300ms) which is great.

The only line from your previous notes which I don't understand is:
 'Context':'NoProject' Undefined * (final)

I can see there is a system defined Software State called 'Context' but I do not understand how you planned to use it within the handshaking protocol?

Regards
Robert

Hi Alexey

Now I understood your line:
'Context':'NoProject' Undefined * (final)

Sorry to be so slow in my understanding!

Now I have working handshaking exactly as you described, that is:
 * if there is no project, nothing will happen
 * if there is project and we are 'Offline' (initially, but can be set later), AZ Controller will periodically send "Hello"
 * in case AZ Controller receives reply and project is loaded, it change the state to 'Online'.

When you have time, could you explain Mark's comment:

'Sonar calls the Connect method on the DLL, and the DLL should return almost immediately with a success code.

If you've got some sort of a handshake, the connect method typically sends a short MIDI message to your device, but should always return immediately - otherwise Sonar will wait for the Connect method. to complete. You definitely don't want this.

Your device may then respond, but this should be handled by the control surface DLL as it would handle any message coming in on MIDI IN.  The connect method should never wait for a reply.'

So my question from Mark's comment is:
Does AZ controller provide the immediate response to Sonar's call (Connect method) on the DLL?
Or is this something I have to do from my PIC?

Best regards
Robert

I have checked.

Connect/Disconnect methods of surface DLL are not related to "surface connection". It is to make a "connection" between Sonar and DLL, in other words "Connect" means "Hey, Sonar is ready for communication". And "Disconnect" means "Please no longer communicate with me".

So, it is about "programming interfaces" communication. In case you know a bit Windows programming, surface DLL is COM. In this method all surface DLLs (including AZ Controller) get pointers to "Sonar interfaces", and finally can do some initialization (access information from Sonar).

So you should not do anything in PIC for that.

If you want send "Disconnect" to PIC, you can monitor Context and check for "NoProject" state. AZ Controller does "final monitor loop" in Disconnect method, to be sure you have a change to send "Goodby" if required.
Obviously you can't expect there will be a chance to receive reply in this case.
Long time ago I have decided there is no reason to process "AZ Controller is removed from surfaces list", "Sonar is closed", "Project is closed" separately. So all that just trigger "NoProject" in Context (the project is always closed when Sonar is closing, if AZ Controller is removed it will be unable to send anything, without loaded project AZ Controller can't do much, if I remember correctly even surface idle is not called).

Hi Alexey

I have the handshaking working with my two PICs and two instances of AZ Controller (RECORD and REPLAY).

I have one problem.

When I load Sonar, but before loading a project, the two instances of AZ Controller are active.  This is good!

Then I load a project and I send 'Hello' message to PIC.  I send just one message rather than using slow timer which just clutters up PIC input.

'Hello' message
Logic actions:
'Context:NoProject' - Undefined*
'Device Offline' - Reset Hello: : Parameter Value Monitor
'Device:Offline' - Parameter Value Monitor

Feeback
-Ch:15 CC9 Value(14bit): 380

I respond from the PIC to set 'Device:Online'
All is good!

Here is the issue:
I close the project
I use 'Device Online' - State Monitor to send shut down message to both PICs.
The two PICs are put in standby mode.
The two instances of AZ Controller become inactive.

However, when I load the next project (Sonar is still open), there is no trigger of the 'Hello' message, even though the AZ Controller instances become active again.

Can you help with this please.

Regards
Robert

"Parameter Value Monitor" need parameter... If it is not defined in the same logic control, it takes whatever was set before. That can produce random results.
So I guess you have not listed your real "Hello" actions.

If you want send just one time, use "Context" Set Monitor. But it included several States which can be switched while project is open, you you need extra set,
let say "Project" with "Opened"  and "Closed". Then:
In "Context" monitor:
Context:NoProject Project:Opened -> Set Project Closed (set engine state, final)
Context:NoProject -> Undefined (final)
# for all other states
Project:Closed -> Set Project Opened (set engine state, final)

Then you need "Project" set monitor, but it will only trigger when project is opened or closed, so you can:
Project:Opened -> ... send hello to pic ...
Project:Closed -> ... send goodby to pic ...
# or  Project:Closed Device:Online -> ...send goodby... with "race case" protection in Hello reply processing (when Project:Closed)
Project:Closed -> Set Device Offline

It may sense to give "Context" monitor highest priority and "Project" monitor next to highest, so they are executed in correct order in the same cycle and before any "Value" monitors.

Hi Alexey
Thanks for the suggestions.  I'll do some updates and let you know how it goes.
I'm off travelling for a week so it will have to be when I get back.
Best regards
Robert

Hi again Alexey
As I'll have some thinking time when I'm travelling, it would be really helpful if you could clarify some points about the functioning of the Software Set 'Context'.

1)
If I set up a logic action: 'Monitor state', 'Context'
Then some kind of Feedback event.
Will the feedback event be triggered by any change of the 'Context' state?
To ask the question another way:
If I load Sonar (but not yet a project), is there would this trigger a change of 'Context' 'Monitor State'?
If I then load a project, does this trigger a further change of 'Context' 'Monitor State'?

2)
When I set an action condition using 'Context (NoProject) as part of a control Logic,
Is AZ Controller able to differentiate between no project loaded and project loaded?  Or is it only sensing a change of state of 'Context'?

Regards
Robert