Re: Sibiac: Single Image Blob Interface Accessible Control

[crozell25]

A screen reader I have used is Autoit. it is able to detect color and images and is programmable. I do not know all of the programming capabilities but it is excellent for automation.

[azslow3]

Thank you for the comment. If I understand correctly, Autoit is not a Screen Reader. But some used technic in it can give me fresh ideas.

[crozell25]

While on the surface it may not seem like a screen reader it is capable of looking at a window and searching for images. When image is found a command can be executed.
Since I commented I realized that with music software latency is an issue. Autoit is basically a loop based script the less latency you define by smaller delays in between script commands the increase in cpu resources can become rather significant. 
Originally it was developed to install software automatically. When an image was found on the screen the script knew what command to proceed to next.
Hopefully what I said makes sense.

[azslow3]

In fact I could not understand the problem till I installed NVDA. It is free and support portable installation (without bloating the system). For anyone with monitor the situation is hard to realize (see my general post). Latency and CPU are not an issue, it is not about speed, it is about theoretical possibility.

All screen readers have advanced scripting with a huge set of information extraction features. For some reason they have decided to "stop" on non image information, except manual OCR call for the whole element (f.e. in browsers). I do not exclude relatively trivial explanation, the development is driven by people who have never seen the screen, for them that is hard to imagine what picture interface really is. For someone who could see before, the idea is almost trivial. That is how this project was born.

And that is why originally sight based tools like Autoit can contribute. Even in case they have useful features, they was not considered by "mainstream" accessibility developers.
But that work till some degree. Primary purpose of a screen reader is to describe current interface state to the user (in strictly text information) and allow to decide what to do as next. I mean not simple "when - then" behavior.

[crozell25]

I read your post again and I understand what you mean by a screen reader. So no autoit doesn't read the screen. I was thinking of a screen reader as a program to look at the screen find a control that is not capable of being assigned to a midi controller via a screen grab.
This is the direction I was going toward since I have software that is not able to be controlled via midi or osc. The knobs are on the screen and can be manipulated with the mouse but as far as I can tell, without exhausting my time , they cannot be controlled otherwise.
Another point you talked on was colors. I have a color deficiency myself which renders me unable to interpret most shades of the same color.
I commend your your work in the community and thank you for your plugin.

[azslow3]

Yes, the primary target for this board is "Accessibility".

But the methods can be shifted from one world to the other. I am thinking about adding "clicking" to my AZ Controller. With many Synthes I have mapped MIDI controls to normal keyboard arrows and enter, but I still have to click first on required region to choose the preset. Unlike you, I do not use VST with mouse only parameters, but that can be the "next level" in the same direction.

Coming from accessibility world back, there are already useful features:
* in AZ Controller that is an audition. F.e. I (and some other) have found it useful to hear markers when they are passed during playback, much less distracting then looking at screen while playing with the question "hmm... which verse that is going to be?".
* MarKo has written an utility to convert AZ Controller preset into text file, parsing GUI where preset is loaded. I must admit, I have no skills to do this in reasonable time

That is why I think interchanging technologies between accessibility and visual worlds can have benefits for both sides.

[azslow3]

The first post is updated. Future development will target plug-ins used from REAPER.