[[apologies if this is a dupe. gmane is a bit odd sometimes]]
I need some help on a handicap accessibility project. It's really great for
people like me that naturally speaking is working in wine (mostly). One
important shortcoming is getting our dictated text into linux. Today, in order
to copy text from the wine environment and place it in the Linux environment, we
need to dictate into the wine version of Notepad, cut the contents of Notepad
into the cut/paste buffer, then navigate to the "active" application
and paste.
It's tedious but it's also significantly better than running in XP
and
dictating into Linux applications using a Windows X11 server.
What I want to do is potentially very simple. I want to be able automate what
I'm doing by hand. To dictate into an editor just like wine notepad and then
on
a command, transfer the contents of the editor buffer into the
"active" application.
Unfortunately, simple doesn't mean easy. we need to make sure Naturally
speaking
sees the right dictation target and that we can cut from the DNS dictation
target and paste into the "active" app in linux.
A secondary wish is to be able to run the natpython environment such that when
recognition happens in wine and the python code invoked by DNS grammars runs in
linux. This would help disabled folks like myself by giving us a powerful tool
for command and control of applications.
this is a very rough sketch of what I want to do and some help with design,
implementation details, and coding would be welcome. Like many users of DNS my
hands were broken by coding and python is the only code spoken here (and not all
that quickly to boot because the tools aren't all that friendly).
--- eric
On Sat, May 30, 2009 at 11:14 PM, Eric S. Johansson <esj at harvee.org> wrote:> [[apologies if this is a dupe. ?gmane is a bit odd sometimes]] > > I need some help on a handicap accessibility project. It's really great for > people like me that naturally speaking is working in wine (mostly). One > important shortcoming is getting our dictated text into linux. ?Today, in > order to copy text from the wine environment and place it in the Linux > environment, we need to dictate into the wine version of Notepad, cut the > contents of Notepad into the cut/paste buffer, then navigate to the "active" > application and paste. ? It's tedious but it's also ?significantly better > than running in XP and dictating into Linux applications using a Windows X11 > server. > > What I want to do is potentially very simple. I want to be able automate > what > I'm doing by hand. To dictate into an editor just like wine notepad and then > on > a command, transfer the contents of the editor buffer into the "active" > application. > > Unfortunately, simple doesn't mean easy. we need to make sure Naturally > speaking sees the right dictation target and that we can cut from the DNS > dictation target and paste into the "active" app in linux. > > A secondary wish is to be able to run the natpython environment such that > when > recognition happens in wine and the python code invoked by DNS grammars runs > in > linux. ?This would help disabled folks like myself by giving us a powerful > tool > for command and control of applications. > > this is a very rough sketch of what I want to do and some help with design, > implementation details, and coding would be welcome. ?Like many users of DNS > my hands were broken by coding and python is the only code spoken here (and > not all that quickly to boot because the tools aren't all that friendly). > > --- eric > >The active app is never a windows application, right? Is the application always launched before notepad, or can it be launched after notepad? -- -Austin
Eric S. Johansson wrote:> Austin English wrote: > > > You may be able to use AutoHotKey to cut and paste for you more > > automatically (http://www.autohotkey.com/). > > > > I don't believe a hotkey will work in this application. It appears it would > only work in the wine environment and not have any effect on the linux side. The > timeline and contexts shifts outlined below might make this clearer. > > [Linux] > Switch to application thunderbird > Bring up the message composer > click on the compose window > > [wine] > switch to first notepad > Dictate message > copy the buffer > > [Linux] > switch to Thunderbird > click on compose window > paste buffer into compose window > > [wine] > Switch to first notepad > dictate more text > > [wine] > start second notepad > switch to second notepad > dictate more text > copy the buffer > > [Linux] > Switch to instant messenger > select message window > paste buffer > > [wine] > switch to first notepad > copy buffer > > [Linux] > switch to Thunderbird compose > select message window > paste buffer > > ------------------------- > > In order to make this work, I need to know the context of the active application > at the time dictation starts. Then during the dictation time, switch context to > the editor window associated with the application, injected text into the editor > if it's plain dictation other wise take the output from a grammar activator > program and stuff it in the X11 keystroke stream. When dictation finishes, > return focus to the formerly active application. > > In thinking about what would be the minimum acceptable setup, a simple ping-pong > between an active application and an editor should be okay about 80% of the time > if something like natlink+python was available. Activating adding and > terminating editing by explicit commands shouldn't be too onerous as that's what > we use today with the current dictation box capability. > > Shifting the topic slightly, natlink is a wrapper around a com interface to > NaturallySpeaking. How difficult would it be to make that something that can > cross the wine/ linux boundary?Not an ideal solution, but have you tried running Windows Thunderbird under wine? Firefox works pretty well, I guess Thunderbird can't be that bad.
If I understand correctly, you need something that can automatically copy the text from any wine app into the currently active Linux app, right? Seems to be a tough task, here's my idea. If you build an application using Winelib, you can get access to both native Linux code and native Windows code. This would be ideal for you to write an application that would do this. The question here is the implementation. You need something that can detect what is being typed in the currently selected textbox, textarea, whatever, using Windows/Wine code (WinAPI). Then, you need to detect which application/window was last active (because the current active window is owned by a Windows App) and figure a way to detect where you were writing and paste it there. One way to do this would be by opening this app at the moment of writing. Another one would be to have this application constantly running. This really is a tough thing, if not in a certain way impossible. You'd need to get an experienced programmer here and I am by no means that experienced, though I would gladly like to help in any way I can. Maybe there is a simpler way to do this (which may be tougher on you, I am not sure). You could have an app that requested a source window and a target window (consider a window everything - button, scrollbar, etc...) - you would select them (My problem here is how you would) and it would gladly redirect the text from one to the other, and that really isn't that hard, AFAIK at the most it will envolve running parallell threads. Jorl17