wine users - Jul 2009 - Re: How to mix (naturally speaking) win32 and native (python) li

Eric S. Johansson wrote:
> vitamin wrote:
> 
> > Eric S. Johansson wrote:
> > 
> > > Is it possible to make this code (natlink) talk to naturally
speaking in
> > > wine using Windows 32 but speak to Python in Linux so we can do
all our fun
> > > command-and-control stuff there.
> > > 
> > 
> > No, Linux's python has no notion of COM (which is obviously a
win32 only
> > thing). And windows python doesn't work all that well on Wine.
> > 
> 
> no surprise.  so if I write a linux python C extension that uses wine libs
to
> make the com connection it won't work?  I guess this means getting
natspeak/wine
> to see the taskbar/window id of the active app will be dammed near
impossible?
> 
> (see what twisted trouble wine can get you into)
> 
> You probably deserve a bit of back story at this point. I'm trying to
build a
> speech recognition friendly dictation target that can cut or paste between
the
> dictation target and the application target. Our shorthand for it is
"persistent
> dictation box". This derives from the feature in NaturallySpeaking
called
> "dictation box". You can use it to dictate text and be able to
edit it using
> speech recognition. On completion of dictation/editing, you can then paste
it
> into any Windows application using standard Windows cut-and-paste
sequences. The
> downside is that the dictation box goes away when you paste the text.  I
figure
> that if you can leave a dictation box up and associated with an
application,
> it'll be an easier environment to work with especially with short burst
> dictation like Instant Messenger.
> 
> To do this the "easy way" I need something like Notepad created
with the
> NaturallySpeaking friendly edit controls. Once I give the command to
transfer, I
> need to show the data into a cut-and-paste buffer, shift focus to the right
> Linux application and paste.
> 
> I would like to do as much of this as I can in Python because my hands are
> broken and Python is probably the only language I can dictate using speech
> recognition unassisted. I also have a couple of people who have expressed
> interest in helping building a speech recognition friendly editing
environment
> but, I need to finish the UI design before we can move forward.
> 
> The next example comes from this speech friendly editing environment. At
the
> very least, to make an editor usable with speech recognition, you need to
be
> able to generate keystrokes that activate keyboard shortcuts to various
> functions. Emacs is wonderful because if there isn't a keystroke
already
> defined, you can define one and get the functionality you need. VI is
> particularly hideous because well, if you misrecognize command and generate
> text, God knows what's going to happen to your work.
> 
> Hopefully this back story helps you understand what I'm up against.
I'm really
> hoping to make Linux more friendly to disable programmers like myself but
it's
> going to take help from developers whose hands still work otherwise the job
> won't get done because it's too big for someone with broken hands.

Hi, if I understand it correctly, your idea is quite related (but not the same
as) to: http://forum.winehq.org/viewtopic.php?t=5048

My lack of experience with python does not let me know how well you can handle
events there, I suppose as well as in C.

So, to get it right:

> I figure
> that if you can leave a dictation box up and associated with an
application,
> it'll be an easier environment to work with especially with short burst

You want to have a dictation box associated with a native app once you select
it, is that right? For starters, I see that this box would have to be made using
the Win32API, so that you can dictate onto it with NatSpeak. Then you'd need
to have some kind of thread that monitored it for changes and then flushed them
to the native window. I'm thinking C here.

(This is all theory which can fully be wrong) So I guess that if you used native
code to get info on the Window (as well as box positioning), you could use it
to, then, create the box and the associated thread which would keep monitoring
and flushing the data. But more than that, you'd have to forward individual
keypresses. (so that you could have CTRL, ALT, etcetera working)

Plus, this all would *proably* have to be run as separate process that waits
until it is 'called' (when you decide to copy-and-paste-create-box.

I don't know much about X, but I suppose there's a way to determine the
currently selected widget inside a window, if it has a representable ID (I
don't know about X, does it), you could then store it as the target ID. Then
you could also store a handle to the dictation box and that would probably be
half-way through. Again, thinking C here...and Winelib.

Sorry if this didn't help, but I figured that storing all these ideas
somewhere might be useful either for now or later,

Jorl17

The only way you can do what you want is to write a simple bridge program using
winelib which communicates with your Win32 app (e.g. using COM) and you access
the winelib bridge from your python script using e.g. shared memory or some
socket (e.g. a unix socket, TCP/IP or whatever).