Frumpagumpus · 2022-09-23 03:02
so, openai just open sourced "whisper" which apparently is human level speech to text software.
i've wanted to do hands free emacs for a while now but existing software like dragon naturally speaking, just didn't seem up for it. Is now the time? Could we design some kind of conlang (constructed language) to interact with emacs with? Would it be like the reverse version of how the blind use a computer (where the words blend together at a seemingly impossible speed for those untrained)?
lebensterben · 2022-09-23 03:06
from my limited experience with my Alexa, I fear that I will soon start cursing at Emacs ...
ideasman_42 · 2022-09-23 03:24
I had a look at whisper and it seems they dont (yet?) support realtime dictation. Unless I missed something, nevertheless pretty cool.
For over a year I've been using [nerd-dication](https://github.com/ideasman42/nerd-dictation/) with emacs, see [demo](https://www.youtube.com/watch?v=T7sR-4DFhpQ).
gvol · 2022-09-23 03:55
You might want to check out [Cursorless](https://github.com/cursorless-dev/cursorless) which has an experimental/in-progress [Emacs version](https://github.com/cursorless-everywhere/emacs-cursorless). Do watch some of their videos. Cursorless is based on [Talon](https://talonvoice.com/) which does most of the hard work of getting voice input (among other things) and is a cool project in it's own right.
The Cursorless team is pretty active on the Talon slack. They have regular meetings on Discord. I don't really follow the development, but I think it would be cool to try out once it's usable and I have some free time.
To your original question, it seems that aegis (the main Talon dev) thinks whisper is probably too slow at the moment to be used for controlling your computer. That is from the [Talon slack](https://talonvoice.slack.com/archives/C9LG499EE/p1663790279747279).
[deleted] · 2022-09-23 04:19
[deleted]
sg_9 · 2022-09-23 04:25
I feel like efficient use of emacs with just voice may still be difficult. Potentially including eye tracking may help as you could essentially have mouselike input with it
indigo0086 · 2022-09-23 12:22
it will be achieved
paretoOptimalDev · 2022-09-23 16:27
It's easiest with good fuzzy finding like consult+orderless.
Then you want something like cursorless because avy is not fun via voice. Trust me, I've used emacs full time via voice like this before.
For voice you need to be able to chain many actions to a location and not have to "hit enter".
Things like (in talon/cursorless speak) "red air embark eww isearch emacs" bringing you to an Aww buffer with an active isearch.