Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix audio device switching #32

Closed
louis030195 opened this issue Jul 11, 2024 · 7 comments
Closed

fix audio device switching #32

louis030195 opened this issue Jul 11, 2024 · 7 comments
Assignees
@louis030195

Comments

@louis030195
Copy link
Collaborator

atm if you change audio device it will crash the process

example:

listening to music with airpods with screenpipe on

turning off airpods and boom

easy solvable probably
@louis030195
Copy link
Collaborator Author

also sometimes i cannot use my microphone anymore (need to reboot computer) - gotta fix asap

@louis030195
Copy link
Collaborator Author

RustAudio/cpal#898

@louis030195
Copy link
Collaborator Author

actively working on this today (07/12/24)

@louis030195 louis030195 self-assigned this Jul 12, 2024
@louis030195
Copy link
Collaborator Author

broke down in first:

allow starting & stopping audio & video
allow starting & stopping each audio device

from api

second:

still need to figure out the thing that turn off mic but maybe first solution will solve this

louis030195 added a commit that referenced this issue Jul 12, 2024
@louis030195
80% #32
7cfa08c
@louis030195
Copy link
Collaborator Author

figured out the audio "jumping" issue: it's just that sometimes gpu spikes at 100% and i guess audio uses GPU

fix: balance more evenly the compute on whisper (say shorter batches)

if theory is true using:

--audio-chunk-duration 5

instead of 30 should fix (but lsos of whisper quality unti lwe impl overlap: #42)

@louis030195
Copy link
Collaborator Author

Screenshot 2024-07-15 at 11 54 03 this

@louis030195
Copy link
Collaborator Author

this is good news (thought it was a bug in cpal audio lib, we can always easily optimise perfs.

louis030195 added a commit that referenced this issue Jul 15, 2024
@louis030195
80% #32
050aed5
louis030195 added a commit that referenced this issue Jul 15, 2024
@louis030195
Merge pull request #44 from louis030195/d-#32
8043f71 
#32

- use tokio in screenpipe-vision
- use n parallel tasks (should be same behaviour than before)
- (new) skip frame if cpu overloaded for OCR (configurable)
- can now turn on/off devices from screenpipe-audio using API
- can now turn on/off vision using API
- disconnecting audio device in use stop listening. User need to make API request to enable any new device (could be improved later)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@louis030195 louis030195

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@louis030195