this post was submitted on 14 May 2024
151 points (98.7% liked)

Open Source

28934 readers
381 users here now

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

founded 4 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[โ€“] [email protected] 5 points 1 month ago (1 children)

While this is certainly a cool concept, local voice assistants like this are currently a novelty. Cool to play around with, though!

You can expect around 5 seconds processing time to start generating the response to a basic question on a very basic model like Llama 3 8B.

For context, using Moondream2 (as recommended) on a RasPi 5, it takes around 50 seconds to process an image taken by the Camera and start generating a description.

[โ€“] [email protected] 2 points 1 month ago

Interesting, using whisper-fast on Home Assistant on my server computer takes like 2-3 seconds to process and delivery an output in English.

Useful in the smart home space.

Laughably broken in most other languages other than English, but then again, google and Alexa barely work in other languages.