search
Semantic file search using FastEmbed and sqlite-vec
Installation
pip install -r requirements.txt
For Nvidia GPUs, install fastembed-gpu. For Intel GPUs, install OpenCL drivers (for example intel-compute-runtime) and onnxruntime-openvino.
Usage
This program uses a client-server architecture to watch directories with inotify and keep the model in memory so the client doesn't have to wait several seconds to load the model. It uses file inodes and modification times to avoid unnecessary re-indexing.
Run python server.py DIRS_TO_INDEX to start the server. Make sure you don't include nested directories or weird stuff will happen. The server currently only indexes images although more modalities may be supported in the future. There are probably some weird race condition bugs if you modify a lot of files at the same time.
Then run python client.py SEARCH_TEXT NUM_RESULTS to get a list of the most similar files. You can pass this list to an image viewer such as Gwenview to view image results, although note that Gwenview doesn't preserve the order of the images. Alternatively, add a third parameter to python client.py and it will create a temporary directory containing symlinks to the search results and return the directory name.
Check out TIDY (for Android) and rclip for similar projects, although this one is probably fastest!
TODO
- Write a
pyproject.tomland properly package this - Support OSes other than Linux? It should be fairly portable now other than a few things like
os.environ["XDG_RUNTIME_DIR"]. - GUI
- Investigate race condition bugs and stress test this more
- Detect if
DIRS_TO_INDEXcontains nested dirs