文章目录
macos 使用brew instll fass, 后python3.12执行引用faiss包功能出现的问题
安装时遇到问题如下
ModuleNotFoundError Traceback (most recent call last)
File ~/Src/ai/framework/langchain/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py:55, in dependable_faiss_import(no_avx2)
54 else:
---> 55 import faiss
56 except ImportError:
ModuleNotFoundError: No module named 'faiss'
During handling of the above exception, another exception occurred:
ImportError Traceback (most recent call last)
Cell In[1], line 19
14 texts = text_splitter.create_documents(
15 [page.page_content for page in pages[:4]]
16 )
18 embeddings = OpenAIEmbeddings(model="text-embedding-ada-002")
---> 19 db = FAISS.from_documents(texts, embeddings)
21 retriever = db.as_retriever(search_kwargs={"k": 5})
23 docs = retriever.invoke(query="llama2有多少参数?")
File ~/Src/ai/framework/langchain/.venv/lib/python3.12/site-packages/langchain_core/vectorstores.py:635, in VectorStore.from_documents(cls, documents, embedding, **kwargs)
633 texts = [d.page_content for d in documents]
634 metadatas = [d.metadata for d in documents]
--> 635 return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)
File ~/Src/ai/framework/langchain/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py:931, in FAISS.from_texts(cls, texts, embedding, metadatas, ids, **kwargs)
912 """Construct FAISS wrapper from raw documents.
913
914 This is a user friendly interface that:
(...)
928 faiss = FAISS.from_texts(texts, embeddings)
929 """
930 embeddings = embedding.embed_documents(texts)
--> 931 return cls.__from(
932 texts,
933 embeddings,
934 embedding,
935 metadatas=metadatas,
936 ids=ids,
937 **kwargs,
938 )
File ~/Src/ai/framework/langchain/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py:883, in FAISS.__from(cls, texts, embeddings, embedding, metadatas, ids, normalize_L2, distance_strategy, **kwargs)
871 @classmethod
872 def __from(
873 cls,
(...)
881 **kwargs: Any,
882 ) -> FAISS:
--> 883 faiss = dependable_faiss_import()
884 if distance_strategy == DistanceStrategy.MAX_INNER_PRODUCT:
885 index = faiss.IndexFlatIP(len(embeddings[0]))
File ~/Src/ai/framework/langchain/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py:57, in dependable_faiss_import(no_avx2)
55 import faiss
56 except ImportError:
---> 57 raise ImportError(
58 "Could not import faiss python package. "
59 "Please install it with `pip install faiss-gpu` (for CUDA supported GPU) "
60 "or `pip install faiss-cpu` (depending on Python version)."
61 )
62 return faiss
ImportError: Could not import faiss python package. Please install it with `pip install faiss-gpu` (for CUDA supported GPU) or `pip install faiss-cpu` (depending on Python version).
直接使用本地 pip安装fass-cpu会遇到如下问题
venwei@svenweis-MacBook-Pro langchain$ pip3 install faiss-cpu
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
xyz, where xyz is the package you are trying to
install.
If you wish to install a Python library that isn't in Homebrew,
use a virtual environment:
python3 -m venv path/to/venv
source path/to/venv/bin/activate
python3 -m pip install xyz
If you wish to install a Python application that isn't in Homebrew,
it may be easiest to use 'pipx install xyz', which will manage a
virtual environment for you. You can install pipx with
brew install pipx
You may restore the old behavior of pip by passing
the '--break-system-packages' flag to pip, or by adding
'break-system-packages = true' to your pip.conf file. The latter
will permanently disable this error.
If you disable this error, we STRONGLY recommend that you additionally
pass the '--user' flag to pip, or set 'user = true' in your pip.conf
file. Failure to do this can result in a broken Homebrew installation.
Read more about this behavior here: <https://peps.python.org/pep-0668/>
```
大概意思是让尝试使用brew来安装, 安装完,再使用python来执行文件就出现文章头的错误. 提示内容也说如果非要使用pip安装就使用python虚拟环境. 目前使用python版本为python3.12, 于是就创建了虚拟环境,并且再次使用pip install faiss-cpu, 又遇到如下列表问题,
* 缺少其它安装包
* swig 执行时出现不同的错误
* ...
中间考虑过是否为python版本问题,运行看pypi上看下说明,只是大于Requires: Python >=3.8, 这句直接导致我浪费大量时间,中间也切换过python3.11遇到是swig执行时遇到问题.
于是考虑是不是swig版本与faiss的版本有相关的,又尝试不同的swig版本,结果仍出问题.
最后想编译faiss,在打算着手做时,才发现python的版本要求是3.8~3.10..WTF
下载并使用python3.10安装faiss-cpu,没用几分钟成功安装