This following describes available embeddings configuration. These parameters are set via the Embeddings constructor.
Sets the configuration storage format. Defaults to
Sets the path for a vectors model. When using a transformers/sentence-transformers model, this can be any model on the Hugging Face Hub or a local file path. Otherwise, it must be a local file path to a word embeddings model.
Sentence embeddings method to use. If the method is not provided, it is inferred using the
words require the similarity extras package to be installed.
Builds sentence embeddings using a transformers model. While this can be any transformers model, it works best with models trained to build sentence embeddings.
Same as transformers but loads models with the sentence-transformers library.
Builds sentence embeddings using a word embeddings model. Transformers models are the preferred vector backend in most cases. Word embeddings models may be deprecated in the future.
Enables copying of a vectors model set in path into the embeddings models output directory on save. This option enables a fully encapsulated index with no external file dependencies.
A scoring model builds weighted averages of word vectors for a given sentence. Supports BM25, TF-IDF and SIF (smooth inverse frequency) methods. If a scoring method is not provided, mean sentence embeddings are built.
Removes n principal components from generated sentence embeddings. When enabled, a TruncatedSVD model is built to help with dimensionality reduction. After pooling of vectors creates a single sentence embedding, this method is applied.
Sentence embeddings are loaded via an external model or API. Requires setting the transform parameter to a function that translates data into vectors.
When method is
external, this function transforms input content into embeddings. The input to this function is a list of data. This method must return either a numpy array or list of numpy arrays.
Sets the transform batch size. This parameter controls how input streams are chunked and vectorized.
Sets the encode batch size. This parameter controls the underlying vector model batch size. This often corresponds to a GPU batch size, which controls GPU memory usage.
Enables string tokenization (defaults to false). This method applies tokenization rules that only work with English language text and may increase the quality of English language sentence embeddings in some situations.
instructions: query: prefix for queries data: prefix for indexing
Instruction-based models use prefixes to modify how embeddings are computed. This is especially useful with asymmetric search, which is when the query and indexed data are of vastly different lengths. In other words, short queries with long documents.
E5-base is an example of a model that accepts instructions. It takes
passage: prefixes and uses those to generate embeddings that work well for asymmetric search.
Approximate Nearest Neighbor (ANN) index backend for storing generated sentence embeddings.
Defaults to faiss. Additional backends require the
similarity extras package to be installed. Add custom backends via setting this parameter to the fully resolvable
Backend-specific settings are set with a corresponding configuration object having the same name as the backend (i.e. annoy, faiss, or hnsw). None of these are required and are set to defaults if omitted.
faiss: components: comma separated list of components - defaults to "Flat" for small indices and "IVFx,Flat" for larger indexes where x = 4 * sqrt(embeddings count) nprobe: search probe setting (int) - defaults to x/16 (as defined above) for larger indexes quantize: store vectors with 8-bit precision vs 32-bit (boolean) defaults to false mmap: load as on-disk index (boolean) - trade query response time for a smaller RAM footprint, defaults to false sample: percent of data to use for model training (0.0 - 1.0) reduces indexing time for larger (>1M+ row) indexes, defaults to 1.0
See the following Faiss documentation links for more information.
hnsw: efconstruction: ef_construction param for init_index (int) - defaults to 200 m: M param for init_index (int) - defaults to 16 randomseed: random-seed param for init_index (int) - defaults to 100 efsearch: ef search param (int) - defaults to None and not set
See Hnswlib documentation for more information on these parameters.
annoy: ntrees: number of trees (int) - defaults to 10 searchk: search_k search setting (int) - defaults to -1
See Annoy documentation for more information on these parameters. Note that annoy indexes can not be modified after creation, upserts/deletes and other modifications are not supported.
Enables content storage. When true, the default storage engine,
sqlite will be used. Also supports
duckdb. Add custom storage engines via setting this parameter to the fully resolvable class string.
List of functions with user-defined SQL functions, only used when content is enabled. Each list element must be one of the following:
- callable object
- dict with fields for name, argcount and function
query: path: sets the path for the query model - this can be any model on the Hugging Face Model Hub or a local file path. prefix: text prefix to prepend to all inputs maxlength: maximum generated sequence length
Query translation model. Translates natural language queries to txtai compatible SQL statements.
graph: backend: graph network backend (string), defaults to "networkx" batchsize: batch query size, used to query embeddings index (int) defaults to 256 limit: maximum number of results to return per embeddings query (int) defaults to 15 minscore: minimum score required to consider embeddings query matches (float) defaults to 0.1 approximate: when true, queries only run for nodes without edges (boolean) defaults to true topics: see below
Enables graph storage. When set, a graph network is built using the embeddings index. Graph nodes are synced with each embeddings index operation (index/upsert/delete). Graph edges are created using the embeddings index upon completion of each index/upsert/delete embeddings index call.
Add custom graph storage engines via setting the
graph.backend parameter to the fully resolvable class string.
Defaults are tuned so that in most cases these values don't need to be changed.
topics: algorithm: community detection algorithm (string), options are louvain (default), greedy, lpa level: controls number of topics (string), options are best (default) or first resolution: controls number of topics (int), larger values create more topics (int), defaults to 100 labels: scoring index method used to build topic labels (string) options are bm25 (default), tfidf, sif terms: number of frequent terms to use for topic labels (int), defaults to 4 stopwords: optional list of stop words to exclude from topic labels categories: optional list of categories used to group topics, allows granular topics with broad categories grouping topics
Enables topic modeling. Defaults are tuned so that in most cases these values don't need to be changed (except for categories). These parameters are available for advanced use cases where one wants full control over the community detection process.