Skip to content


pipeline pipeline

The Textractor pipeline extracts and splits text from documents. This pipeline uses either an Apache Tika backend (if Java is available) or BeautifulSoup4.


The following shows a simple example using this pipeline.

from txtai.pipeline import Textractor

# Create and run pipeline
textract = Textractor()

See the link below for a more detailed example.

Notebook Description
Extract text from documents Extract text from PDF, Office, HTML and more Open In Colab

Configuration-driven example

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.


# Create pipeline using lower case class name

# Run pipeline with workflow
      - action: textractor

Run with Workflows

from import Application

# Create and run pipeline with workflow
app = Application("config.yml")
list(app.workflow("textract", [""]))

Run with API

CONFIG=config.yml uvicorn "txtai.api:app" &

curl \
  -X POST "http://localhost:8000/workflow" \
  -H "Content-Type: application/json" \
  -d '{"name":"textract", "elements":[""]}'


Python documentation for the pipeline.

__init__(self, sentences=False, lines=False, paragraphs=False, minlength=None, join=False, tika=True) special

Source code in txtai/pipeline/data/
def __init__(self, sentences=False, lines=False, paragraphs=False, minlength=None, join=False, tika=True):
    if not TIKA:
        raise ImportError('Textractor pipeline is not available - install "pipeline" extra to enable')

    super().__init__(sentences, lines, paragraphs, minlength, join)

    # Determine if Tika (default if Java is available) or Beautiful Soup should be used
    # Beautiful Soup only supports HTML, Tika supports a wide variety of file formats, including HTML.
    self.tika = self.checkjava() if tika else False

__call__(self, text) special

Source code in txtai/pipeline/data/
def __call__(self, text):
    Segments text into semantic units.

    This method supports text as a string or a list. If the input is a string, the return
    type is text|list. If text is a list, a list of returned, this could be a
    list of text or a list of lists depending on the tokenization strategy.

        text: text|list

        segmented text

    # Get inputs
    texts = [text] if not isinstance(text, list) else text

    # Extract text for each input file
    results = []
    for value in texts:
        # Get text
        value = self.text(value)

        # Parse and add extracted results

    return results[0] if isinstance(text, str) else results