Uploading Documents

Add documents to your agent's knowledge base.

CrabShack supports over 40 file types, from standard office documents to code files and images. This page covers what you can upload, how to do it, and what to expect during processing.

Supported File Types

Category	Formats
Documents	PDF, DOCX, DOC, PPTX, XLSX, XLS, CSV, TXT, MD, RTF, ODT, ODS, EPUB
Email	EML
Web	HTML, HTM
Data & Config	XML, JSON, YAML, YML, LOG, CONF, INI, TOML
Code	PY, JS, TS, SH, SQL
Images (with OCR)	PNG, JPG, JPEG, TIFF, TIF, BMP, GIF, WEBP
Archives	ZIP (contents are extracted and each file is processed individually)

note

Image files require OCR to be enabled in settings. Without OCR, images are accepted but no text is extracted from them. See OCR for Images and Scanned PDFs below.

Upload Limits

Limit	Value
Maximum file size	100 MB per file
Files per upload	Up to 10 files at once
Total documents	500 by default (configurable up to 10,000)
Total storage	Automatic — uses up to 50% of available disk space (or set a manual limit)

How to Upload

1. Navigate to Knowledge Base in the CrabShack sidebar
2. Drag and drop files onto the upload area, or click the upload area to select files from your computer
3. Wait for processing to complete — the status badge will change from Processing to Ready

You can upload multiple files at once (up to 10 per batch). Each file is processed independently, so if one file fails, the others will still complete.

ZIP Files

When you upload a ZIP archive, CrabShack extracts the contents and processes each supported file individually. This is a convenient way to upload many files at once. Unsupported file types within the archive are skipped.

Document Categories

After processing, each document is automatically categorized:

Category	Typical Content
Technical	API documentation, code, database schemas, deployment guides
Business	Revenue reports, marketing plans, strategy documents, budgets
Legal	Contracts, compliance documents, terms of service, agreements
Operations	Workflow guides, schedules, logistics documents, vendor information
Research	Studies, findings, methodology documents, white papers
General	Anything that doesn't fit the categories above

Categories are assigned automatically and help with organization. They don't affect whether a document is searchable — all documents are searchable regardless of category.

OCR for Images and Scanned PDFs

By default, OCR (optical character recognition) is disabled. When enabled, CrabShack can extract text from:

• Image files (PNG, JPG, TIFF, BMP, GIF, WEBP) — photos of documents, screenshots, diagrams with text
• Scanned PDFs — PDFs that contain images of text rather than selectable text

To enable OCR:

1. Navigate to Knowledge Base in the sidebar
2. Open Settings (gear icon)
3. Toggle OCR for images to on
4. Optionally select a specific model for OCR processing

CrabShack uses a two-stage OCR approach: a fast initial pass extracts text, and if the quality is below a threshold, a more capable AI-powered pass fills in the gaps. This ensures good results even with low-quality scans or photographs.

tip

Enable OCR only if you regularly work with scanned documents or images containing text. It adds processing time but unlocks content that would otherwise be invisible to your agents. For standard digital documents (typed PDFs, Word files, etc.), OCR is not needed.

Previewing Documents

To see how CrabShack has processed a document:

1. Navigate to Knowledge Base and find the document in the list
2. Click the document name to open the preview drawer
3. You'll see the full extracted text, broken into chunks with visible boundaries

This is useful for verifying that text extraction worked correctly, especially for PDFs and scanned documents.

Reindexing Documents

If you change Knowledge Base settings (like search precision or context window), existing documents aren't automatically reprocessed. To apply new settings to an existing document:

1. Find the document in the list
2. Click the Reindex button (refresh icon)

Reindexing re-chunks and re-indexes the document without re-uploading it. The document remains available to agents during reindexing.

Failed Documents

If a document fails to process, it appears in the Failed tab of the Knowledge Base dashboard. Each failed document shows an error message explaining what went wrong. You can:

• Retry — attempt to process the document again (useful if the failure was temporary)
• Delete — remove the failed document

Common reasons for failures:

• The file is corrupted or password-protected
• The file type isn't actually what the extension suggests
• The file is empty or contains no extractable text