Searching & Deep Research
How your agent finds and uses information from your documents.
Once documents are in your Knowledge Base, agents can access them in several ways — from automatic background retrieval to targeted deep research across multiple documents.
Automatic Search
When auto-search is enabled (it is by default), CrabShack automatically checks the Knowledge Base during every conversation. If the topic is relevant to your documents, the most relevant excerpts are quietly included in the agent's context.
This means your agent can reference your documents without being explicitly asked to search. If a user asks "What's our refund policy?" and you've uploaded your company's policy handbook, the agent will find and use the relevant section automatically.
How Semantic Search Works
Unlike traditional keyword search, CrabShack's Knowledge Base uses semantic search — it matches by meaning, not just by words. This means:
- Searching for "refund policy" will find content about "return and reimbursement procedures"
- Searching for "how to set up the database" will find installation guides even if they don't use the word "database"
- Technical jargon and plain-language descriptions of the same concept both match
This makes the search significantly more useful, especially when your documents use different terminology than your users.
Search Precision
The search threshold controls how closely a document excerpt must match the conversation topic to be included. Think of it as a dial between broad and strict:
- Lower threshold (toward "Broad") — more results, wider net. Good when you want the agent to find tangentially related information or when your documents cover many topics.
- Higher threshold (toward "Strict") — fewer results, more precise. Good when you have many documents and want only the most directly relevant excerpts.
The default setting works well for most use cases. If you find the agent is pulling in irrelevant information, raise the threshold. If it's missing relevant documents, lower it. See Configuration for exact settings.
Deep Research
For complex questions that span multiple documents, agents can perform deep research. This is a more comprehensive search mode that:
- Searches across all documents in the Knowledge Base
- Groups results by document, so you can see which sources contributed
- Returns more results (up to 50) than a standard search
- Can filter by document type, source, or filename
Deep research is particularly useful for:
- Comparing information across multiple sources ("What do our different policy documents say about remote work?")
- Synthesizing knowledge from several documents ("Summarize all the technical requirements from our vendor contracts")
- Broad investigations where the answer isn't in a single document
Agents activate deep research automatically when a question is complex enough to warrant it, or when explicitly asked to research a topic thoroughly.
Reading a Full Document
Sometimes an excerpt isn't enough — if a question needs the complete context of one specific document (like reading an entire contract rather than the sections that best match a search), your agent can read that document in full instead of relying on search excerpts. Ask your agent to look at a specific document by name if you want this.
Context Window
When an agent retrieves a relevant excerpt, CrabShack can also include adjacent sections — the content immediately before and after the match. This provides additional context that helps the agent understand the excerpt in its full setting.
The context window setting controls this:
| Setting | What's Included |
|---|---|
| 0 | Only the matched excerpt |
| 1 (default) | The matched excerpt plus one section before and after |
| 2 | The matched excerpt plus two sections before and after |
A wider context window gives the agent more surrounding information but uses more of the context budget. The default of 1 works well for most documents.
Tuning Search Quality
If your agent isn't finding the right information, here are the most common adjustments:
| Problem | Solution |
|---|---|
| Agent isn't finding relevant documents | Lower the search threshold (make it broader) |
| Agent is pulling in irrelevant documents | Raise the search threshold (make it stricter) |
| Answers are missing important context | Increase the context window (include adjacent sections) |
| Agent only finds partial information | Increase chunks per query (return more excerpts per search) |
| Search results are too generic | Check that your documents contain the specific information you expect |
Use the Retrieval Test tool to see exactly what your agent would find for a given query. This is the fastest way to understand how your settings affect search results and to fine-tune them before they impact real conversations.
Knowledge Injection Budget
The max knowledge context setting limits how much document content is injected per conversation. This prevents the agent's context from being overwhelmed by long document excerpts, leaving room for the actual conversation and any relevant memories.
The default budget is generous enough for most use cases. If your agents work with very detailed documents, you may want to increase it. If conversations are short and focused, you might reduce it to keep responses snappy. See Configuration for details.