> ## Documentation Index
> Fetch the complete documentation index at: https://aidocs.ethanbragdon.icu/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge Source Ingestion

> Understand the lifecycle from file or source upload through extraction, chunking, embeddings, indexing, and retrieval readiness.

# Knowledge source ingestion

A Knowledge source is not ready for semantic retrieval the instant it appears in storage. Starfire processes durable sources through an ingestion pipeline.

## Ingestion lifecycle

```text theme={null}
Source added
   ↓
Validate source
   ↓
Extract / normalize content
   ↓
Chunk content + metadata
   ↓
Create embeddings / index entries
   ↓
Mark retrieval-ready
```

## Add a source

1. Open the Knowledge or project Knowledge surface.
2. Choose the owning personal, project, or organization context.
3. Add a supported file, repository, URL, or other available source type.
4. Wait for processing/indexing state to complete.
5. Test retrieval with a query that should clearly match the source.

## Extraction

Extraction converts the original source into material Starfire can index. A source can fail here because the format is unsupported, the content is corrupt, or the extractor cannot read useful text.

## Chunking

Large content is divided into retrievable units. Chunk metadata should preserve enough source location and identity for Starfire to connect retrieved material back to its source.

## Embeddings and indexing

Configured embedding/index services turn chunks into searchable representations. Provider health or indexing-worker health can delay this stage even when the upload itself succeeded.

## Ready state

A source should be considered reliable for Knowledge queries only after the relevant indexing state is ready.

<Warning>
  Do not diagnose a retrieval problem by repeatedly uploading duplicate copies of the same source. First inspect whether the original source failed extraction, indexing, permissions, or retrieval.
</Warning>

<Card title="RAG & retrieval" icon="magnifying-glass-chart" href="/knowledge/rag-retrieval">See how indexed content is selected at query time.</Card>