RAG (Retrieval-Augmented Generation) is an AI method that combines a large language model (LLM) with an external knowledge base to produce more accurate, context-aware answers. The idea is simple: first we retrieve relevant information from documents or data sources, then we pass this information to an LLM to generate the final response. This approach reduces hallucinations, improves accuracy, and allows you to update the knowledge base without expensive retraining.

Today, we’ll look at how to build a basic RAG system in PHP (yes, really!) using the Neuron AI framework. This will be a small proof-of-concept: minimal, but fully functional.

Ready to generate something useful?

1. What RAG Is and Why You Need It

In short: RAG helps an AI system avoid guessing by fetching real data before generating an answer.

The classical flow has two steps:

• Retrieval — find relevant document chunks using vector search.
• Generation — create an answer using the retrieved data.

There are many variations of RAG — from simple “vector search + LLM” to complex systems with re-ranking, context caching, and chain-of-thought (better not use that in production). If you want a deep dive, the internet is full of good articles explaining the history and evolution of RAG (the approach started taking shape around 2020).

Typical Use Cases

• internal chatbots for documentation search
• voice assistants
• helpdesk support bots
• or simply impressing your colleagues

2. Why PHP and Neuron AI?

Good question.

Of course, you can build RAG in Python using LangChain, LlamaIndex, Milvus, Chroma, etc. There are plenty of tutorials. But if your whole web project already runs on PHP, why bring in Python just for vector search?

Neuron AI is a lightweight PHP framework that brings embeddings, LLMs, and even a local VectorStore into the PHP world. I wrote about it earlier, and this article continues with a real, practical example.

It’s simple, easy to integrate, and fits naturally into existing PHP applications — very much in the spirit of Laravel, but for AI.

3. What We’re Building

We’ll create an AI agent that can answer questions using your internal knowledge base.

Technically, we’ll build a basic RAG system that:

• creates a vector store from documents
• searches relevant chunks (returns the top-K closest vectors)
• generates an answer using the retrieved data

Imagine you have internal Wiki or Confluence docs and want a bot that can answer questions about them. This is exactly what RAG was designed for — especially useful for newcomers who still see your documentation as “Terra Incognita”.

4. Step-by-Step with Code

4.1 Requirements

• PHP 8.2+
• Composer
• Neuron AI (composer require neuron-ai/neuron)
• LLM API key (OpenAI, etc.)

4.2 Why FileVectorStore?

For this example, we won’t use vector databases like Faiss or Pinecone. A simple FileVectorStore is enough — it stores everything in a .store file.

It’s not scalable, but perfect for demos or small local projects with a few thousand documents.

4.3 Installation and Setup

Install packages:

composer require neuron-ai/neuron
composer require openai-php/laravel

Project structure:

/demo/
├── store/
│    ├── docs/
├── src/
│    ├── Commands/
│    ├── Classes/
└── index.php

4.4 Preparing the Documents

Place four Markdown documents inside store/docs/. They describe a company: culture, overview, services, and technical expertise.

4.5 Creating the VectorStore

Example class: src/Classes/PopulateVectorStore.php

namespace App\demo\src\Classes;
require_once __DIR__ . '/../../../../vendor/autoload.php';

use NeuronAI\RAG\DataLoader\FileDataLoader;
use OpenAI\Factory;

class PopulateVectorStore {
public static function populate(): void {
$vectorDir = __DIR__ . '/../../store';
$storeFile = $vectorDir . '/demo.store';
$metaFile = $vectorDir . '/demo.meta.json';

// Ensure directory exists
if (!is_dir($vectorDir)) {
mkdir($vectorDir, 0755, true);
}

// Clear existing store
file_put_contents($storeFile, '');

// Initialize OpenAI client
$apiKey ='<your-OPENAI_API_KEY-here>';
if (!is_string($apiKey) || trim($apiKey) === '') {
throw new \RuntimeException('OpenAI API key not configured. Ensure OPENAI_API_KEY is set.');
}
$client = (new Factory())
->withApiKey($apiKey)
->make();

$model = 'text-embedding-3-small';

// Probe expected dimension once
$expectedDim = 1536;

// Docs
$documents = FileDataLoader::for($vectorDir . '/docs')->getDocuments();

$written = 0;
// Generate embeddings and write to store
foreach ($documents as $document) {
$content = $document->getContent();

// Get embedding from OpenAI
try {
$response = $client->embeddings()->create([
'model' => $model,
'input' => $content,
'dimensions' => $expectedDim,
]);

// SDK v0.12+ exposes embeddings via `$response->embeddings`
if (isset($response->embeddings[0]->embedding)) {
$embedding = $response->embeddings[0]->embedding;
} else {
// Fallback for array casting if SDK shape changes
$arr = method_exists($response, 'toArray') ? $response->toArray() : [];
if (isset($arr['data'][0]['embedding'])) {
$embedding = $arr['data'][0]['embedding'];
} else {
throw new \RuntimeException('Unable to parse embedding from OpenAI response');
}
}
} catch (\Throwable $e) {
throw new \RuntimeException('Failed to generate embedding: ' . $e->getMessage());
}

// Normalize and validate embedding
if (!is_array($embedding)) {
echo "! Skipped document due to invalid embedding type.\n";
continue;
}
$embedding = array_map(static function ($v) {
return is_numeric($v) ? (float)$v : 0.0;
}, $embedding);

if (count($embedding) !== $expectedDim) {
echo "! Skipped document due to dimension mismatch (got " . count($embedding) . ", expected $expectedDim).\n";
continue;
}

// Write as JSON line to store file (strict JSONL)
// FileVectorStore expects all fields at top level
$jsonLine = json_encode([
'embedding' => $embedding,
'content' => $content,
'sourceType' => $document->getSourceType(),
'sourceName' => $document->getSourceName(),
'id' => md5($content),
'metadata' => [],
], JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);

file_put_contents($storeFile, $jsonLine . "\n", FILE_APPEND);
$written++;

echo "✓ Added embedding ($written) for: " . $storeFile . " | " . str_replace("\n", ' ', substr(trim($content), 0, 70)) . "...\n";
}

// Write metadata file for consistency checks
$meta = [
'model' => $model,
'dimension' => $expectedDim,
'generatedAt' => date(DATE_ATOM),
'count' => $written,
];
file_put_contents($metaFile, json_encode($meta, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE | JSON_PRETTY_PRINT));

echo "\n✓ Vector store populated with $written documents (dimension: $expectedDim)\n";
}
}

Run the script via index.php:

use App\demo\src\Classes\PopulateVectorStore;

require_once __DIR__ . '/src/Classes/PopulateVectorStore.php';

PopulateVectorStore::populate();

You should see logs like:

php app/demo/index.php
✓ Added embedding (1) for: /app/demo/src/Commands/../../store/demo.store | # Linx Team - Company Culture & Values  ## Core Values  ### Innovation...
✓ Added embedding (2) for: /app/demo/src/Commands/../../store/demo.store | ## Work Environment  ### Remote-Friendly We offer flexible work arrang...
✓ Added embedding (3) for: /app/demo/src/Commands/../../store/demo.store | ## Benefits  ### Competitive Compensation We offer competitive salarie...
✓ Added embedding (4) for: /app/demo/src/Commands/../../store/demo.store | # Linx Team - Company Overview  ## About Us Linx Team is a leading sof...
✓ Added embedding (5) for: /app/demo/src/Commands/../../store/demo.store | # Linx Team - Services & Portfolio  ## Services Offered  ### Custom So...
✓ Added embedding (6) for: /app/demo/src/Commands/../../store/demo.store | ### DevOps & Infrastructure Managing cloud infrastructure, implementin...
✓ Added embedding (7) for: /app/demo/src/Commands/../../store/demo.store | # Linx Team - Technical Expertise  ## Core Technologies  ### Backend D...
✓ Added embedding (8) for: /app/demo/src/Commands/../../store/demo.store | js** - React framework with server-side rendering  ### Cloud & DevOps ...

✓ Vector store populated with 8 documents (dimension: 1536)

Two new files will appear:

• demo.meta.json — metadata
• demo.store — actual vector entries

Each line in demo.store is a JSON object:

{
"embedding": [...],
"content": "....",
"sourceType": "files",
"sourceName": "company-culture.md",
"id": "28b40662dad319d6f5718881af03283b",
"metadata": []
}

Why 8 entries from 4 documents?

Because the documents are automatically split into chunks before generating embeddings. Each chunk gets its own vector.

Reasons:

• embeddings work best on short, coherent text pieces
• LLMs have token limits
• retrieval becomes more precise

The dimension: 1536 value comes from the OpenAI embedding model (text-embedding-3-small).

4.6 Creating the ChatBot

Create the agent class: src/Commands/ChatBot.php.

<?php

namespace App\demo\src\Commands;
require_once __DIR__ . '/../../../../vendor/autoload.php';

use NeuronAI\Providers\AIProviderInterface;
use NeuronAI\Providers\OpenAI\OpenAI;
use NeuronAI\RAG\Embeddings\EmbeddingsProviderInterface;
use NeuronAI\RAG\Embeddings\OpenAIEmbeddingsProvider;
use NeuronAI\RAG\RAG;
use NeuronAI\RAG\VectorStore\FileVectorStore;
use NeuronAI\RAG\VectorStore\VectorStoreInterface;

class ChatBot extends RAG
{
private string $apiKey ='<your-OPENAI_API_KEY-here>';
private string $model = 'gpt-4o-mini';

protected function provider(): AIProviderInterface
{
if (!$this->apiKey) {
throw new \Exception('OPENAI_API_KEY environment variable is not set');
}

return new OpenAI(
$this->apiKey,
$this->model,
);
}

protected function embeddings(): EmbeddingsProviderInterface
{
if (!$this->apiKey) {
throw new \Exception('OPENAI_API_KEY environment variable is not set');
}

return new OpenAIEmbeddingsProvider(
key: $this->apiKey,
model: 'text-embedding-3-small',
dimensions: 1536
);
}

protected function vectorStore(): VectorStoreInterface
{
$vectorDir = __DIR__ . '/../../store';

// Ensure the vectors directory exists
if (!is_dir($vectorDir)) {
mkdir($vectorDir, 0755, true);
}

// Ensure the store file exists with at least one empty line to prevent parsing errors
$storeFile = $vectorDir . '/demo.store';
if (!file_exists($storeFile) || filesize($storeFile) === 0) {
// Create an empty store file - FileVectorStore will populate it when documents are added
file_put_contents($storeFile, '');
}

return new FileVectorStore(
directory: $vectorDir,
name: 'demo',
topK: 3
);
}
}

Then update your index.php to run the chatbot:

use App\demo\src\Classes\PopulateVectorStore;
use App\demo\src\Commands\ChatBot;
use NeuronAI\Chat\Messages\UserMessage;

require_once __DIR__ . '/../../vendor/autoload.php';

// Populate the vector store if it doesn't exist
$storeFile = __DIR__ . '/store/demo.store';
if (!file_exists($storeFile) || filesize($storeFile) === 0) {
PopulateVectorStore::populate();
echo "Vector store populated successfully.\n";
} else {
echo "Vector store found, start handling...\n";
}

$chatBot = ChatBot::make();
$response = $chatBot->chat(
new UserMessage('How many employees and managers does the company have?')
);

echo "\n" . $response->getContent() . "\n";

Example output:

The company has over 27 employees and 2 managers, making a total of more than 29 team members.

Another example question

new UserMessage(
'How many employees and managers does the company have? ' .
'Provide links to most relevant documents.'
);

returns not only the answer but also recommended documents:

The company has over 27 employees and 2 managers.
For more detailed information, you can refer to the following documents:

Company Overview: company-overview.md
Company Culture: company-culture.md

About topK: 3

topK determines how many closest vectors to return from the vector store. More chunks = richer context, but also slower queries.

So, after all this, the final structure of the project looks like this:

/demo/
├── store/
│    ├── docs/
│    │    ├── company-culture.md
│    │    ├── company-overview.md
│    │    ├── services-portfolio.md
│    │    ├── technical-expertise.md
│    │    ├── ...
│    ├── demo.meta.json
│    ├── demo.store
├── src/
│    ├── Commands/
│    │    ├── ChatBot.php
│    ├── Classes/
│    │    ├── PopulateVectorStore.php
└── index.php

5. What Can Be Improved

This was just a basic example. You can add:

• real databases (PostgreSQL, Pinecone, Qdrant)
• automatic indexing of new Wiki pages
• caching for common questions
• request logging for analytics and debugging

6. Advanced Version: Add Re-Ranking

For higher accuracy, add document re-ranking using a model like bge-reranker-base. Neuron AI supports this, and the improvement in answer quality is quite noticeable.

7. Modular Architecture: When You Need It

If RAG is just a small feature, keep things simple. But if the project grows, separate:

• VectorStoreService
• EmbeddingPipeline
• RAGPipeline
• ChatController

This lets you switch components easily — e.g., move from OpenAI to Ollama, or from FileVectorStore to Qdrant.

8. Conclusion

RAG is not magic — it’s a clear, well-defined pattern. Neuron AI finally lets PHP developers play with modern AI tools without switching to Python, Docker, or extra servers.

Yes, FileVectorStore is basic, but perfect for demos and local experiments. Once you understand the principles, you can integrate RAG into any PHP framework and upgrade the components whenever you’re ready.