Are you a Rust developer eager to build powerful and intelligent applications using Large Language Models (LLMs)? Look no further. Introducing Helios Engine, a flexible and robust Rust framework designed to streamline the creation of LLM-powered agents. Whether you’re crafting a sophisticated chatbot, a tool-wielding assistant, or an application that leverages the privacy and power of local models, Helios Engine provides the scaffolding for your next big idea.

This post will dive into the core features of Helios Engine, showcasing how it can accelerate your journey into the world of AI with Rust.

What is Helios Engine?

Helios Engine is a versatile Rust crate that simplifies the development of applications powered by Large Language Models. It’s built to be both a command-line tool for rapid prototyping and a library crate for seamless integration into your own Rust projects.

One of the standout capabilities of Helios is its intelligent and automatic handling of different language models. You can effortlessly switch between powerful online APIs and offline, GGUF-based models running locally on your machine. This dual-mode support offers the flexibility to build applications that can function with or without an internet connection, prioritizing privacy and cost-effectiveness when needed.

A Universe of Features

Helios Engine is packed with features designed to make your development experience as smooth as possible:

Seamless Online and Offline Model Support: Helios Engine shines with its ability to automatically switch between online API providers (like OpenAI or any OpenAI-compatible endpoint) and offline GGUF-based models.[1] This allows you to build resilient applications that can leverage remote models for heavy lifting and local models for privacy and offline functionality. The engine can be configured to run in “auto” mode, intelligently selecting the best available model, or locked to “online” or “offline” modes to suit your needs.

Flexible Agent System: At its core, Helios Engine provides a powerful agent system. You can create multiple agents, each with its own unique personality, capabilities, and context.

Extensible Tool Registry: Breathe life into your agents by giving them tools. The extensible tool system allows you to add custom functionalities, enabling your agents to interact with external APIs, perform complex calculations, or access proprietary data sources.

Serve Your Own Agents with Custom Endpoints: Helios Engine is designed to be a core component of your larger applications. You can build and serve your own agents, and with its configurable nature, you can define custom endpoints in a config.toml file. This means you can point Helios to your own self-hosted models or other OpenAI-compatible APIs, giving you complete control over your AI infrastructure.

Advanced Agent Capabilities: The flexibility of the agent system opens up possibilities for more complex interactions, such as chaining agents together. You could have a primary agent that delegates specific tasks to specialized agents, creating sophisticated workflows to tackle complex problems.

Comprehensive Chat Management: With built-in conversation history and session management, Helios Engine makes it easy to maintain context in your chat applications, leading to more natural and engaging user experiences.

Real-time Interaction with Streaming Support: For a more dynamic and interactive feel, the engine supports real-time response streaming. It can also detect and display “thinking tags,” offering a glimpse into the model’s reasoning process as it formulates a response.

Built for the Rust Ecosystem: Helios Engine is asynchronous by design, built on top of the high-performance Tokio runtime. It leverages Rust’s strong type system to ensure your code is both safe and reliable.

Getting Started with Helios Engine

You can start using Helios Engine in two main ways: as a command-line interface for quick interactions and management, or as a library integrated into your Rust project.

For a quick start with the CLI, you can engage in interactive chat sessions and manage your configurations directly from your terminal.

To build your own applications, you can add Helios Engine as a dependency in your Cargo.toml file. Here’s a quick example of how you can use the library for direct LLM calls:

use helios_engine::{LLMClient, ChatMessage};
use helios_engine::config::LLMConfig;

#[tokio::main]
async fn main() -> helios_engine::Result<()> {
let llm_config = LLMConfig {
model_name: "gpt-3.5-turbo".to_string(),
base_url: "https://api.openai.com/v1".to_string(),
api_key: std::env::var("OPENAI_API_KEY").unwrap(),
temperature: 0.7,
max_tokens: 2048,
};

let client = LLMClient::new(helios_engine::llm::LLMProviderType::Remote(llm_config)).await?;

let messages = vec![
ChatMessage::new("system", "You are a helpful assistant."),
ChatMessage::new("user", "Hello, who are you?"),
];

let response = client.chat(messages, None).await?;
println!("Response: {}", response.content);

Ok(())
}

For more in-depth examples, including how to set up and utilize the agent system and local models, be sure to check out the official documentation.

Why Helios Engine?

Helios Engine offers a compelling and comprehensive solution for Rust developers looking to build sophisticated AI-powered applications. Its seamless integration of online and offline models, coupled with a flexible agent and tool system, provides a powerful foundation for a new generation of intelligent software.

If you’re ready to start building with LLMs in Rust, give Helios Engine a try. Head over to the Github : https://github.com/Ammar-Alnagar/Helios-Engine and the crates.io https://crates.io/crates/helios-engine page to learn more and get started on your journey.