Abstract interface for AI model runners. More...
#include <ai_runner.h>
Public Member Functions | |
| virtual | ~ai_runner ()=default |
| Virtual destructor. More... | |
| virtual std::string | process (const std::string &input)=0 |
| Synchronously process input and return generated text. More... | |
| virtual std::vector< double > | embed (const std::string &input)=0 |
| Generate an embedding for the given input. More... | |
| virtual void | unload ()=0 |
| Unload the model and free associated resources. –!Must be thread-safe!– and safe to call concurrently with process()/embed(). | |
Detailed Description
Abstract interface for AI model runners.
Implementations load / run / unload AI models and expose a minimal synchronous API for text processing and embedding generation.
Thread-safety requirements (MANDATORY for all derived classes):
- All public virtual methods (process, embed, unload) MUST be safe to call concurrently from multiple threads.
- Implementations must internally synchronize access to shared resources (model handles, contexts, samplers, caches, global backend state, etc.).
- unload() may be called concurrently with process()/embed(); implementations must either:
- defer actual teardown until in-flight calls complete (reference counting, call guards, condition variables), or
- make unload() idempotent and safe to call while other threads are active.
- The destructor MUST NOT cause undefined behavior when other threads are making calls; prefer explicit lifetime management (guards) or documented external synchronization.
Definition at line 42 of file ai_runner.h.
Constructor & Destructor Documentation
◆ ~ai_runner()
|
virtualdefault |
Virtual destructor.
Implementations should ensure safe destruction semantics in the presence of concurrent calls (see class-level thread-safety requirements).
Member Function Documentation
◆ embed()
|
pure virtual |
Generate an embedding for the given input.
Must be thread-safe.
Implemented in docwire::ai::llama::llama_runner, and docwire::ai::ct2::ct2_runner.
◆ process()
|
pure virtual |
Synchronously process input and return generated text.
Must be thread-safe.
Implemented in docwire::ai::llama::llama_runner, and docwire::ai::ct2::ct2_runner.
The documentation for this class was generated from the following file:
