docwire::ai::model_inference_config Struct Reference

Public Attributes
std::string	model_path
context_size	n_ctx {4096}
batch_size	n_batch {1024}
thread_count	n_threads {4}
token_limit	max_tokens {512}
temperature	temp {0.2f}
min_p	min_probability {0.05f}
bool	verbose = false
std::string	system_prompt = "Do NOT repeat the input. Answer concisely and directly."
std::string	grammar {}
std::string	grammar_root = "root"

Detailed Description

Definition at line 22 of file model_inference_config.h.

---

The documentation for this struct was generated from the following file:

• model_inference_config.h