Gayatri Local LLM API
High-Performance, Drop-In OpenAI API Replacement Powered by Local GGUFs
Architecture & Core Mechanics
For developers who want to run local language models with standard OpenAI-compatible client libraries without dealing with Ollama or heavy cloud runtimes. Gayatri Local LLM API inspects GGUF metadata to calculate RAM footprints, evicts models using LRU policies, and supports native XML and grammar-constrained function calling.
Engineering Highlights & Capabilities
-
✓
100% OpenAI Specification: Implements /v1/chat/completions, /v1/models, and /v1/embeddings. Works seamlessly with official OpenAI SDKs.
-
✓
Dynamic VRAM/RAM Memory Management: Automatically calculates memory requirements from GGUF metadata with LRU eviction and idle unloading.
-
✓
Native Tool Calling: Translates OpenAI function schemas into GGUF grammar constraints or XML routing (<tool_call>) for models like Qwen 2.5.
-
✓
True Thread Concurrency: Safely queues overlapping generation requests with threading locks preventing memory segmentation faults.
Verification & Test Telemetry
● 1-Click Windows run.bat · Zero external subscriptions · OpenAI SDK compatible
Technology Blueprint
Repository Access
Access source code, architectural test harnesses, and documentation under the Gayatri-Education organization.
Inspect on GitHub →Explore Other Systems
Interested in deploying or contributing?
Our products are deployed across universities and enterprise pilots. Get in touch with our engineering team.