BotPsy: On-Device Personal Assistant with a Sub-Billion-Parameter Agentic Language Model

BotPsy: A Small‑Scale, On‑Device Personal Assistant

BotPsy is an open‑source, 0.8 billion‑parameter language model built on Qwen 3.5‑0.8B, tuned specifically for function‑calling and agentic workflows. Its design targets on‑device deployment, offering a privacy‑first alternative to large cloud LLMs while delivering state‑of‑the‑art tool‑use performance for its size class.

---

Why a Sub‑Billion‑Parameter Assistant?

Frontier LLMs consume significant compute, incur unpredictable operational costs, and expose users to vendor lock‑in and data‑privacy risks. Many everyday tasks—search, calendar management, simple commerce queries—do not require the raw power of multi‑billion‑parameter models. A compact model that runs locally can:

• Cut costs – inference runs on‑device with minimal energy and no recurring cloud fees.
• Protect privacy – user prompts, credentials, and tool outputs never leave the device.
• Enable customization – developers can fine‑tune or adapt the model to niche workflows without proprietary restrictions.

BotPsy embodies this philosophy, delivering a capable assistant that fits comfortably on smartphones, edge servers, or embedded hardware.

---

Core Capabilities

| Capability | What BotPsy Does Well | |------------|-----------------------| | Tool Use | Executes single‑tool calls reliably across domains (web search, calendar, file handling, device control). | | Multi‑Turn Reasoning | Holds longer conversations, especially in commerce and device‑management scenarios, outperforming larger peers on several benchmarks. | | Instruction Following | Strong adherence to explicit prompts, with a notable jump in IFEval and IFBench scores after fine‑tuning. | | Safety | Reduces unsafe responses to 21 % of those produced by its base model, the lowest rate among ≤1.2 B‑parameter models. | | Efficiency | Generates answers with ~3× fewer reasoning tokens than the base model, cutting latency and memory use on‑device. |

The model still shows weaknesses in complex multi‑step information retrieval and in low‑resource languages, reflecting the natural limits of its parameter budget.

---

Training Pipeline in a Nutshell

BotPsy’s performance stems from a four‑stage training recipe:

1. Supervised Fine‑Tuning (SFT) – Ingests curated tool‑use dialogues, teaching the model when and how to call functions.
2. Multi‑Teacher On‑Policy Distillation (MOPD) – Three specialist teachers (tool use, reasoning, instruction) provide token‑level feedback, restoring knowledge lost during SFT.
3. First RL Stage – Rewards correct tool‑call format and partial credit for near‑correct arguments, reinforcing reliable function execution.
4. Second RL Stage – Broadens the reward landscape to include math, code, and multiple‑choice knowledge tasks, polishing general reasoning.

Each stage builds on the previous checkpoint, yielding a model that balances tool proficiency with broader language abilities.

---

Practical Deployment Tips

• Model Formats – BotPsy ships as Hugging Face safetensors and GGUF (bf16, Q8, Q4) for easy integration with llama.cpp, the QVAC SDK, or custom runtimes.
• Hardware Targets – Tested on modern smartphones and edge GPUs; quantized Q4 variants run comfortably within 2 GB RAM.
• Safety Guardrails – Pair the model with a lightweight policy filter to catch prompt‑injection attempts, especially when parsing untrusted tool outputs.
• Fine‑Tuning – Use the QVAC Fabric pipeline to adapt the assistant to domain‑specific tools (e.g., home‑automation APIs) without retraining from scratch.

---

Looking Ahead

Tether AI Research plans to expand BotPsy’s ecosystem:

• Decentralized P2P Search – Private, encrypted web queries that bypass corporate trackers.
• Cross‑Device Collaboration – Secure agent‑to‑agent communication across trusted local networks.
• Dynamic Offloading – Seamless delegation of heavy computations to a nearby PC or secure node when the device is offline.
• Enhanced Personalization – Deeper local fine‑tuning to capture individual user preferences while keeping data on‑device.

These road‑maps aim to transform BotPsy from a single‑device assistant into a fully autonomous, privacy‑preserving AI ecosystem.

---

Bottom Line

BotPsy demonstrates that a sub‑billion‑parameter model can deliver competitive tool‑use and agentic performance while offering the privacy, efficiency, and cost advantages of on‑device inference. For developers seeking a customizable, open‑source personal assistant that respects user data, BotPsy provides a compelling foundation.