Optional packages: Voice & Grammar
Enhanced abilities that ship with Embodied. Free, off until you turn them on, and model-free on the robot.
Embodied comes with two optional packages, Voice and Grammar. They give a machine a way to answer questions about its own experience in plain language, so a robot that already remembers what it has done can also tell you about it. You do not build this layer from the ground up. The base work is done and it travels inside the Embodied package, so turning it on is a setting, not a project.
Start here: a runnable example
The fastest way in is the talking-robot example that ships in the package. It needs no model and no robot. It seeds a robot's morning into episodic memory, turns Voice and Grammar on, asks it about its day, teaches one correction, adds a move of its own, and points you at the reason CLI. It is the whole loop in one file.
python3 examples/talking_robot.py
Read it top to bottom and you have the shape of a build: brain.remember(...) as the robot perceives and acts, converse_from_config(...) to turn talking on, voice.answer(...) to ask, the overlay to teach a correction, and an extra move for a question the base does not ship. The episodic memory is created with a retention cap so it stays fast and bounded as the robot runs for months. The sections below are the same steps in depth.
What each one does
Voice: the machine answers from its own memory
Voice is the reasoning layer that turns a question into an answer grounded in what the machine has recorded: where a thing is, when it last did something, how much of something is left, whether a reading is under a limit, whether a value moved, how many times something happened. It reads the robot's own episodic memory through one small seam, so it speaks for the real memory with no copy. When the memory does not hold the answer, it says so plainly instead of guessing, the same rule the rest of NeuraFrame follows.
where is the cup -> The cup is at the desk. is the battery under 50 -> Yes. Battery is at 40%, under 50. how much milk is left -> Milk: 2 left. where is the dog -> I looked, but I could not find anything about that.
Voice is the experience layer, not the controller. It reads memory and speaks. It never drives perception, planning, or actuation, and the safety gating in the core is untouched.
Grammar: the answers read cleanly
Grammar is the expression layer. It takes what Voice worked out and makes it read the way a person would say it: the right article, sensible plurals, a clean list, a finished sentence. It only ever polishes wording. It changes how an answer sounds, never what the answer is, so it can never turn a correct answer into a wrong one.
Each is its own package, trained on its own
Voice and Grammar are separate abilities, not a single bundle. Each is taught on its own board against the big model in simulation, and each carries its own correction: the loop that catches a wrong answer and fixes it for next time. Those corrections stay inside the package. Voice's correction never edits the core's verified memory, and Grammar's correction never touches Voice. Each layer owns its own truth and cannot corrupt the one beneath it, the same separation the core and the gateway already keep.
Because they are taught separately, they grow separately. A phrase Voice reads awkwardly is taught once and remembered. A wording Grammar gets wrong is corrected in a local overlay that cannot alter the shared baseline. You improve one without disturbing the other.
Turn them on when you are ready
Everything needed is already on the device inside the Embodied package. To turn the machine's voice on, wire the memory to Voice and, if you want it, Grammar. One call does the wiring and enforces the one rule.
# the robot writes what it sees and does into its episodic memory ... from nf_embodied_voice import EmbodiedBrain, converse_from_config brain = EmbodiedBrain("/var/lib/neuraframe/episodic.db") brain.remember("placed the cup on the desk") brain.remember("battery level 40%") # ... then build a voice from config. voice is off by default. voice = converse_from_config(brain, { "voice": True, # turn the robot voice on "grammar": True, # optional; needs voice "trace": "/var/lib/neuraframe/robot.trace" # optional reason log }) if voice: print(voice.answer("where is the cup")["answer"]) # The cup is at the desk.
With no flags it returns nothing and the robot simply does not converse. Set voice to true to switch it on later, add grammar when you want the wording polished, and set trace to keep a reason log. Nothing else in your stack changes.
The reason CLI: dig into any past answer
When a reason log is on, every answer is recorded with the reasoning behind it. The reason CLI that ships with the packages lets you go back and ask why the machine said what it said: which move fired, the terms it searched, and the memory the answer stood on. It is the deep-dive tool for a talking robot, so a past answer can be explained instead of guessed at.
python3 nf_voice_why.py --db robot.trace recent 10 # the last answers python3 nf_voice_why.py --db robot.trace find battery # answers that mention battery python3 nf_voice_why.py --db robot.trace why 42 # the full reasoning for answer 42
Foundations, not limits
Voice and Grammar are bases, not finished lists of tricks. They are the first two expression layers on the same seam and the same rules, and the seam does not care whether the memory underneath is a document, an account, or a robot's day. What they become is set by design, not by a ceiling in the software: their reach expands as far as the intent behind them does. More layers will sit on the same foundation over time. The base is deliberately open so it can grow.
How to train and extend them
The base is meant to be built on, so here is how. Growing Voice or Grammar uses the same teach, verify, correct loop as the rest of NeuraFrame, and it happens where teaching is cheap: at build time, against your model, which is then removed so nothing ships on the machine. There are three ways in, smallest to largest.
1. Teach it your words
Voice broadens its search using a short list of domain words, the cues. Add yours and it reads your vocabulary with no other change. A warehouse robot learns pallet, bay, forklift; a kitchen robot learns fridge, shelf, tray. This is the lightest way to point the same reasoning at a new domain.
ROBOT_CUES = ["battery", "level", "left", "location", "seen",
"pallet", "bay", "forklift"] # your words, added
2. Correct it on a board
Put questions and the memory they run against on a board and run the teacher. Voice answers, your model answers the same board grounded so it cannot bluff, and the harness shows where they differ. A wrong or awkward answer is corrected once and verified, and that is how the board grows. The verify gate matters: on one Grammar run the model was wrong on all twelve of its own article suggestions, and the gate caught every one, so a bad lesson never lands.
# build time only, on your training machine. no model ships. python3 voice_teach_r.py --llama-bin=... --llama-model=... # the robot voice python3 nf_grammar_teach.py --llama-bin=... --llama-model=... # grammar
3. Add a move
A move is a small, self-contained skill: it reads the facts and composes one kind of answer. Voice ships with where, quantity, threshold, trend, recall, occurrence, event, and one aggregation move (average and total over an optional time window, for example "the average battery today"). That aggregation move is written to be copied: an inventor uses the same shape to add sum, minimum, maximum, count, or any window they need. When your domain needs a question type that is not there, a "how far" question, a "who did I pass" question, you add a move and route to it. The engine and the seam do not change. You are adding a vocabulary of thought, not rebuilding the base, which is why the reach expands as far as the intent behind it. You do not fork the core to do it: pass your move in the moves list to converse_from_config and it is tried after the built-ins, so it can never override or break what already answers. The example's how_far move is exactly this.
4. Correct it in the field, and curate it across a fleet
After it ships, a correction does not need a rebuild. A wrong phrasing or a wrong plural is written to a local overlay that layers on top of the baseline and never edits it, so the fix takes effect at once and the baseline stays intact. With a Fleet, that correction can ride the same curated round trip as the core memory: it is collected, you review and approve it beside your other learnings, and it spreads to every robot, or you hold it local. Voice and Grammar are two of the three engines a Fleet curates. See Embodied Fleet.
Building on it: the controls you have
Everything an inventor needs is a handful of calls. Here is the whole surface, walked through. For a terse signature reference and the field-by-field event schema, see the CLI reference.
Record what it sees, and keep it bounded
The robot writes observations to its episodic memory as it perceives and acts. The retention cap keeps that history fast and bounded over months. Mark a record private and it is kept for the robot but left out of spoken answers by default, so the robot never volunteers it, and it is never shared to a fleet. Reach it only from your own trusted path with search(..., include_private=True).
brain = EmbodiedBrain("/var/lib/neuraframe/episodic.db",
retain={"max_events": 200000, "max_age_days": 90})
brain.remember("placed the cup on the desk")
brain.remember("resident in room 4 fell at 3pm", private=True) # kept, not spoken or shared
Record typed events, when you want reliability
Text is the anchor, but you can attach typed fields so answers stop depending on how a record was worded. The moves read value, unit, place, and subject directly when they are present and fall back to parsing the text when they are not, so nothing breaks and there is no migration. The six fields we use, etype, subject, action, place, value, and unit, are indexed, so field queries stay fast for years. The extra field is yours: any structured data you want, stored and handed back on every record, never touched by a built-in move.
brain.remember("battery status nominal",
etype="reading", subject="battery", value=40, unit="%",
extra={"cell_temps": [31, 32, 30]}) # extra is your toolkit, not ours
brain.remember("set it down",
etype="event", action="placed", subject="cup", place="desk")
Now "is the battery under 50" reads 40 from the field even though the text never says it, and "where is the cup" answers "the desk" from the place field. The record is also findable by its typed subject, place, or type, not only its wording.
Turn talking on, in your vocabulary
One call wires it. Add your domain words with cues, your own question types with moves, a reason log with trace, and a guard that decides what may be said aloud.
from nf_overlay import Overlay
overlay = Overlay("/var/lib/neuraframe/overlay.json") # where corrections live
voice = converse_from_config(brain, {
"voice": True,
"grammar": True,
"trace": "/var/lib/neuraframe/robot.trace",
"cues": ["pallet", "bay", "aisle"], # your vocabulary, merged with the defaults
"moves": [how_far], # your own move functions (a move returns an answer or None)
"speech_guard": my_guard, # a function that decides what may be said aloud
}, overlay=overlay) # overlay is a keyword arg, not a config key
Gate what it says
A speech guard sees every question and answer before it is spoken. Return a changed string to redact, or None to refuse. This is where private things stay private to the wrong audience.
def my_guard(question, answer):
if "where" in question.lower() and not viewer_is_trusted():
return None # refuse
return answer.replace(resident_name, "the resident") # or redact
Teach and undo corrections
A correction lands in the local overlay, on top of the baseline, and takes effect at once. You can list, remove, and clear them, so a wrong lesson is never stuck.
overlay.set("voice_answer", "how much milk is left", "There are 2 cartons of milk.")
overlay.counts() # what is held, by facet
overlay.remove("voice_answer", "how much milk is left") # undo one
overlay.clear() # wipe them all
Ship the whole mind
One file carries the core memory, the episodic experience, and the corrections, so a robot raised in simulation arrives acting, remembering, and speaking as it learned. This is "ship the memory" for all three stores, not just the core.
# after training, export the whole mind ExperiencePack.export("walker.mind", core_store_dir=STORE, episodic_path=EPI, overlay_path=OVL) # on the robot, restore all three ExperiencePack.import_("walker.mind", core_store_dir=STORE, episodic_path=EPI, overlay_path=OVL)
Ask why it answered
python3 nf_voice_why.py --db robot.trace recent 10 python3 nf_voice_why.py --db robot.trace why 42
Test your build
Before you trust a robot in the field, prove it answers your questions correctly. The check harness that ships with the package runs a board of your own questions against your Voice and reports pass or fail, with no model needed: you say what each answer should contain, and what it must not. Run it after any change to catch a regression.
from nf_voice_check import check
check(voice, [
{"q": "where is the cup", "expect": "desk"},
{"q": "is the battery under 50", "expect": ["40", "under"]},
{"q": "where is the dog", "expect": "could not", "reject": "desk"},
])
# PASS or FAIL per line and a count at the end; the exit code is nonzero on any
# failure, so it drops straight into a build step or a nightly check.
Where they come from
You do not download Voice and Grammar separately. They are inside the Embodied package, so any current Embodied build already carries them, ready to switch on. See the download page for the current builds, and CLI reference for the reason CLI in context.