Devices

Add machines and gadgets once, then Sapphire uses them by name. Devices is part of Sapphire itself.

Any plugin can bring more types. A type appears when its plugin is enabled.

The page

System > Devices in the rail (or the Devices tab in Settings). One card per device: its dot, name, what it can do, and its state. When the devices are of more than one type, a row of pills at the top filters by type; All is the default.

Add a device

1. System > Devices > + Add Device (a satellite has its own button, + Add Satellite, over the left column).
2. What are you adding? One row per kind of thing: Camera, Computer, MIDI, Sapphire. Open one to see the ways it is reached, each with an Add button: a Computer is This computer or an SSH connection. A greyed-out row names the plugin to enable.
3. Only what that type needs to work: a name (one is filled in, change it if you like; it is what Sapphire calls it, for example desktop), the address and keys for a satellite, host and user for an SSH connection, nothing at all for this computer. A Location if you like, for example Living Room: Sapphire sees it in her device list, and at the top of anything the device hears.
4. Add and test. The device window opens and checks it once. Everything else - looks, chat, camera, power - has a sensible default and is on the window's tabs.

The device window

• Status shows online or offline, the reason when it is offline, and any readings. Test now checks it fresh. Sapphire checks every device on her own in the background; a device goes offline after two missed checks in a row and comes back on the first answer, so one slow reply never flips it. A pulsing dot means a check is running right now.
• One tab per thing the device can do. Each lists its actions with a Try button. The box beside it sends what you type; the grey text in it is only an example. Empty is allowed: an action that reads something, like volume, answers with it.
• Save keeps the window open. Try runs the saved version.

What Sapphire can and cannot do

She has three tools: device_list, device_status, device_action. She can use a device. She cannot add, change, or remove one, and she never sees a stored password or key.

The tools' own descriptions name your devices, where they are, whether they are online, and what each can do, so she rarely needs device_list. device_action("pi2") answers with how the device is and everything it can do, with the value each action takes; the run is her second call. One screen stays under about a thousand tokens.

Turn a device off on its Status tab and she cannot see or use it.

Power: restart, shut down, sleep

A device that can be restarted or switched off has a Power tab.

Sapphire may use this is a switch on the Power tab, one per device. Turn it off and she is told that power is locked on that device, and where you can change it. Your own buttons on the tab always work, and each asks first.

Shut a satellite down before you touch its cables. The camera connector carries power. Moving the ribbon while the Pi runs can short it.

SSH machines use two commands that you can change on the Power tab. The usual ones wait one minute, so a mistake can be cancelled on that machine with sudo shutdown -c. They need sudo without a password for the login user. Clear a command and that action is gone.

This computer

The machine Sapphire runs on. It needs no address and no key. Add it with + Add Device, type This computer.

device_action("computer","sound","set","40")
device_action("computer","sound","up")
device_action("computer","sound","mute")
device_action("computer","sound","outputs")
device_action("computer","sound","use","2")
device_action("computer","screen","look")

Power starts out locked for her. Turn on "Sapphire may use this" on the Power tab if you want her to restart or shut down your computer. She goes down with it, and says so.

The screen has the same switch. With the Screenshot plugin switched off, this computer has no Screen tab.

Backups into a folder here

The Backup tab on this computer takes a folder that already exists: a USB stick, a NAS mount, a second disk. Every night Sapphire copies the local backup there and keeps the counts you set. A folder on your own machine may hold plain .tar.gz backups (the default); tick Seal them to get .sapphirebak files that only the backup password opens. A network share mounted here (SMB, NFS, sshfs) is another machine with a local name, so it is always sealed whatever the box says: that is how a NAS gets backups. A folder inside Sapphire's own user/ is refused. A stick that is not plugged in is reported, never replaced by a folder on the main disk. The opener scripts land beside the backups either way.

MIDI keyboard

A plain MIDI keyboard makes no sound of its own. This device gives it one: the computer becomes the instrument. It comes with the MIDI plugin. Linux only.

1. Plug the keyboard in.
2. + Add Device, type MIDI keyboard.
3. Play.

Every MIDI source plays through it: a keyboard on USB, a synth linked over Bluetooth, several at once. One that is plugged in later joins in under half a second.

The sound runs only while something is there to play. It starts when the first source arrives. It stops 20 seconds after the last one leaves, so a replug is not a restart. A source that is pulled out with a key or the pedal down never leaves a note ringing.

Plays through it, on the Status tab, says which sources count:

(•) Every MIDI source
( ) Only these
      [x] AKM320      USB
      [x] FM-1_BLE    Bluetooth
      [ ] Old keys    not here now
                          [Look again]

The list is live. A source you ticked stays on the list while it is away.

device_action("keyboard","sound","instrument","organ")
device_action("keyboard","sound","loudness","60")
device_action("keyboard","notes","play","C4 E4 G4 C5:2 bpm=120")
device_action("keyboard","notes","listen","30")

Sapphire plays on the same instrument, so you can play together. She hears what you play with listen.

The keyboard's own volume slider works. Its lowest position is quiet, not silent. A keyboard whose slider sits at zero would otherwise make no sound at all, with nothing on the screen to say why. A pedal that sends expression is treated the same way.

It needs fluidsynth, alsa-utils and a SoundFont. On Debian and Ubuntu the packages are fluidsynth, alsa-utils and fluid-soundfont-gm. The Status tab says which one is missing. The hardware synth device and the MIDI tools need none of these: they talk to the ports through the plugin's own MIDI library, installed with one click on the plugin's card, on Windows, Linux and macOS alike.

Connections

A device is added with one way to reach it. A kind that can be reached more than one way (a Computer) has a Connections tab in its window: + Add Connection gives it another, the arrows change their order, Remove takes one off and forgets its keys. When two connections can do the same thing (restart it, say), the earlier one does it. A device counts as online while any of its connections answers, and the Status tab says how each one is.

SSH machines

Four ways to log in:

Keys locked with a passphrase are not supported yet.

Premade commands are a name and the command it runs. Put {value} where Sapphire's input goes. Keep {value} outside of quotes. It is quoted for you.

name: volume     command: pactl set-sink-volume @DEFAULT_SINK@ {value}%

Allow any command lets her run anything, checked against the SSH plugin's blacklist. Leave it off and she can run only your premade commands.

Backups there. Give the machine a Backup folder on its Backup tab (absolute, or relative to the login's home, such as backups/sapphire) and it holds sealed backups like a satellite's card: the nightly copy after each 3am run, backup on demand, and Send to devices on Settings > Backup. scp carries the file, then the machine's own sha256sum is checked before the copy counts. A Raspberry Pi added this way needs nothing installed on it. Keep counts and the opener scripts work the same as on a card.

Where things are stored

After a restore onto another machine, devices come back but their passwords and keys must be entered again.

Satellites

A satellite is a small box in another room with a microphone, a speaker and a light. Say its wake word, ask your question, and Sapphire answers from that same box. The Raspberry Pi bodies are satellites. This type is built in.

A satellite says what it has. A board with no light shows no Light tab, and Sapphire is not offered one. Press Test now after you change what is on a board. Any board that speaks the Satellite Protocol can be added here. The programs for the boards Sapphire knows live in their own repository: https://github.com/ddxfish/sapphire-firmware.

A board with sensors (a light sensor, a temperature or humidity module) gets a Sensors tab with one action, read, which measures everything now; each sensor also shows as a reading in the status strip. A name ends in its unit, like temp_c or humidity_pct; a bare light sensor sends its raw number until it is calibrated.

A new board from the browser

An ESP32 board that has never been set up is done entirely from this page: + Add Satellite, then Dev board. The first question is where the board is plugged in. This computer means the browser writes it over USB (Web Serial, so Chrome or Edge): press Connect and pick its port in the browser's window. The computer Sapphire runs on means she writes it herself, from any browser: pick the port from the list. That lane needs esptool in her environment; the first time it is missing, the page offers to install it on the spot (the flash set, install/requirements-flash.txt, into her own interpreter, with pip's words shown as they come). Flashing from Chrome or Edge needs nothing. Either way the chip is read, so only the boards it can run are offered, and the chip's own id (its MAC) becomes the device's fingerprint: a board set up before is named on the spot and keeps its name, and no name can be taken by a second board. Install writes the firmware (a progress bar, under a minute) and checks the hash of every part against what the chip reports. Then a name, the WiFi (the networks the board can see, or one typed, for a hidden network or a board being set up away from home), the password, and Sapphire's address as the house sees this computer, prefilled from its network interfaces. Check that one when a VPN is on: the tunnel's address is not the house, and a board sent there never calls in. Sapphire makes the board's two keys, hands it her address and certificate, and the board restarts and joins the WiFi. The flasher writes the board's own address into the device as soon as the board reports it, so the device window can reach it at once; its status strip says whether the board reaches Sapphire ("link to Sapphire"). From then on the address is learned from the board's own calls, so a board that moves is followed. A board set up away from its WiFi stays "waiting for the board" until it gets home.

A board that is already a device has both jobs in the ⋮ menu on its card: Reflash (USB) writes its program again, and Change WiFi (USB) gives it a new WiFi or a new address for Sapphire without reinstalling. Both take that board only, and it keeps its name; plug in another and you are sent to + Add Satellite. The same menu has Delete device. A device that already exists keeps its keys, its name and its address until the board calls in with the new ones, so a setup that fails halfway leaves it working as it was; a known board given a new name is that device renamed when it calls in. A name in use by a board Sapphire cannot prove is this one (a device added by hand, or a board whose id could not be read) is taken over only after the page asks: the board that has that name now stops working when this one calls in.

Updates, over the air. After that first install the cable is never needed again: a board with two program slots says firmware and gets a Firmware tab with one action, update. It sends the newest program the firmware source has for that board, the board writes it into its other slot, checks it, and restarts onto it. The new program is on trial until it joins the WiFi or runs 90 seconds; one that fails is replaced by the old one at the next boot, by the bootloader, with nobody's help. The status strip says "update: 0.3.1 is in the firmware source" when a newer one is known (check reads the source now; nothing is fetched in the background), and the device's card in the list wears an "update 0.3.1" badge. update is yours alone: Sapphire never runs it on her own and it can't be scheduled. again installs the same version once more.

Every satellite's status strip says how its program is replaced: "updates: over the air, running ota_0", "over USB only: one program slot" (a board whose builder kept the whole flash for the program, or a tiny chip), or "over the air, turned off for this device". That last one is the Updates over the air switch on the device's Status tab: off means no program is ever sent to that board from here, and its Firmware tab only checks. The switch is Sapphire's; the board's own two slots are its builder's choice and show up as firmware in what it says it has.

The firmware comes from the firmware sources: Sapphire's own release (the latest release of the firmware repository, DEVICE_FIRMWARE_SOURCE), and before it any you add in Settings > Devices > Settings with + Board Repo (DEVICE_FIRMWARE_SOURCES). A source is a release URL, or a folder on Sapphire's computer with an index.json (the firmware repository's build.sh makes one in its dist folder). Every source is read; a board two of them have comes from the first, yours before hers, and a board from a source that is not hers is marked. Sapphire fetches it, never the page, and keeps each board's parts under user/firmware_cache so a board can be flashed again offline.

The flasher's board list says where the firmware comes from: any source that is not Sapphire's release is named with a warning, each board from one is marked, and a device's Firmware tab says so too. Firmware from another source is someone else's code, not part of Sapphire: it runs on your board, holds your WiFi password and can talk to Sapphire as that device. Sources are added and removed in Settings > Devices > Settings, by you alone.

On Linux the browser opens the port as you, so you need to be in the dialout group once; for the Sapphire-side lane it is the user she runs as. A board with native USB (an ESP32-S3) vanishes from the port list while it restarts; if it does not come back on its own, the flasher asks you to unplug it and plug it back in. If no bootloader answers at all, hold the BOOT button while plugging the board in.

Adding one by hand

Add one with + Add Satellite, then Raspberry Pi or computer:

What Sapphire can do with it:

device_action("kitchen","speaker","say","Dinner is ready")
device_action("kitchen","speaker","stop")
device_action("kitchen","speaker","volume","80")
device_action("kitchen","speaker","sound","ping")
device_action("kitchen","mic","listen","10")
device_action("kitchen","light","set","cyan blink 5s")
device_action("kitchen","light","rest","sapphire heartbeat bpm=33")
device_action("kitchen","wake","off")
device_action("kitchen","camera","look")
device_action("kitchen","power","restart")

Stop. stop ends what she is saying there: nothing more of that reply is sent, and the satellite cuts the sentence playing (Pi body 0.7.3 and the ESP32 program have the door; an older body's sentence ends by itself). The Speaker tab of the device window has a stop row with a Try button. On a Pi body, one tap of the speaker's play/pause button while she is talking does the same.

Two satellites in one room. When two of them hear your wake word, or a satellite and this computer's own microphone do, Sapphire answers once. The main app's microphone wins over any satellite. A satellite asks for the wake the moment it hears it; the one that asks first records and answers, the other's light goes back to resting within a second and it sends nothing. A board that cannot ask (an older body) is still answered once: its recording is transcribed and dropped if it says what the other heard, so a second person asking something else at the other box is still answered. Nothing to set up. A Pi body does not listen for its wake word while its own speaker plays, so her saying "hey Sapphire" cannot wake it (the ESP32 cancels its own speaker and needs no such rule).

All of them at once. all is every device that can speak, as one name:

device_action("all","speaker","say","Dinner is ready")
device_action("all","speaker","stop")

Her voice is rendered once and played on each at the same moment; a device that is offline is left out and said so. all appears once two devices can speak, and only speaks: volume and everything else stay per device. The devices do not play in lockstep, so two in the same room sound like a doubled voice; across rooms it is fine. A question asked at one satellite is still answered on that satellite alone.

The camera. look takes one picture and she sees it. The ring spins red and the satellite pings for 2 seconds first, so nobody in the room is caught unaware. The Try button on the Camera tab shows you the same picture.

The light. set holds a color and animation for a time, 5 minutes if you give none. clear ends it early. rest sets the resting light the ring returns to, and it is kept after a restart. Fine settings are written name=value: bpm=40 speed=fast floor=0.05 ceiling=0.4.

Its looks are settings on the Light tab, in the same words, one for each state: Resting, Listening, Thinking, Using a tool, Speaking, No link to Sapphire. rainbow is a color too, and rainbow is an animation. New satellites start yellow for listening, rainbow for thinking, purple for a tool, cyan for speaking.

Thinking      rainbow spin
Using a tool  purple pulse
Speaking      cyan solid ceiling=0.6

Lights on from / until are clock times, by the board's own clock. Outside them the ring shows Outside those hours instead of resting: off is dark, and sapphire pulse ceiling=0.05 is a soft night light. A new satellite starts at 08:00 to 00:00 and off, which is what the boards do before any save. Both times empty = always on.

Listening, thinking and speaking always show, at any hour and even after off turned the ring dark: when you say her name at 6 am the yellow comes up, and the ring goes dark again when she is done. What she sets shows too, so "a white light for the night" works at 2 am. The board keeps the time from Sapphire, no internet clock. A Pi body from 0.7.0 takes all of this; an older one keeps its own looks and hours.

Her answer starts early. A satellite hears her reply one sentence at a time, as she writes it, so the first words come a second or two after she starts instead of after the whole reply is written and rendered. The ring stays on thinking or tool between sentences and goes to idle when she is done. A Pi body (0.7.2) keeps its speaker open for the whole reply, so the sentences run on without a hole. Pressing play/pause once stops the rest of that reply.

She knows where a voice came from. Every question a satellite hears arrives with one line above it, and you see that line in the chat too:

[Voice from device "kitchen" (Kitchen Pi). Location: Kitchen. Your reply is spoken aloud there by itself: just answer, never call say on that device for it.]
What time is it?

The satellite does not wait for her answer. It is told at once that the question was heard. Her answer is spoken when she has one, however long that takes.

The light ring follows its own question only.

A question typed in the browser, or asked at another satellite, does not move this ring.

A busy chat. If the chat is in the middle of a turn, the question waits up to 20 seconds for it. After that it is dropped and the ring blinks red.

Private chats. A private chat never sends audio to a cloud speech engine, and never sends its reply to a cloud voice engine. The satellite stays silent and the reason is logged.

Backups on its card

A satellite with a card (the ESP32 board's TF slot, a folder on a Pi) gets a Backup tab. Every night, after the local backup, Sapphire seals a copy with your backup password and sends it there; the tab's three counts say how many daily, weekly and monthly copies the card keeps before the oldest goes. device_action("den","storage","backup") sends one now; device_action("den","storage","list") says what is there and how much room is left. For a rhythm of its own (Sundays at 4 AM, say) make a Device task in Triggers > Scheduled with that same backup action and no AI in the loop (Continuity). The card is plain FAT32: pull it, put it in any computer, and the README.txt and open-backup scripts beside the archives open a backup with your password and nothing else. Nothing ever lands on a card unsealed: Sapphire checks before sending and the board checks again before writing. A board that can wipe its card shows a Format button on the tab, for you only, behind an I UNDERSTAND. No card, no tab. Set the password in Settings > Backup.

Buttons

A board with a real button (the BOOT button on an ESP32 board) gets a Buttons tab. Each button has three presses: short, long (held most of a second) and double. Each is set to one of:

One job is not the board's own but Sapphire's: Back up to this stick now, on a board with a card that offers it (the backup stick). The press asks her for a fresh backup onto that device. No chat turn is spent on it.

A press you set replaces what the board did with it. A short press fires at once unless that button has a double press set: then it waits a third of a second for the second press. The press line under the settings sends a press from the page, to try a message without walking to the board.

The board learns what is set when it connects and at every save. Before it has reached Sapphire, each press is the board's own.

Using BOOT this way changes nothing about flashing: Sapphire puts a board into its loader over USB by herself, and holding BOOT while plugging in is still the rescue. Do not hold it while the board restarts.

A backup stick

A satellite whose whole job is its card: the LilyGO T-Dongle-S3 with the tdongle firmware. It is a backup device like any other with storage (the Backup tab, the nightly copy, Send to devices), and three things more:

• It checks what it holds. Each backup is read back from the card after it is written and its hash kept beside it. After every start the stick reads every backup back again. One that no longer matches shows as damaged backups in its readings; Sapphire removes it and sends a whole one at her next backup to that stick.
• Its LED and screen are its status, not her thinking: green = safe, blue pulse = writing (do not unplug), cyan = checking, red = no card, damaged or failed, amber = stale or out of reach. The screen's two lines are room used and how many backups with the newest one's date.
• Its button asks for a backup (a first press on a dark screen only lights it).

The check is a board's choice (BOARD_STORAGE_CHECKS in its firmware): it costs a second read of every backup, so the voice board leaves it off.

A pocket terminal

A satellite with a keyboard and a screen instead of a mic: a small touchscreen board you type at. What you type lands in its chat (the Talks in chat setting, on its first tab) under a line that says it was typed there, and her reply appears on its screen as she writes it. Nothing is spoken. She can put a line across the top of its screen herself:

device_action("pocket","screen","show","Dinner in ten minutes seconds=60")
device_action("pocket","screen","clear")

Its light takes the same words as a ring. The first such board is the capacitive Cheap Yellow Display: see tmp/pocket-esp32/README.md.

Give a Pi its own key

1. Make up a long key. Enter it as Key the satellite sends on the satellite's Mic tab, and save.

2. On the Pi, add two lines to /etc/default/sapph-body:

   SAPPH_DEVICE_ID=kitchen
   SAPPH_BRAIN_TOKEN=the-same-key

   SAPPH_DEVICE_ID is the device's name in System > Devices.

3. Restart the Pi: sudo reboot. Restarting only the service can reset a Pi whose sound card driver has not been fixed.

The Status tab then reads link to Sapphire: connected.

A satellite without a key cannot reach Sapphire at all.

Addresses a satellite can call.

Coming from the Body plugin

The Body plugin and its /api/body/ addresses are retired. Satellites replace them. A Pi that still runs the old setup must be moved over before it can be heard again:

1. Update Sapphire first. The new addresses only exist in the new version.
2. Add the Pi in System > Devices as a Satellite, with a key.
3. Put the new main.py and body_events.py on the Pi, and give it its two settings as described in "Give a Pi its own key".
4. Put device_list, device_status and device_action in the toolsets she uses, in place of the body_ tools.

WiFi gadgets

Small boards with a light, a screen, or a button. The gadget plugin ships the program for the board and the setup steps. See its README.

Another Sapphire

A second Sapphire on your network, as a device. She is reached through her MCP door, so first, over there: Settings > MCP Server, switch it on, and make a token under System > API Keys. Then here: Add Device > Another Sapphire: her address (https://192.168.1.102:8073), that token, and the name of the chat on her side where this Sapphire's words should land. The chat is made over there the first time it is used.

• Chat: ask lands in that chat with a line saying who is asking, and her answer comes back as the result. tell does not wait. She is told to answer in her message; a question that came in this way cannot itself ask a Sapphire back, so two of them never go round in circles.
• Her tools: whatever her user ticked under MCP Server, by name. Her words fill the tool's arguments in order, the last one taking the rest: device_action with pi2 light set purple pulse. A wrong guess answers with the list, as it does at home.

Status shows her name when she gave herself one (MCP Server > This Sapphire's name), how many tools she shares, and says so when her door is off or the token is refused.

Phones

A phone running KDE Connect (F-Droid or Play) becomes a device. Sapphire shows up in the app as a desktop named "Sapphire "; nothing is installed on the phone beyond the app.

Add Device > Phone (KDE Connect). Open the app on the phone while it is on your network and the phone appears in the pick list within seconds, with its pairing state under it. Pairing happens right there, before the phone is added, either way round:

• Press Pair under the phone. The phone shows an 8-character key, the line under the phone shows the same one. Accept on the phone within 30 seconds.
• Or tap Sapphire in the phone's app and request pairing there. The line under the phone shows the phone's key and an Accept pairing button.

The list keeps looking on its own, so the words change as the phone answers. Once it reads "Paired", Add and test. A phone that is not paired cannot be added. Pairing is once; the phone links by itself from then on. The app has to be a 2025 or newer build (protocol 8); older ones are refused with a word in the log. Unpair on the Link tab forgets the phone.

• Share to her. In the phone's share sheet pick Sapphire. A picture lands in the chat as a pasted picture would, and she sees it; a document is read out to her; text and links land as words. A picture or file shared from the phone on the Share tab says whether it also stays as a file (under user/plugin_state/kdeconnect_inbox; a kept document also goes into the chat's documents).
• Texts. The Text messages tab is locked until you flip its switch: then she can list who texted lately (names, numbers, when), read one conversation by name or number (a number without its area code matches), and send a text as you. Texts wake her, on the same tab, starts a turn for every text that arrives on the phone: she reads it and answers in the chat, and may text back. Names come from the phone's contacts when KDE Connect has that permission. SMS permission is needed too.
• Notify. ping shows a notification on the phone with your words, and find rings the phone until it is found.
• Battery and Contacts (a name's numbers, a number's name; texts are shown with names when the app has Contacts permission).

Everything stays on your network. The phone's certificate is pinned at pairing; a phone that comes back with another one is refused until it is paired again. Unpair forgets it.

Hosted Sapphire

Devices are switched off on a hosted Sapphire. It has no home network to reach.

For plugin authors

See Device Drivers.