Out of Sight

A robot looks around a room. Its camera sees only part of it. Hide a hand behind a pillar, and the robot’s brain keeps track of it anyway.

What it shows about CadenceA Cadence brain keeps a map of what is where in its own state: real memory of the room that lasts from moment to moment and runs in real time, like the brain of a living being keeps track of the things around it. Nothing outside the brain keeps a list.

What you see

The left half of the page is a room seen from above. The right half is the robot’s brain, live.

• The robot carries a camera. The lit area is what the camera sees, and in view it reports each thing’s position about a centimetre off.
• Out of view are the hatched shadows of the pillar and the arm, and everything beside or behind the robot. The camera reports nothing there.
• The robot’s arm has two links on a base. Its joints carry sensors that read their angles.
• Three things stand in the room: a hand, an apple and a slice of bread.
• The switch above the room picks the hand the brain follows: a person’s hand, which moves by itself, or the robot’s own hand at the end of its arm.
• The line under the room lists everything on the brain’s map and how the map holds it.

Why it matters

A robot that works near people has to know where their hands are, including the moments its camera cannot see them. Out of Sight shows a Cadence brain doing that with a map in its own state. The map is real memory: it lasts from one moment to the next without being fed the past again, and it changes only when the robot senses something.

Several things, each tied to its kind. The map holds an apple behind the robot and a slice of bread in front of it at the same time. Turn the robot away from the apple, and the apple stays on the map at its place. The bread joins the map the moment the camera first sees it. Nobody tells the brain what is in the room: it finds out by looking.

The robot’s own body. The robot feels its joints, the way you know where your hand is with your eyes closed. Those readings place its own hand on the map whether the camera sees it or not.

Questions by settling. Ask where the apple is, and the brain answers the way it does everything: its neurons drive each other until their activity rests, and the answer is read from the resting state.

What you can do

• Drag the hand behind the pillar, then bring it back. The map holds it near where it vanished.
• Drag the floor to turn the robot away from the apple, then toward the bread.
• Hide the apple in a shadow while the robot looks away, then turn back: the map finds its old place empty and holds it near there.
• Click a thing to ask the brain where it is; click the floor to ask what is there.
• Click the robot to switch its camera off; move things; switch it on again.
• Drag the pillar to make a new shadow.
• Choose “The robot’s own hand” and click a spot in a shadow: the arm reaches there. The arm moves only when you move it this way, or when a loaded scene file moves it.
• Rest the mouse anywhere in the room: a few words beside it say what a drag and a click do there. Rest it on a part of the brain, or on a ?, for an explanation.
• Tutorial walks through the main steps; Details opens the comparison chart; Load a scene (in Details) replays a recorded scene file (cadence-tracking-scene/1).

How it works

The map

The map is a sheet of 32 by 26 cells laid over the room, each about 6 cm across. Every cell has 16 units, and their pattern names a kind of thing: hand, apple or bread. A thing on the map is a small patch of active cells at its place, carrying the pattern of its kind. The map starts empty, and only the senses change it:

1. A thing enters the map the first time a sense reports it.
2. Seen: a thing the camera sees is written where it is seen.
3. Out of sight: a thing the camera does not see stays at its last known place. Nothing moves it.
4. Seen empty: if the camera sees that last place and the thing is gone, it moved while unseen. The map spreads it over the cells out of sight within 15 cm of that place. If there are no such cells, the map keeps the kind with no place.
5. Sight only takes away: once a thing is spread over a patch, every cell of the patch the camera sees is removed from it. The patch only shrinks.
6. Found: when the camera reports the thing again, the patch clears and the thing is written where it is seen.

The line under the room

seen

The camera reports it at this moment.

felt

The robot’s joint readings place it at this moment. This is the robot’s own hand.

heard

A sound places it. Sounds come only from loaded scene files.

held · 3 s

Kept at its last known place. The time is how long ago a sense last placed it.

near · 2 s

Its last place was seen empty, and it lies somewhere on the patch of hidden cells close by.

not known where

The map knows the thing by its kind and holds no place for it.

The brain

The right half draws every neuron and every synapse of the map: 29,168 neurons and 39,936 synapses, none of them taught. Each region sits where the part whose role it plays sits in a brain. Every region’s name carries a small question mark: rest the pointer on it and its card says what the region does at this moment. “Follow the position” walks through the nine steps a position takes through the map.

The map

The sheet of cells and kind units that holds what is where.

Replay

The map replayed for a question.

Kind

The kind a question asks about, such as “the apple”.

Where

One neuron a cell: where the asked kind is.

What

One neuron a kind unit: what stands at an asked place.

Clock

One neuron a cell: how long ago a sense wrote the cell.

Doubt

One neuron a cell: the cell holds a thing that was not seen to be there.

No place

One neuron a kind unit: a thing of the kind is known with no place.

Each moment of the room, 33 ms, the map settles in two sweeps; the time it took shows at the bottom of the brain.

The numbers

Averages over the page’s recorded test scene, three runs alike (evidence/promise_runs-20261113-0225-1879e461.json). The map’s patch held the true hand at every hidden moment. The page check: evidence/page_check-20261113-1051-1879e461.json.

Limits

The map moves nothing it does not sense. For a hidden person’s hand it holds the area where the hand can be, and the exact point inside that area is unknown. Predicting where an unseen thing goes, from movements seen before, is learned physics, planned for a later Cadence release. The room is flat and seen from above; a real robot sees in three dimensions.

Keys

With the room in focus: