Atari Arcade

Two Atari games run in your browser, Freeway and Atlantis. In each, a Cadence brain is born knowing nothing. It watches a teacher play, takes the joystick once it agrees with the teacher, and then plays the game alone.

What it shows about CadenceA Cadence brain learns during its life. Nothing is trained before the page opens: the brain learns each game in your browser in about a minute, from the screen's pixels alone, and then plays it on its own. It has a real state that carries from one screen to the next, and it runs in real time like the brain of a living being: the game never waits for it. In Freeway its own games score above its teacher's; in Atlantis they match the teacher's.

What you see

The page has one row per game. Each row is one game and its own brain.

• The row's header names the game, shows the brain's skill badge and its phase (being born, watching the teacher, playing for itself), and, on the right, the game number, the brain's best game and the teacher's average.
• Five numbers: how often the brain agrees with the teacher, the reward its critic expects, its lessons from the teacher, how many sweeps the last settle took, and the score of the game in progress.
• The game, live on an emulated Atari 2600. In Freeway the brain is the chicken crossing the road; in Atlantis it fires the three gun bases.
• The brain, drawn as a brain seen from the side, front to the left. Every lit dot is a neuron's measured activity; the fibres are synapses, lit by the signals they carry. The chips on top say what the brain is doing.
• The status line under the row: progress toward taking the controls, and then the brain's decisions and its time per screen.

Why it matters

It learns during its life. Game-playing networks are usually trained before anyone sees them play. This brain is born when you open the page, with random synapses. It learns the game in front of you, in about a minute, from the pixels of the screen, with no description of the game and no game memory. Then it plays alone and keeps learning from the game's reward.

It can do better than its teacher. In Freeway the teacher always presses up. The brain learns to press up too, but its own play is not an exact copy: over three measured brains of twelve games each, its games averaged 22.0 to 22.5 crossings against the teacher's 21.4, with single games up to 27. In Atlantis it matches the teacher's 2,000 points.

It runs in real time, like an animal's brain. The game does not wait for the brain. While the brain settles on the next action, the game goes on with its last one. Its working trace carries its state from one screen to the next, so each answer starts from where the last one ended.

One brain all along. The brain that watches is the brain that plays: there is no second model and no swap. For a builder of games or robots this means a character that learns on the spot from a demonstration, then carries on by itself.

What you can do

• Watch the takeover. At first the phase reads watching the teacher play and the teacher agreement climbs. When it reaches 90% after at least 500 lessons, and the brain has seen one whole game of the teacher, the phase changes to playing for itself.
• Compare scores. The header of each row shows the brain's best game and the teacher's average. The badge turns legend when the brain's last five games average above the teacher's.
• Slow the games down with the speed slider at the top, to watch single screens and the brain's answers.
• Rest the pointer on a question mark, or on any part of the brain, to see what it does. Click a question mark to keep its card open.
• Start again. Reloading the page gives birth to two new brains: nothing is saved. On a narrow screen only Freeway starts by itself; a button starts Atlantis.

How it works

The eye

7,056 sensory neurons, one per pixel of the screen shrunk to 84 by 84 grey pixels, less a fixed background taken before birth. One screen at a time, nothing else.

Association cortex

64 neurons that read the eye and settle together with the motor strip.

Working trace

64 neurons that hold the association cortex's activity of the moments before: the brain's short-term state.

Motor strip

One neuron per joystick action (Freeway: no move, up, down; Atlantis: hold fire, or fire the centre, right or left gun). The strongest one in the settled brain is the action.

Critic

The reward the brain expects from the screen.

Dopamine

The reward that came less the reward expected: the signal that changes the synapses at the controls.

Associative memory

The reward recalled for each action on screens like this one.

Settling. For each screen the whole brain, 7,187 neurons and 456,064 synapses for Freeway, settles: every neuron adjusts to the neurons it is connected to, sweep after sweep, until no neuron moves by more than a small tolerance. That agreement is the answer. A settle that does not reach it is refused: the game keeps the last action and the brain is credited with nothing.

Watching. While the teacher plays, the brain first gives its own answer to each screen (not played, only compared), then learns the teacher's action for that screen. That is one lesson.

The takeover. After at least 500 lessons, once 90% of its last 120 answers agreed with the teacher (or after 2,000 lessons at the latest), and after watching one whole game of the teacher, the brain takes the controls.

At the controls. Each decision is one call: the brain receives the screen and the reward since its last decision, learns from that outcome, and settles on its next action.

The brain is the released cadence-net 0.95.2 library, running in Python inside your browser (Pyodide), in its own thread. The game runs on an Atari 2600 emulator in a second thread.

The numbers

Three brains per game, in Python

The Python edition, brains from seeds 0, 1 and 2, each beside a frozen twin: evidence/headless-0952/.

On this page

The games: evidence/promise-20261113.json, one run of the page by tools/promise_test.py in a headless browser on a machine shared with other jobs (the takeover at 56 and 71 seconds, 60 to 63 ms per screen). The other takeover and timing readings: evidence/page_check.json (39 and 52 seconds, 79 to 81 ms). All files are in the atari-arcade folder of the repository.

Limits

• The teachers are simple. Freeway's teacher always presses up and Atlantis's always fires the centre gun. Matching them shows that the brain learns from pixels which action the teacher holds; it does not show that the brain understands the game.
• The edge over the teacher comes from what the brain learned by watching. A twin of each brain that stops learning at the takeover scores as well in Freeway and Atlantis. Learning from reward after the takeover keeps the skill; it adds no measured gain.
• Only some games. The Python edition also plays Space Invaders, where the brain's last five games beat the teacher's 225 in two brains of three (303 and 339) and fall short in the third, and Carnival, where all three brains fall below the teacher. Those games are not on this page.
• Atlantis is a tie. The brain scores the teacher's 2,000 in most games; a game can end lower.
• Timing depends on your machine. The game does not wait for the brain, so a slower device means fewer screens seen and a later takeover.
• Nothing is kept. The brain lives as long as the page is open. A reload is a new birth.
• A bootstrap. A lesson on every watched screen is a way to start a brain quickly. It is not the general way Cadence brains are meant to learn, which is from witnessed surprises during a continuing life.