Two Sessions at Once: Isolating Parallel Turns in an Agent Dashboard

People multitask with an agent the way they do with a browser: start something long in one tab, ask something short in another. In most agent products the second request waits for the first, or worse, its output appears in the wrong tab. This paper measures Sentinel after two changes: two inference slots on the model server, and a dashboard in which each stream is bound to its session. The claim under test is that two turns can run at once without one slowing the other to a stop and without their events mixing.

What came back

Session A was given a 220-word writing task. Session B was given "reply with one word" 1.5 seconds later. On the laptop, B's first token arrived at 3.3 seconds and B finished at 3.3 seconds while A was still generating; A finished at 7.2 seconds. On the Mac mini, where the model had just been reloaded, B finished at 57.5 seconds against A's 111.2, both paying a cold start and still overlapping. In both runs A's text did not appear in B's session and B's did not appear in A's.

Wall-clock timeline of the two turns on the laptop run. A (charcoal) starts at 0 and ends at 7.2 s; B (steel) starts at 1.5 s and ends at 3.3 s, inside A's run. Times are from tests/test_multitask.py, which records the first and last event of each stream.

What was serialising before

Two things, and neither was the agent loop, which has run one server-owned task per session since the start. The model server was configured with a single slot, so the second request queued behind the first and the second session saw nothing until the first finished. And the dashboard had one global "streaming" state: a single reader rendered events into whatever session was on screen, so switching tabs while a turn ran put the first session's tokens into the second's view, and sending from the second tab was refused as "busy".

The isolation test

The browser half of the test does what a person does. Start A, wait until it is streaming, switch to B, confirm B's input is enabled and A shows a working indicator in the list, send in B, read B's answer, switch back to A. Three checks have to hold: no word of A's topic appears in B's view; A's view shows each of its turns exactly once after re-attaching; A's final answer contains no word of B's. All three hold on both machines. Re-attaching works because finished turns are rendered from the saved session and the running turn is replayed from the server's live buffer, so nothing is rendered twice.

How far this is from a general result

• Two slots, not N. A third concurrent session queues. On a 16 GB machine two slots at 24,576 tokens each is the memory limit for the 8B model with an 8-bit KV cache.
• Two slots share one GPU: each stream generates more slowly than it would alone. We measured overlap, not throughput.
• Tool execution is not isolated between sessions: two sessions driving the same browser automation server can interfere. Files and shell commands are independent.
• One benchmark pair per machine, run twice. The timings vary with cold starts; the ordering (B inside A) did not.

What it means

Multitasking is a property of three layers at once: the loop must own the turn, the server must have more than one slot, and the client must bind streams to sessions. Fix two of three and it still feels serial. The decision rule for an operator: set the model server's parallel slots to the number of people (or tabs) you expect to overlap, size the per-slot window to fit memory, and test with two real sessions, not one.

The determination

A short turn sent 1.5 seconds into a long one answered in 3.3 seconds while the long one kept writing, and zero events crossed between sessions in either direction. Two slots and session-bound streams did that; the model did not change. People can work in two tabs now, and the test that proves it runs with every release.