package-language-reader-runtime/README.md

# Reading package

This package implements the Chinese `language-reader` workspace. Its independent
contracts are LanguageReader, ReadingReader, ReadingDocument, ReadingFeature, ReadingJob,
ReadingVocabulary, ReadingWord, and ReadingOwner. Baseline Studio, the workspace agent, and checked
source activation remain available. Reading entities expose AgentDocumentation;
read `guide` to find this resource and its extension points from an entity.

## Run the workspace

From the Quixos repository:

```sh
nix run .#camino-dev -- start --template language-reader --dir /home/timothy/qx-language-reader --agent
```

The command prints a local browser launch link. Reusing the directory resumes
the same workspace and its independent authoring workbench. A different
directory creates a fresh workspace from the Chinese template.

## Entities

LanguageReader owns documents, annotation features, vocabulary, and durable jobs.
Documents preserve their exact original text and source attribution. Their
Canonical sourceRevision is a content hash; variant revisions identify an immutable
adaptation. All ranges use original UTF-16 offsets in that document.
Creating a rewritten document retains the original document. Features contain
presentation preferences and a processor key; schemas and implementations belong
in code. Feature changes use expectedRevision to protect concurrent edits.

Inference is admitted through a persisted narrow Agent. Each scope request owns
a thread and request; managed graph edges retain those associations. Completion
subscriptions validate structured output and record results without an open
reader tab. Reading annotations default to Luna without reasoning and fast
mode. Cache keys include source revision, semantic configuration, scope, instructions,
output schema and actual context. Model routing/profile and scheduling preferences
do not invalidate completed semantic results. Results persist indefinitely until
explicit regeneration or deletion; attempt history is separate from the indexed
canonical successful result. Declarative model processors retain
results across unrelated runtime and presentation edits. Local processors and
prompt closures conservatively include the package source fingerprint. `ReadingFeature.processorRevision` exposes the checked processor identity
without copying its implementation into graph state. Old processor results are
hidden and old in-flight completions become stale after source activation.
Duplicate requests reuse the job, including a recorded
failure; explicit retries are bounded. Old feature results cannot replace newer
configuration. The document source is untrusted data, not developer instructions.

## Add an annotation

Edit `src/processors.ts` or import a new implementation there. A processor
declares its label, computation strategy, scope, and instructions. A local
processor returns source ranges directly. Section processors produce at most
180 ranges in one pass over a bounded section. Focus processors explain a word
or sentence and use neighboring text as context. Chinese initial features are
in `src/examples/chinese.ts`; French implementations are source examples in
`src/examples/french.ts`. Choosing another language is a workspace template or
source change, not an in-app preset switch.

Generation and reveal are independent. A single section-wide tagging pass can
prepare before reading. A feature requiring one model call per word must be
focused, cached, and lazy. Heavy hover requests should wait for deliberate dwell;
cached or local annotations reveal immediately. Never start a per-word background
fan-out to satisfy a precomputation setting. Leaving hover discards interest in
its result; the durable request may still finish and populate its cache.

Reader settings use a large modal with a reader selector, General settings,
annotator navigation, and breadcrumbs. A document attaches to a ReadingReader;
settings apply to all its attached documents. Cloning copies settings and creates
independent annotator objects. General settings include size, maximum width
(0 is unbounded), CSS font family/fallback list, and background.

Premades insert enabled copies of existing processors. New annotators start with
a visual type, optional title and type-specific color/font suggestions, then a
natural-language behavior request. There is no model/content chooser. The agent
creates checked code, generates a title if needed, chooses preparation strategy,
and supplies any implementation-specific configuration controls.

One processor can present a cached item through several rendering hooks. Declare
`Processor.renders`, for example:

```ts
renders: [
  { style: "replace", reveal: "always" },
  { style: "tooltip", reveal: "hover", content: "source" },
]
```

Each hook keeps the original source range and data source. A replacement's color
already controls its ink. A source tooltip slices the original French text
locally, with no extra inference. `annotation-display.ts` filters/decorates the
cached facts before all hooks; `render-hooks.ts` projects their appearances.
Separate annotator objects can also share facts through `cache.identity` and the
same semantic scope/options, as pinyin and hover pinyin do. Generation stays with
the processor; reveal belongs to each rendering hook.

Premades currently create one feature from code-owned catalog metadata. QX object
initialization can install several edges from supplied references, including their
reciprocal relationships. It does not clone neighboring objects. To instantiate
a processor/renderer bundle as separate graph objects, create all of them with
distinct creation keys in one checked transaction and pass references between
seeds. `addFrenchReader` and `addReader(copy)` are working examples. No second
construction mechanism is needed for multi-edge initialization.

Text content loads before annotation history. Preparation waits for that history
to prevent cache misses while it is still loading. Planning and display-mode
changes have checked JavaScript implementations and run against the local graph;
server validation remains the same. Measurement and segmentation caches survive
page and annotation changes. Package source changes do not invalidate declarative
model results unless instructions, schema, semantic options or actual context
change. Update the task version in `processor-identity.ts` when changing the model
request wire shape or shared task construction; the text planner is fingerprinted
automatically.

Styles are overhead, highlight, underline, color, tooltip, sidebar, replace, bold,
and italic. Hover must not reflow text: labels, replacement, bold, italic, font
changes, sizes and spacing use permanent or click activation instead. This also
applies to custom render hooks. Tooltip/sidebar use hover or click. The renderer
preserves canonical source mapping for replacement spans:
selection/copy returns source text. Fitting words and short labels move together;
oversized spans may wrap. Processing contours cover intervening rows and paragraph
gaps with straight connectors, without diagonal spikes. The embedded feature author makes actual checked
source changes, activates a successful candidate, and registers the new feature
through LanguageReader.addFeature. It has source and graph capabilities; normal
annotation inference has neither. Their prompts and capability settings are
captured by the existing Agent admission implementation.

## Check and activate

Inside an authored workbench, use `qx-workspace check` in each changed resource
and then `root/`. Activate its successful report with `qx-workspace activate`.
Inspect the selected identity and new feature afterward. Preserve unrelated
source and concurrent edits. Never report successful activation after merely
writing files. Failed compilation or activation must preserve the working app.

Renderer code and source-selection tests live in
LanguageReaderPresentation/src/canvas and tests/canvas.test.ts. Follow its
RENDERER_PROVENANCE.md when maintaining the recovered measured canvas approach.
Output schema changes require updating validation and consumer code together.

## Sources and read aloud

Import pasted text or UTF-8 text files, Chinese Wikipedia articles, or opening
excerpts from the existing Gutenberg catalog. Wikipedia imports retain their
source URL and CC BY-SA attribution; requests use TextExtracts, a descriptive
User-Agent, bounded results, caching, and timeouts. Difficulty rewriting creates
a new document, leaving the imported original intact.

Read-aloud settings store an ElevenLabs key and voice once in private workspace
state. The public graph returns configured status, never the key. Playback uses
the streaming text-to-speech endpoint for a selected passage, limited to 1,200
codepoints per request; it does not automatically narrate an entire book. The
network implementation is covered with mocked API responses; live paid audio
requires a configured key and voice.

Primary API references: [MediaWiki TextExtracts](https://www.mediawiki.org/wiki/Extension:TextExtracts),
[Wikimedia User-Agent policy](https://foundation.wikimedia.org/wiki/Policy:Wikimedia_Foundation_User-Agent_Policy),
and [ElevenLabs streaming text-to-speech](https://elevenlabs.io/docs/api-reference/text-to-speech/stream).

## Verification

`nix run .#language-reader-qualification` from the monorepo runs native graph
integration and real Chromium interaction tests. Package checks also run
processor/output validation, mocked network tests, canvas collision/Unicode
selection tests, and hover scheduling tests. The browser suite checks actual
CJK canvas metrics, pointer and keyboard copy, scrolling, bounded hover
admission/cache reuse, the visual editor, imports, and the embedded feature
conversation at desktop and narrow widths.

## Inline annotator configuration

Configuration belongs to annotator code and appears on its page in Reader settings.
`Processor.config` is a list of `ConfigField` controls, or a function returning that
list. Supported controls: `color`, `number`, `select`, `toggle`, `text`, with a key,
label, typed default, optional numeric bounds and select choices. The grammar
processor generates one color control per part of speech; names does the same for
entity types. Declare every option in code so the server can validate it.

`ReadingFeature.options` persists the JSON object. `configureOptions` checks
`expectedRevision`, validates options and increments `configRevision` atomically.
The UI edits a draft and saves all processor options together; appearance/reveal
controls save directly when the processor explicitly declares them. `addFeature` requires `options` (use `"{}"` for defaults) and optional `authoring` (use null for premades).
Concurrent edits reject cleanly. Changing display settings preserves raw cached results; only semantic inputs invalidate computation.

`Processor.prompt(config)` can include validated config in developer instructions.
Local processors receive config as their third argument.
The presentation `annotationDisplays` registry can return per-item `color`, `fontFamily`, or
`fontSize`, applied after strict model output validation. Model output stays the
four-field range/text/tag contract; rendering choices are owned by checked code.
Common options `fontFamily` and `fontSize` configure labels. Their style remains
`overhead`; `options.placement` chooses `"above"` (default) or `"below"`. A
`Processor.renders` hook can set `placement` independently. Both positions use
fixed source lanes, and display changes reuse semantic caches. Wide labels gain
faint alignment guides; hovering a label emphasizes its source without invoking
word-hover tooltips or processors. Different
fonts and sizes in the same annotation source stay on one row; each physical line
reserves its maximum ascent/descent and shares the resulting baseline.

The presentation renders declared fields automatically. For richer controls, add
a React component to `annotatorConfigViews[processor]` in
`LanguageReaderPresentation/src/annotator-config.tsx`; it receives `fields`,
`values`, `disabled`, and `change(key, value)`. This component edits the same draft
and validated options. Built annotators have no universal output-type editor.
`Processor.settings` explicitly opts into color, font, or timing controls;
`description` explains the behavior and `preset` defines a premade output.
The New annotator panel can reuse any installed processor type with its code-owned
controls, or describe a new type to the author. `LanguageReader.archiveType` takes
`{processor, archived}` and updates `archivedTypes`, hiding a type from the creation
library without changing its instances, implementation or caches. Restore it to
make it discoverable again. `ReadingFeature.remove` removes only an instance.
Creation choices are authoring instructions, not a universal runtime schema.
Set `addFeature.authoring` to the supplied `authoringJob` when registering a new
annotator; this links its original request and followups to that annotator.
`customize.feature` links a change request to an existing annotator.
`ReadingFeature.setEnabled` runs locally and preserves configuration revision
and cached annotations across toggles.
Do not bypass persistence or create per-word inference from a config component.

The French word-gloss and passage-translation processors declare `profile: "cerebras-light"`
to use GPT OSS 120B with low reasoning. Processor-owned profiles override the
workspace default and participate in cache identity. Cerebras must be connected.

The default inference profile is `quick-fast`: GPT-6 Luna without reasoning and
fast mode. `LanguageReader.inferenceProfile` is configurable through the graph and
captured for each request. Model-backed cache identity includes the effective
profile; local processors use `local`. Feature authoring remains on balanced.

Pending jobs mark the exact text segment with animated contours. This is
a rendering overlay, independent of layout, and respects reduced-motion settings.
Hover word feedback uses cached segmentation/layout immediately; heavyweight hover
work still waits for dwell and reuses durable results.

## Text processing and reading windows

`src/text-processing.ts` is the reusable source-indexed planner for words,
sentences, chunks and context. `ReadingDocument.plan` returns target ranges for
`kind: "focus"` or `"nearby"`; it never starts inference. The UI reports visible
and nearby source ranges, then uses the same admission/cache path for automatic
preparation, clicks, dwell hover and retries. Pages are the default view; the
Reading layout control also offers scrolling. Navigation persists mode and source
position independently of pixel layout. Selection and cache ranges always refer
to the original document, across pagination, font changes and reflow.

Defaults: focus targets at most 400 graphemes; context includes up to two sentences
before and after, capped at 1,600 graphemes. This also bounds unpunctuated paragraphs.
Automatic preparation uses stable chunks of at most 800 graphemes, preferring
sentence/word boundaries. Pages prepare the current page plus its neighbors;
scrolling prepares roughly one viewport above and below. Only two background
model jobs are admitted concurrently. Independent scopes from the same annotator
may use both slots; duplicate scopes reuse their existing requests. Unsent queued
scopes and admitted work both show processing outlines.
Moving away replaces unsent plans; admitted results finish and remain cached.
Local passes use the same chunks without model admission. Errors do not cause an
automatic retry loop. Focused interactions are independent of preparation slots.

Authors can override the defaults with `Processor.processing`, a partial policy
or `(config) => policy`. Available fields: `chunkChars`, `focusChars`,
`contextChars`, `beforeSentences`, `afterSentences`, `prefetchChunks`, and
`context: "nearby" | "document"`. For long-range references, a processor might
choose `{ context: "document", contextChars: 12000, chunkChars: 2000 }`.
Sizes have a generous 80,000-grapheme ceiling and radii a 20-unit ceiling.
These are author choices, not extra user settings. Large prompts/results still
obey transport and output bounds; choose batch outputs responsibly. Generation
rejects oversized targets so callers cannot accidentally bypass the planner.

`passageTranslation` demonstrates lazy article translation, producing per-sentence
sidebar notes in nearby chunks. Add it from Premade annotators. Adapt text creates
a linked lazy text variant for either translation or difficulty changes.
Hover feedback changes source ink immediately and never draws a word outline.

## Tooltip presentation

Features can use `style: "tooltip"` with `reveal: "hover"` or `"click"`.
Choose Tooltip in the annotation editor, then choose its reveal trigger. Existing
processors and generated/local content work unchanged; this is a presentation
choice. `always` is unsupported for tooltips. Hover inference retains dwell,
nearby preparation and cache behavior. Ready hover content opens immediately.
The source word remains highlighted by ink on hover, without changing layout.

`LanguageReaderPresentation/src/canvas/Tooltip.tsx` renders selectable React
content in a native manual popover (the browser top layer). It escapes canvas,
scroll-pane and component clipping while retaining component styles. Positioning
uses source glyph rectangles in viewport coordinates, flips above/below and clamps
to the viewport. Cards combine matching annotations and identify their sources.
Hover cards allow a brief pointer crossing to select/copy their text. This grace
period does not extend model hover intent. Click cards remain until dismissed or
the anchor leaves view. Escape, outside clicks and the close button dismiss them;
pagination, scroll and resize update or clear the anchor. No model work is owned
by the tooltip component and no HTML from model output is interpreted.

## Lazy text variants

ReadingDocument.rewrite creates a variant linked by canonical; it does not start
inference. adaptation stores immutable canonical ranges and completed outputs.
prepareVariant admits only the next section, bounded to 1400 graphemes, with
nearby context. Requests deduplicate and explicit retries are bounded to three.
Validated completions append in canonical order. Output length changes do not
change canonical section identities or invalidate ready prefix annotations.
The reader prepares ahead near the current page; ordinary annotators then work
on generated text. This path serves both difficulty adaptation and translation.
A failed section retains all completed text and offers an explicit retry. Closing
the UI does not discard an admitted request. Variants are separate documents and
inherit the source's reader, which can subsequently be changed.

## Embedded authoring

customize captures reader, requested configuration, and prose intent separately.
Jobs retain the reader link and a read-only configuration snapshot. Their threads
are collapsed beneath that reader's settings; opening one shows a React request
card rather than raw orchestration text, then ordinary follow-ups and questions.
ConversationTranscript separates rendering from its graph controller, and the
standard agent ThreadView also accepts renderItem for embedded request views.
Before follow-ups, the author must re-read current reader settings and feature
options; a request card records history and is not authoritative live state.

## Product language

Use concrete labels and direct copy. Avoid slogans, inspirational filler,
decorative all-caps branding, and boilerplate AI disclaimers. Explain empty states
with a useful next action. Caveats should identify a specific limitation briefly.
These directions also live in the default Workspace Agent prompt and harness.

## Semantic annotation caching

Raw model/local facts are cached separately from display preferences. The pure
cache.ts wrapper serializes processor identity, scope, generation and explicitly
selected semantic options. Source revision/range and model profile complete the
cache identity. Processor.cache.inputs opts into semantic fields; the default
selects non-color config fields unless computation:false. prompt(config) sees
only these semantic values. Changes to color, fonts, enabled state or display
filters must never admit another model request. Install custom filtering/styling
in LanguageReaderPresentation/src/annotation-display.ts, not model output.

Pinyin batches every word in stable 160-grapheme nearby chunks, with tone marks
and an estimated HSK level per token. The HSK slider filters cached facts locally.
Hover pronunciation uses the same computation identity, including sentence context,
so it does not request inference per word or duplicate the overhead pass. Two
background slots bound preparation; visible chunks precede neighboring chunks.
Context-sensitive annotation caches stay scoped to the canonical source/range;
only introduce bare-word reuse for a processor whose context-independence is
explicit. For ambiguous pronunciations/definitions retain context.

## Text library and generation

The library lists texts; readers are reusable configurations attached to them.
`ReadingDocument.archived` hides a text from the active collection without deleting
its source, variants, annotations or threads. Restoring it changes only that flag.
`LanguageReader.generateDocument` captures a reader and a bounded prose request,
creates a durable `ReadingJob` of kind `text`, and uses the quick-fast inference
agent. Only a validated `{title,text}` result creates and attaches the new text.
The UI can close while the request runs; completion survives independently.

`addFrenchReader` creates an independent reader with two click-only overhead
sources and a small French sample. `src/examples/french.ts` supplies per-word
contextual glosses for the selected sentence and a separate translation lane for
a few sentences. Focus planning bounds the source range and supplies neighboring
context. Clicking another passage replaces both visible scopes immediately.
Ready results remain cached; revisiting a passage reuses them. This scope-based
exclusivity belongs to click/focus annotations; ordinary word/sidebar annotations
continue to follow their source word. Hover tooltip dismissal clears only native
selection owned by the tooltip, so it does not intercept later canvas copying.

Model annotations echo `source` alongside `start`/`end`; it must equal the exact
original UTF-16 slice, including whitespace and punctuation. `text` remains the
rendered pronunciation, gloss or replacement. `validateAnnotations` collects all
indexed errors and strips the echo after validation. The job calls the shared
`AgentRequest.correct` operation for at most two same-thread corrections, staying
pending until valid or exhausted. Invalid results never enter the display cache.
Local processors still receive range/content checks without an echo. An author
may document a code-owned `verifySource:false` exception; `annotationSchemaFor`
then supplies the corresponding schema. This affects semantic cache identity.

The feature author uses `balanced-fast` (Sol, medium reasoning, fast mode).
Inference-based features must be tried on actual reader text or representative
sample text. Start with `cerebras-light` when connected; try Luna fast, then Sol
fast if the output needs a stronger model. Check returned ranges and content,
not just request completion. These are authoring choices, not reader settings.

## Reader extensions and vocabulary

Sidebar panels and word/selection context-menu actions are ordinary React reader
extensions, separate from annotators. `src/extensions.tsx` in the presentation
provides a shared `ReaderContext` and `ReaderAction`; `reader-extensions.tsx` is the
installation point. Actions can inspect the exact original range or cached annotation
tags. Context menus use the browser top layer and keyboard focus/dismissal. Drag
selection does not generate click annotations. Panels can include arbitrary UI.

`ReadingVocabulary` is a named graph list, shared through each reader’s optional
`vocabulary` edge (otherwise the library default). Its standalone editor supports
search, add/remove and rename. The SDK `ObjectPicker` presents checked paginated
lists and explicit selection, creation, preview and open-in-new-tab. Existing words
remain in their original storage; `ensureVocabulary` attaches unlisted entries to
the first default list through public contracts. Cloning a reader shares its list.

Settings show the current text alongside configuration. This preview projects
cached facts through current appearance/display options, never initiates inference.
Author new data models simply, and refactor to richer domain entities when needed
while preserving user values via supported transformations. Independently scoped UI
work can use bounded Codex subagents; join/check before activation.