When I read a technical paper, the interruption is often more expensive than the question itself. I may only want a term explained or a paragraph connected to an earlier idea, but answering it usually means leaving the page, copying text, opening another app, and then rebuilding my concentration.

I built AI Reading Teacher to make that interaction feel more natural. It is a small native Android companion that can see the page I have chosen to share and discuss it with me by voice. It works alongside an ordinary PDF reader, browser, or Zotero, so it does not require a new document library, import workflow, or reading interface.

The project is open source: No-drink/AI-Reading-Teacher.

What the experience is like

After adding an OpenAI API key in Settings, I tap Start Teacher and approve microphone and screen-sharing access. I can then return to whatever I was reading.

The teacher quietly follows the visible page. When I ask a question, it answers using both the conversation and the latest stable screen as context. The reply is played as realtime audio, while its transcript appears inside the app and, optionally, in a draggable floating subtitle over the reading app.

Several details make the interaction feel much closer to speaking with a tutor than operating a chatbot:

  • A page change updates context but never causes an unsolicited answer.
  • The teacher stays silent until voice activity detection determines that I have spoken.
  • I can speak over an answer to interrupt it and immediately ask a follow-up question.
  • The floating subtitle can be moved or hidden without leaving the document.
  • It works with the reader I already use instead of keeping a second copy of the document.

From a changing screen to useful context

Continuously streaming full-resolution screenshots would waste bandwidth, increase cost, and repeatedly send almost identical information. AI Reading Teacher therefore performs the first stage of screen analysis locally.

Every 100 ms, it compares a tiny 96 × 54 representation of the approved screen. When a change settles for roughly 250 ms, the app captures one full frame, scales its longest edge to at most 1600 pixels, and adaptively compresses it to fit the realtime data-channel limit. If I turn another page while that image is being prepared, the newer page supersedes the stale pending frame.

The important separation is that updating visual context and requesting a response are different events. A stable screenshot is sent as the newest context, but it never includes a request for the model to answer. Speech remains the only trigger for a response.

PDF / browser / Zotero
          |
          v
Android screen capture
          |
          v
local change + stability detection
          |
          v
latest compressed page context ----+
                                     |
microphone -- WebRTC realtime voice +--> spoken answer + live subtitles

Why WebRTC

The live session uses WebRTC for microphone input and model audio. On Android, this provides the pieces that are easy to underestimate in a voice product: echo cancellation, jitter buffering, speaker playback, and interruption handling.

When I start speaking during an answer, playback stops quickly. The realtime service also truncates audio that was generated but not actually played, so the conversation history stays aligned with what I heard rather than what the model attempted to say.

The transport is isolated behind a small provider interface. Screen capture, settings, the foreground service, and UI do not need to know the details of the Realtime protocol, which leaves room for another provider later.

Privacy and credential boundaries

Screen sharing is always initiated through Android’s system permission dialog and is released when the teacher stops. Screen-change detection happens on the device; only stable context frames are sent. Android secure-window and DRM protections still apply, so protected content may intentionally appear black.

The API key is encrypted at rest with AES-GCM using a non-exportable key generated by Android Keystore. Backups are disabled, and no credential is stored in the repository.

Version 1.0.0 is deliberately a no-backend MVP, which means the tablet uses a standard API key to create the realtime call. Encryption protects the key while stored, but it cannot make a client-side secret safe on a rooted or compromised device. I recommend using a restricted project key with a spending limit and revoking it if the device is lost. A production deployment should place the standard key on a backend and issue short-lived client credentials instead.

Try it

The source, build instructions, architecture notes, and tablet test checklist are all in the GitHub repository. The current build targets Android SDK 36 and uses gpt-realtime-2.1 by default; the model and API base URL remain editable because availability may vary by account.

The shortest test is:

  1. Build and install the debug APK on an Android tablet.
  2. Save an OpenAI API key in Settings.
  3. Start the teacher and approve microphone and screen sharing.
  4. Open a normal PDF or web page, wait for the page to settle, and ask a question about what is visible.
  5. Interrupt the answer with a follow-up question, then change pages and ask another one.

For the underlying realtime implementation, the most useful references are OpenAI’s Realtime WebRTC guide and Realtime conversations guide.

What comes next

This release intentionally avoids becoming another ebook platform. There is no document library, OCR pipeline, RAG database, account system, analytics service, or cloud synchronization. The narrow goal is to add a conversational layer to reading tools that already work well.

The most valuable next step is automatic realtime reconnection, followed by a backend flow for short-lived credentials. Beyond that, I want to learn from actual reading sessions before adding more features. A good reading tutor should reduce interruptions—not become one more complicated interface competing for attention.