Markdown edition

# Use Gemini 3.1 Flash Live to talk about a shared screen

> Build a local Gemini 3.1 Flash Live voice session that sends combined-desktop frames, checks spoken answers against the screen and stops capture.

By BIGCHANGE.AI Editorial

Published: 2026-10-24T22:47:38.174Z
Updated: 2026-10-24T22:47:38.174Z
Canonical: https://bigchange.ai/blog/gemini-31-flash-live-screen-voice-guide

![Conceptual charcoal illustration from behind one headset-wearing person facing two blank monitors on a shared desk.](https://bigchange.ai/api/media/file/gemini-live-dual-desktop-hero-v2.png)
AI-generated conceptual illustration by BIG CHANGE; no actual Gemini interface, session, screen capture, spoken reply or successful test is depicted.

Google's Gemini 3.1 Flash Live Preview can take microphone audio and still images during a Live API session and return spoken audio. This guide adapts a pinned Google Python example to send a JPEG of the combined desktop once per second. It is for developers who can review the entire desktop before sharing it and who have access to the exact model. BIG CHANGE checked the source and local setup, but did not connect to the model or test a device.

## The big change

**What:** A local Python session can pair spoken questions with desktop frames sent to `gemini-3.1-flash-live-preview` and play the model's audio reply.

**How:** Google's Live API accepts 16-kHz PCM microphone input and JPEG frames at no more than one frame per second; the example plays 24-kHz PCM output and stops on `q`.

**Why it matters:** The reader can check a reply against a visible screen change and end both capture streams. The whole combined desktop, including any visible notification, is in scope while the program runs.

## Choose the route that matches your task

Google's [Gemini Live Android help](https://support.google.com/gemini/answer/15274899) explains how to share a phone screen and stop sharing with the Live control or notification. It says the full device screen is shared, and muting the microphone may leave screen sharing active. That help page does not identify the model behind the consumer app, so it does not establish a Gemini 3.1 session. Camera sharing is a separate control.

The [3.1 model page](https://ai.google.dev/gemini-api/docs/models/gemini-3.1-flash-live-preview) also links a [3.1 AI Studio session](https://aistudio.google.com/live?model=gemini-3.1-flash-live-preview). If you use Studio, check the displayed model and whether your account shows both microphone and screen controls before sharing. The controls and account access are conditional. The local API procedure below pins the model in code and supplies the full microphone, screen and audio path.

Google currently calls 3.1 Flash Live Preview a legacy preview and recommends 3.8 for most new work. The [model page](https://ai.google.dev/gemini-api/docs/models/gemini-3.1-flash-live-preview) still lists the 3.1 model ID and Live API support. This procedure uses 3.1 expressly; a 3.8 session would answer a different task.

## 1. Check access, privacy and local tools

You need Python 3.11 or newer, PowerShell 7.1 or newer, a working microphone and audio output, a Google AI Studio API key with access to 3.1, and permission to share everything visible on the combined desktop. PowerShell's [`Read-Host -MaskInput`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/read-host?view=powershell-7.6) was added in version 7.1. Check `python --version` and `$PSVersionTable.PSVersion` before following the commands.

Review every monitor that contributes to the combined desktop. Close private files, password prompts, chats and notification previews before capture starts. The example uses `mss.monitors[0]`; it has no monitor picker. Use a non-sensitive visible element for the first check. Keep the API key in the local PowerShell environment, never in the script or a page you publish. Check your project's [current limits](https://ai.google.dev/gemini-api/docs/rate-limits), billing tier and data-use terms before connecting.

## 2. Reproduce the reviewed Google example

In a new local folder, download Google's [Apache-2.0 cookbook example at commit `16d412d`](https://github.com/google-gemini/cookbook/blob/16d412d09c84030c996327d8c7728a4412f16719/quickstarts/Get_started_LiveAPI.py). Keep its copyright and license notice. The patch below accepts only the original SHA-256 `74574e03f9490ddc39be0e48910f9c26f5f8adfdd98d51eb4f2183c4953abcba` and verifies the adapted SHA-256 `42ad50634d67822687bbb60e0ddb1d1183f8d8873b8ecd6b8ddb918184d3620e`. It changes the model ID, closes each screenshot handle, scales frames within 1024 × 1024 before JPEG encoding, and guards cleanup if the microphone never opened. Inspect the patch and [Apache license](https://www.apache.org/licenses/LICENSE-2.0) before use.

```powershell
Invoke-WebRequest 'https://raw.githubusercontent.com/google-gemini/cookbook/16d412d09c84030c996327d8c7728a4412f16719/quickstarts/Get_started_LiveAPI.py' -OutFile 'Get_started_LiveAPI.py'
```

Save the following complete code as `patch_live31.py` in that folder, then run `python .\patch_live31.py`. Stop if either hash check fails. The patch reads and writes local files only; it does not contact the model.

```python
"""Apply the four reviewed changes to Google's pinned Apache-2.0 cookbook file.

This script only reads and writes local files. It does not connect to Gemini.
"""
from hashlib import sha256
from pathlib import Path

original = Path("Get_started_LiveAPI.py")
adapted = Path("Get_started_LiveAPI_3_1.py")
expected_original = "74574e03f9490ddc39be0e48910f9c26f5f8adfdd98d51eb4f2183c4953abcba"
expected_adapted = "42ad50634d67822687bbb60e0ddb1d1183f8d8873b8ecd6b8ddb918184d3620e"
data = original.read_bytes()
if sha256(data).hexdigest() != expected_original:
    raise SystemExit("Source hash differs. Stop; do not patch an unreviewed version.")

changes = [
    (
        'MODEL = "gemini-3.8-live"',
        'MODEL = "gemini-3.1-flash-live-preview"',
    ),
    (
        '        sct = mss.mss()\n        monitor = sct.monitors[0]\n        \n        i = sct.grab(monitor)\n        \n        img = PIL.Image.frombytes("RGB", i.size, i.rgb)',
        '        with mss.mss() as sct:\n            monitor = sct.monitors[0]\n            i = sct.grab(monitor)\n        img = PIL.Image.frombytes("RGB", i.size, i.rgb)\n        img.thumbnail((1024, 1024))',
    ),
    (
        '        except ExceptionGroup as EG:\n            self.audio_stream.close()\n            traceback.print_exception(EG)',
        '        except ExceptionGroup as EG:\n            if self.audio_stream:\n                self.audio_stream.close()\n            traceback.print_exception(EG)',
    ),
]
for before, after in changes:
    old = before.replace("\n", "\r\n").encode("utf-8")
    new = after.replace("\n", "\r\n").encode("utf-8")
    if data.count(old) != 1:
        raise SystemExit("Expected one matching source block. Stop; source changed.")
    data = data.replace(old, new, 1)

if sha256(data).hexdigest() != expected_adapted:
    raise SystemExit("Adapted hash differs. Stop; do not run it.")
adapted.write_bytes(data)
print("Verified adapted file:", expected_adapted)
```

Create `requirements.txt` with these exact lines. They pin the package versions checked on 24 October 2026; an individual platform may still need a compatible audio-device build.

```text
google-genai==2.33.0
opencv-python==5.0.0.93
PyAudio==0.2.14
Pillow==13.0.0
mss==10.2.0
```

```powershell
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -r .\requirements.txt
python -m pip freeze > installed-versions.txt
$env:GEMINI_API_KEY = Read-Host -MaskInput 'Google AI Studio API key'
python .\Get_started_LiveAPI_3_1.py --mode screen
```

The pinned example captures 16-kHz mono PCM through PyAudio and sends it with screen JPEGs through the Google GenAI Live SDK. It iterates every returned part, plays 24-kHz PCM, records input/output transcription, clears queued playback on interruption and cancels the running tasks when you enter `q`. It also enables context compression. Google's [Live API overview](https://ai.google.dev/gemini-api/docs/live-api) describes the audio and image formats; its [best practices](https://ai.google.dev/gemini-api/docs/live-api/best-practices) explain interruption handling and context costs.

## 3. Check the answer and stop capture

Ask a spoken question about the non-sensitive visible element. Change that element and ask a follow-up that depends on the new view. Check whether the spoken reply matches what is actually on the screen; transcription printed by the example is a secondary trace and can be wrong. This is a reader check, not an outcome BIG CHANGE observed.

Type `q` in the terminal and press Enter. Confirm the process exits, the microphone indicator turns off and screen capture stops. If the terminal closes unexpectedly, check that no Python process or capture indicator remains before reopening private content. The frame loop can send a frame every second even during silence. Do not leave the session open while you are not using it.

## Cost, limits and recovery

Google's [pricing page](https://ai.google.dev/gemini-api/docs/pricing), checked 24 October 2026, groups 3.1 and 3.8 Live. It lists a limited free tier and paid rates per million tokens of $0.75 for text input, $3 for audio input, $1 for image/video input, $4.50 for text output and $12 for audio output. Actual cost depends on your account and usage, including frames, transcription and accumulated context in a longer session. The page distinguishes free-tier and paid-tier data-use terms. Check the live pricing, limits, terms and account settings before starting; this guide cannot quote a universal session price.

If the connection rejects `gemini-3.1-flash-live-preview`, record the response without the key and check that account's model access and quota. Do not quietly change `MODEL` to 3.8. If installation fails at PyAudio, or the microphone or screenshot call fails, stop and inspect the local device, permissions and package environment. If playback lags after an interruption, use headphones and a short session, and verify a fresh response against the current screen. After an uncertain disconnect, inspect account usage before reconnecting. The example is local developer code; a browser-distributed client would need the separate [ephemeral-token design](https://ai.google.dev/gemini-api/docs/live-api/ephemeral-tokens).

BIG CHANGE verified the adapted file's Python syntax, dependency imports and Live configuration construction in a separate local environment. It did not run the program's main function, open a microphone or screen, establish an API session, hear output, measure quality, or incur a model charge.

## Sources & further reading

- [Gemini 3.1 Flash Live Preview model page](https://ai.google.dev/gemini-api/docs/models/gemini-3.1-flash-live-preview): exact model ID, legacy-preview status, supported inputs and Live API route.
- [Google's pinned Python cookbook example](https://github.com/google-gemini/cookbook/blob/16d412d09c84030c996327d8c7728a4412f16719/quickstarts/Get_started_LiveAPI.py): the attributed Apache-2.0 source for the local program; the hash-checked changes are printed above.
- [Live API overview](https://ai.google.dev/gemini-api/docs/live-api) and [best practices](https://ai.google.dev/gemini-api/docs/live-api/best-practices): media formats, interruption and session-cost behavior.
- [Live API pricing](https://ai.google.dev/gemini-api/docs/pricing) and [rate limits](https://ai.google.dev/gemini-api/docs/rate-limits): account-dependent cost and access to check immediately before use.
- [Gemini Live Android help](https://support.google.com/gemini/answer/15274899) and [AI Studio quickstart](https://ai.google.dev/gemini-api/docs/ai-studio-quickstart): separate consumer and Studio routes, neither proof of the local API result.

## Sources

- [Gemini 3.1 Flash Live Preview](https://ai.google.dev/gemini-api/docs/models/gemini-3.1-flash-live-preview) — Exact model, legacy preview, Live support and Studio route.
- [Google cookbook pinned example](https://github.com/google-gemini/cookbook/blob/16d412d09c84030c996327d8c7728a4412f16719/quickstarts/Get_started_LiveAPI.py) — Apache-2.0 original, pinned and hash guarded.
- [Live API overview](https://ai.google.dev/gemini-api/docs/live-api) — Media transport and formats.
- [Live API best practices](https://ai.google.dev/gemini-api/docs/live-api/best-practices) — Interruptions, transcription and context costs.
- [Gemini API pricing](https://ai.google.dev/gemini-api/docs/pricing) — Tier/data use and usage-based rates, checked 24 October 2026.
- [Gemini API rate limits](https://ai.google.dev/gemini-api/docs/rate-limits) — Account/project access and limits.
- [Gemini Live Android help](https://support.google.com/gemini/answer/15274899) — Consumer route, without exact model identity.
- [AI Studio quickstart](https://ai.google.dev/gemini-api/docs/ai-studio-quickstart) — Studio testing route; controls conditional.
- [Microsoft Read-Host](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/read-host?view=powershell-7.6) — MaskInput available in PowerShell 7.1+.
- [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0) — Original code redistribution terms.