Google's Gemini 3.1 Flash Live Preview can take microphone audio and still images during a Live API session and return spoken audio. This guide adapts a pinned Google Python example to send a JPEG of the combined desktop once per second. It is for developers who can review the entire desktop before sharing it and who have access to the exact model. BIG CHANGE checked the source and local setup, but did not connect to the model or test a device.

The big change

What: A local Python session can pair spoken questions with desktop frames sent to gemini-3.1-flash-live-preview and play the model's audio reply.

How: Google's Live API accepts 16-kHz PCM microphone input and JPEG frames at no more than one frame per second; the example plays 24-kHz PCM output and stops on q.

Why it matters: The reader can check a reply against a visible screen change and end both capture streams. The whole combined desktop, including any visible notification, is in scope while the program runs.

Choose the route that matches your task

Google's Gemini Live Android help explains how to share a phone screen and stop sharing with the Live control or notification. It says the full device screen is shared, and muting the microphone may leave screen sharing active. That help page does not identify the model behind the consumer app, so it does not establish a Gemini 3.1 session. Camera sharing is a separate control.

The 3.1 model page also links a 3.1 AI Studio session. If you use Studio, check the displayed model and whether your account shows both microphone and screen controls before sharing. The controls and account access are conditional. The local API procedure below pins the model in code and supplies the full microphone, screen and audio path.

Google currently calls 3.1 Flash Live Preview a legacy preview and recommends 3.8 for most new work. The model page still lists the 3.1 model ID and Live API support. This procedure uses 3.1 expressly; a 3.8 session would answer a different task.

1. Check access, privacy and local tools

You need Python 3.11 or newer, PowerShell 7.1 or newer, a working microphone and audio output, a Google AI Studio API key with access to 3.1, and permission to share everything visible on the combined desktop. PowerShell's Read-Host -MaskInput was added in version 7.1. Check python --version and $PSVersionTable.PSVersion before following the commands.

Review every monitor that contributes to the combined desktop. Close private files, password prompts, chats and notification previews before capture starts. The example uses mss.monitors[0]; it has no monitor picker. Use a non-sensitive visible element for the first check. Keep the API key in the local PowerShell environment, never in the script or a page you publish. Check your project's current limits, billing tier and data-use terms before connecting.

2. Reproduce the reviewed Google example

In a new local folder, download Google's Apache-2.0 cookbook example at commit 16d412d. Keep its copyright and license notice. The patch below accepts only the original SHA-256 74574e03f9490ddc39be0e48910f9c26f5f8adfdd98d51eb4f2183c4953abcba and verifies the adapted SHA-256 42ad50634d67822687bbb60e0ddb1d1183f8d8873b8ecd6b8ddb918184d3620e. It changes the model ID, closes each screenshot handle, scales frames within 1024 × 1024 before JPEG encoding, and guards cleanup if the microphone never opened. Inspect the patch and Apache license before use.

PowerShell
Invoke-WebRequest 'https://raw.githubusercontent.com/google-gemini/cookbook/16d412d09c84030c996327d8c7728a4412f16719/quickstarts/Get_started_LiveAPI.py' -OutFile 'Get_started_LiveAPI.py'

Save the following complete code as patch_live31.py in that folder, then run python .\patch_live31.py. Stop if either hash check fails. The patch reads and writes local files only; it does not contact the model.

Python
"""Apply the four reviewed changes to Google's pinned Apache-2.0 cookbook file.

This script only reads and writes local files. It does not connect to Gemini.
"""
from hashlib import sha256
from pathlib import Path

original = Path("Get_started_LiveAPI.py")
adapted = Path("Get_started_LiveAPI_3_1.py")
expected_original = "74574e03f9490ddc39be0e48910f9c26f5f8adfdd98d51eb4f2183c4953abcba"
expected_adapted = "42ad50634d67822687bbb60e0ddb1d1183f8d8873b8ecd6b8ddb918184d3620e"
data = original.read_bytes()
if sha256(data).hexdigest() != expected_original:
    raise SystemExit("Source hash differs. Stop; do not patch an unreviewed version.")

changes = [
    (
        'MODEL = "gemini-3.8-live"',
        'MODEL = "gemini-3.1-flash-live-preview"',
    ),
    (
        '        sct = mss.mss()\n        monitor = sct.monitors[0]\n        \n        i = sct.grab(monitor)\n        \n        img = PIL.Image.frombytes("RGB", i.size, i.rgb)',
        '        with mss.mss() as sct:\n            monitor = sct.monitors[0]\n            i = sct.grab(monitor)\n        img = PIL.Image.frombytes("RGB", i.size, i.rgb)\n        img.thumbnail((1024, 1024))',
    ),
    (
        '        except ExceptionGroup as EG:\n            self.audio_stream.close()\n            traceback.print_exception(EG)',
        '        except ExceptionGroup as EG:\n            if self.audio_stream:\n                self.audio_stream.close()\n            traceback.print_exception(EG)',
    ),
]
for before, after in changes:
    old = before.replace("\n", "\r\n").encode("utf-8")
    new = after.replace("\n", "\r\n").encode("utf-8")
    if data.count(old) != 1:
        raise SystemExit("Expected one matching source block. Stop; source changed.")
    data = data.replace(old, new, 1)

if sha256(data).hexdigest() != expected_adapted:
    raise SystemExit("Adapted hash differs. Stop; do not run it.")
adapted.write_bytes(data)
print("Verified adapted file:", expected_adapted)

Create requirements.txt with these exact lines. They pin the package versions checked on 24 October 2026; an individual platform may still need a compatible audio-device build.

Text
google-genai==2.33.0
opencv-python==5.0.0.93
PyAudio==0.2.14
Pillow==13.0.0
mss==10.2.0
PowerShell
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -r .\requirements.txt
python -m pip freeze > installed-versions.txt
$env:GEMINI_API_KEY = Read-Host -MaskInput 'Google AI Studio API key'
python .\Get_started_LiveAPI_3_1.py --mode screen

The pinned example captures 16-kHz mono PCM through PyAudio and sends it with screen JPEGs through the Google GenAI Live SDK. It iterates every returned part, plays 24-kHz PCM, records input/output transcription, clears queued playback on interruption and cancels the running tasks when you enter q. It also enables context compression. Google's Live API overview describes the audio and image formats; its best practices explain interruption handling and context costs.

3. Check the answer and stop capture

Ask a spoken question about the non-sensitive visible element. Change that element and ask a follow-up that depends on the new view. Check whether the spoken reply matches what is actually on the screen; transcription printed by the example is a secondary trace and can be wrong. This is a reader check, not an outcome BIG CHANGE observed.

Type q in the terminal and press Enter. Confirm the process exits, the microphone indicator turns off and screen capture stops. If the terminal closes unexpectedly, check that no Python process or capture indicator remains before reopening private content. The frame loop can send a frame every second even during silence. Do not leave the session open while you are not using it.

Cost, limits and recovery

Google's pricing page, checked 24 October 2026, groups 3.1 and 3.8 Live. It lists a limited free tier and paid rates per million tokens of $0.75 for text input, $3 for audio input, $1 for image/video input, $4.50 for text output and $12 for audio output. Actual cost depends on your account and usage, including frames, transcription and accumulated context in a longer session. The page distinguishes free-tier and paid-tier data-use terms. Check the live pricing, limits, terms and account settings before starting; this guide cannot quote a universal session price.

If the connection rejects gemini-3.1-flash-live-preview, record the response without the key and check that account's model access and quota. Do not quietly change MODEL to 3.8. If installation fails at PyAudio, or the microphone or screenshot call fails, stop and inspect the local device, permissions and package environment. If playback lags after an interruption, use headphones and a short session, and verify a fresh response against the current screen. After an uncertain disconnect, inspect account usage before reconnecting. The example is local developer code; a browser-distributed client would need the separate ephemeral-token design.

BIG CHANGE verified the adapted file's Python syntax, dependency imports and Live configuration construction in a separate local environment. It did not run the program's main function, open a microphone or screen, establish an API session, hear output, measure quality, or incur a model charge.

Sources & further reading