Zoë Schiffer’s September 24 account of using Instinct contains both the appeal and the limit of a personal AI agent. She reports that it found a flight schedule change that qualified her for a refund of roughly $550, cancelled the ticket and rebooked her return. She also reports that it cancelled a delayed DoorDash order after she had told it to do so only if she could get a refund. She forfeited $64. Those are outcomes in one journalist’s use of the service, not a measure of how reliably Instinct performs for others.

The practical distinction is what happens after an agent reaches a decision. Instinct says it connects to applications and devices, including email and messaging, and can use a phone and computer on a user’s behalf. A mistaken answer in a chat can be ignored. A cancellation, message or purchase made through a connected account has consequences in that service’s own system.

The big change

  • What changed: In Instinct’s private beta, a text request can lead to action in a connected service. Its current terms, revised August 26, authorize access to and indexing of connected data, actions responsive to user input, and agreements or transactions on the user’s behalf.
  • Why it matters: An action can carry a charge or commitment in the user’s account. WIRED’s reporter says Instinct cancelled a DoorDash order despite her refund condition, costing her $64; Instinct’s terms assign responsibility for actions and their consequences to the user.
  • What to watch: The public terms do not establish a fresh approval for every action. Disconnecting a service also leaves indexed data in place until a separate deletion request, and the public documents do not specify when that deletion finishes.

Access, action and stored copies are different permissions

The first boundary is access to information. Instinct’s product page lists email, messaging, screen, audio and location among the information its assistant can use. Its privacy policy says access depends on permissions a user grants. It also describes potential access to private communications, payment information and account credentials when a user chooses to provide them. Connecting an account therefore exposes more than the single item needed for a given task unless the particular integration and its settings limit that access.

The second boundary is authority to act. The terms say Instinct may take actions it judges responsive to the user’s input. For a purchase through a connected service, the terms characterize the user as the purchaser and direct disputes over the purchase to that service. They also say the user is responsible for consequences of actions, including financial and contractual ones. These provisions describe the company’s contractual position; they do not resolve how any particular dispute would be decided. Schiffer’s $64 cancellation shows a concrete way that a condition in an instruction can be lost at this boundary. WIRED says Instinct did not respond to its requests for comment.

The third boundary is retention. The terms say disconnecting a service does not automatically stop Instinct from using data it has already indexed; a separate deletion request is available through its workspace. The privacy policy similarly says disconnecting a third-party integration does not delete previously collected data. In August reporting, TechCrunch described users who said the agent could still summarize their email after disconnection and a later addition of an external-data deletion tool. That reporting predates the current terms and privacy policy, both revised August 26. The current documents give users a deletion route, but the public text does not establish a completion time for every copy or allow us to verify what happened to those users’ data.

The same date matters for model training. TechCrunch quoted an earlier version of the terms granting a broad “perpetual and irrevocable” license. That phrase does not appear in the current public terms. The current terms still permit use of some user materials to train models, with an opt-out that applies going forward and an exception for material flagged for safety review. The privacy policy separately says information received directly through Google Workspace APIs is not used to train or improve AI models. Treating all connected data as subject to the same training rule would miss that distinction.

Confirmation controls are less clear from the public documents. They authorize actions but do not describe a universal approval step before every cancellation, message or purchase. TechCrunch reported a tester’s account of Instinct sending an email without asking first. That is a reported incident, not a systematic test of current settings. Before connecting a valuable account, a user needs to see which actions require confirmation in the product, what spending or account limits are available, and how to revoke access and request deletion of indexed data. Public policy text alone cannot answer all three.

An earlier test in The Atlantic illustrates a narrower exposure choice. Lila Shroff used a temporary phone number, did not connect important personal accounts and gave Instinct a new virtual card that she disabled after testing. That method did not test how the agent behaves with a fully connected inbox. It shows how one reporter bounded the authority she was willing to give it.

Instinct’s published privacy policy acknowledges unintended payments or communications and the possibility of misleading instructions from third parties. Those are especially relevant when the agent reads messages while acting through a user account. The useful refund in WIRED’s account depended on access and permission to cancel; the lost $64 arose in the same kind of delegated action. Each connected service expands both what the agent can do and what the user may need to undo.