On October 28, Kindo announced Deep Hat Sift, a filter intended to reduce AI-agent runs on alerts, tickets and logs. Whether it saves money without withholding useful work depends on the stream and the quality of the decisions it learns.

The big change

  • An organization can place a decision before an agent run, making the choice to skip work a measurable part of the workflow. That changes the cost question from the price of each run to the number and kinds of runs avoided.
  • A saved run has an opportunity cost if it would have revealed something worth investigating. The useful comparison is therefore not savings alone: it is savings alongside an estimate of what the filter withheld incorrectly.
  • That estimate has a reference point. Agreement with an agent can show how closely the filter follows its judgments, while the correctness of those judgments requires separate evidence.

Learning from an agent

The self-hosted filter learns from its paired agent’s verdicts. Kindo’s documentation requires human-approved examples before shadow and separate threshold approval with a written reason.

In shadow, Sift scores events while the agent supplies verdicts. Events ordinarily reach the agent; optional budgets and sampled shadow can withhold events. A preview can help an operator judge a proposed threshold on the available stream; it is not proof that future events will resemble those observations.

After activation, Sift selects a random 3% of otherwise withheld events by default; agent budgets can queue or discard them. Some selected events may therefore remain unseen by the agent. This continued sample matters because a filter that never checks what it suppresses loses a way to compare those decisions with the agent’s response. The sample is still only a sample: rare mistakes can be difficult to assess, and the number of reviewed events matters when interpreting a rate.

What a miss rate can establish

The reported comparison is with the paired agent’s verdict, not an independently established truth. A filter may agree with an agent that an alert needs no action even when both are wrong. Conversely, disagreement identifies a case for investigation without itself establishing which judgment was correct. For security teams, a trusted investigation outcome or another suitable reference would be needed to evaluate the underlying decision, not merely the match between two models.

The distinction reflects a wider research problem. In work on selective labels, researchers explain how decisions affect which outcomes become visible, including in settings involving bail decisions. That study does not test Sift. It provides a useful reason to examine the coverage of an evaluation: the records available after a decision may not represent all the cases a system encountered.

Kindo’s operating documentation says drift or an hourly trigger ceiling ordinarily returns Sift to shadow; operators may choose filtering with an alert. Missing approval and stalled delivery stop filtering. These are stated controls, not independently observed performance results. The choice of fallback matters because a warning leaves an operator with a different response obligation from an automatic return to shadow.

Testing the operating fit

A pilot can compare the agent workload and the decisions made on a representative stream before relying on the threshold. The comparison needs enough observations to interpret both the avoided runs and the sampled disagreements. It also needs to account for the cost of the filter and the continued sample; counting skipped agent runs alone does not establish net savings. A meaningful evaluation of consequential misses requires evidence beyond the paired agent’s verdict.

The NIST AI Risk Management Framework playbook recommends comparing production indicators with pre-deployment results and investigating shifts in inputs and outputs. Applied to this filter, that general guidance points to a continuing comparison after a pilot: a threshold selected on one period’s events may need reassessment when the stream changes. NIST does not evaluate Sift or establish that its safeguards work in a particular deployment.

The current product limits also shape that decision. Kindo lists one host without failover and local accounts without MFA. Its pages differ on sign-in, leaving SSO unclear. Local scoring does not establish that downstream model calls stay local. The paired agent retains its own permissions, which still need review.

BIG CHANGE has not deployed Sift or measured its performance. Kindo’s announcement and documentation describe the proposed operation; they do not establish a universal saving or an independently measured rate of real-world incidents missed.