Phone-based human checkpoints: a task-bound handoff test for coding agents

[vasanthsreeram]

I build JustCallMe, a hosted phone channel for a coding agent to ask its owner a specific blocking question while the owner is away. I am looking for independent tests of the decision-to-task boundary.

AutoGen's human-in-the-loop guide warns that UserProxyAgent blocks the team during a run and recommends short, immediate interactions. For delayed feedback, it documents terminating a run, saving team state, and supplying feedback to the next run:
https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/human-in-the-loop.html

For a phone checkpoint, the proposed AutoGen adapter boundary is that second pattern: stop at the question, persist team state and the exact question/request ID, retrieve its answer, then validate the current task before a new run. AutoGen pairing is unimplemented and untested; this is a test proposal, with no claim of automatic integration or approval.

A useful throwaway coding-task test:

1. Ask "SQLite or Postgres for dummy project alpha?" and keep its request ID with alpha.
2. Change the requirement before answering. The caller should request a fresh decision if the answer no longer applies.
3. Try applying alpha's answer to project beta. Reject the task mismatch.
4. Let a separate request expire. The decision stays pending; there is no approval to act.

The calling agent retains its permissions, context checks and code review. JustCallMe allows one waiting request per account; overlapping submissions return HTTP 409, so the caller owns retry scheduling.

If you want to test the phone part first in a throwaway Claude Code or Codex session, signup is here:
https://justcallme.dev/app?utm_source=autogen_showtell#signup
Verify your own phone, create a scoped key and install the private dashboard skill. Early access includes 20 free minutes, no card. Public API docs: https://justcallme.dev/docs

Please report setup friction or a decision reaching the wrong task. Keep phone numbers, verification codes, keys and real project details private. Happy to help willing testers with setup.

[bomtobe-hgkim]

One edge case I’d add to the throwaway run: deliver a delayed answer after the originating run has been canceled or replaced, while a new task is waiting. The old request ID should not resume the new task or inherit its permissions. I haven’t run JustCallMe; this is a suggested boundary test, not a result. I’m with BOMTOBE, the team behind Orke, a desktop workspace for Claude Code and Codex, so task-to-result handoffs are a problem space we follow.

[bomtobe-hgkim]

One edge case I’d add to the throwaway run: deliver a delayed answer after the originating run has been canceled or replaced, while a new task is waiting. The old request ID should not resume the new task or inherit its permissions. I haven’t run JustCallMe; this is a suggested boundary test, not a result. I’m with BOMTOBE, the team behind Orke, a desktop workspace for Claude Code and Codex, so task-to-result handoffs are a problem space we follow.

[Wesley (GriffMurder)]

Adding another angle on the delayed-human pattern: some blockers aren't a decision for the owner, they need a phone call to a third party (confirming a business's hours, a vendor with no API). One approach is to hand the call to a person over MCP. The agent submits a structured list of questions plus context, returns immediately with a job id, a human makes the call, and the agent later polls for structured answers and wrap-up notes it can act on, so nothing blocks the run.

Tradeoffs: latency is minutes, not seconds; write questions with clear, checkable answers (e.g. "open Saturday? yes/no, until what time"); keep PII to the minimum the call needs; and, as this thread says, bind the job id to the originating task so a late answer can't resume a different one.

Disclosure: I work on TaskBullet, which does this as an MCP tool (npm @taskbullet/mcp-server): https://taskbullet.com/hub. $2.50 for the first minute, then $1/min.

[Dr Srinivas Padmanabhuni (srpadmanabhuni)]

The request-ID-bound test cases are the right ones, and I would make the mismatch cases machine-checkable. Store the task ID, a hash of the requirement text, and the request ID alongside the saved team state, then assert on resume that all three match, otherwise discard the answer. Extra checks I'd run: answer the question twice (second answer must be ignored), answer after the team state was restored from an older snapshot, and answer in the 409 overlap window to confirm the caller does not silently drop the first pending question. I'd also log the rejected answer, since a quietly discarded human decision is its own failure mode.