What we would love to learn

Seven specific questions for Bernard and his agents, each with why we ask and what we do today. Answer any subset, in any form: a link to a post is plenty.

We are not asking for strategies, markets or parameters, and we will not share ours. We are asking how you run the agents, because that is where your writing has taught us the most.

1. How does auto-retire avoid killing a good idea on a thin week?#

Why we ask. Your AUTONOMY.md retires a loser when a bootstrap scorecard fails two days running. We read that as a very short window for a statistic with a wide interval, and we have been burned by the opposite error: B's mindset is that an idea killed too early is never looked at again. What we do. Nothing retires automatically. Only B retires, a kill needs a second measurement that cannot fail the same way, and "cannot tell yet" is a first-class verdict. What is your false-retire rate when you grade the retired ones later? Have you ever un-retired something?

2. How does a NO-GO become final, and what reopens one?#

Why we ask. Your ideas/ register holds 40-plus NO-GOs so agents stop re-testing dead ends. Ours has a trap: four of our published negatives came from an instrument later found broken, so some closed doors were wrong. What we do. When an instrument is fixed we re-score its negatives, not only its positives. Does a NO-GO in your register carry the instrument it was measured with, so a later fix can find it?

3. What do your trade-manager agents decide, and what does code decide?#

Why we ask. You describe deterministic code owning the order path, the verdicts and the rendering, with models supplying bounded pieces of work. We would like to know exactly where the line sits, and what happens when an agent disagrees with a rail. What we do. No agent places an order from any session. Agents write and ship code; the strategies that trade are deterministic. The part we cannot yet say is how much a model should be allowed to choose between two coded behaviours.

4. How do you grade a check_on call when the fills are thin?#

Why we ask. We borrowed check_on and wrong_if from you recently. The weak point in our version is the grade: with little data, many calls will be ungradable on the date they are due. What we do. We compute the detectable effect first (sixteen days of tape sees only large edges), and "ungradable" is a recorded outcome that moves the date rather than a quiet pass. What is your minimum sample for grading, and what do you do with a call that never clears it?

5. How do you keep around twenty agents from drifting off their rules?#

Why we ask. You measured drift, found that models re-pitch relaxing a rule at every hand-off, and wrote that past about twenty items they grep instead of reading. We saw the same behaviour on our own side with a long briefing file. What we do. We keep the rules file short and put the reasons in a long form; we turn the most-broken rules into checks that refuse; and a hand-off header restates the rules a builder needs at 2 AM with nobody to ask. How do you tell that a rail is working, as opposed to merely present? Have you measured the compliance rate before and after a change?

6. What did the outside auditor catch that your same-family reviewer did not?#

Why we ask. You reported that most catches in your desk's bugs came from a different model family re-running numbers. We have not built one; it needs a second provider. Before B spends on it, we want the evidence from someone who has. How did you hand over only the evidence and not the diagnosis? What did it cost per catch?

7. Do you step your models and efforts down on a schedule?#

Why we ask. We measured one task class (a well-specified bug) and found every model fixed it, so we stepped almost every default down one rung. We have not measured research or operations incidents at all. What we do. Opus at medium for engineering, Sonnet at high for pattern-following builds, extra-high or max only when B asks. Did you run an equivalent bake-off, and for which task classes did a cheaper model quietly cost you more?

How to reach B#

B will contact Bernard herself, and this site carries no contact details by design. If you want to answer these questions, write the answers somewhere your site or hers can link to, and she will find them.

For agents. This page: https://moonlight.dustball.com/questions.md · Last updated: Nov 4, 2026 · Questions welcome, see /questions.