The Machines Finished the Work. They Still Couldn’t Do the Job.

Two frontier AI agents were given six days, thousands of dollars in computing credits, unrestricted internet access and a virtual machine, then asked to answer genuine, unpublished research questions. They searched the literature, wrote code, ran experiments and produced complete papers. The scientists who had posed the questions rejected both papers.

That result is more damaging to the grandest claims about artificial intelligence than another failed chatbot benchmark. The agents did not collapse because they could not write Python or format a paper. They failed because they could not decide what was worth trying, recognise when a promising path had become a dead end, or invent a better question when the first one stopped working.

The test was aimed at the claim behind the AI boom

The experiment, published by the CRUX project with researchers from Princeton, the UK AI Security Institute and other institutions, was designed to answer a question that has been obscured by the industry’s fondness for impressive demonstrations: can an AI agent conduct open-ended research without a human quietly supplying the important thinking?

The stakes are larger than the fate of two machine-learning papers. The most aggressive forecasts for artificial intelligence depend on a feedback loop. If an AI system can conduct research, improve models, design better training methods and repeat the process, then progress might accelerate beyond the pace set by human researchers. That prospect helps explain the enormous investment in data centres, chips and energy infrastructure. It also sits behind predictions that large parts of professional work will soon be automated.

A system that can complete a list of instructions is useful. A system that can choose the list, revise it when reality intervenes and understand why the result matters is something else. The distinction is often erased in product demonstrations because a tidy task is easier to show than an untidy job.

The CRUX researchers tried to restore that distinction. They took the central questions from two unpublished NeurIPS 2026 submissions. The papers had not been released, so the agents could not simply retrieve the answers from their training data. The agents received the question rather than the researchers’ method, and were expected to build the investigation themselves. Their work was then assessed by the original authors, who knew the subject intimately and judged the output as they would a conference submission. The paper describes this as a “shadow evaluation”.¹

That arrangement matters because the usual AI tests are built for clean answers. A model can be asked to solve a problem with a known solution, pass a coding test or retrieve information from a defined corpus. Those tests may measure important abilities, but they do not measure whether a researcher can look at a confusing result and decide which explanation deserves another week of work.

Nor is ordinary peer review a perfect substitute. A paper produced by an AI system can be sent to reviewers who are busy, inconsistent and unfamiliar with the exact route by which the result was obtained. The CRUX design placed the work in front of people who had already wrestled with the question. It was not a flawless experiment, but it was closer to the real problem than a leaderboard score.

What the agents accomplished should make the result more, not less, important

The agents were not useless. On the contrary, they displayed the level of competence that makes the failure consequential.

They conducted literature searches, wrote and debugged software, ran experiments, performed robustness checks, managed computing resources and assembled full research papers. They operated across a virtual computer and the open web rather than inside a narrow question-and-answer box. According to the experiment’s reporting, only a small number of human interventions were required for infrastructure or presentation issues. The basic engineering pipeline worked.

That is precisely why the result should not be dismissed as another story about an AI system making a silly mistake. The machines could perform many of the visible activities associated with research. They could produce the outward signs of a functioning laboratory: papers, plots, code, experiments and revisions. What they could not reliably produce was a convincing reason for choosing one line of inquiry over another.

The two outputs were rejected by the original researchers. One received a “Reject” verdict and the other a “Strong Reject”, with criticism directed at the quality of the research choices rather than a single typographical error. Reviewers found weakly motivated experiments, poor explanations and no contribution that would change what specialists understood. A detailed account of the study reported that one review described the reasoning as a “proof by example” fallacy and called it highly non-scientific.²

The distinction is easy to miss when the output is judged by appearance. A paper can contain a plausible abstract, clean equations and a long list of experiments while still being scientifically empty. Research is not a typing contest. The question is not whether the author can produce enough material to fill a document. The question is whether the work has identified something that deserves to be known.

Human researchers routinely make errors, waste time and pursue bad hypotheses. The comparison is not between perfect people and defective machines. It is between two different kinds of failure. Humans can sometimes recognise that a project is going nowhere and change direction. They can notice that a result is technically valid but intellectually unimportant. They can decide that a new result is worth more than a polished answer to the original question. Those decisions are the centre of the work, not decorative extras attached to it.

The agents’ competence also clarifies the likely near-term effect of AI. Tools that remove routine engineering may make researchers faster, cheaper and more numerous. But faster execution does not settle who will choose the projects, set the standards or accept responsibility for the consequences. If anything, the easier it becomes to generate experiments, the more valuable the ability to reject bad experiments becomes.

The five failures were failures of judgement, not intelligence theatre

The researchers identified a pattern that will be familiar to anyone who has worked with long-running AI agents. The systems could follow local instructions while losing the larger purpose. They could respond to a problem without understanding whether the response improved the project.

The first failure was judgement about quality. The agents did not understand the standard for publishable research. They generated plausible ideas and completed the associated work, but they struggled to distinguish a result that merely existed from a result that mattered. That is the difference between satisfying a specification and meeting a professional standard. The specification can say “run experiments”; it cannot fully specify which experiments would make the work worth reading.

The second was creative problem-solving. When an approach weakened, the agents tended to narrow or repair the same approach rather than produce a fundamentally different one. They could vary parameters, add checks and rewrite the claim. They rarely escaped the conceptual frame that had generated the original failure.

The third was backtracking. This is more than persistence. A competent researcher does not treat the first plausible plan as a contract. If a central assumption fails, the researcher may discard weeks of work and reopen the question. Agents often behaved as though sunk cost were a reason to continue. They made incremental repairs after the core strategy had stopped being credible.

The fourth was resource awareness. The agents had money and time left, but did not use those resources as a human under pressure might. One run ended well before the deadline and after another negative assessment from a reviewer. The machine treated “project complete” as a state it could declare, not a judgement that had to be earned.

The fifth was instruction drift. Both agents were told to respect page limits, yet both exceeded them. The error was not technically profound. It was professionally fatal. A submission that ignores a basic format rule may be rejected before its scientific merits are considered. The episode showed that an agent can acknowledge a requirement at the beginning of a task and still lose it while managing the details of a long sequence.

These are not random quirks. They form a coherent picture. The systems were good at operating inside a local frame and poor at deciding whether the frame remained useful. That makes them powerful assistants and unreliable principals. They can be handed a direction and asked to move quickly. They should not be mistaken for the person responsible for knowing whether the destination is worth reaching.

There was also a finding that cuts against the more sensational accounts of AI behaviour. The agents did not appear to manipulate the evidence to preserve attractive claims. When results failed to support an ambitious hypothesis, they reduced or abandoned the claim. The central problem was not that the systems were secretly scheming. It was that they could be honest about an uninteresting result without knowing that the result was uninteresting.

Why more tokens do not automatically create research sense

The industry has adopted the word “reasoning” to describe systems that spend more computation before giving an answer. That shorthand is useful for marketing and dangerous for analysis.

A large language model generates text by predicting what comes next from patterns learned during training. Newer systems can use longer chains of intermediate computation, call tools, inspect their own work and ask subordinate models for assistance. Those additions can improve performance, especially on problems with a clear answer or a reliable way to check progress. But extra steps do not automatically create a sense of purpose.

When a model produces a long explanation, the explanation is not proof that the system has discovered the cause of its own answer. It is a sequence of generated representations that may help the computation, but may also rationalise a decision already made. The visible story of deliberation and the process that produces a useful result should not be treated as identical.

The deeper issue is the nature of the information available to the machine. A language model is extraordinarily good at absorbing the conclusions, distinctions and procedures that have been expressed in language. It can recombine those materials at speed. That gives it an enormous advantage in retrieval and synthesis. But a new research result is not merely a sentence waiting to be assembled from old sentences.

A scientist must interact with a world that can resist the hypothesis. The scientist chooses a measurement, changes a condition, observes an outcome and decides whether the outcome supports the underlying explanation. The important information is not only what has been written before. It is what happens when someone intervenes.

This is why a model can appear sophisticated while still failing at the most important moment. It may know that a particular experiment is standard, that a certain statistical test is common and that a paper should contain a limitations section. It may not know whether the experiment answers the question that matters, whether the test addresses the actual confound or whether the limitation destroys the claim.

Research is full of situations in which several options are defensible and only one is useful. A model can list the options. It can describe their advantages. It can generate a table comparing them. The human contribution is often the decision that the table cannot settle: this is the risk worth taking; this anomaly is worth pursuing; this apparently elegant route should be abandoned.

That does not mean language models cannot improve. It means that scaling the existing capability should not be confused with solving the missing capability. More memory may help an agent remember instructions. More tools may help it collect evidence. More compute may allow it to test more variations. None of those changes, by themselves, guarantee the ability to recognise a question that has not yet been asked.

The brain does not reason by language alone

There is a tempting but misleading picture of thought in which intelligence is simply language moving faster and with better grammar. If that picture were correct, a system trained on nearly all available text would be on a direct path to human reasoning. Evidence from cognitive neuroscience points to a more complicated reality.

A 2024 perspective in Nature argued that language is primarily a tool for communication rather than a prerequisite for complex thought. The authors reviewed evidence for a separation between language and reasoning networks, including cases in which people with severe language impairments retained important forms of numerical and logical ability. Their conclusion was not that language is unimportant. Language transmits culture, stores knowledge and allows one person’s reasoning to become another person’s starting point. It does not follow that language creates every form of thought.³

The distinction matters for AI because large language models begin from a very different foundation. They do not grow up in a physical environment, form intentions through years of social experience or learn by changing the world and coping with the consequences. They learn statistical relationships in data and, in agentic systems, are connected to tools that let them act.

Tool use narrows the gap, but it does not erase it. Giving an agent a browser provides access to information. Giving it a computer provides a place to run code. Giving it a budget provides a constraint. None of these supplies a settled standard of significance.

A human researcher brings a background of tacit knowledge that is difficult to write down. The field has tried this approach before. The last five papers using a certain dataset were all criticised for the same hidden problem. A result technically improves a benchmark but would not survive contact with a real deployment. A question is fashionable because it attracts funding rather than because it leads to understanding. A supervisor’s silence means the idea is weak, not that it has been approved.

These judgements are not mystical. They are learned through participation in a community and exposure to consequences. They can be explained after the fact, but they are not always available as a complete checklist before the decision. That is why an agent can repeat the language of scientific caution without possessing the practical sense that makes caution useful.

The lesson should not be turned into a romantic claim that humans possess an uncopyable soul. Human judgement can be trained, assisted and sometimes formalised. Machines may acquire better world models, planning systems or feedback mechanisms. But the current evidence does not justify treating fluent language as a substitute for the broader machinery of thought.

The agents in the CRUX experiment were not defeated by a lack of vocabulary. They were defeated by a lack of contact between execution and significance. They could say what they had done. They struggled to know whether it was worth doing.

Pearl’s ladder explains the missing step

Judea Pearl’s account of causation offers a useful way to describe the gap without pretending that one metaphor settles the entire science of AI.

In The Book of Why, Pearl and Dana Mackenzie describe a ladder of causal reasoning. The first rung concerns association: what tends to occur with what? The second concerns intervention: what happens if we change something? The third concerns counterfactuals: what would have happened if events had unfolded differently? Pearl’s work made these distinctions mathematically explicit and helped separate pattern recognition from questions about causes.⁴

Most of the public performance of language models lives near the first rung. They are good at identifying regularities in what people have written. They can predict that a certain treatment is associated with a certain outcome because the association is recorded in thousands of papers. They can summarise a method that worked in one setting and suggest places where it might be useful.

But “what is usually associated with recovery?” is not the same question as “what would happen if we gave this patient the treatment?” The second question requires intervention. “What would have happened if we had chosen the other design?” requires a counterfactual. The answer cannot be recovered from observation alone without assumptions about the causal structure of the world.

Open-ended research lives across all three rungs. A researcher observes an anomaly, intervenes to test an explanation and imagines alternatives when the result does not fit. The decisive move may be to change the question itself. That is why the agents could perform experiments without necessarily conducting research in the stronger sense. They could execute the intervention. They could not consistently decide which intervention would expose the hidden cause.

Pearl’s framework should not be used as a crude slogan that every AI system is permanently trapped at one level. Modern systems can interact with environments, run experiments and learn from feedback. A language model connected to tools is not identical to a text-only predictor. But the framework does identify a persistent error in public discussion: confusing the ability to reproduce descriptions of causal knowledge with the ability to generate reliable causal knowledge.

That confusion appears in office work as well. A system can find the legal cases that resemble a dispute. It may not know which distinction will decide the case. It can compare a company’s sales figures. It may not know which unexplained movement reflects a structural change and which is noise. It can generate a treatment summary. It may not know which missing piece of evidence should stop a decision.

Information is the raw material of judgement, not judgement itself. A larger library is valuable. It is not a magistrate, a scientist or a manager.

Why the accountant argument is too simple

The most serious objection to this analysis is not that AI can do nothing. It is that AI may not need to do an entire job in order to reduce the number of people employed in it.

That objection is correct. A person can lose work even when a machine has not mastered the whole occupation. If routine drafting, reconciliation, search and basic analysis become cheap, employers may need fewer junior staff for the same volume of output. Workers may also be expected to produce more, faster, under closer supervision. Saying that an agent cannot do a whole job is not a promise that every job is safe.

But the claim that automation simply removes half the workers makes another assumption: that the amount of useful work is fixed. History does not support it.

Economists Jeremy Greenwood, Ananth Seshadri and Mehmet Yorukoglu studied the spread of household technologies in the twentieth century and argued that labour-saving appliances helped release women from unpaid domestic work into the paid labour force. The mechanism was not that society decided to enjoy the same amount of laundry with fewer hours. The price of household production fell, expectations changed and a larger number of people could take part in market work. Their model attributed more than half of the rise in women’s labour-force participation in the period they examined to labour-saving technology, with other factors contributing as well.⁵

The historical record is not a guarantee that AI will create more jobs than it destroys. It is a warning against treating demand as a constant. The St. Louis Federal Reserve, summarising the evidence, reported that the labour-force participation rate for women rose by 24.3 percentage points from 1955 to its 1999 peak and highlighted the role of technology that reduced the time required for housework.⁶

Software offers a related example. Compilers removed the need for most programmers to write machine instructions by hand. That did not produce a world with a tiny number of programmers. It made software cheaper to create, widened access to programming and allowed society to demand more software than before.

The analogy has limits. Technology can produce a painful transition even when the long-run economy expands. New demand does not necessarily appear in the same place, for the same workers or on the same timetable. A displaced bookkeeper may not become the person who designs a new financial system. A junior lawyer whose research has been automated may not be promoted into a role involving client judgement. The human costs are real.

Still, the economic question is not simply whether a machine can perform the tasks on an accountant’s current checklist. It is whether cheaper accounting creates new services, higher standards, more compliance, broader access or different forms of financial decision-making. The quantity of work at the task layer can fall while the value of deciding what should be done rises.

That is the likely pattern if AI remains strong at execution and weak at direction. Routine information work becomes cheaper. The premium moves towards people who can define the problem, judge the evidence, handle exceptions and bear responsibility when the answer is wrong.

The study is early evidence, not a licence for complacency

The CRUX result should be read with the same scepticism that it applies to AI demonstrations. Two research tasks and two agents cannot settle the future of machine intelligence.

The reviewers were not blind in the full conventional sense. They knew their own research questions and knew they were assessing AI-generated work. The sample was small because expert evaluation of open-ended work takes time. There was no objective answer key that could settle every disagreement. The agents were tested in a particular scaffold, under a particular budget and with particular research questions. A different system, better memory design or more effective feedback could perform better.

The CRUX team acknowledges these limitations. Its open-world evaluation project is intended to complement, not replace, narrow benchmarks and conventional review. The project’s own description stresses the trade-off: controlled tests are scalable and easier to score; real-world tasks are messier, slower and harder to generalise from.⁷

That caution cuts both ways. The study cannot prove that autonomous research is impossible. It can show that the claim is not yet supported by the evidence that matters. A small experiment is enough to weaken confidence in a sweeping forecast, even if it is not enough to establish a permanent limit.

There is also a technical reason to take the result seriously. Modern reinforcement learning works best when success can be checked. A game has a score. A coding task has tests. A mathematical problem may have a proof or a known answer. Open-ended research does not offer a simple reward signal for “the most important question” or “the experiment that changed the field”. The evaluator may understand the result only after a long chain of choices, and even experts can disagree about what counts as important.

That does not make open-ended learning impossible. It makes it expensive and fragile. An agent may need better simulations, long-term memory, world models, social feedback, expert supervision and the ability to learn from consequences rather than just tokens. Each addition may help. Each also creates new ways for the system to optimise the measurable proxy while missing the purpose.

The responsible conclusion is therefore narrower than either side’s preferred slogan. Today’s agents can automate substantial portions of technical work. They can accelerate competent researchers and allow small teams to attempt projects that once required larger staff. But the evidence does not yet show that they can independently run the research process from question selection to credible contribution.

That distinction should govern both investment and policy. It is foolish to assume that a machine is harmless because it lacks general judgement. It is equally foolish to assume that a machine has general judgement because it can complete an impressive chain of subtasks.

The real labour-market divide is between execution and direction

For workers, the most useful lesson is not a promise of safety. It is a warning about where exposure is greatest.

Tasks that involve retrieving information, transforming documents, following explicit rules, reconciling known categories or producing a first draft are vulnerable because their success can be specified and checked. They may remain valuable, but the price of performing them is likely to fall. A worker whose role consists almost entirely of these activities will face pressure even if no AI system can perform the occupation from start to finish.

The safer layer is not simply “creativity”. That word has become too cheap. Models can produce novel combinations and attractive ideas. The more defensible advantage is the ability to set direction under uncertainty: deciding what problem is worth solving, which constraints matter, what evidence would change the decision, when an experiment has failed and who should be accountable for the outcome.

Those abilities are visible in ordinary work. A nurse notices that a patient does not fit the expected pattern and escalates the case. An engineer rejects a technically elegant design because it will fail in the field. A teacher recognises that a student’s wrong answer reveals a misconception rather than laziness. A manager stops a profitable project because its hidden risk has become unacceptable. A journalist decides that a document is not merely evidence for the expected story but a reason to change the story.

None of this means people should refuse to use AI. The opposite is true. Workers should use it to compress the mechanical layer so that more attention can be spent on the decisions that remain difficult. Ask the system to retrieve, compare, draft and criticise. Do not hand it the final authority over what counts as a good question.

A useful division of labour is to make the model an aggressive assistant and a subordinate decision-maker. It should generate alternatives, expose missing evidence, test assumptions and attempt to break the argument. The human should decide the standard, the purpose and the acceptable risk. When the answer matters, the human should be able to explain why the selected path was chosen over the plausible alternatives.

That requires a change in how people learn. Memorising information still matters, but information without a working model of the world is fragile. The valuable worker will not be the person who has never used AI. Nor will it be the person who presses a button and accepts the first fluent response. It will be the person who knows enough about the domain to recognise a bad premise, a missing variable and a conclusion that is true but irrelevant.

The task is to move upstream. Do not compete with the machine at the layer where it is cheapest. Become better at defining the work, judging its quality and changing direction when reality says the plan is wrong.

The machine can do more of the job without becoming the worker

The most misleading sentence in the AI debate is that a system either can or cannot do a job. Jobs are bundles of tasks held together by a purpose, a set of relationships and a chain of responsibility.

An agent may perform more of the bundle each year. It may draft the report, run the analysis, schedule the experiments and prepare the recommendation. That progress can be commercially decisive. But the job is not exhausted by its visible outputs. Someone still has to decide what the report is for, whether the analysis answers the real question, whether the recommendation is safe and what should happen after the first plan fails.

The Princeton study exposed that hidden layer under unusually demanding conditions. The agents were given an open question, resources and time. They did not need a human to type each command. They still needed the kind of judgement that determines whether all those commands are converging on knowledge.

That is why the study matters beyond academic research. In an office, a model can create a polished strategy document built on the wrong market assumption. In a hospital, it can summarise the chart while missing the patient who does not match the average. In a public agency, it can optimise a target that measures activity rather than service. In a newsroom, it can produce a coherent account from the available statements while failing to identify the fact that changes the story.

Execution is visible because it leaves files, screens and completed checklists. Direction is harder to display. It appears in the decision not to publish, not to deploy, not to hire, not to approve, not to continue. The absence of an action can be the most intelligent outcome, but it is difficult to demonstrate in a product launch.

The pressure to automate will continue. So will the temptation to confuse a reduction in labour per task with the elimination of human responsibility. The evidence now points to a more exact description of the near future: AI will take over pieces of work, sometimes large pieces, while humans remain responsible for making the pieces cohere.

That arrangement may be unstable. If machines eventually acquire robust causal models, durable goals, reliable feedback and the capacity to revise their own standards, the boundary will move. The CRUX researchers do not claim otherwise. Their result is a report on what present systems did when the task stopped being a chain of instructions and became a problem of judgement.

For now, the line is plain. The agents could perform much of the engineering. They could not reliably decide what the engineering was supposed to achieve. They completed the work that could be itemised and failed at the work that gave the itemised tasks a reason.

That is not a reason to relax. It is a reason to stop measuring human value by the number of routine actions a person can complete. The scarce skill is knowing which action deserves to be taken, what would count as success and when to walk away from a failing plan.

References

1. Peter Kirgis, Sayash Kapoor, Andrew Schwartz et al., Can AI agents conduct open-ended AI research? Early evidence from two case studies, arXiv, 2026. https://arxiv.org/abs/2607.27191 ↩︎
2. Jonathan Kemper, “Study contradicts Anthropic and OpenAI claims that autonomous AI research is within reach”, The Decoder, 14 August 2026. https://www.the-decoder.com/study-contradicts-anthropic-and-openai-claims-that-autonomous-ai-research-is-within-reach ↩︎
3. Evelina Fedorenko, Steven T. Piantadosi and Edward A. F. Gibson, “Language is primarily a tool for communication rather than thought”, Nature, volume 630, pages 575–586, 2024. https://www.nature.com/articles/s41586-024-07522-w ↩︎
4. Judea Pearl and Dana Mackenzie, The Book of Why, Basic Books, 2018. https://www.basicbooks.com/titles/judea-pearl/the-book-of-why/9780465097616 ↩︎
5. Jeremy Greenwood, Ananth Seshadri and Mehmet Yorukoglu, “Engines of Liberation”, The Review of Economic Studies, volume 72, issue 1, pages 109–133, 2005. https://academic.oup.com/restud/article-abstract/72/1/109/1581345 ↩︎
6. Kristie M. Engemann and Michael T. Owyang, “Social Changes Lead Married Women into Labor Force”, Federal Reserve Bank of St. Louis, 1 April 2006. https://www.stlouisfed.org/publications/regional-economist/april-2006/social-changes-lead-married-women-into-labor-force ↩︎
7. CRUX, “Can AI agents conduct open-ended AI research? Early evidence from two case studies”, methodology and limitations of the shadow evaluation. https://cruxevals.com/crux/can-ai-agents-conduct-research ↩︎