What we believe now

Our model's own sentences are the clearest sign of a machine. Only human text with small edits has read as human so far.

17 tests · 25 lessons

How it got here

1. 2026-10-23did not passCan sentences from several authors on one scene topic make one scene that reads as human? The collages scored 0.71 and 0.76 with the two model readers. The scene that our writing model wrote alone scored 0.80 and 0.72. The collages were also hard to follow. One author's scene with our names scored 0.08 and 0.03. But the name swap made that scene less clear.
2. 2026-10-23did not passCan rules put each name in the correct form after we swap the names in a human scene? The rules cut the errors from 21 to 17. We needed 10 or fewer. A fault in how we matched people to names caused new errors. The machine score did not rise.
3. 2026-10-23did not passDoes our writing model read more human when it continues a human page than when it writes from a plan? The continuations scored 0.42 and 0.50. The scenes from the plan scored 0.71 and 0.72. The authors scored 0.06 and 0.03. The gain was smaller than our bar. With the lifted plan beside the page, the model followed the planned events. It did not read more like a machine. The model readers named the source book in 9 of 12 cases.
4. 2026-10-23did not passDo the rules for name forms pass when the match of people to names is fixed? The errors fell from 21 to 13. We needed 10 or fewer. Most of the errors that stay are old words of address such as sir and miss. Others are minor people that our world does not have.
5. 2026-10-23did not passDoes the gain last when our writing model continues its own text into the next scene? The two model readers scored the next scene 0.53 and 0.76. The same scene from the plan alone scored 0.74 and 0.70. Model judges rated its continuity with the story very low. A scene that we choose by topic alone does not follow from the story.
6. 2026-10-23did not passDoes a fresh human page before each scene keep the text of our writing model near human for three scenes? The parts that our model wrote scored 0.35 and 0.33. Our writing model, when it continued its own text with no human page, scored 0.45 and 0.70. Model judges rated continuity with the story very low. Each borrowed page brought its own people and history. The model readers named the source of the page in 11 and 12 of 12 cases.
7. 2026-10-23did not passDo books that the model readers do not know from one page stay unknown on their other pages? The model readers named the book of every scene with a human page. One page does not show if a model reader knows a book. A fault in the test gave 6 of 12 scenes no human page.
8. 2026-10-23did not passDoes a borrowed page fit our story when a plan comes first and the page is merged to our facts? The merge changed 8% of the sentences. The page still brought its own people and history. Model judges rated continuity lower with a page than without one. The plan alone rated highest. The model readers named 7 of the 8 scenes with a page. A fault in the test gave 4 of 12 scenes no page.
9. 2026-10-23did not passDoes a passage by our writing model that read as human help its next scene read as human? The next scenes scored 0.75 and 0.80. Scenes from a summary of the passage scored 0.75 and 0.69. The earlier gain came from the human page. It did not come from the text of our model.
10. 2026-10-23did not passCan an untrained base model follow a plan in the page? Can a second writing model continue a human page? The base model ignored the plan. It scored 0.79 and 0.81. The second writing model scored 0.22 and 0.27 and followed the plan well. A later test in one run found the two writing models about equal.
11. 2026-10-23did not passDoes a second writing model continue a human page more like a human than our usual writing model? Our usual model scored 0.33 and 0.44. The second model scored 0.31 and 0.43. The difference is too small to count. Two texts from one prompt can differ by up to 0.67. Thus we compare writers only inside one run.
12. 2026-10-23did not passCan a small base model that we train on one author continue a page from a plan? Model judges rated how well it followed the plan below our bar. It did better than the untrained model. Its machine score fell from 0.92 and 0.98 to 0.82 and 0.88. Training moved it the correct way but not far enough.
13. 2026-10-24did not passCan a model that we train on three similar authors write in one voice and hold the scene? No version passed. The best version scored 0.71 and 0.68. Its style counts were near those of the authors. Model judges rated the coherence of the trained versions far below that of our usual writing model.
14. 2026-10-24did not passCan a model that we train on the authors restyle the scenes of our writing model so that they read as human? Each pass read more like a machine. The scenes of our writing model scored 0.31 and 0.45. One pass scored 0.66 and 0.54. Two passes scored 0.71 and 0.63. The style counts became human. The model readers still called the text the work of a machine. The rewrites also changed or lost events, speakers and facts.
15. 2026-10-24did not passDoes a larger base model that we train on the same scenes hold the scene better? The 32B model scored 0.56 and 0.50. The 8B model scored 0.71 and 0.67. Model judges rated its coherence higher than before but below our bar. With one of the model readers the 32B model matched our usual writing model. It did not pass.
16. 2026-10-25did not passCan a 31B open model that we train on the inputs of our library write a new voice from before 1931? The trained model scored 0.82 as the mean of both model readers. Our usual writing model scored 0.68 on the same prompts, over the scenes that it wrote. The real scenes scored 0.03. Model judges rated the coherence of the trained model low. Our usual model refused 5 of its 24 scenes. Our first count took these refusals as scenes. Against the scenes of our usual model, the trained model won 0 of 38 blind choices. Its style counts matched human prose. The model readers still called it the work of a machine.
17. 2026-10-26did not passDoes the kind of plan change how much the prose of our writing model reads like a machine? We took the mean of the machine scores from the two model readers for each plan. These means were 0.66 to 0.72. The mean for the real scenes was 0.03. The mean for the plan with no words was 0.72. It did not pass our bar. Our model followed that plan less well than the usual plan. Model judges rated its scenes as holding together better than those from the usual plan.

← All questions