Untitled

It really hit home for me when Google shipped their 'writing help' feature or whatever in gmail. Pretty much every sentence I wrote, it had useful tips for me like "make this more concise" and "reduce run-on sentence". No joke, I averaged about 1.3 suggestions per sentence. And pretty much universally, they were awful. It was like somebody had blindly taken an English writing textbook and applied every little helper rule to every clause of every sentence, paying no attention whatsoever to deeper concepts like pacing, drawing attention to key words, and moderating emotion. And I think that's basically what's going on in all of the frontier models as well. When they are being fine tuned, some reward score is telling them to be concise, not to use filler words, to avoid run-on sentences, etc. And there's no counter-balance in the fine-tuning that looks at the writing and flow as a whole.

Yea the hypothesis seems the exact opposite of what I find with LLM writing. It’s not that it’s overly dense instead I find that it’s hard to read because it’s so information sparse. Like you’re saying it feels optimized to “good writing” not to conveying information.

I'm a writer, a technical writer, and a highly technical person in the executive space. Communication has always been one of my strong points, and the blathering on that AI does drives me to distraction. It's impossible to really tailor the output of LLMs very much. I value information dense, direct writing when it comes to technical things and they're always embellishing and going off on side quests and it's obnoxious.

As both a writer and SWE the problem with LLM output is the same. They can achieve localized goals but have trouble juggling multiple concerns at the same time. Things that are easy for humans, like remembering a character is holding a gun during a conversation, gets forgotten, because there is no clear architectural/spacial understanding of how the words are describing an underlying reality. And if the LLM gets this right it might instead forget which character has a Southern drawl, or make an illogical metaphor. This is all more clear with programming, but the problem is the same - a lack of interwoven semantic understanding and friction between the real world and the self-contained world of the LLM's context.

With ChatGPT, I notice it tends to use abstract subjects. E.g., ChatGPT wrote "Texas allows treated sewage sludge to be used as agricultural fertilizer, yet the proposed law requiring routine PFAS testing failed to advance". I would have written "Texas allows treated sewage sludge to be used as agricultural fertilizer, yet the state failed to advance the proposed law requiring routine PFAS testing". No matter how many times, I ask it to avoid fluff; it keeps writing it. E.g., Please avoid fluff sentences like "This is where the story gets more interesting." and "Here is how the evidence compares." They are not needed and serve no purpose. I also notice its sentences do not provide a clear sense of who did what. E.g., it wrote "According to pilgrimage organizer Christian Marquant, Bishop Schneider met with Pope Leo in December 2025 and discussed the traditional Latin Mass." I would have wrote "According to pilgrimage organizer Christian Marquant, Bishop Schneider discussed the traditional Latin Mass with Pope Leo on December 2025." The phrase "met with" was extraneous and made it seem like the meeting was by accident.