Personal Log · 2mo · discuss
I am not good at detecting when a piece of text has been produced by a large language model (LLM from now on). Very often I read a post shared on a link aggregator, to later check the comments and learn that it is slop. It is true that there are some stylistic clues that can mean a text was generated (or there was some LLM assistance, at least), but I assume you have to be used to read LLM output to actual spot the more subtle cues. I was reading today in Chris’ Entropic Thoughts: Claude comment detection, and his summary of “clues” is interesting: If we jam all features at once into the model to try to get them to cancel out their redundancies, here is what remains, in order of most predictive isolated feature to least: Claude uses em dashes more than humans. Claude uses the connective interjection “so” more than humans. Claude ends comments with full stops more than humans. Claude uses semicolons more than humans. Claude uses parentheses more than humans. Claude uses the possessive form “its” more than humans. Claude uses adjectives more than humans. Claude uses the word “whether” more than humans. I guess interesting is not the right word. I do write em dashes (to avoid abusing parentheses), I tend to write “so” too much (probably because English is not my first language and it helps me to connect sentences), sometimes I use semicolons (because I tend to write long sentences), and I use parentheses (may be I write like I speak). I’ve been doing this since I started this blog in 2021. Just in case you don’t know: I don’t use LLMs to write posts. May be I should add a disclaimer somewhere, that all these em dashes are mine. Like my sometimes slightly broken English was not enough human quality. Is not that I’m reading too much into Chris’ post. You may have noticed that the AI topic is very divisive, especially online. LLMs can make people angry, in one way or another (pro or against). Sometimes they make me angry, although it is generally just intense dread. If people were happy to disclose when they have used an LLM to write some content, things would be easier. Are people embarrassed to say that they use this type of technology? Is it perhaps that they think that they are doing something wrong or they believe they are being shamed for it? I haven’t decided what to do, if the disclaimer is the right approach, or if I should make the effort to write less LLM-alike.