How To Detect Text Written By AI

How to Detect Text Written by AI

Once my mutuals reminded me not to let fear take over and to get out of my own head, I did. And when I did, I realized something—those who are feeding authors’ fics into these notoriously unreliable AI detectors are actually helping these companies by giving them more data. And most of these companies? They also have their own AI content generation tools, which means they are using the samples of the writing of from those authors’ to improve their own AI generation tools.

So now, I’m not just scared—I’m scared and mad.

Instead of making unproductive call-out posts that don’t actually help the community recognize AI-generated text, let’s do something useful. Let’s talk about ways you can be empowered as a reader to spot AI-generated content.

There is no fool proof method to figure out for sure if a text is AI generated, unless you are literally physically sitting with the writer watching them write like they are a zoo animal.

AI detectors are unreliable and inconsistent. Plus, with how quickly AI-generated content is evolving, these tools are constantly lagging behind, making them outdated and even less effective over time.

Thus far, the best AI detection is YOU. It isn’t easy at first, but the research shows that you can learn how to do this.

One article from the MIT Technology Review (2019) states:

Another study found that untrained humans were able to correctly spot text generated by GPT-3 only at a level consistent with random chance. The good news is that people can be trained to be better at spotting AI-generated text, Ippolito says. She built a game to test how many sentences a computer can generate before a player catches on that it’s not human, and found that people got gradually better over time.  “If you look at lots of generative texts and you try to figure out what doesn’t make sense about it, you can get better at this task,” she says. One way is to pick up on implausible statements, like the AI saying it takes 60 minutes to make a cup of coffee.

Fun fact: This article has the journalist submitting her own work to various AI detectors, one of them being Originality.AI, which indicated her writing had a 50-50 split on being AI and human.

One of the problems with these large language models (LLMs), as this article from Capitol Technology University points out, is that it has made AI generated text more complex, making it harder to tell it apart from human writing. But, they share some signs you can look for, as described from their article below:

Inconsistencies and repetition: Occasionally, AI produces nonsensical or odd sentences which can be a clear indicator of AI-generated text. Abrupt shifts in tone, style, or topic can point to AI that is struggling to maintain coherent ideas. Whereas humans often vary structure to create a better flow, repeated phrases or sentence structures can point to AI relying more on memorized patterns. Occasionally, AI produces nonsensical or odd sentences which can be a clear indicator of AI-generated text.

Context and content: If the text seems to be unable to grasp the larger context of the writing, is missing the point entirely, or references specific details without appropriate context, it could be AI.

Does that mean all inconsistencies, excessive repetition, or overly descriptive writing automatically indicate AI-generated text? No, absolutely not. Writing styles vary, and many human writers naturally have quirks like these.

This article from Forbes (2024) identified 5 ways to help identify AI generated content, but I feel these 3 specific ones best apply to fiction and fanfics:

Language Patterns: AI generated text tends to lack emotional subtlety, be overly formal, or use complex words, leading to the sentences sounding stiff and not flowing well. Alternatively, the over use of cliches. Consistency Issues: AI generated text tends to struggle with narrative details, leading to abrupt changes in the story. For example an abrupt change in the description of a setting without explanation. Unusual Language Errors: Sometimes AI generated text uses odd and unusual phrases that feel out of place.

Again, does this automatically mean that writing lacking emotional subtlety or being overly formal is AI-generated? No, not at all. Writing styles vary, and some authors naturally have a more formal tone or a direct approach to storytelling.

If I find a story that I think is AI-generated, what should I do?

1. Pause and Assess – Don't Jump to Conclusions

AI-generated text can sometimes be hard to distinguish from human writing, and many of the so-called “signs” of AI can also be just someone’s writing style or someone being new to writing. Before assuming a fic is AI-generated, take a step back and look at it critically.

Does it actually feel off in a way that suggests AI (such as major consistency errors, repetitive phrases, or nonsensical sentences)?

Or does it just have a different writing style than you’re used to?

2. Don’t Rely on AI Detectors

As shown in multiple studies and real-world examples, AI detectors are not reliable. They are inconsistent, often outdated, and can flag even completely human-written work as AI. Using them as definitive proof, even with the above information does more harm than good.

Also, some of these AI detectors programs also have AI generation programs. You are literally providing more data to these programs to help improve generate AI text.

If an AI detection program does not appear to have a a sister program that generate AI content, ask yourself:

Could this company be mangaged by a parent company that DOES have an AI generated program?

Have you read the terms and conditions to agree to use this AI detection program? If not, have you considered where does the data you have it analyze go? Is it stored and sold to 3rd parties to be used to improve AI generated content programs?

3. Consider Reaching Out to the Author First

If you still have doubts, and you feel it’s appropriate, you could politely reach out to the author. Instead of accusing them, ask about their writing process or how they developed the story. Most human writers love to talk about their inspiration, research, and creative choices.

4. Avoid Public Call-Outs

Unless you have strong, irrefutable proof (which is very hard to get), publicly accusing someone of using AI can do serious harm—both to the writer and to the community as a whole. False accusations drive real writers away from sharing their work.

5. If You’re Concerned About AI in Creative Spaces, Advocate for Constructive Discussions

Instead of call-outs and AI detectors, push for conversations on how to navigate AI in fandoms and creative writing spaces in a way that doesn’t rely on fear or false accusations. Encourage transparency, but also respect that writers shouldn’t have to prove their humanity just to share their work.

Thank you and I will now go back to playing in my little section of the dragon age sandbox where I make Solas kiss Lavellan.