AI Editors Produce Terrible Suggestions
In all the hype around AI and its capabilities, has anyone taken the time to notice that it produces some absolutely awful editing suggestions?
Every editor makes mistakes. Editors specifically write “stet” for that purpose. However, when it comes to bad editing suggestions, people simply cannot compete with AI.
AI can miss crucial concepts in a sentence. Sometimes an AI can suggest sentences that make no sense at all. The capacity for catastrophically poor editing choices is simply beyond anything a human editor could come up with.
If you’re reading this and know that we produce Draftsmith (an AI writing refiner), you probably expect us to say that Draftsmith is different. You probably expect us to say that, somehow, we have a better AI that doesn’t make mistakes.
We don’t. Draftsmith, like any AI product, is no exception. It can offer truly terrible ideas for how to improve your text!
But that’s what happens when you use an assistant. They have some bad ideas. The question to ask yourself is whether your work is better with an assistant or without one.
We created Draftsmith because your workflow can be far faster with easy access to an AI writing refiner. When the AI is close, you can alter its suggestions. When the AI sparks an idea, you can make use of it. And when the AI produces something that’s completely wrong, you can skip it and have another suggestion in seconds.
We find that results with AI vary by the type of text you’re working on and the type of suggestions you want the AI to give. As an example, in an academic edit using the Simplify function in Draftsmith, we classified results as follows:
Excellent: We accepted the suggestion in full with no alterations required.
Close: We edited the suggestion slightly before using the rest of it.
Mediocre: The suggestion was reasonable but there wasn’t enough improvement to justify the change.
Awful: The suggestion missed an important concept or didn’t make sense at all.
Our results are shown below.
Your mileage may vary. However, in the same way that spell check also makes mistakes, the question we ask is not whether AI has bad suggestions (it clearly does). But what helps you deliver your best work in the time available? For a lot of people, the answer is working with an AI that leaves you in control of every decision.
We released Draftsmith knowing that a 1 in 20 rate of terrible suggestions would lead to a flood of #amediting screenshots that demonstrate the value of a human editor. We know. We agree. Draftsmith is not a replacement for a person.
Editing and writing refinement are, and always will be, human skills. Even as AI improves, the truth will always be that you cannot trust an AI to do editing work on its own. Text requires human checking and it always will.