The Natmaster - Natmaster.com
I might not be the best representation of human writings - so I decided to test the detector against some well known texts. Here are the results.
- Shakespeare's "The Tempest: Act 2, Scene 1" - Inauthentic with a 14% chance of being authentic.
- Shakespeare's "Much Ado about Nothing: Act 4, Scene 1" - Inauthentic with a 38% chance of being authentic.
- The Bible, NVI: Colossians 4 - Inauthentic with a 26.5% chance of being authentic.
- The Bible, NVI: Luke 11 - Inauthentic with a 20% chance of being authentic.
- The Bible, NVI: Romans 10 - Inauthentic with a 20.4% chance of being authentic.
- The Bible, NVI: Exodus 13 - Inauthentic with a 17.6% chance of being authentic.
- Tolkien's "Lord of the Rings: Book 1, Chapter 1" - Inauthentic with a 32.8% chance of being authentic.
- Tolkien's "Lord of the Rings: Book 3, Chapter 4" - Inauthentic with a 35.5% chance of being authentic.
As you can see - either many famous writers are all robots, or this detector is simply bogus. Each text extract generates seemingly random numbers that appear to have no correlation to their author (being human or not). In fact, the average score human-written texts got was lower than my randomly generated texts. (Although my experiments did not have a large enough sample space to be considered scientific evidence.) In the end, I believe this device is simply a glorified random number generator.
The Discovery
Many of you have heard about the MIT students who devised an algorithm to generate fake research papers. These students even got accepted to scientific conferences to give a "randomly generated talk" (as they call it) on their paper.
When I first read about this algorithm, visions of the future dashed in my head. No longer any need for writers wasting their life - computers can write books and articles for them! Naturally, I was crushed to find out that someone else had devised an "Inauthentic Paper Detector" to counter the wonderful possibilities. Just feed your article into the program and it will tell you if there was a soul behind the pen.
Something to it?
Can an algorithm out-smart another? Is there really a way to detect algorithmically generated text? Can't the generator just improve itself to counter detection mechanisms?
These are all great questions. Maybe I'll answer them another time, but for now I am covering THIS inauthentic paper detector. Curious as to the accuracy of this device, I hurried to the site to do some experimenting.
First, I started randomly generating articles written by me at the MIT student's website. I then copied each paper into the detector and crunched away. I was disappointingly surprised to see that it was detecting them all as "inaccurate". Each trial showed the probability of the texts being authetic as approximately 30%.
Disconcerted, I decided to authenticate some articles I had written. I tried some articles on this site, as well as papers I had written for school. It's been confirmed by multiple sources - I'm a robot. My writings consistently placed worse than their randomly-generated peers - usually scoring around 20%.
Conclusion
I haven't confirmed that I'm a robot from a third-party, but I'm guessing there are some improper assumptions the creators made when producing their detector. Obviously, my writing style confuses it. (I didn't know I was that boring.)
I suggest you try it out yourself - determine how robotic your writing is. When you're done, maybe you'll be as skeptical as I am about the accuracy of this device. Obviously, it is not perfect - but can there EVER be a "perfect authentication detector"? In my humble opinion, I think not. Every iteration of detector will spurn trivial (since the detction techniques are known) enhancements to the producer that easily counter the new detction mechanisms.
