Berliner Boersenzeitung - Spotting AI writing: how reliable are the detectors?

EUR -
AED 4.035801
AFN 70.873922
ALL 90.10994
AMD 393.199697
ANG 1.96752
AOA 1008.813772
ARS 1713.49854
AUD 1.638405
AWG 1.980813
AZN 1.866213
BAM 1.959224
BBD 2.211845
BDT 136.718936
BGN 1.948912
BHD 0.414413
BIF 3300.583265
BMD 1.098926
BND 1.416382
BOB 12.865895
BRL 6.471461
BSD 1.098249
BTN 102.621615
BWP 15.514537
BYN 3.294533
BYR 21538.940421
BZD 2.20864
CAD 1.572606
CDF 2511.044887
CHF 0.926279
CLF 0.028107
CLP 1106.200355
CNY 7.327413
CNH 7.328064
COP 3686.57647
CRC 488.248843
CUC 1.098926
CUP 26.357262
CVE 110.461566
CZK 24.417741
DJF 195.553538
DKK 7.474606
DOP 66.395431
DZD 150.61112
EGP 59.150029
ERN 16.483883
ETB 180.246247
FJD 2.51132
FKP 0.835601
GBP 0.835678
GEL 2.845964
GGP 0.835601
GHS 12.892943
GIP 0.835601
GMD 80.772848
GNF 9736.148226
GTQ 8.392935
GYD 229.678712
HKD 8.62078
HNL 29.519859
HRK 7.507821
HTG 143.982285
HUF 369.165393
IDR 20060.885581
ILS 3.60624
IMP 0.835601
INR 102.510576
IQD 1664.855025
IRR 2124403.82307
ISK 137.003159
JEP 0.835601
JMD 174.182825
JOD 0.779156
JPY 175.760502
KES 142.970955
KGS 96.100969
KHR 4473.839038
KMF 493.417659
KPW 989.033363
KRW 1459.585735
KWD 0.342447
KYD 0.915224
KZT 488.406895
LAK 24742.366625
LBP 98453.662215
LKR 365.485368
LRD 184.495791
LSL 18.878853
LTL 3.244841
LVL 0.664729
LYD 7.102477
MAD 10.938164
MDL 20.162696
MGA 4993.86383
MKD 61.501024
MMK 2307.525378
MNT 3957.026494
MOP 8.875554
MRU 44.629779
MUR 51.473589
MVR 16.921006
MWK 1904.210556
MXN 20.704724
MYR 4.520102
MZN 70.21819
NAD 18.878508
NGN 1485.373439
NIO 40.41136
NOK 10.650594
NPR 164.193434
NZD 1.988725
OMR 0.422538
PAB 1.098249
PEN 3.749252
PGK 4.940634
PHP 69.990521
PKR 303.624617
PLN 4.416626
PYG 6874.075997
QAR 4.003725
RON 5.345723
RSD 117.641046
RUB 89.919561
RWF 1625.225528
SAR 4.12249
SBD 8.942002
SCR 15.256719
SDG 660.998503
SEK 11.146748
SGD 1.417153
SHP 0.832015
SLE 25.053508
SLL 23043.922963
SOS 627.591095
SRD 41.082779
STD 22745.539784
STN 24.542892
SVC 9.609056
SYP 14288.230133
SZL 18.875503
THB 37.248629
TJS 10.097923
TMT 3.846239
TND 3.326119
TOP 2.645949
TRY 55.44783
TTD 7.452363
TWD 36.195863
TZS 2917.922262
UAH 49.579396
UGX 4601.187787
USD 1.098926
UYU 44.619197
UZS 13189.470587
VES 1007.305858
VND 28302.827068
VUV 132.914078
WST 3.102576
XAF 655.957
XAG 0.024386
XAU 0.000295365626
XCD 2.969902
XCG 1.979213
XDR 0.776997
XOF 655.957
XPF 119.331742
YER 259.950923
ZAR 18.957108
ZMK 9891.653254
ZMW 22.556215
ZWL 353.853573
SSP 6461.241782
MXV 2.321538
  • CMSD

    -0.1200

    19.81

    -0.61%

  • JRI

    0.0000

    10.77

    0%

  • BCC

    -1.8200

    74.15

    -2.45%

  • CMSC

    -0.2228

    19.53

    -1.14%

  • BCE

    0.2000

    20.34

    +0.98%

  • RIO

    0.0100

    94.17

    +0.01%

  • AZN

    -2.7400

    161.47

    -1.7%

  • GSK

    -1.1800

    48.17

    -2.45%

  • RBGPF

    0.0000

    65

    0%

  • NGG

    0.8800

    75.82

    +1.16%

  • RYCEF

    -0.7300

    19.08

    -3.83%

  • BTI

    -0.4500

    54.4

    -0.83%

  • BP

    0.4700

    43.99

    +1.07%

  • RELX

    -0.0600

    33.18

    -0.18%

  • VOD

    -0.1800

    16.15

    -1.11%

Spotting AI writing: how reliable are the detectors?
Spotting AI writing: how reliable are the detectors? / Photo: Joel Saget - AFP

Spotting AI writing: how reliable are the detectors?

Claims that Canadian-Haitian writer Thelyson Orelien used AI tools to write his best-selling debut novel are rocking the French literary world and posing questions about how AI can be detected.

Text size:

An X account with a handful of followers sparked the controversy by claiming that the AI-detecting program Pangram had concluded the tale of a Haitian migrant travelling to Canada had been largely written by a text-generator such as Claude or ChatGPT.

Orelien, 38, denies the allegations and has said he will prove that he wrote the book, which has been sold to publishers in more than 20 countries.

"I didn't write this novel with my head, but with my guts and my heart," he told AFP recently.

AFP ran excerpts of the book through various AI detection tools and came up with a variety of results -- from it being "very likely" to have been written by a human to "100 percent" AI generated.

- How do AI detectors work? -

Dozens of tools are now available online to detect AI generated text -- many of them designed for use in education or publishing.

They generally have large troves of AI-generated text alongside content written by humans -- and compare the two.

Their performance has improved significantly in recent years.

In a paper published in October 2025 called "Artificial Writing and Automated Detection", researchers at the University of Chicago found Pangram "achieving near-zero error rates" on medium-to-long extracts.

- What are the clues? -

Thierry Poibeau, a researcher at France's national CNRS research institute and specialist in natural language processing, said the programs look for "linguistic tics".

"The classic example is triadic phrasing (using three clauses), or em dashes," Poibeau explained.

These telltale signs used to be common in generated texts because the tools had been trained on American scientific writing or American novels, he said.

Pangram claims it is able to "detect AI text by analysing and understanding subtle cues in the writing".

Some words or phrases show up regularly, while monotonous sentence structures are also a warning sign.

- Are the tools reliable? -

Many tools publish percentage figures about their accuracy, but the figures are impossible to verify and do not take into account the risk of a false positive.

Pangram CEO and co-founder Max Spero stressed to the American outlet The Atlantic that the tool should not be the ultimate arbiter, but rather a starting point for a more in-depth investigation.

According to Thierry Poibeau, these detection systems also run up against a "moving target problem" because AI programs are "constantly evolving".

The expert also points to a potential problem when it comes to detecting the use of AI in novels.

"The tool may pick up something particular in the writing, something repetitive, but that might be exactly what constitutes a writer's style," he said.

Orelien claimed this week his style was based on Haitian and Caribbean traditions, telling Liberation newspaper the software was "used to the French literary tradition and classical authors".

- What do the experts say? -

Benoit Raphael, an entrepreneur and writer specialised in AI, published his own research using Pangram to test Orelien's work -- and the results were also positive for AI.

"The fact that it rated at around 100 percent AI, or even 90 percent, is a pretty telling sign," he told Nouvel Obs magazine.

And although the book was widely praised by critics and was flying off shelves in France before the scandal, some doubts had been raised.

Lea Bory on her podcast Torchon was possibly the first to say publicly earlier this month that she thought it had been written by IA because of the repetitive nature of the sentences and the over-use of analogies that are hallmarks of IA.

Writing in newspaper Canard Enchaine, Fabrice Colin also criticised the formulaic sentence structure and said he "vainly looked for a unique voice" in the prose.

(U.Gruber--BBZ)