M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection

The advent of Large Language Models (LLMs) has brought an unprecedented surgein machine-generated text (MGT) across diverse channels. This raises legitimateconcerns about its potential misuse and societal implications. The need toidentify and differentiate such content from genuine human-generated text iscritical in combating disinformation, preserving the integrity of education andscientific fields, and maintaining trust in communication. In this work, weaddress this problem by introducing a new benchmark based on a multilingual,multi-domain, and multi-generator corpus of MGTs – M4GT-Bench. The benchmarkis compiled of three tasks: (1) mono-lingual and multi-lingual binary MGTdetection; (2) multi-way detection where one need to identify, which particularmodel generated the text; and (3) mixed human-machine text detection, where aword boundary delimiting MGT from human-written content should be determined.On the developed benchmark, we have tested several MGT detection baselines andalso conducted an evaluation of human performance. We see that obtaining goodperformance in MGT detection usually requires an access to the training datafrom the same domain and generators. The benchmark is available athttps://github.com/mbzuai-nlp/M4GT-Bench.

Further reading#

• Access Paper in arXiv.org