Giovanni’s group wrote an ACL-accepted paper on how to evaluate AI generated news vs true news using LLAMA’s generations for the creation of the fake news.
The approach they used is simple:
- If the model is fine-tuned is harder to detect;
- They also used human evaluation to see how human performs in detecting fake news written by AI;
- Then they trained a classifier to detect the AI generated text. It reached high accuracy. Human, however, reached accuracy near 1 (This was done on multiple languages);
It seems that classification models uses very easy (syntactic) structures to identify which texts are fakes. Fake texts seem to be less variable.
Detection seem to work very well on established dataset, and breaking it could be interesting creating more complex dataset to better detect AI generated text.
New stuff & Ideas
xsum (dataset of news on hugginface) has been used to generate fake news from its articles headlines.
With this, they have 100k articles and their correspective fake article.
This article are being analized with a linguistic pipeline to create linguistical features to train a simple model (Like a linear SVM) to classify wheter the text is human or AI written.
The idea is to see which is the most important feature for the SVM to classify articles as either fake or original. Then, we want to alter that feature either:
- Directly: using prompting technique to ask the model to alter that feature when generating texts from the headlines. Then with the new dataset, trying again to see if the SVM performances drops. We can do multiple steps.
- We need to understand how to verbalize the features, wheter to use one or multiple prompt, etc. (prompting engineering sucks);
- Indirectly: using the model prediction to create a pseudo-label on to the dataset. Then, we tag the entry that the SVM thinks are true as the preferred one, and do a DPO fine-tuning on the model.
- We believe that using the SVM as a proxy for human preference, i.e. what human thinks are human generated text, may work.
- We should do this also with a fraction of the dataset and asking real humans what they think is or isn’t generated. We could see an inter-annotator agreement between SVM and human, and see how the model performs after the DPO step done both by humans and the svm labels.
- After each training (may they be the direct approach of altering a feature or the DPO training with the svm preferences) we collect human data on how they perceive the newly generated texts from the headlines.