Eight LLM detector configurations (ChatGPT-3.5, GPT-4, Llama2-7B-chat, Llama2-13B-chat, each with and without zero-shot CoT) were evaluated on PolitiFact, GossipCop, and COAID. Detection accuracy on LLM-generated misinformation is consistently lower than on human-written misinformation. For example, GPT-4 (no CoT) on PolitiFact drops from 48.6% (human-written) to 22.0% (open-ended generation); Llama2-7B-chat (CoT) drops from 47.4% to 27.8% on rewriting generation (19.6% drop). Results hold across all six open-source generators (Llama2-7B/13B/70B, Vicuna-7B/13B/33B) and all three datasets, with most p-values below 0.05.
Zero-shot setting only; average over three runs for API-based detectors due to output variance; GossipCop and COAID results show less consistent drops than PolitiFact for some detector-generator pairs.