PURPOSE: To evaluate the accuracy of ChatGPT references in scientific writing relevant to head and neck surgery. MATERIALS AND METHODS: Five commonly researched keywords relevant to head and neck surgery were selected (osteoradionecrosis of the jaws, oral cancer, adjuvant therapy for oral cancer, TORS, and free flap reconstruction in oral cancer). The AI chatbot was then asked to provide ten complete citations for each of the keywords. Two independent authors reviewed the results for accuracy and assigned each article a numerical score based on pre-selected criteria. RESULTS: Among 50 total references provided by ChatGPT, only five (10 %) were found to have the correct title, journal, authors, year of publication, and DOI. Merely 14 % of the presented references had correct DOI. References regarding free flap reconstruction for oral cancer were the least accurate from all the five categories, with no correct DOI. Complete inter-rater agreement was noted while evaluating the citations. CONCLUSION: Only 10 % of the articles provided by ChatGPT, relevant to head and neck surgery, were correct. A high degree of academic hallucination was noted.
No clinical trial protocols linked to this paper
Clinical trials are automatically linked when NCT numbers are found in the paper's title or abstract.PICO Elements
No PICO elements extracted yet. Click "Extract PICO" to analyze this paper.
Paper Details
MeSH Terms
Associated Data
No associated datasets or code repositories found for this paper.
Related Papers
Related paper suggestions will be available in future updates.