Will Yeadon

Will Yeadon

Auto-generated profile, based on publicly available information

Will Yeadon

Department of Physics Durham University UK

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
5 April 2023 in Physics Education
Verified
Journal Article
Open access
PDF file
WYWill Yeadon
Will Yeadon
OIOto-Obong Inyang
Oto-Obong Inyang
AMArin Mizouri
Arin Mizouri
APAlex Peach
Alex Peach
CTCraig P Testrow
Craig P Testrow
The latest AI language modules can produce original, high quality full short-form (300-word) Physics essays within seconds. These technologies such as ChatGPT and davinci-003 are freely available to anyone with an internet connection. In this work, we present evidence of AI generated short-form essays achieving First-Class grades on an essay writing assessment from an accredited, current university Physics module. The assessment requires students answer five open-ended questions with a short, 300-word essay each. Fifty AI answers were generated to create ten submissions that were independently marked by five separate markers. The AI generated submissions achieved an average mark of 71 ± 2 % , in strong agreement with the current module average of 71 ± 5 % . A typical AI submission would therefore most-likely be awarded a First Class, the highest classification available at UK universities. Plagiarism detection software returned a plagiarism score between 2 ± 1 % (Grammarly) and 7 ± 2 % (TurnitIn). We argue that these results indicate that current natural language processing AI represent a significant threat to the fidelity of short-form essays as an assessment method in Physics courses.

6 February 2024 in Physics Education
Verified
Journal Article
Open access
WYWill Yeadon
Will Yeadon
THTom Hardy
Tom Hardy
With the rapid evolution of artificial intelligence (AI), its potential implications for higher education have become a focal point of interest. This study delves into the capabilities of AI in physics education and offers actionable AI policy recommendations. Using openAI’s flagship gpt-3.5-turbo large language model (LLM), we assessed its ability to answer 1337 physics exam questions spanning general certificate of secondary education (GCSE), A-Level, and introductory university curricula. We employed various AI prompting techniques: Zero Shot, in context learning, and confirmatory checking, which merges chain of thought reasoning with reflection. The proficiency of gpt-3.5-turbo varied across academic levels: it scored an average of 83.4% on GCSE, 63.8% on A-Level, and 37.4% on university-level questions, with an overall average of 59.9% using the most effective prompting technique. In a separate test, the LLM’s accuracy on 5000 mathematical operations was found to be 45.2%. When evaluated as a marking tool, the LLM’s concordance with human markers averaged at 50.8%, with notable inaccuracies in marking straightforward questions, like multiple-choice. Given these results, our recommendations underscore caution: while current LLMs can consistently perform well on physics questions at earlier educational stages, their efficacy diminishes with advanced content and complex calculations. LLM outputs often showcase novel methods not in the syllabus, excessive verbosity, and miscalculations in basic arithmetic. This suggests that at university, there’s no substantial threat from LLMs for non-invigilated physics questions. However, given the LLMs’ considerable proficiency in writing physics essays and coding abilities, non-invigilated examinations of these skills in physics are highly vulnerable to automated completion by LLMs. This vulnerability also extends to pysics questions pitched at lower academic levels. It is thus recommended that educators be transparent about LLM capabilities with their students, while emphasizing caution against overreliance on their output due to its tendency to sound plausible but be incorrect.

21 October 2024 in International Journal for Educational Integrity
Verified
Journal Article
Open access
PDF file
TRT. Revell
T. Revell
WYW. Yeadon
W. Yeadon
GCG. Cahilly-Bretzin
G. Cahilly-Bretzin
ICI. Clarke
I. Clarke
GMG. Manning
G. Manning
FLF. Leneghan
F. Leneghan
Generative AI has prompted educators to reevaluate traditional teaching and assessment methods. This study examines AI’s ability to write essays analysing Old English poetry; human markers assessed and attempted to distinguish them from authentic analyses of poetry by first-year undergraduate students in English at the University of Oxford. Using the standard UK University grading system, AI-written essays averaged a score of 60.46, whilst human essays achieved 63.57, a margin of difference not statistically significant (p = 0.10). Notably, student submissions applied a nuanced understanding of cultural context and secondary criticism to their close reading, while AI essays often described rather than analysed, lacking depth in the evaluation of poetic features, and sometimes failing to properly recognise key aspects of passages. Distinguishing features of human essays included detailed and sustained analysis of poetic style, as well as spelling errors and lack of structural cohesion. AI essays, on the other hand, exhibited a more formal structure and tone but sometimes fell short in incisive critique of poetic form and effect. Human markers correctly identified the origin of essays 79.41% of the time. Additionally, we compare three purported AI detectors, finding that the best, ‘Quillbot’, correctly identified the origin of essays 95.59% of the time. However, given the high threshold for academic misconduct, conclusively determining origin remains challenging. The research also highlights the potential benefits of generative AI’s ability to advise on structuring essays and suggesting avenues for research. We advocate for transparency regarding AI’s capabilities and limitations, and this study underscores the importance of human critical engagement in teaching and learning in Higher Education. As AI’s proficiency grows, educators must reevaluate what authentic assessment is, and consider implementing dynamic, holistic methods to ensure academic integrity.

7 October 2024 in Scientific Reports
Verified
Journal Article
Open access
PDF file
WYWill Yeadon
Will Yeadon
APAlex Peach
Alex Peach
CTCraig Testrow
Craig Testrow
This study evaluates the performance of ChatGPT variants, GPT-3.5 and GPT-4, both with and without prompt engineering, against solely student work and a mixed category containing both student and GPT-4 contributions in university-level physics coding assignments using the Python language. Comparing 50 student submissions to 50 AI-generated submissions across different categories, and marked blindly by three independent markers, we amassed $$n = 300$$ n = 300 data points. Students averaged 91.9% (SE:0.4), surpassing the highest performing AI submission category, GPT-4 with prompt engineering, which scored 81.1% (SE:0.8)—a statistically significant difference (p = $$2.482 \times 10^{-10}$$ 2.482 × 10 - 10 ). Prompt engineering significantly improved scores for both GPT-4 (p = $$1.661 \times 10^{-4}$$ 1.661 × 10 - 4 ) and GPT-3.5 (p = $$4.967 \times 10^{-9}$$ 4.967 × 10 - 9 ). Additionally, the blinded markers were tasked with guessing the authorship of the submissions on a four-point Likert scale from ‘Definitely AI’ to ‘Definitely Human’. They accurately identified the authorship, with 92.1% of the work categorized as ‘Definitely Human’ being human-authored. Simplifying this to a binary ‘AI’ or ‘Human’ categorization resulted in an average accuracy rate of 85.3%. These findings suggest that while AI-generated work closely approaches the quality of university students’ work, it often remains detectable by human evaluators.

1 September 2024 in European Journal of Physics
Verified
Journal Article
Open access
WYWill Yeadon
Will Yeadon
EAElise Agra
Elise Agra
OIOto-Obong Inyang
Oto-Obong Inyang
PMPaul Mackay
Paul Mackay
AMArin Mizouri
Arin Mizouri
This study aims to compare the academic writing quality and detectability of authorship between human and AI-generated texts by evaluating n = 300 short-form physics essay submissions, equally divided between student work submitted before the introduction of ChatGPT and those generated by OpenAI’s GPT-4. In blinded evaluations conducted by five independent markers who were unaware of the origin of the essays, we observed no statistically significant differences in scores between essays authored by humans and those produced by AI (p-value = 0.107, α = 0.05). Additionally, when the markers subsequently attempted to identify the authorship of the essays on a 4-point Likert scale—from ‘Definitely AI’ to ‘Definitely Human’—their performance was only marginally better than random chance. This outcome not only underscores the convergence of AI and human authorship quality but also highlights the difficulty of discerning AI-generated content solely through human judgment. Furthermore, the effectiveness of five commercially available software tools for identifying essay authorship was evaluated. Among these, ZeroGPT was the most accurate, achieving a 98% accuracy rate and a precision score of 1.0 when its classifications were reduced to binary outcomes. This result is a source of potential optimism for maintaining assessment integrity. Finally, we propose that texts with ≤50% AI-generated content should be considered the upper limit for classification as human-authored, a boundary inclusive of a future with ubiquitous AI assistance whilst also respecting human-authorship.

17 June 2020 in Journal of Instrumentation
Verified
Journal Article
Open access
PDF file
LCL. Cooper
L. Cooper
MCM. Crouvizier
M. Crouvizier
SES. Edwards
S. Edwards
RFR. French
R. French
FGF. Gannaway
F. Gannaway
WYW. Yeadon
W. Yeadon
Ultra-thin walled cooling tubes for heat exchangers and condenser units have applications in multiple high-value manufacturing industries. Grade 2 commercially pure titanium (CP-2 Ti) requires far less mass to achieve the same mass flow handling abilities as stainless steel tubing yet it is more challenging to join, particularly at wall thicknesses less than 500 μm (termed ultra-thin walled tube). This paper presents a single-pass joinery method that produces reliable welds on 2.275 mm outer diameter (OD), 160 ± 10 μm wall thickness tubing with a service life of 20 of more years. This is achieved through an automated orbital gas tungsten constricted arc welding (GTCAW) process incorporating enveloping low-mass sleeves used in tandem with a buttressing internal gas pressure to support the molten metal and maintain consistent internal diameter inside the tube. The industrial applicability is demonstrated through the production of a 1:1 scale mock-up of a fixed geometry CO2 cooling circuit for a next-generation particle detector. The tensile strengths of the joints, 403.8 ± 4.2 MPa, exceed the tensile strength of the parent CP-2 Ti.