The rapid proliferation of the Internet of Things (IoT) has transformed modern digital ecosystems but has also introduced critical security vulnerabilities. FL has emerged as a viable solution to many of the issues faced by classical ML models in IoT environments. The growing sophistication of cyber threats, in turn, poses a threat to FL itself. Traditional detection methods, which primarily rely on static signatures, are increasingly inadequate for addressing the dynamic, evolving nature of IoT-based poisoning attacks. This research has leveraged Large Language Models (LLMs) to detect and mitigate model poisoning attacks in IoT environments. The DistilBERT LLM has been fine-tuned to first classify incoming client updates as poisoned or benign, and if an update is classified as poisoned, then optimal updates are generated for it. The model was tested on a publicly available ToN-IoT dataset, which gave an average accuracy of 89.5\% despite poisoning, while a peak accuracy of 93.7\% has also been achieved. On the other hand, Attack Success Rate (ASR) has been reduced to a mere 3.0\%.