This survey provides an in-depth review of large language models (LLMs), highlighting the significant paradigm shift they represent in artificial intelligence. Our purpose is to consolidate state-of-the-art advances in LLM design, training, adaptation, evaluation, and application for both researchers and practitioners. To accomplish this, we trace the evolution of language models and describe core approaches, including parameter-efficient fine-tuning (PEFT). The methodology involves a thorough survey of real-world LLM applications across the scientific, engineering, healthcare, and creative sectors, coupled with a review of current benchmarks. Our findings indicate that high training and inference costs are shaping market structures, raising economic and labor concerns, while also underscoring a persistent need for human oversight in assessment. Key trends include the development of unified multimodal architectures capable of processing varied data inputs and the emergence of agentic systems that exhibit complex behaviors such as tool use and planning. We identify critical open problems, such as detectability, data contamination, generalization, and benchmark diversity. Ultimately, we conclude that overcoming these complex technical, economic, and social challenges necessitates collaborative advancements in adaptation, evaluation, infrastructure, and governance.

Federated learning enables collaborative training of a model while keeping the training data decentralized and private. However, in IoT systems, inherent heterogeneity in processing power, communication bandwidth, and task size can significantly hinder the efficient training of large models. Such heterogeneity would render vast variations in the training time of clients, lengthening overall training and wasting resources of faster clients. To tackle these heterogeneity challenges, we propose Dynamic Tiering-based Federated Learning (DTFL), a novel system that leverages distributed optimization principles to improve edge learning performance. Based on clients’ resources, DTFL dynamically offloads part of the global model to the server, alleviating resource constraints on slower clients and speeding up training.By leveraging Split Learning, DTFL offloads different portions of the global model to clients in different tiers and enables each client to update the models in parallel via local-loss-based training. This helps reduce the computation and communication demand on resource-constrained devices, mitigating the straggler problem. DTFL introduces a dynamic tier scheduler that uses tier profiling to estimate the expected training time of each client based on their historical training time, communication speed, and dataset size. The dynamic tier scheduler assigns clients to suitable tiers to minimize the overall training time in each round. We theoretically prove the convergence properties of DTFL and validate its effectiveness by training large models (ResNet-56 and ResNet-110) across varying numbers of clients (from 10 to 200) using popular image datasets (CIFAR-10, CIFAR-100, CINIC-10, and HAM10000) under both IID and non-IID systems. DTFL seamlessly integrates various privacy measures without sacrificing performance. Extensive experimental results show that compared with state-of-the-art FL methods, DTFL can significantly reduce the training time by up to 80% while maintaining model accuracy.

Machine learning models are the backbone of smart grid optimization, but their effectiveness hinges on access to vast amounts of training data. However, smart grids face critical communication bottlenecks due to the ever-increasing volume of data from distributed sensors. This paper introduces a novel approach leveraging Generative Artificial Intelligence (GenAI), specifically a type of pre-trained Foundation Model (FM) architecture suitable for time series data due to its efficiency and privacy-preserving properties. These GenAI models are distributed to agents, or data holders, empowering them to fine-tune the foundation model with their local datasets. By fine-tuning the foundation model, the updated model can produce synthetic data that mirrors real-world grid conditions. The server aggregates fine-tuned model from all agents and then generates synthetic data which considers all data collected in the grid. This synthetic data can be used to train global machine learning models for specific tasks like anomaly detection and energy optimization. Then, the trained task models are distributed to agents in the grid to leverage them. The paper highlights the advantages of GenAI for smart grid communication, including reduced communication burden, enhanced privacy through anonymized data transmission, and improved efficiency and scalability. By enabling a distributed and intelligent communication architecture, GenAI introduces a novel way for a more secure, efficient, and sustainable energy future.

This survey offers an in-depth review of Large Language Models (LLMs), highlighting the significant paradigm shift they represent in artificial intelligence. Our purpose is to consolidate state-of-the-art advances in LLM design, training, adaptation, evaluation, and application for both researchers and practitioners. To accomplish this, we trace the evolution of language models and describe core approaches, including Parameter-Efficient Fine-Tuning (PEFT). The methodology involves a thorough survey of real-world LLM applications across the scientific, engineering, healthcare, and creative sectors, coupled with a review of current benchmarks. Our findings indicate that high training and inference costs are shaping market structures, raising economic and labour concerns, while also underscoring a persistent need for human oversight in assessment. Key trends include the development of multimodal and agentic systems. We identify critical open problems, such as detectability, data contamination, generalisation, and benchmark diversity. Ultimately, we conclude that overcoming these complex technical, economic, and social challenges necessitates collaborative advancements in adaptation, evaluation, infrastructure, and governance.

Despite‍‍‍ Large Language Models (LLMs) exhibiting outstanding capabilities in various natural language processing tasks, they might still be unreliable. Actually, one of their main sources of unreliability is a phenomenon called hallucination, the creation of reasonable but false pieces of information. This work provides a comprehensive overview of advances in understanding, locating, and reducing hallucinations. We start by considering hallucination as the main obstacle in creating reliable AI, and define a taxonomy that follows the development of factual errors and the notion of unfaithfulness with respect to the model's accessible knowledge. Afterwards, we survey the detection methods that are classified depending on the degree of model access and also, and we also refer to the different cognitive processes used for their comparison, which comprise uncertainty estimation, consistency checking, and knowledge-grounding evaluation. In the end, we offer a well-organized representation of the interventions aimed at the abolition of the model hallucinations employed at various stages of the model lifecycle: (1) data-centric interventions exemplified by high-quality data curation, (2) model-centric alignment through preference optimization and knowledge editing, and (3) inference-time strategies such as retrieval-augmented generation (RAG) and self-correction. We affirm that the multilayer, defense-in-depth framework incorporating these non-overlapping strategies is crucial for robust hallucination abatement. Some of the ongoing difficulties are the scalable data curation, the trade-off between alignment and model capability, and the problem of editing the reasoning pathways instead of the surface ‍‍‍facts.