Activation functions are pivotal in neural networks, determining the output of each neuron. Traditionally, functions like sigmoid and ReLU have been static and deterministic. However, the introduction of parametric activation functions represents a significant paradigm shift by incorporating adjustable parameters, thus enhancing flexibility and optimization capabilities. Furthermore, parametric activation functions can enhance interpretability, helping to solve the black-box problem and providing deeper insights into the functioning of neural networks. They are also essential components of the innovative model of Kolmogorov-Arnold Network, that features learnable activation functions on the edges of the neural network. This review highlights the importance, characteristics, and applications of these parametric functions in neural networks. Notable examples of parametric activation functions are analyzed, with emphasis on their mathematical formulations and their transformative impact on modern neural networks. These functions not only represent an evolutionary step but a substantial shift in the design and optimization of neural network architectures, offering a promising avenue for innovation and advancement in the field.
This paper comprehensively examines symmetric difference operators within logical systems generated by nilpotent t-norms and t-conorms, specifically addressing their behavior and applicability in bounded and Łukasiewicz fuzzy logic systems. We identify two distinct symmetric difference operators and analyze their fundamental properties, revealing their inherent non-associativity. Recognizing the limitations posed by non-associative behavior in practical multi-step logical operations, we introduce a novel aggregated symmetric difference operator constructed through the arithmetic mean of the previously defined operators. The primary theoretical contribution of our research is establishing the associativity of this new aggregated operator, significantly enhancing its effectiveness for consistent multi-stage computations. Moreover, this operator retains critical properties including symmetry, neutrality, antitonicity, and invariance under negation, thus making it particularly valuable for various computational and applied domains such as image processing, pattern recognition, fuzzy neural networks, cryptographic schemes, and medical data analysis. The demonstrated theoretical robustness and practical versatility of our associative operator provide a clear improvement over existing methodologies, laying a solid foundation for future research in fuzzy logic and interdisciplinary applications. Our broader aim is to derive and study symmetric difference operators in both bounded and Łukasiewicz systems, as this represents a new direction of research.