Recent advances in hardware and algorithms have enabled AI systems to tackle tasks once considered uniquely human. However, the growing size of modern AI models brings substantial energy demands, raising environmental concerns. This thesis explores Spiking Neural Networks (SNN) as a promising low-power alternative. Inspired by the brain, SNN can operate 10 to 1.000 times more efficiently than conventional AI when deployed on neuromorphic hardware, though they often fall short in performance on complex tasks. To address this, we propose several biologically motivated extensions to SNN models, focusing on learning mechanisms and performance improvements. First, we propose three biologically inspired enhancements to the Liquid State Machine (LSM) framework: balancing excitatory and inhibitory input currents, incorporating SFA, and introducing neuronal heterogeneity. These extensions collectively enrich the reservoir’s temporal dynamics, leading to improved class separation and enhanced classification performance across multiple speech recognition benchmarks. We demonstrate that the resulting ELSM model approaches state-of-the-art accuracy, rivaling end-to-end trained AI models, while maintaining the efficiency and simplicity of the LSM architecture. Next, we explore end-to-end training of SNN using. We introduce the cAdLIF neuron model, which enables joint optimization of homeostatic adaptation parameters alongside a novel temporally-aware learning rule for synaptic transmission delays. This combination allows neurons to explicitly correlate temporally dispersed input patterns, enhancing temporal processing capabilities. We demonstrate that these biologically inspired enhancements lead to substantial performance gains across benchmark datasets and show that the proposed model approaches or surpasses state-of-the-art results in SNN. Next, we introduce Twin Network Augmentation (TNA), a new method for improved model generalization and compression to binary/ternary weight precision, through a twin-network setup. Two identical architectures are co-trained using individual cross-entropy losses and a shared logit matching loss, encouraging consistent decision boundaries. We show that TNA improves performance over the baseline across all evaluated datasets and outperforms a related technique, knowledge distillation. Finally, we extend the TNA framework to non-spiking ANN, including more complex architectures such as CNN and transformers. We validate the method on popular vision benchmarks and conduct extensive ablation studies to isolate the source of performance gains. Our results show that improvements stem from mutual learning via logit matching rather than an increase in trainable parameters. Furthermore, analysis of internal representations reveals that TNA-trained models develop more discriminative high-level features and exhibit enhanced robustness to input corruptions.