Reduction in system inertia and maintaining the frequency at the nominal value is a staple of today's and future power systems since their operation, stability, and resiliency are degraded by frequency oscillation and cascading failures. Consequently, designing a stable, scalable, and robust virtual inertia control system is highly relevant to skillfully diminishing the deviations during major contingencies. Therefore, considering the potential problems in predesigned nonflexible control systems with offline tuning techniques, we propose a variable fractional-order PID controller for virtual inertia control applications, which is tuned online using a modified neural network-based algorithm. The new proposed tuner algorithm is trained using a deep reinforcement learning strategy with a simplified deep deterministic policy gradient, which considers microgrid uncertainties. Compared with existing methods, all the tuning knobs of the discrete type and fully tunable variable FOPID controller (for both gain and order) can be captured based on the proposed hybrid algorithm, which inherits features from both classical and advanced techniques. To demonstrate the effectiveness of the training of the proposed controller, a comparative analysis with the standard FOPID and PID controllers is given under three different scenarios with a smooth (dis)connection of renewable energy sources and loads.