In Industrial Internet of Things (IIoT), the latency-sensitive tasks can be efficiently performed based on real-time data collection, transmission, and computation. In this paper, we thus propose a hybrid Non-Orthogonal Multiple Access-Rate Splitting Multiple Access (NOMA-RSMA) edge-cloud service computing framework for the IIoT consisting of end devices (EDs), edge servers (ESs), and a cloud server (CS), in which the tasks can be computed at the EDs, the ESs, or the CS. When task computation takes place at the ESs or the CS, tasks would be firstly compressed at the EDs, and then be offloaded to the ESs via NOMA. After that, the ESs offload tasks to the CS via RSMA when the tasks are intended to be processed at the CS. Otherwise, the ESs perform computation locally. Specifically, under the delay constraints, we aim to minimize the total system energy consumption by jointly optimizing the transmit power of the EDs and the ESs, the transmission delays of each NOMA group and the RSMA, the signal splitting ratios of the ESs, the computation power of the EDs, the ESs, and the CS, the compression delay, the offloading decisions of the EDs and the ESs. To address the formulated non-convex problem, we employ a decomposition approach to layer it into a top-level offloading decision problem and a bottom-level resource allocation problem. We design a Block Coordinate Descent (BCD)-based method to solve the bottom-level problem. Additionally, we propose an algorithm based on Deep Reinforcement Learning and Online Offloading (DROO) to obtain the sub-optimal offloading decisions for EDs and ESs in the top-level problem. Numerical results validate the accuracy and effectiveness of our algorithms in terms of the total energy consumption, compared with three heuristic algorithms, i.e., the genetic algorithm, the cross-entropy algorithm, and the Deep Q-Network algorithm.