20 May 2025 in ACM Transactions on Cyber-Physical Systems
Data-driven models for state estimation using measurements obtained over a wireless network are essential to cyber-physical systems. Learning a data-driven model for an estimator using real wireless network deployments is, however, impractical as it would require data that captures varied wireless network conditions and their impact on estimation. We propose and evaluate a simulation to real-world transfer of a data-driven model for state estimation. Specifically, we train an estimator model using only data generated by a low-fidelity simulation of networks. Our choice of network simulation is a first-come-first-served single server queue, which is a network model with the two parameters of arrival rate of measurement packets into the queue and packet service rates. We employ domain randomization to bridge the gap between simulation and the real-world, appropriately randomizing the network model parameters during training. The efficacy of the resulting estimator model is demonstrated by testing it over two deployments of real wireless networks. In one, the estimator model estimates vehicles’ positions and speeds using data from vehicular trajectories received by it over a shared WiFi network, with up to seventy sources sending the measurements. In the other, GPS coordinates are communicated by public transit buses over city-wide cellular networks. The estimator uses the received measurements to estimate locations of the buses.
12 May 2026 in ACM Transactions on Cyber-Physical Systems
As robots (edge-devices, agents) find uses in an increasing number of settings and edge-cloud resources become pervasive, wireless networks will often be shared by flows of data traffic that result from communication between agents and their corresponding edge-cloud nodes (cloud compute or data resource accessed by an agent). In such a setting, any agent communicating with the edge-cloud is unaware of the state of the network resource, which evolves in response to not just the agent’s own communication at any given time but also to communication by the other agents, which stays unknown to the agent. We address the challenge of an agent learning a policy that allows it to decide whether or not to communicate with its cloud node, using limited feedback it obtains from its own attempts to communicate, with the goal of optimizing its utility. The policy must generalize well to any number of other agents sharing the network and must not be trained for any particular network configuration. Our proposed policy is a deep reinforcement learning model Query Net (QNet) that we train using a proposed simulation-to-real framework. Our simulation model has just one parameter and is agnostic to specific configurations of any wireless network. It however allows training an agent’s policy over a wide range of outcomes that an agent’s communication with its edge-cloud node may face when using a shared network, by suitably randomizing the simulation parameter. We propose a learning algorithm that addresses the challenges we observe in training QNet. We validate our simulation-to-real driven approach through experiments conducted on real wireless networks including WiFi and cellular. We compare QNet with other policies to demonstrate its efficacy. Our WiFi experiments involved as few as five agents, resulting in barely any contention for the network, to as many as 50 agents, resulting in severe contention. The cellular experiments spanned a broad range of network conditions, with baseline network round-trip times ranging from a low of 0.07 s to a high of 0.83 s.