23 November 2023 in ACM Transactions on Software Engineering and Methodology
Powered by the rising popularity of deep learning techniques on smartphones, on-device deep learning models are being used in vital fields such as finance, social media, and driving assistance. Because of the transparency of the Android platform and the on-device models inside, on-device models on Android smartphones have been proven to be extremely vulnerable. However, due to the challenge in accessing and analyzing iOS app files, despite iOS being a mobile platform as popular as Android, there are no relevant works on on-device models in iOS apps. Since the functionalities of the same app on Android and iOS platforms are similar, the same vulnerabilities may exist on both platforms. In this article, we present the first empirical study about on-device models in iOS apps, including their adoption of deep learning frameworks, structure, functionality, and potential security issues. We study why current developers use different on-device models for one app between iOS and Android. We propose a more general attack against white-box models that does not rely on pre-trained models and a new adversarial attack approach based on our findings to target iOS’s gray-box on-device models. Our results show the effectiveness of our approaches. Finally, we successfully exploit the vulnerabilities of on-device models to attack real-world iOS apps.
27 June 2024 in ACM Transactions on Software Engineering and Methodology
Mobile apps are ubiquitous in our daily lives for supporting different tasks such as reading and chatting. Despite the availability of many GUI testing tools, app testers still struggle with low testing code coverage due to tools frequently getting stuck in loops or overlooking activities with concealed entries. This results in a significant amount of testing time being spent on redundant and repetitive exploration of a few GUI pages. To address this, we utilize Android’s deep links, which assist in triggering Android intents to lead users to specific pages and introduce a deep link-enhanced exploration method. This approach, integrated into the testing tool Monkey, gives rise to Delm (Deep Link-enhanced Monkey). Delm oversees the dynamic exploration process, guiding the tool out of meaningless testing loops to unexplored GUI pages. We provide a rigorous activity context mock-up approach for triggering existing Android intents to discover more activities with hidden entrances. We conduct experiments to evaluate Delm’s effectiveness on activity context mock-up, activity coverage, method coverage, and crash detection. The findings reveal that Delm can mock up more complex activity contexts and significantly outperform state-of-the-art baselines with 27.2% activity coverage, 21.13% method coverage, and 23.81% crash detection.
22 December 2023 in ACM Transactions on Software Engineering and Methodology
With the increasing interconnection of smart devices, users often desire to adopt the same app on quite different devices for identical tasks, such as watching the same movies on both their smartphones and TVs. However, the significant differences in screen size, aspect ratio, and interaction styles make it challenging to adapt Graphical User Interfaces (GUIs) across these devices. Although there are millions of apps available on Google Play, only a few thousand are designed to support smart TV displays. Existing techniques to map a mobile app GUI to a TV either adopt a responsive design, which struggles to bridge the substantial gap between phone and TV, or use mirror apps for improved video display, which requires hardware support and extra engineering efforts. Instead of developing another app for supporting TVs, we propose a semi-automated approach to generate corresponding adaptive TV GUIs, given the phone GUIs as the input. Based on our empirical study of GUI pairs for TVs and phones in existing apps, we synthesize a list of rules for grouping and classifying phone GUIs, converting them to TV GUIs, and generating dynamic TV layouts and source code for the TV display. Our tool is not only beneficial to developers but also to GUI designers, who can further customize the generated GUIs for their TV app development. An evaluation and user study demonstrate the accuracy of our generated GUIs and the usefulness of our tool.
1 November 2010 in IEEE Transactions on Consumer Electronics
Wireless Multimedia service faces the difficulty of QoS decrease caused by the cache underflow, due to both the unreliable and time-varying nature of the wireless channel. In order to reduce the probability of the cache outage, a statistical model is established to estimate the underflow time of the cache. Based on the estimation of the underflow time, an adaptive playout algorithm with dual-threshold to adjust the frame rate is presented. This approach adjusts the threshold dynamically according to the channel conditions and the status of the cache, and thus regulates the frame rate timely. Moreover, the algorithm possesses scene-aware characteristics, since it adjusts the playout rate according to the motion intensity of the playout scenes to reduce the underflow probability of the cache and provide better QoS. Simulation results show that as compared to other adaptive media playout algorithms, the proposed algorithm efficiently reduces the probability of the cache underflow, especially in low rate channel and timevarying rate environment, and provides better visual quality.
17 June 2026 in ACM Transactions on Software Engineering and Methodology
Android applications (i.e., apps) are indispensable nowadays and are getting bigger and bigger with an increasing number of functionalities. To understand how to access functionalities in an app, prior studies proposed tools to model the transitions between functionalities with the activity transition graph (ATG). ATG is an important data structure and has been used for various Android app analyses, including app design, understanding, and testing. However, there is no benchmarking work on ATG generation. It is still unclear whether the transitions identified by tools are correct and how many transitions are missed. To fill this gap, we manually identified all transitions in 98 applications to build a benchmark. Using the benchmark, we evaluate seven popular ATG generation tools that were used in prior studies. We observe that these tools not only report incorrect transitions but also miss transitions, and different tools do not report the same set of transitions. Compared with the transitions reported by a single tool, the union set of the transitions reported by different tools contains fewer missed transitions but more incorrect transitions. We summarize five potential reasons that explain why GUI testing tools fail to identify transitions in ATG, revealing the limitations in the current design of exploration strategies. For instance, we observe that learning-based tools may overlook subtle distinctions between states, potentially misclassifying different states as identical. This will lead the tool to always focus on the old state while the new state is not fully explored. Based on our findings, we propose a series of suggestions for researchers using and building ATG generation tools. For example, when aiming to identify more transitions, one can combine the results of different tools by running each tool for 10 minutes, which will produce better results than running a single tool for 60 minutes.