27 March 2023 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Nowadays, voice assistants help users complete tasks on the smartphone with voice commands, replacing traditional touchscreen interactions when such interactions are inhibited. However, the usability of those tools remains moderate due to the problems in understanding rich language variations in human commands, along with efficiency and comprehensibility issues. Therefore, we introduce Voicify, an Android virtual assistant that allows users to interact with on-screen elements in mobile apps through voice commands. Using a novel deep learning command parser, Voicify interprets human verbal input and performs matching with UI elements. In addition, the tool can directly open a specific feature from installed applications by fetching application code information to explore the set of in-app components. Our command parser achieved 90% accuracy on the human command dataset. Furthermore, the direct feature invocation module achieves better feature coverage in comparison to Google Assistant. The user study demonstrates the usefulness of Voicify in real-world scenarios.
14 December 2023 in Distributed Ledger Technologies: Research and Practice
Recent years have witnessed a marked increase in both academic proposals and industrial adoptions of blockchain technology. However, a majority of the projects remain at the stage of prototype proposals and their real-world deployment has not met the anticipated level. This gap can be attributed to three major barriers - technical difficulties, human factors, and social context. Most of the existing research leans towards addressing the technical challenges, leaving the human and social aspects inadequately explored. Moreover, a lack of practical insights in the existing blockchain software engineering frameworks further exacerbates the adoption problem. To address these gaps, we introduce a Blockchain-oriented Software Engineering Approach for Higher Adoption Possibility (BOSE-HAP) . This approach emphasizes collaboration, reflective thinking, and iterative development, aiming to bolster implementation consistency and stimulate industry adoption. We have applied this approach in the design, development, and launch of a blockchain credentialing product, CValid.org , in the context of a university-level summer school. The product achieves industry-accepted System Usability Score and has seen successful real-world deployment. In addition, this study embed usability considerations throughout the process, involved a total of 112 stakeholders across different development stages, with 25 of them participating in our in-depth interviews and usability testing. Drawing from our firsthand experience and industrial-grounded findings, we deliver eight reflections and propose five best practice suggestions relevant to blockchain adoption. We believe these insights will provide invaluable guidance for both academic researchers and industry practitioners involved in the field of blockchain technology.
27 June 2024 in ACM Transactions on Software Engineering and Methodology
Mobile apps are ubiquitous in our daily lives for supporting different tasks such as reading and chatting. Despite the availability of many GUI testing tools, app testers still struggle with low testing code coverage due to tools frequently getting stuck in loops or overlooking activities with concealed entries. This results in a significant amount of testing time being spent on redundant and repetitive exploration of a few GUI pages. To address this, we utilize Android’s deep links, which assist in triggering Android intents to lead users to specific pages and introduce a deep link-enhanced exploration method. This approach, integrated into the testing tool Monkey, gives rise to Delm (Deep Link-enhanced Monkey). Delm oversees the dynamic exploration process, guiding the tool out of meaningless testing loops to unexplored GUI pages. We provide a rigorous activity context mock-up approach for triggering existing Android intents to discover more activities with hidden entrances. We conduct experiments to evaluate Delm’s effectiveness on activity context mock-up, activity coverage, method coverage, and crash detection. The findings reveal that Delm can mock up more complex activity contexts and significantly outperform state-of-the-art baselines with 27.2% activity coverage, 21.13% method coverage, and 23.81% crash detection.
18 April 2024 in ACM Transactions on Software Engineering and Methodology
Deep Learning (DL) models have rapidly advanced, focusing on achieving high performance through testing model accuracy and robustness. However, it is unclear whether DL projects, as software systems, are tested thoroughly or functionally correct when there is a need to treat and test them like other software systems. Therefore, we empirically study the unit tests in open-source DL projects, analyzing 9,129 projects from GitHub. We find that: (1) unit tested DL projects have positive correlation with the open-source project metrics and have a higher acceptance rate of pull requests; (2) 68% of the sampled DL projects are not unit tested at all; (3) the layer and utilities (utils) of DL models have the most unit tests. Based on these findings and previous research outcomes, we built a mapping taxonomy between unit tests and faults in DL projects. We discuss the implications of our findings for developers and researchers and highlight the need for unit testing in open-source DL projects to ensure their reliability and stability. The study contributes to this community by raising awareness of the importance of unit testing in DL projects and encouraging further research in this area.