Adolescent online safety researchers have emphasized the importance of moving beyond restrictive and privacy invasive approaches to online safety, towards resilience-based approaches for empowering teens to deal with online risks independently. Unfortunately, many of the existing online safety interventions are focused on parental mediation and not contextualized to teens' personal experiences online; thus, they do not effectively cater to the unique needs of teens. To better understand how we might design online safety interventions that help teens deal with online risks, as well as when and how to intervene, we must include teens as partners in the design process and equip them with the skills needed to contribute equally to the design process. As such, we conducted User Experience (UX) bootcamps with 21 teens (ages 13-17) to first teach them important UX design skills using industry standard tools, so they could create storyboards for unsafe online interactions commonly experienced by teens and high-fidelity, interactive prototypes for dealing with these situations. Based on their storyboards, teens often encountered information breaches and sexual risks with strangers, as well as cyberbullying from acquaintances or friends. While teens often blocked or reported strangers, they struggled with responding to risks from friends or acquaintances, seeking advice from others on the best action to take. Importantly, teens did not find any of the existing ways for responding to these risks to be effective in keeping them safe. When asked to create their own design-based interventions, teens frequently envisioned "nudges" that occurred in real-time. Interestingly, teens more often designed for risk prevention (rather than risk coping) by focusing on nudging the risk perpetrator (rather than the victim) to rethink their actions, block harmful actions from occurring, or penalizing perpetrators for inappropriate behavior to prevent it from happening again in the future. Teens also designed personalized sensitivity filters to provide teens the ability to manage content they wanted to see online. Some teens also designed personalized nudges, so that teens could receive intelligent, guided advice from the platform that would help them know how to handle online risks themselves without intervention from their parents. Our findings highlight how teens want to address online risks "at the root" by putting the onus of risk prevention on those who perpetrate them - rather than on the victim. Our work is the first to leverage co-design with teens to develop novel online safety interventions that advocate for a paradigm shift from youth risk protection to promoting good digital citizenship.

Purpose: The purpose of this paper is to examine the extent to which negative online risk experiences (information breaches, explicit content exposure, cyberbullying and sexual solicitations) cause post-traumatic stress disorder (PTSD) symptoms in adolescents. The study also explores whether teens’ short-term coping responses serve to mitigate PTSD or, instead, act as a response to stress from online events. Design/methodology/approach: The study utilized a web-based diary design over the course of two months. Data were analyzed using hierarchical linear modeling with repeated measures. Findings: The study confirmed that explicit content exposure, cyberbullying and sexual solicitations (but not information breaches) evoke symptoms of PTSD. Analyses also indicated that teens engage in active and communicative coping after they experience post-traumatic stress, regardless of risk type or frequency. Practical implications: The authors found that teens took active measures to cope with online risks soon after they felt threatened (within a week). Actively coping with stressful situations has been shown to enhance adolescent resilience and reduce long-term negative effects of risk exposure. If these early coping behaviors can be detected, social media platforms may be able to embed effective interventions to support healthy coping processes that can further protect teens against long-term harm from exposure to online risks. Originality/value: This is the first study to examine situational PTSD symptoms related to four types of adolescent online risk exposure within the week exposure occurred. By applying two competing theoretical frameworks (the adolescent resilience framework and transactional theory of stress), the authors show empirical evidence that suggests short-term coping responses are likely a stress reaction to PTSD, not a protective factor against it.

We collected Instagram data from 150 adolescents (ages 13-21) that included 15,547 private message conversations of which 326 conversations were flagged as sexually risky by participants. Based on this data, we leveraged a human-centered machine learning approach to create sexual risk detection classifiers for youth social media conversations. Our Convolutional Neural Network (CNN) and Random Forest models outperformed in identifying sexual risks at the conversation-level (AUC=0.88), and CNN outperformed at the message-level (AUC=0.85). We also trained classifiers to detect the severity risk level (i.e., safe, low, medium-high) of a given message with CNN outperforming other models (AUC=0.88). A feature analysis yielded deeper insights into patterns found within sexually safe versus unsafe conversations. We found that contextual features (e.g., age, gender, and relationship type) and Linguistic Inquiry and Word Count (LIWC) contributed the most for accurately detecting sexual conversations that made youth feel uncomfortable or unsafe. Our analysis provides insights into the important factors and contextual features that enhance automated detection of sexual risks within youths' private conversations. As such, we make valuable contributions to the computational risk detection and adolescent online safety literature through our human-centered approach of collecting and ground truth coding private social media conversations of youth for the purpose of risk classification.

Instagram, one of the most popular social media platforms among youth, has recently come under scrutiny for potentially being harmful to the safety and well-being of our younger generations. Automated approaches for risk detection may be one way to help mitigate some of these risks if such algorithms are both accurate and contextual to the types of online harms youth face on social media platforms. However, the imminent switch by Instagram to end-to-end encryption for private conversations will limit the type of data that will be available to the platform to detect and mitigate such risks. In this paper, we investigate which indicators are most helpful in automatically detecting risk in Instagram private conversations, with an eye on high-level metadata, which will still be available in the scenario of end-to-end encryption. Toward this end, we collected Instagram data from 172 youth (ages 13-21) and asked them to identify private message conversations that made them feel uncomfortable or unsafe. Our participants risk-flagged 28,725 conversations that contained 4,181,970 direct messages, including textual posts and images. Based on this rich and multimodal dataset, we tested multiple feature sets (metadata, linguistic cues, and image features) and trained classifiers to detect risky conversations. Overall, we found that the metadata features (e.g., conversation length, a proxy for participant engagement) were the best predictors of risky conversations. However, for distinguishing between risk types, the different linguistic and media cues were the best predictors. Based on our findings, we provide design implications for AI risk detection systems in the presence of end-to-end encryption. More broadly, our work contributes to the literature on adolescent online safety by moving toward more robust solutions for risk detection that directly takes into account the lived risk experiences of youth.

Although youth increasingly communicate with peers online, we know little about how private online channels play a role in providing a supportive environment for youth. To fill this gap, we asked youth to donate their Instagram Direct Messages and filtered them by the phrase “help me.” From this query, we analyzed 82 conversations comprised of 336,760 messages that 42 participants donated. These threads often began as casual conversations among friends or lovers they met offline or online. The conversations evolved into sharing negative experiences about everyday stress (e.g., school, dating) to severe mental health disclosures (e.g., suicide). Disclosures were usually reciprocated with relatable experiences and positive peer support. We also discovered unsupport as a theme, where conversation members denied giving support, a unique finding in the online social support literature. We discuss the role of social media-based private channels and their implications for design in supporting youth’s mental health. Content Warning: This paper includes sensitive topics, including self-harm and suicide ideation. Reader discretion is advised.

We conducted a study with 173 adolescents (ages 13-21), who self-reported their offline and online risk experiences and uploaded their Instagram data to our study website to flag private conversations as unsafe. Risk profiles were first created based on the survey data and then compared with the risk-flagged social media data. Five risk profiles emerged: Low Risks (51% of the participants), Medium Risks (29%), Increased Sexting (8%), Increased Self-Harm (8%), and High Risk Perpetration (4%). Overall, the profiles correlated well with the social media data with the highest level of risk occurring in the three smallest profiles. Youth who experienced increased sexting and self-harm frequently reported engaging in unsafe sexual conversations. Meanwhile, high risk perpetration was characterized by increased violence, threats, and sales/promotion of illegal activities. A key insight from our study was that offline risk behavior sometimes manifested differently in online contexts (i.e., offline self-harm as risky online sexual interactions). Our findings highlight the need for targeted risk prevention strategies for youth online safety.