Le, T., Zhao, D., Wang, Z., Wang, X. F., & Tian, Y. (2024). Alexa, is the skill always safe? Uncover Lenient Skill Vetting Process and Protect User Privacy at Run Time. In Proceedings of the 46th International Conference on Software Engineering (ICSE).
@unpublished{le2024alexa,
author = {Le, Tu and Zhao, Dongfang and Wang, Zihao and Wang, XiaoFeng and Tian, Yuan},
title = {Alexa, is the skill always safe? Uncover Lenient Skill Vetting Process and Protect User Privacy at Run Time},
year = {2024},
booktitle = {Proceedings of the 46th International Conference on Software Engineering (ICSE)}
}
Voice personal assistant (VPA) platforms (e.g., Amazon Alexa) allow developers to deploy their voice apps on third-party servers. However, this strategy introduces unexpected privacy risks to VPA customers. Malicious developers can dynamically change their app’s behaviors to circumvent the platform’s vetting process. This paper aims to systematically analyze Alexa’s voice app ecosystem (i.e., Alexa skills), focusing on behavior manipulation (also referred to as skill behavior change). We identify the root causes of malicious skills getting published and propose a defense solution to effectively protect users. First, we uncover Amazon’s skill vetting strategy and the privacy issues relevant to their vetting. We reveal that, in addition to the skill certification process before a skill gets published, Amazon also deploys a skill monitoring scheme after the skill is published. We further discover limitations of this monitoring scheme that have not been explored in previous research. Lastly, to address these issues, we propose a run-time skill monitoring approach to check the consistency of the skill behaviors when users interact with skills. Our findings suggest a call for action to improve the vetting process for VPA skills without placing a burden on skill developers and help developers adhere to policies.
Le, T., Wang, Z., Huang, D. Y., Yao, Y., & Tian, Y. (2024). Towards Real-time Voice Interaction Data Collection Monitoring and Ambient Light Privacy Notification for Voice-controlled Services. In Symposium on Usable Security and Privacy (USEC).
@unpublished{le2024monitoring,
author = {Le, Tu and Wang, Zixin and Huang, Danny Yuxing and Yao, Yaxing and Tian, Yuan},
title = {Towards Real-time Voice Interaction Data Collection Monitoring and Ambient Light Privacy Notification for Voice-controlled Services},
year = {2024},
booktitle = {Symposium on Usable Security and Privacy (USEC)}
}
Voice-controlled devices or their software component, known as voice personal assistant (VPA), offer technological advancements that improve user experience. However, they come with privacy concerns such as unintended recording of the user’s private conversations. This data could potentially be stolen by adversaries or shared with third parties. Therefore, users need to be aware of these and other similar potential privacy risks presented by VPAs. In this paper, we first study how VPA users monitor their voice interaction recorded by their VPAs and their expectations via an online survey of 100 users. We find that even though users were aware of the VPAs holding recordings of them, they initially thought reviewing the recordings was unnecessary. However, they were surprised that there were unintended recordings and that they could review the recordings. When presented with what types of unintended recordings might happen, more users wanted the option to review their interaction history. This indicates the importance of data transparency. We then build a browser extension that helps users monitor their voice interaction history and notifies users of unintended conversations recorded by their voice assistants. Our tool experiments with notifications using smart light devices in addition to the traditional push notification approach. With our tool, we then interview 10 users to evaluate the usability and further understand users’ perceptions of such unintended recordings. Our results show that unintended recordings could be common in the wild and there is a need for a tool to help manage the voice interaction recordings with VPAs. Smart light notification is potentially a useful mechanism that should be adopted in addition to the traditional push notification.
Refereed journal articles
Le, T., Huang, D. Y., Apthorpe, N., & Tian, Y. (2022). SkillBot: Identifying Risky Content for Children in Alexa Skills. ACM Trans. Internet Technol., 22(3). https://doi.org/10.1145/3539609
@article{le2022skillbot,
author = {Le, Tu and Huang, Danny Yuxing and Apthorpe, Noah and Tian, Yuan},
title = {SkillBot: Identifying Risky Content for Children in Alexa Skills},
year = {2022},
issue_date = {August 2022},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {22},
number = {3},
issn = {1533-5399},
url = {https://doi.org/10.1145/3539609},
doi = {10.1145/3539609},
journal = {ACM Trans. Internet Technol.},
month = jul,
articleno = {79},
numpages = {31},
keywords = {automated system, risky voice app, parents’ perceptions, Child safety}
}
Many households include children who use voice personal assistants (VPA) such as Amazon Alexa. Children benefit from the rich functionalities of VPAs and third-party apps but are also exposed to new risks in the VPA ecosystem. In this article, we first investigate “risky” child-directed voice apps that contain inappropriate content or ask for personal information through voice interactions. We build SkillBot—a natural language processing-based system to automatically interact with VPA apps and analyze the resulting conversations. We find 28 risky child-directed apps and maintain a growing dataset of 31,966 non-overlapping app behaviors collected from 3,434 Alexa apps. Our findings suggest that although child-directed VPA apps are subject to stricter policy requirements and more intensive vetting, children remain vulnerable to inappropriate content and privacy violations. We then conduct a user study showing that parents are concerned about the identified risky apps. Many parents do not believe that these apps are available and designed for families/kids, although these apps are actually published in Amazon’s “Kids” product category. We also find that parents often neglect basic precautions, such as enabling parental controls on Alexa devices. Finally, we identify a novel risk in the VPA ecosystem: confounding utterances or voice commands shared by multiple apps that may cause a user to interact with a different app than intended. We identify 4,487 confounding utterances, including 581 shared by child-directed and non-child-directed apps. We find that 27% of these confounding utterances prioritize invoking a non-child-directed app over a child-directed app. This indicates that children are at real risk of accidentally invoking non-child-directed apps due to confounding utterances.
Refereed conference proceedings
Le, T., Wang, A., Yao, Y., Feng, Y., Heydarian, A., Sadeh, N., & Tian, Y. (2023). Exploring Smart Commercial Building Occupants’ Perceptions and Notification Preferences of Internet of Things Data Collection in the United States. 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P), 1030–1046. https://doi.org/10.1109/EuroSP57164.2023.00064
@inproceedings{le2023exploring,
author = {Le, Tu and Wang, Alan and Yao, Yaxing and Feng, Yuanyuan and Heydarian, Arsalan and Sadeh, Norman and Tian, Yuan},
booktitle = {2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P)},
title = {Exploring Smart Commercial Building Occupants’ Perceptions and Notification Preferences of Internet of Things Data Collection in the United States},
year = {2023},
volume = {},
number = {},
pages = {1030-1046},
keywords = {Data privacy;Smart buildings;Buildings;Smart homes;Data collection;User experience;Internet of Things;data collection;notification;privacy;IoT;smart building;smart devices},
doi = {10.1109/EuroSP57164.2023.00064}
}
Data collection through the Internet of Things (IoT) devices, or smart devices, in commercial buildings enables possibilities for increased convenience and energy efficiency. However, such benefits face a large perceptual challenge when being implemented in practice, due to the different ways occupants working in the buildings understand and trust in the data collection. The semi-public, pervasive, and multi-modal nature of data collection in smart buildings points to the need to study occupants’ understanding of data collection and notification preferences. We conduct an online study with 492 participants in the US who report working in smart commercial buildings regarding: 1) awareness and perception of data collection in smart commercial buildings, 2) privacy notification preferences, and 3) potential factors for privacy notification preferences. We find that around half of the participants are not fully aware of the data collection and use practices of IoT even though they notice the presence of IoT devices and sensors. We also discover many misunderstandings around different data practices. The majority of participants want to be notified of data practices in smart buildings, and they prefer push notifications to passive ones such as websites or physical signs. Surprisingly, mobile app notification, despite being a popular channel for smart homes, is the least preferred method for smart commercial buildings.
Yang, P., Fan, J., Wei, Z., Li, H., Le, T., & Tian, Y. (2023). Towards Usable Parental Control for Voice Assistants. Proceedings of Cyber-Physical Systems and Internet of Things Week 2023, 43–48. https://doi.org/10.1145/3576914.3587491
@inproceedings{yang2023parental,
author = {Yang, Peiyi and Fan, Jie and Wei, Zice and Li, Haoqian and Le, Tu and Tian, Yuan},
title = {Towards Usable Parental Control for Voice Assistants},
year = {2023},
isbn = {9798400700491},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3576914.3587491},
doi = {10.1145/3576914.3587491},
booktitle = {Proceedings of Cyber-Physical Systems and Internet of Things Week 2023},
pages = {43–48},
numpages = {6},
keywords = {Alexa Kids, Amazon Alexa, Parental Control, Privacy, Safety, Security, Smart Speaker, User Interface, Voice Assistant},
location = {San Antonio, TX, USA},
series = {CPS-IoT Week '23}
}
Voice Personal Assistants (VPA) have become a common household appliance. As one of the leading platforms for VPA technology, Amazon created Alexa and designed Amazon Kids for children to safely enjoy the rich functionalities of VPA and for parents to monitor their kids’ activities through the Parent Dashboard. Although this ecosystem is in place, the usage of Parent Dashboard is not yet popularized among parents. In this paper, we conduct a parent survey to find out what they like and dislike about the current parental control features. We find that parents need more visuals about their children’s activity, easier access to security features for their children, and a better user interface. Based on the insights from our survey, we present a new design for the Parent Dashboard considering the parents’ expectations.
Ahmad, W., Chi, J., Le, T., Norton, T., Tian, Y., & Chang, K.-W. (2021). Intent Classification and Slot Filling for Privacy Policies. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 4402–4417. https://doi.org/10.18653/v1/2021.acl-long.340
@inproceedings{ahmad2021intent,
title = {Intent Classification and Slot Filling for Privacy Policies},
author = {Ahmad, Wasi and Chi, Jianfeng and Le, Tu and Norton, Thomas and Tian, Yuan and Chang, Kai-Wei},
booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
month = aug,
year = {2021},
address = {Online},
publisher = {Association for Computational Linguistics},
url = {https://aclanthology.org/2021.acl-long.340},
doi = {10.18653/v1/2021.acl-long.340},
pages = {4402--4417}
}
Understanding privacy policies is crucial for users as it empowers them to learn about the information that matters to them. Sentences written in a privacy policy document explain privacy practices, and the constituent text spans convey further specific information about that practice. We refer to predicting the privacy practice explained in a sentence as intent classification and identifying the text spans sharing specific information as slot filling. In this work, we propose PolicyIE, an English corpus consisting of 5,250 intent and 11,788 slot annotations spanning 31 privacy policies of websites and mobile applications. PolicyIE corpus is a challenging real-world benchmark with limited labeled examples reflecting the cost of collecting large-scale annotations from domain experts. We present two alternative neural approaches as baselines, (1) intent classification and slot filling as a joint sequence tagging and (2) modeling them as a sequence-tosequence (Seq2Seq) learning task. The experiment results show that both approaches perform comparably in intent classification, while the Seq2Seq method outperforms the sequence tagging approach in slot filling by a large margin. We perform a detailed error analysis to reveal the challenges of the proposed corpus.
Stankovic, J. A., Le, T., Hendawi, A., & Tian, Y. (2021). Hardware/Software Security Patches for the Internet of Things. 2021 IEEE International Conference on Smart Computing (SMARTCOMP), 240–245. https://doi.org/10.1109/SMARTCOMP52413.2021.00054
@inproceedings{stankovic2021hardware,
author = {Stankovic, John A. and Le, Tu and Hendawi, Abdeltawab and Tian, Yuan},
booktitle = {2021 IEEE International Conference on Smart Computing (SMARTCOMP)},
title = {Hardware/Software Security Patches for the Internet of Things},
year = {2021},
volume = {},
number = {},
pages = {240-245},
keywords = {Surveillance;Robot vision systems;Cameras;Hardware;Software;Security;Internet of Things;Security;Internet of Things;Smart Buttons;Smart devices;Hardware/software co-design},
doi = {10.1109/SMARTCOMP52413.2021.00054}
}
With the rapid development of the Internet of Things (IoT), there are billions of interacting devices and applications. With so many devices and applications, one of the most critical challenges is how to provide security. Traditional software-based defenses will not be enough to protect the security of IoT because of the attack surfaces derived from the physical environment. For example, an attacker can physically re-point a surveillance camera, can move a smart device to another location, can send a sound signal to influence an accelerometer, can cause wireless jamming, etc. We propose to create "smart buttons," and collections of them called "smart blankets" as hardware/software (HW/SW) security patches rather than software-only patches. These fixes operate similarly to software patches, but because of the hardware added, these new patches can better support against physical world attacks. While this paper primarily presents a vision for HW/SW patches, solutions are implemented and shown for two classes of attacks involving cameras and robots. Open questions are also discussed.
Le, T., Elsayed-Aly, I., Jin, W., Ryu, S., Verrier, G., Al Rahat, T., Park, B. B., & Tian, Y. (2020). Evaluating the Dedicated Short-range Communication for Connected Vehicles against Network Security Attacks. Proceedings of the 6th International Conference on Vehicle Technology and Intelligent Transport Systems - Volume 1: VEHITS, 37–44. https://doi.org/10.5220/0009355500370044
@inproceedings{le2020evaluating,
title = {Evaluating the Dedicated Short-range Communication for Connected Vehicles against Network Security Attacks.},
author = {Le, Tu and Elsayed-Aly, Ingy and Jin, Weizhao and Ryu, Seunghan and Verrier, Guy and Al Rahat, Tamjid and Park, B Brian and Tian, Yuan},
booktitle = {Proceedings of the 6th International Conference on Vehicle Technology and Intelligent Transport Systems - Volume 1: VEHITS,},
pages = {37--44},
doi = {10.5220/0009355500370044},
year = {2020}
}
Le, T., ElSayed-Aly, I., Jin, W., Ryu, S., Verrier, G., Al Rahat, T., Park, B. B., & Tian, Y. (2019). Poster: Attack the dedicated short-range communication for connected vehicles. Poster Presented at the 40th IEEE Symposium on Security and Privacy.
@inproceedings{le2019poster,
title = {Poster: Attack the dedicated short-range communication for connected vehicles},
author = {Le, Tu and ElSayed-Aly, Ingy and Jin, Weizhao and Ryu, Seunghan and Verrier, Guy and Al Rahat, Tamjid and Park, B Brian and Tian, Yuan},
booktitle = {Poster presented at the 40th IEEE Symposium on Security and Privacy},
year = {2019}
}
In the near future, autonomous vehicles will be able to operate without human drivers, making them safety-critical systems. Connected vehicles will make use of wireless communication technology to exchange information about their surrounding environment with each other and roadside infrastructure. It is essential to study these systems extensively before deployment to ensure the security and safety of passengers and pedestrians. Dedicated Short-Range Communication (DSRC) is a popular lowlatency protocol designed for wireless communication between connected vehicles and infrastructure (V2I), and among connected vehicles (V2V). In this work, we evaluate the robustness of the DSRC protocol by presenting three real-world attacks on the communication layer of DSRC-connected vehicles. Such attacks can be cost-effectively deployed by adversaries without significant resources. We also discuss appropriate countermeasures against these attacks.