Sponsored by USENIX, the Advanced Computing Systems Association.
All authors of accepted USENIX Security '24 papers (including shepherd-approved papers) are encouraged to submit artifacts for Artifact Evaluation (AE). Artifacts can be submitted in the same cycle as the accepted paper or in any of the following cycles for 2024. Each submitted artifact will be reviewed by the Artifact Evaluation Committee (AEC). Before submitting your artifact, please check the Artifact Evaluation Information below. Should you have any questions or concerns, you can reach the AEC chairs at sec24aec@usenix.org.
Important Dates
Summer Deadline
- Notification to authors: Friday, September 1, 2023, 11:59 pm AoE
- Final papers due: Tuesday, October 10, 2023
- Artifact Registration deadline: Tuesday, October 17, 2023
- Artifact submission deadline: Tuesday, October 24, 2023
- Author discussion period: Monday, October 30 to Wednesday, November 22, 2023
- Artifact decisions announced: Tuesday, December 5, 2023
- Final appendix files due: Tuesday, December 12, 2023
Fall Deadline
- Notification to authors: Thursday, February 1, 2024, 11:59 pm AoE
- Final papers due: Tuesday, March 5, 2024
- Artifact Registration deadline: Friday, March 8, 2024
- Artifact submission deadline: Tuesday, March 19, 2024
- Author discussion period: Monday, March 25 to Friday, April 19, 2024
- Artifact decisions announced: Tuesday, April 30, 2024
- Final appendix files due: Tuesday, May 7, 2024
Winter Deadline
- Notification to authors: Wednesday, May 8, 2024, 11:59 pm AoE
- Artifact Registration deadline: Monday, June 10, 2024
- Final papers due: Thursday, June 13, 2024
- Artifact submission deadline: Friday, June 21, 2024
- Author discussion period: Thursday, June 27 to Wednesday, July 24, 2024
- Artifact decisions announced: Friday, August 2, 2024
- Final appendix files due: Friday, August 9, 2024
Artifact Evaluation Committee
Artifact Evaluation Committee Co-Chairs
Phani Vadrevu, Louisiana State University
Anjo Vahldiek-Oberwagner, Intel Labs
Artifact Evaluation Publication Chair
Victor Le Pochat, KU Leuven
Artifact Evaluation Committee
Bhupendra Acharya, CISPA Helmholtz Center for Information Security
Shubham Agarwal, CISPA Helmholtz Center for Information Security
Ayomide Akinsanya, Stevens Institute of Technology
Ruba Alsmadi, Louisiana State University
Jessy Ayala, University of California, Irvine
Alessandro Baccarini, University at Buffalo
Xuesong Bai, University of California, Irvine
David Balash, University of Richmond
Luca Baldesi, University of California, Irvine
Hannaneh Barahouei Pasandi, University of California, Berkeley
Yohan Beugin, University of Wisconsin—Madison
Evangelos Bitsikas, Northeastern University
Jakob Bleier, Technische Universität Wien
Clemens-Alexander Brust, DLR Institute of Data Science
Jiahao Cao, Tsinghua University
Yuanhaur Chang, Washington University in St. Louis
Mohamed Hashim Changrampadi, Chalmers University of Technology
Kevin Cheang, Amazon Web Services
Chen Chen, Texas A&M University
Wei Chen, The Hong Kong University of Science and Technology
Xiaoqi Chen, Princeton University
Yanzuo Chen, The Hong Kong University of Science and Technology
Yongting Chen, New York University Shanghai
Mingfei Cheng, Singapore Management University
Pascal Cotret, ENSTA Bretagne
Jian Cui, Indiana University Bloomington
Giulio De Pasquale, King's College London
Zizhuang Deng, SKLOIS, Institute of Information Engineering, Chinese Academy of Sciences
Aolin Ding, Accenture Labs
Antreas Dionysiou, University of Cyprus
Tian Dong, Shanghai Jiao Tong University
Hossam ElAtali, University of Waterloo
Yuzhou Fang, University of Southern California
Yusi Feng, University of Chinese Academy of Sciences
Christof Ferreira Torres, ETH Zurich
Chuanpu Fu, Tsinghua University
Alexander Gaidis, Brown University
Lukas Gerlach, CISPA Helmholtz Center for Information Security
Shlok Gilda, University of Florida
Ravi Prakash Giri, Meta
Sindhu Reddy Kalathur Gopal, University of Wyoming
Sanket Goutam, Stony Brook University
Ziyi Guo, Northwestern University
Florian Hantke, CISPA Helmholtz Center for Information Security
Keno Hassler, CISPA Helmholtz Center for Information Security
Haojie He, Shanghai Jiao Tong University
Ningyu He, Peking University
Yi He, Tsinghua University
Tiago Heinrich, Federal University of Paraná
Hailong Hu, University of Luxembourg
Hongsheng Hu, CSIRO’s Data61
Jingmei Hu, Amazon
Shengtuo Hu, ByteDance
Zhenghao Hu, New York University
Long Huang, Louisiana State University
Qiqing Huang, University at Buffalo
Alfusainey Jallow, CISPA Helmholtz Center for Information Security
Yilin Ji, Karlsruhe Institute of Technology
Hao Jin, Texas A&M University
Yibo Jin, The Hong Kong University of Science and Technology
Jonas Juffinger, Graz University of Technology
Joseph Khoury, Louisiana State University
Soomin Kim, Korea Advanced Institute of Science and Technology (KAIST)
Sungwoo Kim, Purdue University
Konstantinos Kleftogiorgos, Stevens Institute of Technology
Torsten Krauß, University of Würzburg
John Kressel, University of Manchester
Tu Le, University of California, Irvine
Seongmin Lee, Max Planck Institute for Security and Privacy
Caihua Li, Yale University
Peiyang Li, Tsinghua University
Penghui Li, The Chinese University of Hong Kong
Zihao Li, The Hong Kong Polytechnic University
Han Liu, Washington University in St. Louis
Jing Liu, University of California, Irvine
Jinhong Liu, Zhejiang University
Shigang Liu, Swinburne University of Technology
Zhengyang Liu, University of Utah
Eman Maali, Imperial College London
Noah Mauthe, CISPA Helmholtz Center for Information Security
Atefeh Mohseni Ejiyeh, University of California, Santa Barbara
Kunal Mukherjee, The University of Texas at Dallas
Hoang Dai Nguyen, Louisiana State University
Abdun Nihaal, Indian Institute of Technology Madras
Xiaoyi Pang, Zhejiang University
Eric Pauley, University of Wisconsin—Madison
Deepak Pillai, University of Adelaide
Imranur Rahman, North Carolina State University
Mirza Masfiqur Rahman, Purdue University
Vidya Lakshmi Rajagopalan, Stevens Institute of Technology
Prashant Rajput, New York University Abu Dhabi
Jenni Reuben
Ritik Roongta, New York University
Zain Ruan, Massachusetts Institute of Technology
Abhinaya S.B., North Carolina State University
Mohammad Hammas Saeed, Boston University
Amit Samanta, University of Utah
Fan Sang, Georgia Institute of Technology
Benedict Schlüter, ETH Zurich
Till Schlüter, CISPA Helmholtz Center for Information Security
Christoph Sendner, University of Würzburg
Amir Shahatit, New York University
Salvatore Signorello, Telefonica Research
Sachin Kumar Singh, University of Utah
Johnny So, Stony Brook University
Avinash Sudhodanan, HUMAN Security
Yuqiang Sun, Nanyang Technological University
Leon Trampert, CISPA Helmholtz Center for Information Security
Billy Tsouvalas, Stony Brook University
Kevin van Liebergen, IMDEA Software Institute
Arturo Villacañas, CISPA Helmholtz Center for Information Security
Ryan Vrecenar, Sandia National Laboratories
Jinwen Wang, Washington University in St. Louis
Feng Wei, University at Buffalo
Shujiang Wu, F5
Yuhao Wu, Washington University in St. Louis
Qi Xia, The University of Texas at San Antonio
Jiacen Xu, University of California, Irvine
Jinyan Xu, Zhejiang University
Yibin Xu, University of Copenhagen
Jiongchi Yu, Singapore Management University
Zheng Yu, Northwestern University
Bowen Zhang, The Hong Kong University of Science and Technology
Jingyao Zhang, University of California, Riverside
Menghao Zhang, Tsinghua University
Mengya Zhang, The Ohio State University
Qifan Zhang, University of California, Irvine
Ruiyi Zhang, CISPA Helmholtz Center for Information Security
Yifan Zhang, Indiana University Bloomington
Zhiyu Zhang, SKLOIS, Institute of Information Engineering, Chinese Academy of Sciences
Zhiyuan Zhang, The University of Melbourne
Kunsong Zhao, The Hong Kong Polytechnic University
Yu Zheng, The Chinese University of Hong Kong
Chenyu Zhou, University of Southern California
Kyrie Zhou, University of Illinois at Urbana–Champaign
Man Zhou, Huazhong University of Science and Technology
Xia Zhou, Zhejiang University
Xugui Zhou, University of Virginia
Yi Zhou, Carnegie Mellon University
Artifact Evaluation Information
Overview
A scientific paper consists of a constellation of artifacts that extend beyond the document itself: software, hardware, evaluation data and documentation, raw survey results, mechanized proofs, models, test suites, benchmarks, and so on. In some cases, the quality of these artifacts is as important as that of the document itself. To emphasize the importance of such artifacts, the benefits to the authors and the community as a whole, and promote the availability and reproducibility of experimental results, USENIX Security runs an optional AE. The AEC will review each submitted artifact and also grant Distinguished Artifact Awards to outstanding artifacts accepted to USENIX Security '24.
Process
To maintain a wall of separation between paper review and the artifacts, authors will be given the option to submit their artifacts only after their papers have been accepted for publication at USENIX Security. The artifact registration deadline is around the camera-ready deadline, while the full artifact submission is about two weeks afterward. By the artifact submission deadline, authors can submit their artifacts, Artifact Appendix, and other supporting information of their accepted USENIX Security 2024 paper via the submission form using the provided submission instructions.
Authors define the contents of their artifact submission. For example, software, hardware, data sets, survey results, test suites, mechanized (but not paper) proofs, access to special hardware, and so on. Authors choose which badges their artifact should be evaluated towards (one, two, or all three of the following Artifacts Available, Artifacts Functional, and Results Reproduced. In general, good artifacts are expected to be: consistent with the paper, as complete as possible, well documented, and easy to (re)use. The AEC will read the paper and then judge if the artifact meets the criteria for each of the requested badges.
Each artifact submission will be reviewed by at least two AEC members. The review is single-blind and strictly confidential. All AEC members are instructed to treat the artifact confidential during or after completing evaluation, nor retain any part of it after evaluation. Thus, you are free to include models, data files, proprietary binaries, exploits under embargo, etc., in your artifact. Since we anticipate small glitches with installation and use, reviewers may communicate with authors for a period of up to five weeks after the artifact submission deadline to help resolve glitches while preserving reviewer anonymity.
The AEC will then complete its evaluation and notify the authors of the outcome around six weeks after the artifact submission deadline. Please make sure that at least one of the authors is reachable to answer questions in a timely manner.
Acknowledgements
The AE process at USENIX Security '24 is a continuation of the AE process at USENIX Security '20–'23 and was inspired by multiple other conferences, such as OSDI, EuroSys, and several other systems conferences. See artifact-eval.org for the origins of the AE process.