P3129/D3,August2022 Draft Standard for Robustness Testing and Evaluation of Artificial Intelligence (Al)-based Image Recognition Service P3129 TM/D3 Draft Standard for Robustness Testing 2 and Evaluation of Artificial Intelligence 3 (Al)-based Image Recognition Service 4 5 Developed by the 67 Artificial Intelligence Standards Committee of the IEEE Computer Society Approved <Date Approved> 45 IEEESAStandardsBoard 16 Copyright 2021 by The Institute of Electrical and Electronics Engineers, Inc. 17 Three Park Avenue 18 New York, New York 10016-5997, USA 19 All rights reserved. This document is an unapproved draft of a proposed IEEE Standard. As such, this document is subject to change. USE AT YOUR OWN RISK! IEEE copyright statements SHALL NOT BE REMOVED from draft or approved IEEE standards, or modified in any way. Because this is an unapproved draft, this document must not be utilized for any conformance/compliance purposes. Permission is hereby granted for officers from each IEEE Standards Working Group or Committee to reproduce the draft document developed by that Working Group for purposes of international standardization consideration. IEEE Standards Department must beinformed of thesubmission for considerationprior to any reproduction for international standardization consideration ([email protected]). Prior to adoption of this document, in whole or in part, by another standards development organization, permission must first be obtained from the IEEE Standards Department ([email protected]). When requesting permission, IEEE Standards Department will require a copy of the standard development organization's document highlighting the use of IEEE content. Other entitiesseekingpermissiontoreproducethisdocument,inwholeorinpart,mustalsoobtainpermissionfrom theIEEEStandardsDepartment. 343 IEEE Standards Department 445HoesLane Piscataway, NJ 08854, USA 36 P3129/D3,August 2022 Draft Standard for Robustness Testing and Evaluation of Artificial Intelligence (Al)-based Image Recognition Service 1 Abstract: This standard provides test specifications with a set of indicators for common corruption 23 and adversarial attacks, which can be used to evaluate the robustness of Artificial Intelligence- basedImageRecognition services.This standard specifiesrobustness attack threats sand 4 establishesanassessmentframeworktoevaluatetherobustnessofArtificialIntelligence-based 5 Image Recognition service under various settings. 6 7 Keywords: Artificial Intelligence-based services, robustness, common corruption, adversarial 8 attacks,assessment framework 9 10 The Institute of Electrical and Electronics Engineers, Inc. 3 Park Avenue, New York, NY 10016-5997, USA Copyright 2020 by The Institute of Electrical and Electronics Engineers, Inc. All rights reserved. Published <Date Published>. Printed in the United States of America. IEEE is a registered trademark in the U.S. Patent & Trademark Office, owned by The Institute of Electrical and Electronics Engineers, Incorporated. PDF: ISBN 978-0-XXXX-XXXX-X STDXXXXX Print: ISBN 978-0-XXXX-XXXX-X STDPDXXXXX IEEE prohibits discrimination,harassment,and bullying Formore information, visit https://www.ieee.org/about/corporate/govemance/p9-26.html of thepublisher P3129/D3,August2022 Draft Standard for Robustness Testing and Evaluation of Artificial Intelligence (Al)-based Image Recognition Service 1 Important Notices and Disclaimers Concerning IEEE Standards Documents 234 IEEE Standards documents are made available for use subject to important notices and legal disclaimers. These notices and disclaimers, or a reference to this page (https://standards.ieee.org/ipr/disclaimers.html), appear in all standards and may be found under the heading Important Notices and Disclaimers Concerning 5 IEEEStandardsDocuments."" 6 Notice and Disclaimer of Liability Concerning the Use of IEEE Standards 7 Documents 8 IEEE Standards documents are developed within IEEE Societies and subcommittees of IEEE Standards 9 Association(IEEES