Download This PaperRandomized Controlled Trials for Microsoft Copilot for Security
10 Pages Posted: 2 Dec 2023 Last revised: 8 Apr 2024
Date Written: March 27, 2024
Abstract
We conducted randomized controlled trials (RCTs) to measure the efficiency gains from using Copilot for Security, including speed and quality improvements. External experimental subjects logged into a M365 Defender instance created for this experiment and performed four tasks: Incident Summarization, Script Analyzer, Incident Report, and Guided Response. We found that Copilot for Security delivered large improvements on both speed and accuracy. Copilot brought improvements for both novices and security professionals.
Keywords: security, AI, RCT
JEL Classification: O32, L86, J24, C93
Suggested Citation: Suggested Citation
Edelman, Benjamin G. and Bono, James and Peng, Sida and Rodriguez, Roberto and Ho, Sandra, Randomized Controlled Trials for Microsoft Copilot for Security (March 27, 2024). Available at SSRN: https://ssrn.com/abstract=4648700 or http://dx.doi.org/10.2139/ssrn.4648700
Do you have a job opening that you would like to promote on SSRN?
Feedback
Feedback to SSRN