Inside our response to a surge in AI-assisted security reports

Many software companies, Deskpro included, invite independent security researchers to look for vulnerabilities in their products. It’s a way of improving security through crowdsourcing: the researchers help us find vulnerabilities so we can fix them before they become a problem, and they get to claim credit (and potentially a bug bounty) for the discovery. Programs like this have been common in the software industry since the early 2010s, but recently, they’ve taken on a new life because of AI.

Many security researchers now use AI to scale up their ability to find vulnerabilities and write reports. For many engineering teams, this has led to a large increase in vulnerability disclosure reports, and we have seen the same trend. The reports vary in quality: some describe valid issues, some duplicate issues that were already reported, and some cannot be reproduced.

Although I haven’t kept an exact tally, I’d estimate that Deskpro’s engineering team has received well over 100 vulnerability disclosure reports to date this year, many of them likely coming from researchers using AI as an assistant. We’ve learned a lot from these reports, and those takeaways are helping us reshape our processes, resolve vulnerabilities more effectively, and successfully partner with security researchers. I’ll cover five of our biggest lessons below.

AI changed how reports reach us and how we handle them

Deskpro used to receive an easily manageable number of vulnerability reports. Now, we’re seeing a much higher volume of reports coming in bursts (often over the weekend, presumably because a lot of security researchers are finding vulnerabilities as a side project outside their full-time jobs). We’re also seeing multiple reports about the same potential vulnerability coming in around the same time, further increasing the noise in our inbox.

Fortunately, just as security researchers are using AI to speed up the process of finding and reporting vulnerabilities, our team is using AI to speed up the process of triaging reports, verifying vulnerabilities, and producing fixes. (I wrote about this in depth in my article “How AI is transforming the process of finding and fixing security vulnerabilities.”)

We use AI to help us reproduce and verify the vulnerabilities because AI is much faster than humans at reviewing hundreds of thousands of lines of code. It helps us keep pace with the increased volume of reports without sacrificing quality. We always have team members review AI outputs for accuracy, and we have named people behind every decision, from rating the severity of the vulnerability to implementing a fix.

AI-assisted reports are often right about the problem but wrong about its severity

While researchers send us reports in different formats, most include a severity rating based on the Common Vulnerability Scoring System (CVSS). This system uses a standard set of metrics to determine a 0-10 score for software vulnerabilities, where 0 is the lowest and 10 is the highest severity. While AI can help assign a CVSS rating to a reported vulnerability, it often misses some of the nuances that go into determining the score and either overstates or understates the problem.

My team and I are responsible for evaluating the real risk. We reproduce the vulnerability and use the CVSS to determine our own score. We have the benefit of additional context: our team members know the product well, and we may also be reviewing other reports about the same or related issues. One benefit of getting many reports at once is that we’re often able to piece together that while one vulnerability is not that bad in isolation, the potential harm is amplified once we factor in a related vulnerability.

Evaluating the severity level internally helps us prioritize our work so that we can release patches for the most critical vulnerabilities first, mitigating the risk of a serious breach.

Clear communication helps us address reports faster

This might seem obvious, but it’s worth noting: the easier it is for both humans and AI to parse through and understand a vulnerability report, the faster we’re able to address it. When reports come in with a vague description of the issue, our team has to do more work to figure out how to reproduce it. When a report includes a step-by-step guide to reproduce the issue, video recordings, and network requests or other proof that the researcher actually found the vulnerability, it takes our team considerably less time to verify. A well-written report not only makes it easier to review but can be the difference between “we can’t reproduce this” and “let’s get this fix prioritized.”

Our team also aims to be clear when we communicate with security researchers. We respond to all reports with our position, whether we’re confirming we verified the vulnerability and implemented the fix or telling them we could not reproduce it. We also document our position so that we send a consistent response when the same issue is reported multiple times. With duplicates, the earliest report typically gets the main award for finding and reporting the vulnerability, but if someone else has independently found it, we still give them credit where their research helps us.

Clear two-way communication is important to us because responsible reporting helps us keep customers safe. When researchers feel respected, they’re more likely to keep working with us.

If we can’t reproduce something, we don’t just close out the report

With the influx of AI-assisted reports, the easiest path would be to call a report resolved if we were unable to reproduce it, but just because we can’t initially reproduce a vulnerability doesn’t mean it’s not there.

We look at the version of our software the research is on so we can determine if the vulnerability is something we’ve fixed with a newer release. If that’s not the case and we still can’t reproduce the vulnerability, we go back to the researcher and ask for more information. In some cases, they do provide useful information that helps us reproduce the issue. If they can’t, we close out the report. We have a named person who owns that decision, and if the researcher challenges the decision, we do our due diligence and look at the report again. We recognize there are times we can be wrong, and we never want inflexibility to get in the way of us improving our product’s security.

AI is a valuable tool, but it doesn’t have the final say

My biggest takeaway from the influx of AI-assisted reports is that AI is a valuable tool for both software engineers and security researchers, but it can’t replace human expertise and judgment. AI makes mistakes, which is why any responsible software development team should always have humans reviewing and owning what gets shipped. We use AI as an accelerant rather than a replacement for human thinking and contextual knowledge.

We’re rolling this learning up and building a bespoke portal for researchers to use when submitting reports to us. The aim is for us to get more high quality reports more consistently, and reduce the amount of back and forth needed before we get to a validation.

When researchers use AI well and we take their work seriously, our customers are safer. The rise of AI-assisted vulnerability reports is still a relatively recent development, and we’ll continue to learn from this new reality and improve our processes as we go.