The Future of Cybersecurity is Human-centred AI

As AI continues its inexorable advancement and agentic AI makes inroads, change is inevitable. While it's hard to predict exactly what will happen, cybersecurity professionals believe knowledge of AI is no longer optional, observed Jason Lau, a member of the Global Board of Directors at ISACA.

Speaking with GovWare, Lau shared details of a recent survey. He said: “Almost 90% of cybersecurity professionals say increased AI knowledge and skills are needed to retain their job or advance their careers in the next two years. If they don't augment their capabilities, they're going to be at risk of potentially losing their job.”
 

Towards Collaborative Intelligence

Lau isn't suggesting that AI will replace cyber experts. “There's a lot of talk about whether AI is going to replace humans. I don't think AI is going to replace humans at this point. At least for now, AI will augment humans in a collaborative intelligence fashion.”

This collaborative approach reflects a broader shift in how the industry views the use of AI: “Agentic AI is fundamentally reshaping how we respond to threats. Not just for detecting anomalies but being proactive with autonomous actions. What used to take hours of triaging and labour-intensive coordination can be handled by AI within seconds.”

This speed advantage manifests across multiple cybersecurity domains. AI can transform incident response by automating the scanning of vast volumes of log files, identifying anomalies and patterns that might take human analysts hours to detect. It can also enhance insider threat detection and streamline post-incident response triaging, turning what was once a laborious and error-prone process into rapid, systematic analysis.

“In cyber warfare, speed is survival — especially in incident response. Agentic AI is our newest weapon on the digital battlefield,” said Lau, though he added that this assumes AI systems are designed carefully and correctly.

The Human in the Loop

Despite the promise of agentic AI, cybersecurity leaders must be clear-eyed about its limits. Not every task should be handed over to machines, especially when human judgment, accountability, or nuance is required, says Lau.

“AI is great at handling clear-cut incidents where there's a well-defined playbook to follow. But the moment legal, reputational, or ethical issues come into play, it's crucial that humans stay in the driver’s seat. In those situations, AI should inform decisions, not make them.”

He also cautioned against giving AI free rein over high-stakes decisions, such as terminating accounts or granting access to sensitive data – actions that would typically require management approval. He drew a parallel to the risks of automating public communications, noting that an AI-generated crisis response could backfire if not carefully vetted by a qualified communications expert first.

“I would be rather comfortable with AI autonomously helping with well-defined, low-risk, low-variance type of situations,” said Lau. “We have to trust AI to act fast when the risks are low, but never really to act alone when accountability is on the line.”

“We’re not looking for AI to make high-stakes decisions on its own, especially when there’s a risk of data poisoning or model manipulation. The goal is for AI to handle low-risk tasks reliably, and to know when to escalate more complex or sensitive issues to a human,” Lau said.
 

The Hidden Risks of AI

One concern Lau highlighted is when AI is put indiscriminately in control of systems without appropriate oversight. This can result in unintended consequences, especially when those systems interact with others autonomously and without human intervention.

“The danger lies in the black box – when AI interacts with other AIs or systems and decisions are made in the background, without any human in the loop. That’s when things can spiral quickly and unpredictably,” explained Lau.

“We often don’t realise the risks until something goes wrong. Only then do we ask, ‘Why was this AI agent acting this way or tapping into another system via an API in a way it’s not meant to?’ Without full transparency, we’re operating in a black box, and that’s when AI stops being a tool and starts becoming a risk multiplier.”

But can’t organisations simply put extensive logging in place and monitor all AI interactions to stay in control?

“Logging helps, but it’s not a silver bullet. It all comes down to design, transparency, and capacity. Most organisations face resource constraints that limit how much can be monitored in real time. That’s why we need to be deliberate about what we let AI agents do: too much autonomy, too soon, without oversight, is where risk scales fast.”
 

Red Teaming, Testing and Trust

There is no putting AI back in the box now. What are some ways we can ensure that it’s properly secured? Lau recommends that organisations to red team their AI to check for potential vulnerabilities.

“From a practical perspective, you need to have your red teaming and internal teams conduct penetration tests and simulated attacks. We always hear about adversarial inputs and all sorts of different things, so we try our best to carry out as much pen testing as possible.”

How can organisations take the next step with AI? Lau recommends experimenting through a process of continuous iterative improvements, but to always test and verify. Ultimately, AI isn’t a luxury; it’s necessary for cyber defenders to stay ahead.

“Explore how things can be done better, more efficiently and faster. It doesn't necessarily mean you need to take highly risky experiments – but stay within the scope of what you're doing,” he said.

“Organisations must make a deliberate shift to integrate AI into their cybersecurity strategies, because threat actors are already doing so. And if we fail to keep up with their pace, we’re going to expose organisations to even greater and escalating risks.”