Skip to main content

Knowledge Hub

New page title image

Cracking the code for cybersecurity product evaluations

5 min read
Cracking the code for cybersecurity product evaluations
Image Source: Canva

As cybersecurity grows in scope and complexity, cybersecurity practitioners are often scrambling to acquire new and more effective solutions to tackle the constantly evolving landscape of threats and challenges in the digital realm.

In the face of this, evaluation and deployment teams must juggle multiple tasks simultaneously. They include resisting pressure from internal stakeholders, conducting proof-of-concept (POC) evaluations, collaborating with vendors, and gaining the support of users. How can cybersecurity leaders navigate the chaos and emerge with the right tools? 
 

The cybersecurity landscape today

For Hoo Chuan-Wei, the CISO of StarHub, the situation for CISOs is far better today than it was a few years ago. Hoo observed that the CISO function wasn’t heavily emphasised in the earlier days, and cybersecurity decisions were typically driven by CIOs.

Today, CIOs and CISOs work together on the best way to meet operational, compliance, and governance needs. In effect, each office will lobby for support from each other, with the CISO serving as a check and balance when it comes to cybersecurity governance.

“CISOs tend to look at controls that are fit for purpose for business. That means we take a holistic approach to data; everything evolves around data. Most of the controls or the solutions we look for must be able to address data in terms of confidentiality, integrity, availability, sovereignty, and reliability,” Hoo explained.

Yaron Slutzky, the Chief Security Officer at Agoda, says he has witnessed a huge shift in the cybersecurity landscape over the last two years as CISOs seek new solutions to address the changing face of digital systems.

“IT environments are changing so quickly. Organisations are migrating to the cloud and turning to digital transformation. All these have forced companies to look at new solutions to support their new environments, as well as reevaluating existing solutions from a cost perspective,” he said.

Should organisations gravitate towards cybersecurity startups or traditional brands? Slutzky says it’s about achieving the right balance: “If you have a very specific case that you're trying to address, I think it's good to consider cybersecurity startups. For large enterprises, I would probably go with a more mature, traditional provider that offers more support and is more experienced.”
 

Performing the POC 

What should cybersecurity teams look out for during a POC? For Yaron Slutzky, clearly defining the criteria for success is vital. Because the field evolves so quickly, security experts or CISOs are sometimes left uncertain about how to define the KPIs for a successful evaluation. Regardless, these should be nailed down right at the beginning.

Slutzky also cautioned against rushing an evaluation, noting how some vendors might pressure companies with a time-constrained review period. Indeed, the difference between success and failure often hinges on a company’s ability to accurately assess whether a solution meets specified requirements and functions as advertised.

A common mistake is to get carried away and only consider a single solution. This can occur when the organisation becomes enamoured with specific technical capabilities or has developed a deep trust in a cybersecurity brand or its leadership.

“Make sure that it's solving your problem; make sure you're bringing the right solution in. And compare it with other solutions – don't blindly go with one vendor. Do a check with two, three, or even four competing products to ascertain the right solution for your organisation.”
 

Mind the scope creep

Hoo sounded a warning about potential scope creep when evaluating a new solution, noting how this often seeps in unnoticed.

“You’re looking for a particular solution to address a particular need. Then you noticed how this solution stack has other features that could potentially address other concerns. And just because it’s possible to get everything in one solution, the team decides to go with this vendor. But your scope has just increased overnight, and along with it, the budget.”

One danger of scope creep would surely be overlapping controls, where two systems perform the same function. Inevitably, this means the total cost goes up. Hoo puts it this way: “It doesn’t make sense if you are going to spend $5 to protect a $2 asset. But if you are proposing spending $2 to protect $5 of assets, then yes, we should talk.”

Rather than approach each purchasing decision on a case-by-case basis, Hoo suggests working with the CIO to come up with a common network security reference architecture. The idea is to ensure that both the CIO and CISO are aligned in their vision of the company’s architecture.

"Ultimately, the two parties need to come together and say, 'We agree on this common architecture, this common facade.' Only then do you achieve a better outcome, either using a best-of-breed or a more cost-effective approach. That is important. The reference architecture drives a lot of things."
 

Don’t forget the users

Both CISOs agreed that the considerations of users are non-negotiable and must be taken into account. Hoo said, "We need to take into consideration how a user will behave and react. We cannot implement new systems with new processes in a draconian way, unless in a highly sensitive environment such as defence. In commercial organisations, I think there is room for negotiation."

Slutzky echoed Hoo’s viewpoints, noting that it’s a matter of being deliberate to build trust across the organisation. He recommends starting any new deployment with a smaller group, and then expanding it across departments while continually fine-tuning it and ensuring that users remain receptive.

In his view, there are always edge cases, and there would be scenarios where users have completely justifiable reasons to push back. In such situations, the onus is on the cybersecurity team to rethink their strategy as necessary.

“Eventually, if employees trust you and they understand that you are rolling out the new solution because there’s a risk that you are trying to solve – I think they will go along with you. Help them understand it by explaining it better; work with them and not against them,” said Slutzky.

“If employees trust you and they understand that you are rolling out the new solution because there’s a risk that you are trying to solve – I think they will go along with you. Help them understand it by explaining it better; work with them and not against them.”– Yaron Slutzky, Chief Security Officer, Agoda.

Working with vendors

Finally, how should CISOs work with cybersecurity vendors? On his part, Slutzky recommends performing regular quarterly business reviews (QBRs) with vendors to determine what went well and what didn’t.

“Doing a QBR with your vendors is good for relationships. They have an interest in it because they want to improve while we want to get better outcomes. A QBR helps us build a better relationship with our vendors to get the support that we need. At the same time, vendors learn from it to get a better idea of what to improve and what needs to change.”

Hoo says he starts by being transparent in his communications. “I always believe in honesty and transparency in dealing with vendors. You do not want to lead them on – share what you can share. If you're honest and transparent, most of the vendors I have come across will reciprocate in the same way.”

Specifically, this means being quick to acknowledge potential shortfalls in their offerings, allowing for better and faster decision-making. Of course, he doesn’t take everything a vendor says at face value but takes the time to research the solution thoroughly himself. And this should happen even before talking to the vendor, says Hoo.

“Do your solution research first. What are you trying to achieve? Do your own comparison. There are many papers out there. Read them. And because you did your research, you will know whether a vendor is being honest with you in terms of the capabilities of their products,” he said.

“Do your solution research first. What are you trying to achieve? Do your own comparison... And because you did your research, you will know whether a vendor is being honest with you in terms of the capabilities of their products.”– Hoo Chuan-Wei, CISO, StarHub.

Any parting word of advice for aspiring CISOs out there? Hoo has this to say: “Be careful what you wish for. It’s every cybersecurity professional’s dream to become a CISO. But I will tell you this. The CISO role looks glamorous on the outside, much like a fighter pilot in their flight suit. But in combat, you have to make very difficult, split-second decisions. If you are not prepared for that, then it’ll be a challenge.”

 

View All Articles
Loading