Securing AI in the real world: An organisational dilemma

As the industrial wheel of Artificial Intelligence (AI) turns another revolution thanks to the rapid adoption and exponential rise of Generative AI (Gen AI), the opportunity for organisations to embrace AI into their working practices is even greater than at any time before.  

Whilst AI could potentially deliver significant benefits (the use of robotics within technological processes, or Large Language Models (LLM's) and Chat Agents within operations), this has led to an increase in risks and vulnerabilities (for example - data breaches, biased results & misinformation, poisoning of the training models - all of which have been well documented), organisations face a serious dilemma.  

On one hand, organisations feel that they need to adopt AI as they aim to increase competitive advantage through innovation. On the other hand, if organisations overlook the security considerations, or fail to address the risks associated with the potential threats, this could actually outweigh any potential benefits organisations may have initially perceived. 

With AI vulnerabilities on the rise, we’re now in an era where misinformation and disinformation are even easier to sneak into machines and models (whilst also being even harder to distinguish from fact). To compound the challenge, the regulatory landscape is also evolving at great velocity, and with the introduction of the EU AI Act, the landscape is becoming progressively more challenging to navigate. 

This article looks at some of the risks that result from insufficient oversight of cybersecurity in AI deployment and what they could mean to organisations. It also suggests ways in which organisations can embrace established and emerging models effectively whilst ensuring the appropriate mitigations are in place. 

Cybersecurity Risks posed by AI Adoption 

An Increasing Cyber Threat Landscape 

A range of organisations have highlighted the dominance of AI risks in the Cyber Threat landscape and some core areas for concern. The World Economic Forum (WEF) published their ‘Biggest cybersecurity threats for 2025’ in February, with AI featuring twice in their six key vulnerabilities anticipated by leaders in 2025. They cited that “many organisations lack processes to properly assess the security of AI tools before deployment”. In addition, WEF confirmed that (in reference to Gen AI) 75% of organisations report rising cyber risks, with generative AI fuelling more sophisticated social engineering and ransomware attacks.  

The Cloud Security Alliance earlier this year expressed similar concerns in their article on ‘The Emerging Cybersecurity Threats in 2025’, confirming that AI driven attacks are within their top 5 threats. They confirmed that AI allows threat actors to automate and scale their operations, making it easier for them to bypass traditional security measures. 

Bringing the subject closer to home, the UK’s National Cyber Security Centre’s (NCSC) 2024 Annual review states that AI is one of the main tools deployed by nation-state threat actors when carrying out reconnaissance, social engineering and analysis of exfiltrated data. 

Vulnerabilities 

As LLM’s and AI agents become absorbed into organisations’ business processes with use cases ranging broadly from internal/employee-only use, to solutions that are being architected to replace the need for human involvement along the supply chain. With this technology becoming more and more embedded in organisations it is crucial to consider that these models are fallible and subject to vulnerabilities. The data set that feeds these models is known as prompt engineering (which you can read more about here. These models which have been trained (on data sets) to provide a service and are susceptible to vulnerabilities that will affect the outcome of the answers they provide. One such vulnerability is Prompt Injections which has been listed by the Open Web Application Security Project (OWASP) as the no.1 security issue in their “Top 10 for LLM applications 2025” report.  

Biases and Misinformation 

Organisations need to be aware of the biases within any models which they turn to and understand that outputs are shaped by the datasets used during training (which undoubtable will include specific biases). Text generators (such as Chat agents) are only making a probabilistic guess of a sequentially-arrangement of words or algorithm based on the training data which they have absorbed. Sometimes it is easy to overlook this fact and assume that the responses given by such chat agents and generators are accurate and trustworthy.  

An additional threat posed by AI, is that of misinformation. One of the main causes of this resides within the dataset that the AI model has been trained upon.  There have already been numerous examples of LLM’s returning content that has been confirmed as misinformation, demonstrating bias and other anomalies (‘hallucinations’), as a result of flawed training.  AI models can also be susceptible to attacks from bad actors who are able to manipulate the algorithm of the model, to change the output that the model returns to the user, resulting in negative consequences to either the user or the entity in question. 

User Data - Privacy Concerns 

Another core area for concern when deploying AI is data privacy. Employees using a tool will add inputs (which may include personal or confidential data. This data may in turn be used by the supplier of a model to further train or fine tune it. As such this information could be (inadvertently) made available (or alluded to) in the responses the tool generates for other external users. Another privacy concern relating to user data worthy of consideration is that in some instances the data may be made available to foreign governments. A risk highlighted most notably by the arrival of DeepSeek, with the Australian government banning the tool due to the threat it poses to national security. 

Practical Steps to Improve Compliance 

It's highly likely that the subject of AI will arise within your organisation. Whether you procure a third-party AI solution or decide to create your own.  

Whether it’s in the planning, deployment, on-going maintenance or decommissioning of the AI solution, cybersecurity plays a vital role in ensuring protection against threats/vulnerabilities faced within the lifecycle. 

Navigating this journey is going to require more than just ticking the boxes of a vendor/suppler checklist. There are some immediate practical steps that you can take which – if followed, can give an organisation the confidence and assurance (particularly amongst its senior management, employees, suppliers and its customers), which will lead to having a competitive advantage when it comes to utilizing the appropriate AI solution for your organization.  

Some practical steps to consider as an organisation include: 

1. The Business Case: What is the process that you are looking to replace with AI. What are the benefits (cost, capabilities, dependencies, time) and risks (threats/vulnerabilities) that need to be considered and how do these factors compare with the existing process/es?  
2. Due Diligence Process: Ensure there is a clear and compulsory process which must be followed when procuring, developing and implementing AI and ensure controls are in place within all phases of its lifecycle.  Make sure that all relevant stakeholders are engaged in these processes with cybersecurity a key part of the team. 
3. Review Data Sharing Agreements: The due diligence process should include a careful review of all contractual terms provided by the supplier of any LLM that is looking to be procured. This assessment should look at how your data (such as user inputs) will be used by the supplier and who they may grant access to this data.  
4. Responsible and safe AI practices: The process should also include a consideration of the specific use cases, and assess the risk posed by biased outputs and potential hallucinates or delivery of misinformation to consider potential mitigations. Where developing your own model think about the data you’re going to feed the model for training, and if anything can be done to counterbalance potential biases embedded in the data set. Does the data include information of a sensitive or confidential nature? If so, what controls are in place to prevent this information accidently being disclosed.  
5. Training & Awareness: Ensure that suitable training, guidance and policies are provided on the acceptable use and deployment (and development, if relevant) of AI within your organisation and that these are periodically reviewed and updated. 
6. Monitoring and Audits: Once the AI solution is live, ensure it is monitored against suitable metrics and audited appropriately considering the specific use case and risks posed.  
7. Vulnerability Testing: Tools and techniques can be deployed to test how easily AI models can be fooled, manipulated or otherwise compromised in addition to traditional vulnerability scanners used to detect software and API vulnerabilities. The most common vulnerabilities are still often overlooked when it comes to AI deployment. Utilising frameworks such as the OWASP Top 10 for LLM Applications can help form your vulnerability management strategy. 

Of course, regulatory risks and obligations should also be considered. With local laws and regulations recently coming into force (e.g., The EU AI Act), the interplay between technology, privacy and data protection means that a company-wide approach needs to be adopted when rolling out your solution. This also means that cybersecurity plays a crucial role in ensuring that AI systems are resilient against attempts to alter their use, behaviour, performance or compromise of security by malicious third parties. Furthermore, it could be a legal requirement to have the appropriate technical documentation drawn up before that system is deployed.   

Whilst ensuring cybersecurity compliance may seem challenging, our data protection and cyber-risk service have extensive experience in navigating complex compliance requirements and in assisting organisations in adapting to evolving regulatory landscapes. Our expertise ensures that your organisation stays ahead of regulatory changes, fostering resilience and trust in a rapidly changing digital world. If you would like to discuss our services, please contact our advisors.