Artificial intelligence is becoming part of ordinary business life.
Companies are using it to draft documents, analyse customer behaviour, respond to enquiries, detect suspicious transactions and automate routine work. Some use it to screen job applications, review contracts or support financial decisions.
The appeal is understandable. Used properly, AI can save time, reduce costs and help employees work more efficiently. The same capabilities, however, are also available to criminals.
AI can help fraudsters write convincing emails, imitate the voice of a senior executive and collect information about an organisation before launching an attack. It is improving business productivity, while also making deception more polished, personalised and difficult to recognise.
The obvious scam is disappearing
For years, many phishing emails were easy to identify. They contained spelling mistakes, poor grammar or awkward language. The sender might claim to be a bank manager, company director or foreign investor, yet the message rarely sounded convincing.
Generative AI has begun to remove those warning signs.
A criminal can now produce a polished email in seconds, adjusting the language to sound formal, friendly, urgent or authoritative. The message can be written in the style of a particular profession or organisation, then refined repeatedly without requiring strong writing skills.
It can also contain accurate information.
Names, job titles, company projects and supplier details are often available on corporate websites, social media and professional networking platforms. A fraudulent email might therefore mention a genuine project, identify the correct senior manager and refer to a real supplier or expected payment.
The information can be accurate even when the request is not.
This matters because business email compromise is already one of the most damaging forms of cybercrime. According to the United States Federal Bureau of Investigation, cases reported between October 2013 and December 2023 involved about US$55.5 billion in exposed losses worldwide.
The underlying fraud is not new. Criminals have long impersonated executives, suppliers and business partners. What AI changes is the speed, scale and quality of the deception. An attacker no longer needs excellent writing skills or extensive knowledge of a company before producing a credible message.
A familiar voice may not be genuine
Email is only one part of the threat.
Voice-cloning technology can recreate the voice of a senior executive using material taken from interviews, speeches, online videos or recorded meetings. Deepfake technology can also produce realistic video of people appearing to say things they never said.
For businesses, this weakens one of the assumptions on which many decisions have traditionally depended: that recognising a person’s voice or face is enough to establish that an instruction is genuine.
An employee could receive a telephone call that sounds like the managing director, asking for an urgent payment or confidential information. The request might then be supported by an email or an official-looking document. In a more sophisticated attack, the employee might even appear to see the executive on a video call.
Recognition alone is no longer sufficient verification.
Requests involving payments, passwords, customer information or changes to supplier bank details should be checked through a second channel. An employee should call a number already held by the organisation, rather than using contact details included in the message being questioned.
The principle is straightforward: do not rely only on the voice, email address or face. Follow the agreed process.
When productivity creates a security risk
Not every AI-related security problem begins with a criminal. Sometimes the risk comes from employees trying to work faster.
A widely reported case involved Samsung Electronics in 2023. Employees in its semiconductor division reportedly entered sensitive company information into ChatGPT while seeking help with work-related problems. The material was said to include source code and information from an internal meeting.
The employees were not attempting to harm the company. They were using a convenient tool to complete their work more efficiently. Once the information had been entered into an external platform, however, it was no longer entirely within Samsung’s direct control.
The case illustrates a broader problem. A useful tool can become a security risk when employees do not understand what information they are permitted to share with it.
This does not mean that every AI platform handles data in the same way, nor does it mean that employees should stop using AI. It means that confidential information should not be uploaded to an external service merely because doing so is convenient.
Before approving an AI platform, a business should understand how the provider handles the information entered into it. Where is the data stored? Who can access it? How long is it retained? Can it be used to train or improve the provider’s systems?
Those questions should be answered before staff begin using the tool for sensitive work.
The rise of shadow AI
Many employees are already using public AI tools without formal approval.
They use them to draft emails, summarise reports, review contracts, analyse figures or solve technical problems. This informal and often hidden use of artificial intelligence is sometimes described as shadow AI.
The central problem is not simply that employees are using AI. It is that management may not know which tools are being used, what information is being entered into them or whether basic security controls are in place.
An employee could copy customer records, financial reports, contracts, business plans or internal investigations into a public AI platform. Others might enter system details, confidential code or commercially sensitive information without understanding where the data will go.
A company cannot manage the risks of tools it does not know its employees are using.
A complete ban may appear to offer a simple solution, but it can create another problem. Employees who find AI useful may continue using it privately, particularly when the organisation has failed to provide an approved alternative.
Businesses need rules that are practical enough to follow. Staff should know which tools are permitted, what information must never be entered into them and who to contact when they are uncertain.
The policy should be clear and accessible. It should not be buried in a long technology document that few employees will read.
Connecting AI to company systems increases the risk
The risks become greater when an AI tool is connected to internal emails, customer databases, financial systems and company documents.
An AI assistant with access to internal information can save employees considerable time. It can search documents, prepare summaries and answer questions about company records. The same access, however, can also make the system an attractive target.
Weak controls could allow confidential information to be revealed to the wrong employee, supplier or attacker. A system might also act on manipulated instructions without the user recognising what has happened.
One emerging threat is prompt injection. This occurs when someone gives an AI system instructions intended to override or weaken its original rules. An attacker might, for example, place hidden instructions inside a document or webpage that the system has been asked to review. The tool could then follow those instructions without the employee realising it.
There is also the risk of data poisoning, in which false or manipulated information is introduced into the data used by an AI system.
These threats sound technical, yet the management questions are familiar:
Who can access the system? What information can it see? Who approved that access? How is its activity monitored? What happens when it produces a wrong answer or reveals information it should protect?
Businesses should ask these questions of AI systems in the same way they would of any important financial, operational or information system.
External providers require proper scrutiny
Many AI services are supplied by external companies, which means businesses must also understand how those providers handle their information.
A company should know where its data will be stored, whether subcontractors can access it and how the provider will respond to a security incident. It should also examine the contractual position on privacy, data ownership and responsibility when something goes wrong.
The excitement surrounding AI can encourage organisations to move too quickly. A department may subscribe to a service because it appears useful, while questions about security and data protection are postponed until later.
That approach is risky.
An AI provider should be assessed with the same care as any other organisation handling customer data, financial information or important business records. Popularity is not evidence that a service is suitable for every type of information.
The final target is often still a person
Although the technology is becoming more sophisticated, many successful cyberattacks still depend on persuading a person to take an action.
An employee clicks a link. A finance officer changes supplier bank details. A manager sends confidential information. An IT employee resets a password.
AI makes the request more convincing, but the final target is often still a human being.
Cybersecurity training must therefore change. It is no longer enough to advise employees to look for spelling mistakes or poor grammar. A fraudulent message can now be professionally written, highly personalised and supported by convincing audio or video.
Employees should be particularly cautious when a request combines urgency, secrecy, money or sensitive information. A message insisting that a payment must be made immediately and must not be discussed with anyone should raise concern, even when it appears to come from the chief executive.
Businesses also need a culture in which employees are allowed to question unusual instructions.
This is not only an IT issue. It is a management and governance issue. When staff are afraid to delay a payment or challenge a senior executive, criminals can use that fear against the organisation.
A strong control system gives employees both the authority and the responsibility to stop and verify.
AI can also strengthen cybersecurity
Artificial intelligence is not only helping criminals. It can also help businesses identify phishing emails, detect suspicious transactions and examine unusual activity across their systems.
Security teams can use it to review large volumes of information and identify patterns that a human analyst might miss. It can help organisations respond more quickly when something appears abnormal.
The question is therefore not whether AI is inherently good or bad. The more useful question is how it is being used, what information it can access and whether the organisation has retained effective oversight.
Responsibility cannot rest with the IT department alone.
Finance teams need to understand the risk of fraudulent payment instructions, procurement teams must assess AI providers, human resources should ensure that employees understand the rules and legal, risk and compliance teams need to consider privacy, accountability and regulatory exposure.
Internal audit should examine whether the controls are working, while senior management and boards should ask where AI is being used, what information it can access and what would happen if the system were manipulated or produced a damaging answer.
AI now affects too many areas of business to be treated as a narrow technology matter.
Faster business requires stronger controls
Artificial intelligence will continue to shape how companies operate.
Businesses that ignore it could miss opportunities to improve productivity, reduce costs and serve customers more effectively. Those that adopt it carelessly, however, risk exposing themselves to fraud, legal disputes, data loss and reputational damage.
The answer is not to reject AI. It is to use it under clear rules and with controls that reflect the seriousness of the information involved.
Companies need approved platforms, practical staff training and clear restrictions on what employees can enter into external systems. They also need verification procedures that do not depend solely on recognising an email address, voice or face.
AI is making businesses faster and attackers more capable.
The organisations that benefit most will not necessarily be those that adopt it first. They will be those that understand where the risks lie, assign responsibility clearly and retain the discipline to verify before acting.
Sources and further reading
Federal Bureau of Investigation, Internet Crime Complaint Center. Business Email Compromise: The $55 Billion Scam. 2024.
Europol. Facing Reality? Law Enforcement and the Challenge of Deepfakes. 2022.
National Cyber Security Centre. Impact of AI on Cyber Threat from Now to 2027. 2025.
Cybersecurity Dive. Samsung Employees Leaked Corporate Data in ChatGPT. 2023.
National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework (AI RMF 1.0). 2023.
Leave a Reply