How Hackers Can Up Their Game by Using ChatGPT

Consumers, beware: AI chatbots like ChatGPT are likely to drive an increase in the use and effectiveness of online fraud tools such as phishing and spear-phishing messages.

In fact, it could already be happening. Phishing attacks around the world grew almost 50% in 2022 from a year earlier, according to Zscaler, a cloud-security provider. And, some experts say, artificial-intelligence software that makes phishing messages sound more believable are part of the problem. AI reduces or eliminates language barriers and grammatical mistakes, helping scammers impersonate a target’s colleagues, friends or relatives.

“This new era is going to be worse than what we had before,” says Meredith Broussard, research director at the New York University Alliance for Public Interest Technology. “And what we had before was really, really bad.”

High stakes

AI chatbots have exploded in popularity, with perhaps the best-known being ChatGPT, developed by the AI-research company OpenAI, a strategic partner of Microsoft. But dozens of chatbots, using what are referred to as large language models, are becoming more widely available and can closely mimic human communication based on data they amass. These models can be used for many purposes, such as helping office workers create routine memos more quickly. But they can also be used by criminals—to defraud victims, for instance, or to spread malicious viruses.

Telltale signs of a phishing attack have long included mistakes in grammar or spelling. But AI can give a phishing attack more credibility—and reach—not just because of its ability to generate fluent, grammatical messages in many languages, but also because of its ability to mimic the speaking or writing styles of individuals.

“The whole point with large language models is their ability to emulate what humans sound like,” says Etay Maor, senior director of security strategy at Cato Networks, a cloud networking and security provider.

Thus, given the opportunity to learn the style in which a certain person writes emails and texts, Maor says, an AI program can be used to mimic communications from a company executive.

“It’s all about trust, and if I can make you think I’m one of you, you’re going to begin to do things with more trust and less skepticism,” says Roger Grimes, a computer-security professional with KnowBe4, a security-awareness training and simulated-phishing platform.

Using AI, Grimes says, criminals can quickly determine industry-specific terms that give them more ability to target companies such as hospitals, banks and fintech.

Targeted campaigns

AI’s usefulness in phishing and spear-phishing attacks doesn’t stop with its ability to mimic authentic human communication. The analytic skills of machine learning can also be useful in determining who best to target in an organization and how exactly to attack them.

Sean McNee, vice president of research and data at DomainTools, an internet intelligence company, offers a hypothetical example. Say an accountant at a company innocently posts on social media about his frustrations with a recent audit. AI could determine the accountant’s peers, his company’s reporting structure and who else at the company might be most susceptible to an attack. The attacker then could create a spear-phishing email purporting to be from the chief financial officer referring to a discrepancy in the audit and asking the recipient to open an attached spreadsheet that contains a virus.

Ramayya Krishnan, dean of Carnegie Mellon University’s Heinz College, recommends being proactive to protect against such attacks.

First, before acting on something, he says, people should always verify the legitimacy of the request through independent means. This means before clicking on a link or sending money, the recipient should call the individual through a familiar phone number or walk into the person’s office to confirm the request, Krishnan says.

Maintain a healthy dose of skepticism for everything you receive, Maor says. Ask yourself, why is my bank emailing me? Why is there a sense of urgency? Why is there an attachment to click on? It’s also advisable to hover over a link before clicking to see if it leads to an expected URL. “If you have some reason to think something is amiss, don’t click on it,” Maor says.

Other guardrails

Strong regulation of AI could also help, says Broussard, who is also an associate professor at the Arthur L. Carter Journalism Institute of New York University.

AI itself should also be enlisted to help identify malicious content with its origins in AI, says Dave Ahn, chief architect at Centripetal, a cybersecurity company. But first the models for doing so will have to evolve and the data will have to improve. Data on successful AI-based attacks will help cybersecurity experts train new models to identify malicious activity better, says Ahn.

Other possible security measures include giving users a way to distinguish their content as authentic. The use of hidden patterns known as “watermarks,” for instance, can be buried in AI-generated texts to help identify whether the words are written by a human or computer, Krishnan says. But the applicability of these tools is limited.

Says Krishnan, “We’re not near deploying them at scale where it’s a solution to the bad-actor potential we have today.”

For decades, the world’s computing power has quietly expanded at an astonishing pace.

From the first transistor developed at Bell Labs in 1947 to modern processors containing billions and even trillions of transistors, each generation of technology has been faster, smaller and more powerful than the last.

But according to quantum physicist and technology entrepreneur David Reilly, that era of effortless progress is beginning to slow.

Reilly, CEO of Sydney-based Emergence Quantum and Professor of Physics at the University of Sydney, says the computing infrastructure underpinning modern economies is approaching fundamental physical limits.

And that could have enormous implications for finance, artificial intelligence and global investment.

Speaking at an industry event organised by Kanebridge International, Reilly said many critical parts of modern society depend on computing and the infrastructure used to process information.

The slowdown behind the tech boom

For years, the technology industry relied on a steady improvement known as Moore’s Law, where the number of transistors on a chip doubled roughly every two years.

More transistors meant more computing power, allowing faster software, smarter devices and ever-larger data systems.

Today, however, those gains are slowing.

“It feels to me very innate that I’m going to just find that next year there’s going to be another breakthrough,” Reilly said.

“But if you look at the data…there’s a slowing down, a roll off in performance that started some 10, 20 years ago.”

Rather than making chips dramatically faster, manufacturers are now largely increasing computing capacity by packing more transistors onto each processor.

The approach works, but it comes with growing complexity, higher costs and increasing energy demands.

The brute-force race for AI

That challenge is already visible in the massive data centres being built to support artificial intelligence.

In the race to dominate AI, companies are constructing vast computing facilities that consume huge amounts of electricity and water. Reilly described this expansion as a “brute force” approach driven by the global competition to develop advanced AI systems.

Yet the demand for computing power continues to accelerate.

Artificial intelligence, advanced robotics, healthcare research, pharmaceuticals and cybersecurity all require far more processing capacity than today’s systems can easily deliver.

The question now facing the technology sector is whether traditional computing can keep up.

Enter quantum computing

That is where quantum computing enters the conversation.

Unlike conventional computers, which process information using binary switches that represent ones and zeros, quantum computers exploit the unusual behaviour of particles at the atomic scale.

Reilly describes them as a fundamentally different type of machine.

“So a quantum computer is a wave computer,” he said.

Instead of processing information through simple on-off switches, quantum systems can use wave-like properties of particles to process many possible outcomes simultaneously.

Those waves can interact in complex ways, reinforcing correct solutions while cancelling out incorrect ones. In theory, this allows quantum systems to tackle certain types of problems dramatically faster than classical computers.

What it could mean for finance

The concept may sound abstract, but its potential applications are significant.

Quantum computers are expected to transform areas such as materials science, chemical modelling and pharmaceutical development.

They could also help solve complex optimisation problems in logistics, finance and risk management.

For financial institutions in particular, the technology could offer new tools for detecting fraud, analysing market behaviour and optimising portfolios.

But the shift will not happen overnight.

“One message to take away is that quantum is not going to suddenly solve all of your problems,” Reilly said.

Instead, he said quantum systems will likely complement existing computing technologies as part of a broader and more diverse computing ecosystem.

Why data centres may soon “go cold”

One key change already emerging is how computing systems are physically designed.

Many next-generation technologies, including quantum processors, operate far more efficiently at extremely low temperatures. As a result, future data centres may rely heavily on cryogenic cooling systems to manage heat and energy consumption.

Reilly believes that the shift will gradually reshape the computing industry.

“Over the next five years, you’re going to see data centres go cold,” he said.

“And as that happens, they almost drag with them new compute paradigms.”

Emergence Quantum, the company he co-founded, is focused on developing technologies to support that transition, including cryogenic electronics and integrated hardware platforms designed for quantum computing and energy-efficient systems.

A new technological era

For investors and businesses, the technology remains in its early stages. But the scale of global interest is growing rapidly.

Governments, research institutions and technology companies are investing heavily in quantum research, betting it could become a foundational technology for the next generation of computing.

For Reilly, the moment feels similar to earlier technological turning points.

In the 19th century, new discoveries in thermodynamics helped drive the development of steam engines and the Industrial Revolution. In the 20th century, advances in electromagnetism led to radio, television and eventually the internet.

Quantum physics, he suggests, could represent the next chapter in that story.

“Today we have, as a society, in our hands new physics that we’re just beginning to figure out what to do with,” Reilly said.

“But I think it’s an exciting time to be alive and watch what happens over the coming decades.”