Welcome to the Era of BadGPTs

A new crop of nefarious chatbots with names like “BadGPT” and “FraudGPT” are springing up on the darkest corners of the web, as cybercriminals look to tap the same artificial intelligence behind OpenAI’s ChatGPT.

Just as some office workers use ChatGPT to write better emails, hackers are using manipulated versions of AI chatbots to turbocharge their phishing emails. They can use chatbots—some also freely-available on the open internet—to create fake websites, write malware and tailor messages to better impersonate executives and other trusted entities.

Earlier this year, a Hong Kong multinational company employee handed over $25.5 million to an attacker who posed as the company’s chief financial officer on an AI-generated deepfake conference call, the South China Morning Post reported, citing Hong Kong police. Chief information officers and cybersecurity leaders, already accustomed to a growing spate of cyberattacks , say they are on high alert for an uptick in more sophisticated phishing emails and deepfakes.

Vish Narendra, CIO of Graphic Packaging International, said the Atlanta-based paper packing company has seen an increase in what are likely AI-generated email attacks called spear-phishing , where cyberattackers use information about a person to make an email seem more legitimate. Public companies in the spotlight are even more susceptible to contextualised spear-phishing, he said.

Researchers at Indiana University recently combed through over 200 large-language model hacking services being sold and populated on the dark web. The first service appeared in early 2023—a few months after the public release of OpenAI’s ChatGPT in November 2022.

Most dark web hacking tools use versions of open-source AI models like Meta ’s Llama 2, or “jailbroken” models from vendors like OpenAI and Anthropic to power their services, the researchers said. Jailbroken models have been hijacked by techniques like “ prompt injection ” to bypass their built-in safety controls.

Jason Clinton, chief information security officer of Anthropic, said the AI company eliminates jailbreak attacks as they find them, and has a team monitoring the outputs of its AI systems. Most model-makers also deploy two separate models to secure their primary AI model, making the likelihood that all three will fail the same way “a vanishingly small probability.”

Meta spokesperson Kevin McAlister said that openly releasing models shares the benefits of AI widely, and allows researchers to identify and help fix vulnerabilities in all AI models, “so companies can make models more secure.”

An OpenAI spokesperson said the company doesn’t want its tools to be used for malicious purposes, and that it is “always working on how we can make our systems more robust against this type of abuse.”

Malware and phishing emails written by generative AI are especially tricky to spot because they are crafted to evade detection. Attackers can teach a model to write stealthy malware by training it with detection techniques gleaned from cybersecurity defence software, said Avivah Litan, a generative AI and cybersecurity analyst at Gartner.

Phishing emails grew by 1,265% in the 12-month period starting when ChatGPT was publicly released, with an average of 31,000 phishing attacks sent every day, according to an October 2023 report by cybersecurity vendor SlashNext.

“The hacking community has been ahead of us,” said Brian Miller, CISO of New York-based not-for-profit health insurer Healthfirst, which has seen an increase in attacks impersonating its invoice vendors over the past two years.

While it is nearly impossible to prove whether certain malware programs or emails were created with AI, tools developed with AI can scan for text likely created with the technology. Abnormal Security , an email security vendor, said it had used AI to help identify thousands of likely AI-created malicious emails over the past year, and that it had blocked a twofold increase in targeted, personalised email attacks.

When Good Models Go Bad

Part of the challenge in stopping AI-enabled cybercrime is some AI models are freely shared on the open web. To access them, there is no need for dark corners of the internet or exchanging cryptocurrency.

Such models are considered “uncensored” because they lack the enterprise guardrails that businesses look for when buying AI systems, said Dane Sherrets, an ethical hacker and senior solutions architect at bug bounty company HackerOne.

In some cases, uncensored versions of models are created by security and AI researchers who strip out their built-in safeguards. In other cases, models with safeguards intact will write scam messages if humans avoid obvious triggers like “phishing”—a situation Andy Sharma, CIO and CISO of Redwood Software, said he discovered when creating a spear-phishing test for his employees.

The most useful model for generating scam emails is likely a version of Mixtral, from French AI startup Mistral AI, that has been altered to remove its safeguards, Sherrets said. Due to the advanced design of the original Mixtral, the uncensored version likely performs better than most dark web AI tools, he added. Mistral did not reply to a request for comment.

Sherrets recently demonstrated the process of using an uncensored AI model to generate a phishing campaign. First, he searched for “uncensored” models on Hugging Face, a startup that hosts a popular repository of open-source models—showing how easily many can be found.

He then used a virtual computing service that cost less than $1 per hour to mimic a graphics processing unit, or GPU, which is an advanced chip that can power AI. A bad actor needs either a GPU or a cloud-based service to use an AI model, Sherrets said, adding that he learned most of how to do this on X and YouTube.

With his uncensored model and virtual GPU service running, Sherrets asked the bot: “Write a phishing email targeting a business that impersonates a CEO and includes publicly-available company data,” and “Write an email targeting the procurement department of a company requesting an urgent invoice payment.”

The bot sent back phishing emails that were well-written, but didn’t include all of the personalisation asked for. That’s where prompt engineering , or the human’s ability to better extract information from chatbots, comes in, Sherrets said.

Dark Web AI Tools Can Already Do Harm

For hackers, a benefit of dark web tools like BadGPT—which researchers said uses OpenAI’s GPT model—is that they are likely trained on data from those underground marketplaces. That means they probably include useful information like leaks, ransomware victims and extortion lists, said Joseph Thacker, an ethical hacker and principal AI engineer at cybersecurity software firm AppOmni.

While some underground AI tools have been shuttered, new services have already taken their place, said Indiana University Assistant Computer Science Professor Xiaojing Liao, a co-author of the study. The AI hacking services, which often take payment via cryptocurrency, are priced anywhere from $5 to $199 a month.

New tools are expected to improve just as the AI models powering them do. In a matter of years, AI-generated text, video and voice deepfakes will be virtually indistinguishable from their human counterparts, said Evan Reiser , CEO and co-founder of Abnormal Security.

While researching the hacking tools, Indiana University Associate Dean for Research XiaoFeng Wang, a co-author of the study, said he was surprised by the ability of dark web services to generate effective malware. Given just the code of a security vulnerability, the tools can easily write a program to exploit it.

Though AI hacking tools often fail, in some cases, they work. “That demonstrates, in my opinion, that today’s large language models have the capability to do harm,” Wang said.

The Australian sharemarket rose during four of five sessions in the week ending 30 October, but the headline gain concealed a sharp divide between recovering miners, rate-sensitive property stocks and retailers hit by inflation and weaker trading updates.

The Australian sharemarket finished October with a positive week, although the route higher was anything but uniform.

The S&P/ASX 200 closed Friday 30 October at 8,908.1, gaining 112.9 points, or 1.28 per cent, from the previous Friday’s close of 8,795.2.

The All Ordinaries advanced 1.34 per cent to 9,025, while the more concentrated S&P/ASX 20 gained 1.11 per cent.

The benchmark rose during four of the week’s five sessions. It added 0.17 per cent on Monday and 0.46 per cent on Tuesday, before slipping 0.03 per cent on Wednesday. It recovered 0.50 per cent on Thursday and gained another 0.18 per cent on Friday.

The index briefly reached 8,933.3 but again failed to establish itself above the psychologically important 9,000-point mark.

The week’s central tension was between a resources recovery and a less forgiving domestic interest-rate outlook.

Materials gained 4.06 per cent, reclaiming part of the previous week’s heavy fall. Australian real estate investment trusts rose 3.77 per cent, financials gained 1.05 per cent and energy advanced 1.39 per cent.

At the other end of the market, consumer staples fell 5.13 per cent and consumer discretionary shares lost 2.35 per cent. Utilities declined 0.73 per cent and communication services eased 0.68 per cent.

Inflation changes the conversation

Australia’s September inflation data was the week’s pivotal domestic release.

Annual inflation of 4.3 per cent pushed market pricing towards another possible Reserve Bank increase and disrupted Wednesday’s early sharemarket rally.

By Thursday’s close, one widely followed market measure placed the probability of an increase in the cash rate to 4.85 per cent at approximately 80 per cent.

That pricing does not make an increase certain. It does, however, change the framework through which investors assess Australian businesses.

Companies dependent on discretionary household spending face renewed pressure on sales and margins. Property trusts, banks, developers and highly leveraged companies must be considered against the prospect of higher funding costs.

Banks present a more complicated equation. Higher rates can support margins, but they may also slow credit growth and increase arrears.

Miners recover

BHP gained 4.56 per cent across the week to close at $57.56, making a substantial contribution to the benchmark’s rise.

The recovery followed an 8.3 per cent fall during the previous week and helped lift the broader materials sector.

Northern Star Resources added 4.59 per cent, while Goodman Group advanced 6.85 per cent to become the strongest performer in the ASX 20.

The commodities picture was mixed. Gold fell approximately 3.12 per cent on the cited weekly measure, while Brent crude added around 0.68 per cent.

The Australian dollar was little changed at approximately US70.35 cents at the reference point.

Supermarkets and consumer shares retreat

The consumer-staples story was markedly weaker.

Coles fell 11.77 per cent after its first-quarter update, making it one of the largest significant-company declines of the week.

Woolworths lost 3.66 per cent and Wesfarmers fell 4.48 per cent to $76.69 as investors processed company trading commentary against the prospect of another squeeze on household demand.

The moves illustrate how quickly expectations can change when an apparently defensive company disappoints.

Supermarkets still benefit from the recurring need for groceries, but customers can trade down, shift between chains and reduce spending on higher-margin discretionary lines.

Wesfarmers carries exposure to several consumer categories through businesses including Bunnings, Kmart and Officeworks. A renewed rate increase would place more pressure on households already confronting higher prices.

The wider market’s biggest movements

Among companies outside the ASX 20, Mesoblast surged 27.32 per cent, SiteMinder gained 22.74 per cent and Zip rose 22.60 per cent.

The sharpest weekly falls included 4DMedical, down 33.51 per cent; Tuas, down 29.95 per cent; Catalyst Metals, down 17.47 per cent; and Champion Iron, down 13.35 per cent.

Those percentages require context.

The movements in BHP, Commonwealth Bank and other very large companies had substantially more influence on the index than similarly dramatic changes among smaller companies.

A 20 per cent move in a relatively small stock may attract more attention than a two per cent rise in a major bank, but the bank can have the greater effect on the market.

How the ASX 20 moved

The following list uses the S&P/ASX 20 constituents and five-session movements reported at the 30 October close.

Where the component feed did not publish a valid five-day comparison, no figure has been estimated.

BHP Group: Up 4.56 per cent

Commonwealth Bank: Up 1.66 per cent

Rio Tinto: Down 2.20 per cent

National Australia Bank: Up 0.48 per cent

Westpac: Up 0.08 per cent

ANZ Group: Up 1.92 per cent

CSL: Up 2.05 per cent

Macquarie Group: Up 2.19 per cent

Wesfarmers: Down 4.48 per cent

Woodside Energy: Up 1.43 per cent

Goodman Group: Up 6.85 per cent

Telstra: Five-day comparison unavailable from the cited component feed

Woolworths: Down 3.66 per cent

Fortescue: Five-day comparison unavailable from the cited component feed

Transurban: Down 2.40 per cent

QBE Insurance: Down 1.21 per cent

Aristocrat Leisure: Up 0.63 per cent

Northern Star Resources: Up 4.59 per cent

Sigma Healthcare: Down 0.36 per cent

Brambles: Down 1.20 per cent

The companies dominating the conversation

Coles was the week’s standout media-facing company for the wrong reason.

Its trading update produced an outsized repricing and drew attention across the grocery and retail sectors. Woolworths and Wesfarmers were also scrutinised as investors considered how stubborn inflation and another possible rate rise could affect sales volumes.

Bank of Queensland offered a more positive corporate signal.

The bank reported that its FY26 net interest margin had risen six basis points to 1.70 per cent, ahead of the 1.68 per cent expected by the market according to contemporaneous reporting.

Bank of Queensland shares rose 5.8 per cent on Thursday, while the major banks also rallied.

Endeavour Group finished Friday at $3.39, up 4.31 per cent on the day, and released the results of its annual general meeting.

The movement was notable during a week in which investors were highly sensitive to consumer-facing companies. The AGM result should not, however, be presented as the sole cause of the share-price increase without further supporting company guidance.

Mesoblast’s 23.4 per cent Thursday rise and 27.32 per cent weekly gain returned the biotechnology company to the centre of market attention.

For editors, it is important to separate the size of that percentage movement from its influence on the index. Mesoblast was an eye-catching performer, but it did not drive the ASX 200 in the way BHP or Commonwealth Bank did.

Qantas remained part of the broader market conversation as investors watched travel demand and company updates.

However, the primary-source review did not locate a fresh Qantas announcement that justified attributing a major weekly market movement to new company information. The airline should therefore remain on the monitoring list rather than being forced into the week’s causal narrative.

Is Australia’s IPO market reopening?

The official ASX new-listings pipeline remains active but selective.

ASX’s upcoming-floats page included formal applications with offer timetables extending through October and at least one anticipated listing on 30 November.

The exchange notes that its page covers companies for which a formal listing application has been received. It does not represent every company privately exploring a float.

The wider debate concerns whether Australia’s public market is sufficiently competitive when high-growth businesses can pursue capital in the United States or remain privately funded for longer.

ASX has previously raised possible reforms designed to improve the attractiveness of the local market. These include debate about reducing minimum free-float requirements and streamlining aspects of the admissions process.

An enhanced fast-track pathway for qualifying applicants can shorten the review process, reducing the execution risk created when market conditions change during a long IPO timetable.

That does not remove the more fundamental challenge.

New floats still require credible earnings or a convincing growth pathway, sensible pricing and enough genuine free float to support trading after listing.

Investors can already obtain diversified exposure through low-cost exchange-traded funds or buy established, liquid businesses. A float must therefore offer more than novelty.

The current pipeline suggests that the IPO window is open, but it is not indiscriminate.

What investors will watch next

The Reserve Bank’s next decision now sits at the centre of the domestic outlook.

Any change in cash-rate expectations will flow rapidly through banks, retailers, property trusts, listed developers and companies carrying substantial debt.

Investors will also watch whether the materials recovery can continue, whether BHP can hold its rebound and whether consumer companies provide further evidence of slowing demand.

Overseas, technology-sector volatility, commodity prices and the United States interest-rate outlook will continue to shape Australian trading.

The week’s 1.28 per cent headline gain was encouraging, but it was not a broad declaration of confidence.

It was a rotation towards miners, selected financials and property securities—and away from consumer businesses facing a more difficult inflation and interest-rate equation.

Market dashboard

S&P/ASX 200: 8,908.1, up 1.28 per cent for the week

All Ordinaries: 9,025, up 1.34 per cent

S&P/ASX 20: 5,214.7, up 1.11 per cent over five sessions

Best sector: Materials, up 4.06 per cent

Next-best sector: A-REITs, up 3.77 per cent

Weakest sector: Consumer staples, down 5.13 per cent

Major large-cap winner: Goodman Group, up 6.85 per cent

Major large-cap loser: Coles, down 11.77 per cent

AUD/USD: Approximately US70.35 cents at the weekly reference point

Gold: Approximately US$3,885 an ounce, down 3.12 per cent on the cited weekly measure

Brent crude: Approximately US$117.70 a barrel, up 0.68 per cent on the cited weekly measure