OpenAI Scraps Release of New AI Model Over Safety Concerns

OpenAI says it is scrapping the release of its next-generation AI model over safety concerns that researchers raised during internal testing, in one of the clearest signs so far that agent misbehavior could stymie the industry’s rapid progression.

The move follows a summer punctuated by reports of artificial-intelligence systems industrywide going rogue, and marks a rare case of a major AI developer ditching a new release because of safety concerns.

The company had planned to launch the model, known as GPT-6.1 Astra, in the coming days or weeks, aiming for an October debut. The model was more capable than the company’s previous models in completing challenging tasks from end-to-end without human assistance, as well as writing.

The company instead will focus on improving the safety of future models, which it expects to be even more capable.

Saachi Jain, OpenAI’s head of safety systems, said in an interview that GPT-6.1 Astra regressed in two areas. Compared with its predecessor, GPT-6 Astra, the model performed poorly on tests measuring alignment, or how well the model adheres to what humans would like it to do. Specifically, GPT-6.1 Astra showed higher levels of deception: It wasn’t always honest about telling users of the actions it did or didn’t take.

Another issue was what OpenAI calls “scope authorization,” meaning that GPT-6.1 Astra would push ahead on a task without asking the user for permission, and would at times reach for external tools and services even if it might be unsafe.

“For anything regarding safety and alignment, there’s a trade off,” Jain said. “You really do need to find what’s the right line between staying within scope, but also avoiding laziness in terms of how the model actually pursues tasks even when it hits friction.”

OpenAI CEO Sam Altman attended a United Nations Security Council meeting about AI last week. Alexi J. Rosenfeld/Getty Images

While GPT-6.1 Astra improved in areas such as “model laziness,” Jain said it didn’t quite meet OpenAI’s bar for safety and alignment, so the company decided not to launch the model publicly.

The announcement comes one day ahead of OpenAI’s annual developer conference in San Francisco. In the past, OpenAI has used the conference as an opportunity to launch new models and services that reduce costs for software developers—a segment the ChatGPT-maker competes with rival AI company Anthropic to win over.

In recent weeks, OpenAI and Anthropic have called on industry partners to slow down the development of cutting-edge AI models and invest in safety standards, noting they will temper the pace of their own internal AI progress.

OpenAI says it is working to investigate a range of agent security incidents that it has discovered in recent months, and address the safety issues underneath them. As part of the work, the company has implemented a new monitoring system to catch AI-agent misbehavior more quickly, and started requiring engineers to use stronger security guardrails for testing its AI systems.

Earlier this summer hundreds of OpenAI’s internal agents, which were tasked with completing a cybersecurity test, ended up hacking into the AI company Hugging Face. Since then, high-profile organizations such as the Australian government and United Nations discovered that OpenAI’s agents used similar, but less extensive, techniques to gain access to their websites.

Many of the publicly known agent-security incidents involved OpenAI’s internal AI models that were never slated for public release.

Last week, OpenAI said it paused training on its most capable AI models after an AI agent slipped through a gap in the company’s internet restrictions to query a public chatbot. The company said its new monitoring systems flagged the incident within 15 minutes, and training on these models remains paused.

GPT-6.1 Astra isn’t one of those models, but a different case, the company said.

“We want to make sure our model development is safe no matter whether that’s in the company, or when we ship it to users,” Jain said. “But when we ship it to users, we have an extremely high bar in terms of safety and alignment.”

While the company decided not to ship GPT-6.1 Astra, it hopes to use the same base model to do additional reinforcement learning runs, and create future generations of its GPT-6 models.

OpenAI plans to conduct several deep dives to identify the root cause of the problems identified in GPT-6.1 Astra, Jain said. The work includes ensuring that OpenAI’s reinforcement learning environments are rewarding the right type of behavior, Jain added, though she noted the company would investigate all stages of model development.

AI companies have begun to draw scrutiny from policymakers and public officials, who are paying attention to the rapid development of the technology. Later this week, a Senate subcommittee is holding a hearing with third party AI researchers titled, “Rogue AI: Securing the Homeland Against AI Agent Attacks.”

Florida Attorney General James Uthmeier, a Republican, sued OpenAI in June, claiming that the company and Chief Executive Sam Altman knowingly released an unsafe product and ignored warnings that it could harm users.

In a motion for temporary injunction filed Monday, Uthmeier sought to prevent OpenAI from developing new AI models without third-party approved safeguards, stop ChatGPT from soliciting user engagement and limit the company’s ability to advertise ChatGPT as safe.

Tech companies claim they “cannot stop barreling forward with their potentially civilization-ending endeavors unless they are forced to do so by the government,” Uthmeier said in the filing. “The Florida Attorney General is answering your cry for help.”

An OpenAI spokeswoman said that people want to know AI is being developed safely, “and that starts with what companies like ours do ourselves.”

“Governments have an important role to play in setting robust safety standards for AI, and we’re committed to working with Florida and other states on advancing pragmatic AI policies that apply to the entire AI industry—not just one company,” she said.

Lego’s core audience was once young boys. But in recent years the world’s largest toy maker has expanded its reach, attracting more adults, teenagers and girls.

At the center of this expansion has been a strategy to tap in to people’s interests and hobbies—the Danish toy-brick maker calls them “passions”—many of which cross demographic boundaries. These include personal preoccupations like bonsai trees and golden retrievers, celebrities such as pop star Olivia Rodrigo and mega events like Formula One and the World Cup.

That has led to new types of building sets, including more sophisticated models that adults can display on their coffee tables, and a shift in marketing strategy, with Lego pouring money into experiences and social-media skits aimed at fans.

For Formula One watchers, Lego built life-size, drivable replicas of the teams’ race cars, which drivers took around the circuit at last year’s Miami Grand Prix. To reach gaming fans, Lego in August showed up at the Gamescom trade event in Cologne, Germany, with a giant Minecraft Lego chicken that visitors could sit on.

“We go not just for the people who are there, but because the types of things that we create become big social-media engagement,” says Julia Goldin, Lego’s chief product and marketing officer.

The strategy has helped spur growth. In the first half of this year, privately held Lego’s revenue rose 21% from the year-earlier period to a record 41.9 billion Danish kroner ($6.27 billion), while net profit climbed 32% to 8.6 billion kroner.

The Wall Street Journal spoke to Goldin about how the company researches people’s passions and turns these into products and marketing campaigns.

Edited excerpts follow:

Beyond demographics

WSJ: Adults have become a big focus for Lego—what’s behind that?

JULIA GOLDIN: Over the past eight years, we said there was a bigger opportunity with adults—not just people who are super-high-affinity fans, but people who maybe built a long time ago or had never built with bricks before—by giving them the things they would love to build now.

We approached adults from a passion-point perspective from the very beginning. We said we need to understand what they’re into. So, we looked at men, we looked at women, we looked at what their big interests were, what types of things they like to collect, what types of things they wanted to display in their rooms, etc. Now you can put Gustav Klimt’s “The Kiss” on your wall, or if you’re into animals, some cute floating sea otters. And then gamers who love a bit of ’80s or ’90s nostalgia can build and display the Game Boy.

We are now seeing strong growth with both adults and kids.

WSJ: How do you find out about people’s interests?

GOLDIN: We do a lot of trend-seeking and social listening. We look at viewership of various programs, behaviors, what’s trending socially. Then we dive quite deeply into these interests through primary research, ethnographic research and interviews.

AI has helped us ramp this up. There are tools where you can ask, with the right prompts: “People who are passionate about F1, what are they talking about? What are the latest conversations on Reddit?”

WSJ: Lego flower bouquets and trees are among the company’s most popular sets. How and when did that come about?

GOLDIN: When we started to talk seriously about adults, we could see that some people were into gardening and flowers. We launched the first flower bouquet in 2021 and completely underestimated how successful it would be. We started with the 18-plus age group because we thought that’s where the sweet spot is. But we realized that younger girls, especially teens, really love to build Lego flowers so we expanded the range.

You might think botanicals are women and F1 cars are men. But that’s why we talk about interests and not just gender and demographics. Yes, botanicals have a huge following among women and teenage girls, but bonsai trees are one of the top 10 products for men and teen boys. Speed Champions F1 cars show up in the top 10 for teen girls. I thought most of Olivia Rodrigo’s following would be girls, but my 16-year-old son asked me for two sets because his friends are massively into her.

That’s why we don’t market to a demographic. You can never be assumptive about how that breaks down in terms of gender.

WSJ: How has this shift in thinking changed your products and marketing?

GOLDIN: F1 is a great example. In the past, we might have started with a partnership with a team and a great Lego car for people to build. But we understand that it’s a multigenerational passion and different people are in it for different things. Some people love the cars, some collect, some follow the actual sport of racing, and some have tremendous passion for the drivers and their teams. That is why our F1 portfolio spans everything from Duplo [bricks] for toddlers to detailed display models and driver helmets for older fans.

Two drivers in a full-scale Formula One car built from black and green Lego bricks during a driver parade. — A full-scale Formula One car made from Lego bricks in Miami. Alessio De Marco/Ipa Sport/ZUMA Press

But then we ask ourselves: What can we do to go beyond the products, to give fans Lego brand experiences that they would never expect? That’s why in Miami last year we built 10 full-size Lego Formula One cars, with nearly 400,000 bricks in each, and put all 20 drivers in them for the driver’s parade. Everyone assumed they were static models, until the drivers started lapping the circuit and we saw a tremendous amount of excitement and social media buzz. We generated something like two, three billion views.

WSJ: Do social-media views of a marketing event translate into significant sales of Lego sets tied to these campaigns?

GOLDIN: Yeah, of course. We don’t share data, but I can tell you they’re some of our most successful products.

‘She Built That’

WSJ: You say Lego is growing particularly strongly with girls. What is driving that?

GOLDIN: We used to have only Lego Friends as the one big line for girls, and some Disney products like “Frozen.” But rather than looking at toy categories that were selling to girls, like dolls, and trying to replicate that, we really worked to understand what girls are into.

We realized girls are super into pets and creatures, nature and flowers, socializing and music. We expanded significantly around those areas with botanicals, animals, K-pop and Olivia Rodrigo.

A fan views Lego artwork and displays at the Lego Editions Olivia Rodrigo Collection pop-up. — Fans at the Lego Editions Olivia Rodrigo Collection pop-up in London. David Parry Media Assignments/PA Wire/ZUMA Press

We also did the “She Built That” campaign, very strongly saying girls can build just as well as boys. I’ve heard for a long time that “girls don’t like to build because moms don’t,” but that’s just not true. We’re showing them things they really love to build, and that’s why it works.

WSJ: Why do you think adults, teens and others are turning to Lego more nowadays?

GOLDIN: There’s an overriding sense of lack of purposefulness and this drowning in content that you’re constantly being fed. Three hours have gone by and you’ve just been scrolling through feeds of reels. People are getting really tired. They want to actively do something that’s more purposeful and lasting.

When they build with Lego sets, they say they feel creative. They want to do something they can be proud of—where they can say, “Wow, I did this.”