Sam Altman, the maker of ChatGPT, says the A.I. future is both awesome and terrifying. If it goes badly: ‘It’s lights-out for all of us’

Dystopian fears are never far when the conversation turns to advanced artificial intelligence. And even the CEO of the hottest A.I. company around right now says the tech’s future is a toss-up.

A.I. may have finally gone mainstream last November, when the San Francisco-based startup OpenAI followed up on its earlier success with text-to-image generator DALL-E 2 by launching its latest creation for the public to test out: ChatGPT, arguably the most advanced chatbot in existence. Trained on troves of data, the bot garnered a million users in a matter of days.

The possibilities of ChatGPT are virtually endless. It’s already posing a potential threat to Google’s search engine hegemony, and the technology could revolutionize code-writing and meal planning as well as hacking and cheating on school tests. Not only does its potential have Google on its toes, but rival Microsoft has already swooped in with what’s reportedly a $10 billion investment in OpenAI. That capital infusion comes on top of $3 billion that Microsoft has already invested since 2019. The Microsoft money, combined with other purchases of existing shares by venture capital firms and other investors, has reportedly brought the company’s valuation to just under $29 billion.

OpenAI and its CEO and co-founder Sam Altman seem on top of the world right now, so much so that the 37-year-old will grace the cover of the next issue of Fortune magazine. He’s also the focus of a feature by Jeremy Kahn, with reporting by Michal Lev-Ram and Jessica Mathews, that looks back on OpenAI’s achievements and what is to come.

But there will be no shortage of challenges for Altman in the years ahead as he attempts to make artificial general intelligence—top-level A.I. with shades of science fiction that some experts interviewed by Kahn say is unattainable—a reality. Even Altman himself admits that for all the promise of A.I.’s future, there is just as much uncertainty and risk.

“I think the good case [for A.I.] is just so unbelievably good that you sound like a crazy person talking about it,” Kahn reported Altman saying during a VC event in San Francisco on Jan. 12.

“I think the worst case is lights-out for all of us,” he added.

OpenAI shift

OpenAI has gone through big changes since a group of early backers including Altman, Elon Musk, and Peter Thiel launched the company in 2015.

The company started as a research lab in response to Google’s $500 million purchase in 2014 of London-based A.I. startup DeepMind. DeepMind has made its own massive strides in machine learning, including predicting and cataloging the structure of almost every known protein in the human body two years ago.

But OpenAI started with a different mission. “The new lab aimed to be everything DeepMind and Google were not,” Kahn writes. “It would be run as a nonprofit, explicitly dedicated to democratizing the benefits from advanced A.I.”

OpenAI and Microsoft intend to continue “democratizing” A.I. by making its tools widely available in the new phase of their relationship, Microsoft CEO Satya Nadella said in a statement announcing the new investment on Monday. “We formed our partnership with OpenAI around a shared ambition to responsibly advance cutting-edge A.I. research and democratize A.I. as a new technology platform,” he said.

Microsoft’s deal with OpenAI, the details of which were seen in documents earlier this month by Mathews and Kahn, is complicated. In return for the huge investment, Altman and his team will effectively have to hand over control of OpenAI to Microsoft until the A.I. company makes enough money to pay back the tech giant’s capital, and then some.

“In essence, OpenAI is lending the company to Microsoft—for how long depends on how quickly OpenAI can make money,” Kahn wrote.

OpenAI has more projects in the pipeline, including a text-to-video system, according to Kahn. But if ChatGPT is the entry point to artificial intelligence, there are still some important snags, including evidence of racial bias and mixing up information when it comes to niche topics. Several former employees—many of whom worked in A.I. safety— left the company in 2021 as a result of “cultural and strategic shifts” at OpenAI, Kahn reports.

With OpenAI falling under Microsoft’s wing for the foreseeable future, the startup may behave more like a profit-seeking company too. In December, Altman wrote on Twitter that he will “have to monetize it somehow at some point.” This week, OpenAI launched a new subscription plan for ChatGPT that allows users to interact with the chatbot at faster speeds and during times of high demand. That in turn will help the technology evolve faster–and as Altman notes, that could be a good thing, a bad thing, or both.

This story was originally featured on Fortune.com

More from Fortune:
Olympic legend Usain Bolt lost $12 million in savings to a scam. Only $12,000 remains in his account
Meghan Markle’s real sin that the British public can’t forgive–and Americans can’t understand
‘It just doesn’t work.’ The world’s best restaurant is shutting down as its owner calls the modern fine dining model ‘unsustainable’
Bob Iger just put his foot down and told Disney employees to come back into the office