13: India

This Changes Everything . “A.I. is probably the most important thing humanity has ever worked on. I think of it as something more profound than electricity or fire.” ....... What is hardest to appreciate in A.I. is the improvement curve. ....... I find myself thinking back to the early days of Covid. There were weeks when it was clear that lockdowns were coming, that the world was tilting into crisis, and yet normalcy reigned, and you sounded like a loon telling your family to stock up on toilet paper. ....... There is a natural pace to human deliberation. A lot breaks when we are denied the luxury of time. ......... the people working on A.I. ...... a community that is living with an altered sense of time and consequence. They are creating a power that they do not understand at a pace they often cannot believe. ......... Would you work on a technology you thought had a 10 percent chance of wiping out humanity? ...... They believe they might summon demons. They are calling anyway. ........ This was true among cryptocurrency enthusiasts in recent years. The claims they made about how blockchains would revolutionize everything from money to governance to trust to dating never made much sense. But they were believed most fervently by those closest to the code. ......... Crypto was always a story about an unlikely future searching for traction in the present. With A.I., to imagine the future you need only look closely at the present. ........ In 2021, a system built by DeepMind managed to predict the 3-D structure of tens of thousands of proteins, an advance so remarkable that the editors of the journal Science named it their breakthrough of the year. ....... “Within two months of downloading Replika, Denise Valenciano, a 30-year-old woman in San Diego, left her boyfriend and is now ‘happily retired from human relationships’” ........ Could it help terrorists or antagonistic states develop lethal weapons and crippling cyber attacks? ........ These systems will already offer guidance on building biological weapons if you ask them cleverly enough. ........ A.I. is already being used for predictive policing and judicial sentencing. ........ The “thinking,” for lack of a better word, is utterly inhuman, but we have trained it to present as deeply human. And the more inhuman the systems get — the more billions of connections they draw and layers and parameters and nodes and computing power they acquire — the more human they seem to us. .......... “as A.I. continues to blow past us in benchmark after benchmark of higher cognition, we quell our anxiety by insisting that what distinguishes true consciousness is emotions, perception, the ability to experience and feel: the qualities, in other words, that we share with animals.” ......... The major tech companies are in a race for A.I. dominance. The U.S. and China are in a race for A.I. dominance. Money is gushing toward companies with A.I. expertise. ....... Slowing down “would involve coordinating numerous people .

The Return of the Magicians people talk increasingly about the limits of the scientific endeavor — the increasing impediments to discovering new ideas, the absence of low-hanging scientific fruit, the near impossibility, given the laws of physics as we understand them, of ever spreading human civilization beyond our lonely planet or beyond our isolated solar system. ....... — namely, beings that can enlighten us, elevate us, serve us and usher in the Age of Aquarius, the Singularity or both. ........... a golem, more the embodied spirit of all the words on the internet than a coherent self with independent goals. .......... With the emergent forms of A.I., they argue, we have created an intelligence that can yield answers the way an oracle might or a Magic 8 Ball: through processes that are invisible to us, permanently beyond our understanding, so complex as to be indistinguishable from action in a supernatural mind. ...... the A.I. revolution represents a fundamental break with Enlightenment science, which “was trusted because each step of replicable experimental processes was also tested, hence trusted.” .......... the spirit might be disobedient, destructive, a rampaging Skynet bent on our extermination. ....... we would be wise to fear apparent obedience as well. .

Should GPT exist? Gary Marcus asks about Microsoft, “what did they know, and when did they know it?”—a question I tend to associate more with deadly chemical spills or high-level political corruption than with a cheeky, back-talking chatbot. ........ in reality it’s merely a “stochastic parrot,” a glorified autocomplete that still makes laughable commonsense errors and that lacks any model of reality outside streams of text. ....... If you need months to think things over, generative AI probably isn’t for you right now. I’ll be relieved to get back to the slow-paced, humdrum world of quantum computing. ....... if OpenAI couldn’t even prevent ChatGPT from entering an “evil mode” when asked, despite all its efforts at Reinforcement Learning with Human Feedback, then what hope do we have for GPT-6 or GPT-7? ....... Even if they don’t destroy the world on their own initiative, won’t they cheerfully help some awful person build a biological warfare agent or start a nuclear war? ......... a classic example being nuclear weapons. But, like, nuclear weapons kill millions of people. They could’ve had many civilian applications—powering turbines and spacecraft, deflecting asteroids, redirecting the flow of rivers—but they’ve never been used for any of that, mostly because our civilization made an explicit decision in the 1960s, for example via the test ban treaty, not to normalize their use. ........

GPT is not exactly a nuclear weapon. A hundred million people have signed up to use ChatGPT, in the fastest product launch in the history of the Internet. ... the ChatGPT death toll stands at zero

....... The science that we could learn from a GPT-7 or GPT-8, if it continued along the capability curve we’ve come to expect from GPT-1, -2, and -3. Holy mackerel. ....... I was a pessimist about climate change, ocean acidification, deforestation, drought, war, and the survival of liberal democracy. The central event in my mental life is and always will be the Holocaust. I see encroaching darkness everywhere. .......... it’s amazing at poetry, better than most of us. .

The False Promise of Chomskyism . .
Why am I not terrified of AI? “I’m scared about AI destroying the world”—an idea now so firmly within the Overton Window that Henry Kissinger gravely ponders it in the Wall Street Journal? ....... I think it’s entirely plausible that, even as AI transforms civilization, it will do so in the form of tools and services that can no more plot to annihilate us than can Windows 11 or the Google search bar......... the young field of AI safety will still be extremely important, but it will be broadly continuous with aviation safety and nuclear safety and cybersecurity and so on, rather than being a desperate losing war against an incipient godlike alien. ........ In the Orthodox AI-doomers’ own account, the paperclip-maximizing AI would’ve mastered the nuances of human moral philosophy far more completely than any human—the better to deceive the humans, en route to extracting the iron from their bodies to make more paperclips. And yet the AI would never once use all that learning to question its paperclip directive. ........ from this decade onward, I expect AI to be woven into everything that happens in human civilization ........ Trump might never have been elected in 2016 if not for the Facebook recommendation algorithm, and after Trump’s conspiracy-fueled insurrection and the continuing strength of its unrepentant backers, many would classify the United States as at best a failing or teetering democracy, no longer a robust one like Finland or Denmark ....... I come down in favor right now of proceeding with AI research … with extreme caution, but proceeding.

The Chomsky et al. opinion piece in the @nytimes about ChatGPT is making the rounds. Rather than trying to deconstruct their argument, I asked @bing what it thinks of it.

Now you can judge for yourself who has the moral high ground 😂. pic.twitter.com/itdh1VOatl

— Sebastien Bubeck (@SebastienBubeck) March 10, 2023

Planning for AGI and beyond Our mission is to ensure that artificial general intelligence—AI systems that are generally smarter than humans—benefits all of humanity.