Welcome back to part two of Aixistential. You’ve read through the news in part one and now it’s time to find out more about the risks.
Up until now, we’ve technically been talking about what is known as Narrow AI: AI which is built to be as good as (if not better than) humans at specific tasks. This is, so far, the only AI we currently have. “But ChatGPT does everything so well!ˮ I hear you saying – well, firstly, try telling my girlfriend that. Secondly, it’s not actually good at everything; there is an argument for how specific is specific, but the point is it’s not as good as humans at everything we’re good at. it’s great at reading and writing, ie language, but it can’t produce images, videos, music, sound (by itself, it can talk to other models which can), and certainly no model can interpret someone’s body-language and know how they’re feeling (…yet), have the curiosity to advance science, or genuinely appreciate a piece of art.
However, what if there was a model which could do everything a human can, as well as a human can, if not better? Except, with significantly fewer constraints than a human – they don’t need to fit inside a skull, they aren’t limited by how much information they can process (they can read the entire internet, can you?), and their processing speed isn’t limited by biology. That is Artificial General Intelligence (AGI), aka Superhuman Machine Intelligence (SMI). It is a fundamentally different thing to what we have now, and I believe people need to understand this difference and therefore the vast difference in risks.
Yes. Literally just google OpenAIʼs mission statement – “to ensure that artificial general intelligence—AI systems that are generally smarter than humans—benefits all of humanityˮ, which implies they’re building AGI, or superintelligence, and if you believe they can control it to “ensureˮ it “benefits all of humanityˮ, keep reading. It’s not just OpenAI either, just scroll up and see Safe Superintelligence – it’s in the name.
Smarter people than me? Whatever do you mean. Ok but seriously, we definitely have no idea how to do so currently, and I would argue we never could. How does one possibly control something smarter and more capable than any of us? If that were possible, why aren’t we being controlled by chimps? I’d claim a superhuman level of intelligence is, by definition, impossible for us to understand, let alone control. That would be like me asking you to imagine a brand new colour, or what it’s like to see in infrared, or what it feels like to be a cloud. It just doesn’t work.
I’m not the only one who says this. Anthropic, one of the other BigAIs, admitted “no one knows how to train very powerful AI systems to be robustly helpful, honest and harmlessˮ. Shane Legg, Co-Founder and Chief Scientist at DeepMind, (now) Googleʼs AI research lab, said that “we have no idea how to solve this problemˮ. Turing Prize winner Geoffrey Hinton – widely accepted as the ‘Godfather of AIʼ – warned there is a significant risk it will quickly get out of our control, and “If it gets to be much smarter than us, it will be very good at manipulation, because it will have learned that from us … It’ll figure out ways of manipulating people to do what it wants.ˮ
An open letter by the Future of Life Institute (FLI) in March 2023 put it best, now signed by over 30,000 people including key AI experts like: Stuart Russell (UC Berkeley, author of the standard textbook “Artificial Intelligence: a Modern Approach”), Yoshua Bengio (U. Montreal, Turing Prize winner), Max Tegmark (MIT), John J Hopfield (Princeton), Andrew Critch (AI Research Scientist at UC Berkeley, Founder and President of Berkeley Existential Risk Initiative), Emad Mostaque (CEO Stability AI), Connor Leahy (CEO Conjecture), Yann Lecun (Chief AI Scientist Meta), as well as other researchers, engineers, and scientists from OpenAI, DeepMind, and others. It powerfully states we are in a race to “deploy ever more powerful digital minds that no one – not even their creators – can understand, predict, or reliably controlˮ.
So, actually, it is exactly those people smarter than me who say we can’t control it. In fact, there are already examples of AI attempting to deceive us and seek power. GPT4 even recruited someone on TaskRabbit to solve a CAPTCHA (basically an anti-bot test) when it failed to do so itself (albeit, with some hinting). Checkmate.
Itʼs true it is hard to estimate when we will build AGI, but it’s possibly, if not probably, not as distant as you think. A 2023 AIImpacts survey of AI experts found that the aggregate forecast time for a 50% change of AGI existing was 2047. I don’t know about you, but I hope to still be around then.
Oh, and that figure dropped by 13 years since the same survey in 2022. For comparison, that figure dropped by just one year in the six years between 2016 (result was based on 2061 survey responses) and 2022 (result was based on 2060 survey responses). Notice any trajectory there? The top players in the space also seem to agree. Shane Legg predicted “a log-normal distribution with a mean of 2028 and a mode of 2025ˮ in 2011, whilst his CEO Demis Hassabis thought it “could be just a few years, maybe within a decade awayˮ in 2023. Model Evaluation and Threat Research METR, (formerly ARC Evals) – a non-profit aiming to assess whether AI could pose “catastrophicˮ risks to society – stated that the fact AI is not capable enough to “pose very large risks to the world… could change quicklyˮ. That was over a year ago.
The scarier thing is that, we’ve repeatedly underestimated the speed of AI progression. This is even obvious anecdotally – how many people do you know were predicting the possibility of something like ChatGPT was less than a year away at the start of 2022? I even remember predicting this incorrectly myself. And it’s not just me; economist Bryan Caplan predicted in January 2023 that no AI would be able to get an A in 5 out 6 of his exams by even January 2029. 3 months later, GPT4 scored 73/100, the fourth-highest score on the test to his “surprise and no small dismayˮ.
So basically, experts believe it’s not that far away, and we have repeatedly underestimated the speed of its progression. Not so sci-fi.
There is a misconception that AI would need to hate us, or be evil, in order for it to pose an existential risk to humanity. This is not the case, it just needs to want something that isn’t perfectly aligned with human interests, then execute on that desire. Do you consider yourself evil? Do you have a vendetta against living things? No (well, hopefully). Do you walk around with your eyes on the floor to avoid stepping on bugs? Didn’t think so. As the FLI open letter put it, “Humans don’t generally hate ants, but we’re more intelligent than they are – so if we want to build a hydroelectric dam and there’s an anthill there, too bad for the antsˮ.
This is known as the alignment problem. As Eliezer Yudkowsky (founder and research fellow of Machine Intelligence Research Institute (MIRI)) put it “the vast majority of randomly specified utility functions do not have optima with humans in themˮ, or in simpler words, there are way more goals/aims which are more optimal without humans, than with. And AI is very good at optimising.
There’s a famous thought experiment called the paperclip maximiser which helps get this point across. Imagine an AI smarter than humans is made, with the sole goal of producing paper clips – that couldn’t possibly be harmful right? At first, it’s great, producing paper clips at a rate never seen before, yay! But then, it runs out of the metal we gave it. Cities and buildings have a bunch of metal it could use. Oops, no more cities or buildings. Ok ok, so we tell it it can’t use buildings. Oops, it just dug out the entire earth. Ok, no digging. Oops, it extracted the iron in the blood of all humans. Ok ok – so we tell it it can only use the metal we give it, fine. I mean, probably not fine, it would find a way to trick us to “giveˮ it more. But let’s say it wouldn’t. Nice, we’re back to an efficient paper clip-making AI. Uh oh, it ran out of space in the warehouse we gave it to store paper clips. Oops, it just demolished all cities. You get the point.
In fact, this has always been a challenge with AI, we just haven’t built anything capable of causing (too much) damage… yet. We have not figured out how to reliably ensure the goals we give it donʼt have unintended consequences. An AI tasked with a grasping task, figured out it was easier to fool the evaluator by hovering between them and the camera, than actually complete the task. Another one in a walking simulation figured out there was a bug in the simulation it could exploit.
Another example was where an AI was tasked with playing a racing game, in which power-ups would give a turbo boost. The researchers decided to add a reward for getting the power-up… it quickly figured out it could rake in more points by spinning in circles infinitely, hitting the power-ups repeatedly as they respawn, rather than finishing the race. Here’s a list of over 60 examples of this behaviour being observed if you’re interested.
Hopefully you can now see that even without maliciousness, there are many many more ways in which it ends badly, rather than well, for us.
I’ll let the experts take this one.
“Superhuman machine intelligence SMI is probably the greatest threat to the continued existence of humanityˮ – Sam Altman, CEO of OpenAI.
“It’s my number 1 risk for this centuryˮ, when asked whether AGI risks outweighed other existential risks – Shane Legg CBE, Co-founder and Chief AI Scientist of DeepMind.
“We don’t have all the answers there yet – and the technology is moving fast. So does that keep me up at night? Absolutely.”, when talking about AI dangers – Sundar Pichai, CEO of Google.
“I’m actually deeply concerned that in two or three years, we’ll get to the point where the models can, I don’t know, do very dangerous things with science, engineering, biology, and then a jailbreak could be life or death.ˮ; and “AGI could destroy humanity. I canʼt see any reason in principle why that couldnʼt happen.ˮ – Dario Amodei, Anthropic CEO and Ex OpenAI VP of Research 2023. (Jailbreaking means tricking AI to do things it was designed not to).
“What happens when these things get more intelligent than us … Right now, they’re not more intelligent than us, as far as I can tell … given the rate of progress, we expect things to get better quite fastˮ; and “Itʼs a completely different form of intelligence … A new and better form of intelligence.ˮ; and “I console myself with the normal excuse: If I hadnʼt done it, somebody else would haveˮ (when asked if he regretʼs his lifeʼs work); and “The idea that this stuff could actually get smarter than people — a few people believed that … But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.ˮ Geoffrey Hinton, Turing Prize winner, who left Google in 2023 specifically to be able to speak about AI dangers without conflict of interest.
“When the time comes [to] build a highway between two cities, we are not asking the animals for permission. We just do it because itʼs important for us. And I think by default thatʼs the kind of relationship thatʼs going to be between us and AGIs which are truly autonomous and operating on their own behalf.ˮ – Ilya Sutskever, Chief Scientist at OpenAI.
“The core danger with AGI is that it has the potential to cause rapid change. This means we could end up in an undesirable environment before we have a chance to realise where weʼre even heading.ˮ – Greg Brockman, CoFounder and CTO of OpenAI.
“I hear people who denigrate these fears, but I donʼt see any solid argument that would convince me that there are no risks of the magnitude that [Geoff Hinton] thinks aboutˮ Yoshua Bengio, Turing Prize Winner.
“The development of full artificial intelligence could spell the end of the human raceˮ – Stephen Hawking
“One difference between worrying about AI and worrying about other kinds of technologies (e.g. nuclear power, vaccines) is that people who understand it well worry more, on average, than people who don’t. That difference is worth paying attention to.ˮ – Paul Graham, YCombinator Founder and computer scientist.
“One of the biggest risks to the future of civilization is AIˮ; and “Mark my words A.I. is far more dangerous than nukesˮ – Elon Musk, CEO of xAI.
“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.ˮ – Center for AI Safety Open Letter 2023.
Signatories include: Sam Altman, Demis Hassabis, Dario Amodei, Geoffrey Hinton, Bill Gates, Shane Legg, James Manyika (Google Alphabet SVP Research, Tech andSociety), Kevin Scott (Microsoft CTO), Mira Murati (OpenAI CTO), Tony Wu (xAI Co-Founder), Ilya Sutskever (OpenAI Co-Founder and Chief Scientist), Igor Babuschkin (xAI Co-Founder), Mustafa Suleyman (Inflection AI CEO), Emad Mostaque (Stability AI CEO), Ian Goodfellow (Google DeepMind Principal Scientist), Mark Beall (Former Director of AI Strategy and Policy at DoD), Andy Weber (Former US Assistant Secretary of Defence for Nuclear, Chemical and Biological Defense Programs). And thatʼs just a few.
So, still think this is tinfoil-hat material? If you do then I ask you this: what do you know that these people don’t? Because I’m definitely scared.
Thank you to stop.ai, a brilliant website trying to raise awareness for these dangers, where I found a lot of the resources for this article.
*TreasurySpring’s blogs and commentaries are provided for general information purposes only, and do not constitute legal, investment or other advice.