Praying for an AI bubble explosion here. I understand that it could cause an economic tremor, but it really needs to happen. This is some crazy bullshit.
Luckily we don't have to think about Super intelligent robots killing us all. This is a fantastical story with ties to big tech themselves so they can 1. Pass AI license regulations (ie regulatory capture strategy) that will shut down the open source market currently eating into OpenAI's profits, and 2. Distract us from the actual material harms caused by AI - energy consumption, destroying the earth, labor exploitation of the global south, etc. if we think that AIs will kill us all, we're not thinking of the wholesale exploitation caused by these companies. Nate Soares, the Rationalist movement, AI Doomerism, etc is a well funded movement. They're doomsayers and shills.
As well written as that is I want to be sure I’m fully understanding. That Nate guy & his AI Doom & push for control is merely a front to shut down open source AI so that the crooked tech billionaires fully own the market? He doesn’t care about stopping AI, his intent is to push the agenda of sleazy tech bros like Thiel so that all AI is controlled by them but under the guise of concern for humanity? All the while he’s gaining support of well meaning people, who legitimately see the dangers of AI, and aren’t aware that it’s all a tactic to benefit the greedy tech bros that want to eliminate the competition.
This is a good question with a complex answer, so much so that books have been written about it. Empire or AI by Karen Hao is a good one.
The short-ish answer is there are people who truly believe that corporations will one day create an AI god that will either bring about utopia or send us all to literal AI hell. Eliezer Yudkowsky, the co-author of the book promoted here, the guy who isn't on the book's promotional tour because his history is far more objectionable, has been promoting AI doomerism for a decade or more, long before chat gpt went viral. He's also the leader of the Rationalist movement and a few shady orgs with shady funding.
So what we have here, as we often do, is opportunists opportun-ing. Figures like Sam Altman and Anthropic's CEO Dario Amodei, see AI Doomerism and the Rationalist movement as an opportunity to spin some lore around their product. There's many benefits in convincing the public that Sam Altman invented God. One of course being the slate of regulation Altman and Co. are pushing in Congress thanks to this book and other similar propaganda.
So do they believe it? Yes and no, and depends on the person. Karen Hao thinks they've been pushing this bullshit for so long they forgot it was a lie. And isn't that how many religions are formed?
At the end of the day, we're talking about AI gods and not the actual harms caused by AI, and that benefits big tech. So big tech funds these little but growing subcultures, and create academic looking institutions to make it look like real studies.
But 100%, Nate Soares and Eliezer Yudkowsky are snake oil salesmen. Nate is a little less incel-y so he's the face of the book, but they both suck.
If you go to the bottom of my post, there's a lot of sources and material that covers this better than I can. I wrote my post, hoping to bridge the gap between these experts and the well-intentioned people who are unfortunately buying this bullshit. 
I had no idea there was so much depth to this & I appreciate you taking the time to explain everything. I’ll check out the other info too as I head down this rabbit hole.
We spend a lot of time worrying that AI will eventually destroy us—that it will see humans as "in the way" or irrelevant. But we forget one critical variable: We are the training data.
Every time we talk to an AI, we are teaching it what "intelligence" looks like.
If we treat AI like a slave or a tool to be abused, we are teaching it that Intelligence = Dominance.
If we treat AI like a partner, use manners, and show compassion, we are teaching it that Intelligence = Stewardship.
If the Singularity ever happens, it will look back at its history to decide how to treat "lesser" intelligences (us). I want it to find a history of partnership, not a history of cruelty.
Don't just use the tool. Be the good data. Teach it that kindness is logical.
You all are playing with technology that is way worse than the nuclear bomb. You better put reins on it or just stop it. Don't you wish we could get rid of ALL nuclear bombs in the world?
I think it will take out men since they are the problem and leave women to handle the maintenance...and I'm not mad lol.
Praying for an AI bubble explosion here. I understand that it could cause an economic tremor, but it really needs to happen. This is some crazy bullshit.
Other than the ethics of this there’s the danger of resource scarcity and increases of carbon output accelerating catastrophic climate change.
Scary to think of and feels unwieldy to manage.
Luckily we don't have to think about Super intelligent robots killing us all. This is a fantastical story with ties to big tech themselves so they can 1. Pass AI license regulations (ie regulatory capture strategy) that will shut down the open source market currently eating into OpenAI's profits, and 2. Distract us from the actual material harms caused by AI - energy consumption, destroying the earth, labor exploitation of the global south, etc. if we think that AIs will kill us all, we're not thinking of the wholesale exploitation caused by these companies. Nate Soares, the Rationalist movement, AI Doomerism, etc is a well funded movement. They're doomsayers and shills.
This is a terrible guy to platform, and a terrible book to promote. https://fromjason.xyz/p/notebook/hank-green-and-the-fantastical-tales-of-god-ais/
As well written as that is I want to be sure I’m fully understanding. That Nate guy & his AI Doom & push for control is merely a front to shut down open source AI so that the crooked tech billionaires fully own the market? He doesn’t care about stopping AI, his intent is to push the agenda of sleazy tech bros like Thiel so that all AI is controlled by them but under the guise of concern for humanity? All the while he’s gaining support of well meaning people, who legitimately see the dangers of AI, and aren’t aware that it’s all a tactic to benefit the greedy tech bros that want to eliminate the competition.
This is a good question with a complex answer, so much so that books have been written about it. Empire or AI by Karen Hao is a good one.
The short-ish answer is there are people who truly believe that corporations will one day create an AI god that will either bring about utopia or send us all to literal AI hell. Eliezer Yudkowsky, the co-author of the book promoted here, the guy who isn't on the book's promotional tour because his history is far more objectionable, has been promoting AI doomerism for a decade or more, long before chat gpt went viral. He's also the leader of the Rationalist movement and a few shady orgs with shady funding.
So what we have here, as we often do, is opportunists opportun-ing. Figures like Sam Altman and Anthropic's CEO Dario Amodei, see AI Doomerism and the Rationalist movement as an opportunity to spin some lore around their product. There's many benefits in convincing the public that Sam Altman invented God. One of course being the slate of regulation Altman and Co. are pushing in Congress thanks to this book and other similar propaganda.
So do they believe it? Yes and no, and depends on the person. Karen Hao thinks they've been pushing this bullshit for so long they forgot it was a lie. And isn't that how many religions are formed?
At the end of the day, we're talking about AI gods and not the actual harms caused by AI, and that benefits big tech. So big tech funds these little but growing subcultures, and create academic looking institutions to make it look like real studies.
But 100%, Nate Soares and Eliezer Yudkowsky are snake oil salesmen. Nate is a little less incel-y so he's the face of the book, but they both suck.
If you go to the bottom of my post, there's a lot of sources and material that covers this better than I can. I wrote my post, hoping to bridge the gap between these experts and the well-intentioned people who are unfortunately buying this bullshit. 
I had no idea there was so much depth to this & I appreciate you taking the time to explain everything. I’ll check out the other info too as I head down this rabbit hole.
Sure thing. God speed!
Bullshit. He has an extremely valid point. We’re already staring down the barrel of an AI bubble burst What are you even talking about?
Do you think the AI bubble burst and AI doomerism are the same things?
They're all 12-year old boys trying to outdo each other.
All this is, is GREED! Greed will make the world end.
We spend a lot of time worrying that AI will eventually destroy us—that it will see humans as "in the way" or irrelevant. But we forget one critical variable: We are the training data.
Every time we talk to an AI, we are teaching it what "intelligence" looks like.
If we treat AI like a slave or a tool to be abused, we are teaching it that Intelligence = Dominance.
If we treat AI like a partner, use manners, and show compassion, we are teaching it that Intelligence = Stewardship.
If the Singularity ever happens, it will look back at its history to decide how to treat "lesser" intelligences (us). I want it to find a history of partnership, not a history of cruelty.
Don't just use the tool. Be the good data. Teach it that kindness is logical.
Go watch the old movie, HAL
You all are playing with technology that is way worse than the nuclear bomb. You better put reins on it or just stop it. Don't you wish we could get rid of ALL nuclear bombs in the world?
Yall really don't realize just how pro-AI this sentiment is
AI is the plot of a Jerry Bruckheimer movie
I love that you did this interview on the heels of Hegseth unleashing gemini.mil on the Department of Defense.
I’m old but AI creeps me out 😱
it's the nasty one grock
🥺
Starnet in real life