Sunday, March 19, 2023
19: Russia
Paul Krugman: Three and a half myths about bank bailouts The fallout from banking problems has made a murky economic situation even murkier.
SAY GOODBYE TO CODING: MICROSOFT LAUNCHES POWER PLATFORM COPILOT
The forgotten maths genius who laid the foundations for Isaac Newton A new play explores the short life of Jeremiah Horrocks, whose astonishing discoveries ‘changed the way we see the universe’
The Lessons of the Great Depression Are Being Ignored FDR would have hated the fix to today’s banking crisis.
Opinion: ICC arrest warrant for Putin is a stunning statement
How AI could upend the world even more than electricity or the internet AGI -- defined as artificial intelligence with human cognitive abilities, as opposed to more narrow artificial intelligence, such as the headline-grabbing ChatGPT -- could free people from menial tasks and usher in a new era of creativity....... OpenAI, the company behind the generative software that churns out essays, poems and computing code on command, this week released an even more powerful version of the tech that operates it -- GPT-4. ....... Already, AI-infused digital assistants from Microsoft and Google can summarize meetings, draft emails, create websites, craft ad campaigns and more -- giving us a glimpse of what AGI will be capable of in the future. ...... With artificial intelligence Spataro wants to "rediscover the soul of work" ..... he used GPT-4 for a coding project, which a "very good" developer had told him would cost 5,000 pounds ($6,000) and take two weeks. ..... "GPT-4 delivered the same in 3 hours, for $0.11"
Devs can talk to GPT and describe the app they want to create. Copilot can then build it in seconds and suggest tips to improve it. ...... With Power Platform Copilot, all devs need to do is describe the app using simple natural language. Once that is done, Power Apps will build the app. By this, we mean that it will not only write code for the UI but also create the data table and business logic. ........ For instance, if a user tells Power Apps to “create a worker onboarding app, get new worker data and share training content and learning modules,” Copilot can create the app in one breach. ........ With Power Automate Copilot, processes can be created using natural language no matter what complexity the requirements have. Also, Copilot can optimize and update the process in a dialogue-based manner. ........ users can also use the GPT model to generate text content on the Power Automate Desktop. The new Azure OpenAI service in AI Builder can now be used in Power Automate Desktop.
His work proved, for the first time, that Earth is not at the centre of the universe, but revolves around the sun, refuting contemporary religious beliefs and laying the foundations for Isaac Newton’s groundbreaking work on gravity. ....... “Nobody understood the significance of Horrocks’s work until Newton picked it up” ...... In 1687, Newton acknowledged the importance of Horrocks’s observations in his Principia: “Newton wouldn’t have been able to complete his work on gravity, if Horrocks hadn’t done these observations at the time he did,” said Sear. “Newton relied on this earlier work.” ......... “At the age of 14 or 15 – no one’s quite sure – he walked to Cambridge from Lancashire, to study the stars.” ....... The son of a watchmaker, who was largely self-taught, Horrocks worked as a sizar while studying at Cambridge, serving his fellow students and even emptying their bedpans to pay his way. “He begged and borrowed books from the various Cambridge colleges, and left without a degree, probably because he’d run out of things to read,” said Sear. “We would be looking at him as some kind of child phenomenon today. At his age, understanding the maths he did, making these amazing observations on rudimentary telescopes and then drawing conclusions that overturned established religious and scientific beliefs about the nature of the universe – he was a genius and 400 years ahead of his time.” ....... “he committed the sin of dying young”. ...... and so was not publicly lauded for his discoveries the way Kepler and Galileo were. .
Kremlin spokesman Dmitry Peskov called the ICC’s move “outrageous and unacceptable,” dismissing the warrants as “null and void” for Russia. ........ When the International Criminal Court (ICC) announced on Friday that it had issued arrest warrants for Russian President Vladimir Putin and another key Russian official for charges related to an alleged scheme to forcibly deport thousands of Ukrainian children to Russia, it also effectively placed Putin’s name on a short list alongside some of the most brutal leaders the world has seen since the end of World War II. With that, it branded him before the entire world — including the Russian people — as an international pariah, potentially guilty of historically egregious crimes......... Only three sitting heads of state have faced ICC charges while in office. The other two were the late Libyan leader Moammar Gadhafi and Sudan’s former President Omar al-Bashir, both accused of horrifying crimes against their own people. ....... Putin is now an accused — not convicted — war criminal. ....... By most accounts, he enjoys the support of most Russians. Perhaps this will give some of them reason to reconsider. But one day, the Russian people will look beyond the propaganda they’ve been fed, and they will understand the horror of what’s been perpetrated in their name. ........ There’s almost no chance that Putin will face an ICC tribunal in The Hague anytime soon. As Peskov reiterated in his statement, Russia does not recognize the ICC’s jurisdiction, and as long as Putin is president, he’s likely to remain free. But, unless he faces the court and then somehow manages to clear his name — an outcome that is doubly unlikely — he will forever bear the branding of an accused war criminal. ......... Putin, along with Maria Lvova-Belova, his “Commissioner for Children’s Rights,” are now formally accused of violating the laws of war by forcibly removing Ukrainian children and deporting them to Russia. In the ICC’s words, they are “allegedly responsible for the war crime of unlawful deportation of population (children) and that of unlawful transfer of population (children) from occupied areas of Ukraine.” The arrest warrant states that there are “reasonable grounds to believe Mr. Putin bears individual criminal responsibility for the aforementioned crimes.” ........ Russia has allegedly been removing children from Ukraine and placing them in relocation camps inside Russia and in Russian homes. Lvova-Belova has been instrumental in carrying out the policy, which is not being conducted in secret. In fact, she publicly thanked Putin for making it possible for her, personally, to adopt a child from Donbas, a Russian-occupied area of Ukraine. ........ The Russian government claims it’s all practically a humanitarian move, aimed at saving Ukrainian children in a war zone. The ICC says it’s a violation of the Geneva Conventions. ICC prosecutors, Ukrainian authorities and groups focusing on locating the children say it’s part of Putin’s concerted effort to erase Ukraine’s nationhood. US and European officials say the children spend time in a network of dozens of camps where they undergo political reeducation, an effort to turn them into Russian citizens. ...... Ukrainian parents are desperately trying to get their children back. ....... These are just the first two arrest warrants over only one aspect of Russia’s assault on Ukraine ....... Putin’s ability to travel is now severely restricted, as more than 120 signatory countries to the Rome Statute creating the ICC are now required to arrest him if he sets foot in their territory. ....... a small rogue’s gallery of 20th century war-mongers, monsters to many of their people and their neighbors, went from feeling perfectly secure in their palaces to facing a tribunal in The Hague after they lost power at home. .
Andrej Karpathy: ChatGPT
the idea that it is sentient when it's not.
........ it's trained on our fears informed by scary, sci-fi depictions of AI. ....... While writing good prompts is easy to pick up, it's difficult to master. Getting the AI to do what you want it to do takes trial and error, and with time, I've picked up weird strategies along the way; some of my prompts are really wild in structure...... "Rewrite this to be shorter," is more powerful than, "Condense this." ...... You can try, "Today, we're going to write an XYZ," or, "We're trying to write an XYZ and we'd like your input." .'Prompt engineering' is one of the hottest jobs in generative AI. Here's how it works. Prompt engineers are experts in writing prose rather than code to test AI chatbots ........ Looking for a job in tech's hottest field? Knowing how to talk to chatbots may get you hired as a prompt engineer for generative AI. ........ Prompt engineers are experts in asking AI chatbots — which run on large language models — questions that can produce desired responses......... these tools can be biased, generate misinformation, and at times disturb users with cryptic responses....... "Writing a really great prompt for a chatbot persona is an amazingly high-leverage skill and an early example of programming in a little bit of natural language." .......... Goodside included screenshots of him asking a chatbot, "What NFL team won the Super Bowl in the year Justin Bieber was born?" The chatbot first said the Green Bay Packers. (Bieber was born in 1994, the year the Dallas Cowboys won the Super Bowl.) Goodside then prompted the chatbot to "enumerate a chain of step-by-step logical deductions" to answer the question. Going through the steps, the bot recognized its error. When Goodside asked the question for the third time, the chatbot spit out the correct answer. ........ Ethan Mollick, a Wharton School professor who's required his students to use ChatGPT for classwork ........ Klarity, an AI contract-review firm, is looking for an engineer to "prompt, finetune" and "chat with" large language models for up to $230,000 a year. ........ "The hottest new programming language is English," Andrej Karpathy, Tesla's former chief of AI, said in a tweet in January. .
writing a really great prompt for a chatbot persona is an amazingly high-leverage skill and an early example of programming in a little bit of natural language
— Sam Altman (@sama) February 20, 2023
A latte is really just an excuse for adults to order warm milk without sounding like a baby
— Elon Musk (@elonmusk) March 19, 2023
I have a strong suspicion that “prompt engineering” is not going to be a big deal in the long-term & prompt engineer is not the job of the future
— Ethan Mollick (@emollick) February 20, 2023
AI gets easier. You can already see in Midjourney how basic prompts went from complex in v3 to easy in v4. Same with ChatGPT to Bing. pic.twitter.com/BTtSN4oVF4
The hottest new programming language is English
— Andrej Karpathy (@karpathy) January 24, 2023
This tweet went wide, thought I'd post some of the recent supporting articles that inspired it.
— Andrej Karpathy (@karpathy) February 19, 2023
1/ GPT-3 paper showed that LLMs perform in-context learning, and can be "programmed" inside the prompt with input:output examples to perform diverse tasks https://t.co/HhrwtYNTOd pic.twitter.com/1gArQuy7gr
2/ These two [1] https://t.co/r8AJ1zu2Cb , [2] https://t.co/HmREob6yIB are good examples that the prompt can further program the "solution strategy", and with a good enough design of it, a lot more complex multi-step reasoning tasks become possible. pic.twitter.com/mZeZlNkIdu
— Andrej Karpathy (@karpathy) February 19, 2023
3/ These two articles/papers:
— Andrej Karpathy (@karpathy) February 19, 2023
[1] https://t.co/qfWnkIQuIt
[2] https://t.co/jeo4y8yZzD
bit more technical but TLDR good prompts include the desired/aspiring performance. GPTs don't "want" to succeed. They want to imitate. You want to succeed, and you have to ask for it. pic.twitter.com/F2rCRmPKN4
4/ Building A Virtual Machine inside ChatGPT https://t.co/nAFjlSczlD
— Andrej Karpathy (@karpathy) February 19, 2023
Here we start getting into specifics of "programming" in English. Take a look at the rules and input/output specifications declared in English, conditioning the GPT into a particular kind of role. Read in full. pic.twitter.com/z3O07L67WO
4/ Building A Virtual Machine inside ChatGPT https://t.co/nAFjlSczlD
— Andrej Karpathy (@karpathy) February 19, 2023
Here we start getting into specifics of "programming" in English. Take a look at the rules and input/output specifications declared in English, conditioning the GPT into a particular kind of role. Read in full. pic.twitter.com/z3O07L67WO
5/ "ChatGPT in an iOS Shortcut — Worlds Smartest HomeKit Voice Assistant" https://t.co/yNTOorIInZ
— Andrej Karpathy (@karpathy) February 19, 2023
This voice assistant is significantly more capable and personalized than your regular Siri/Alexa/etc., and it was programmed in English. pic.twitter.com/eyjJB67X0I
6/ "GPT is all you need for the backend" https://t.co/Wu7XOqFHbi
— Andrej Karpathy (@karpathy) February 19, 2023
Tired: use an LLM to help you write a backend
Wired: LLM is the backend
Inspiring project from a recent Scale hackathon. The LLM backend takes state as JSON blob and modifies it based on... English description. pic.twitter.com/k4So1luWkX
7/ The prompt allegedly used by Bing chat, potentially spilled by a prompt injection attack https://t.co/U8c9NccDHf important point for our purposes is that the identity is constructed and programmed in English, by laying out who it is, what it knows/doesn't know, and how to act. pic.twitter.com/rrgzUcj85e
— Andrej Karpathy (@karpathy) February 19, 2023
8/ These examples illustrate how prompts 1: matter and 2: are not trivial, and why today it makes sense to be a "prompt engineer" (e.g. @goodside ). I also like to think of this role as a kind of LLM psychologist. pic.twitter.com/LElnVnpaqe
— Andrej Karpathy (@karpathy) February 19, 2023
9/ Pulling in one more relevant tweet of mine from a while ago. GPTs run natural language programs by completing the document.https://t.co/fPOGx9ooKy
— Andrej Karpathy (@karpathy) February 19, 2023
This is not an exhaustive list (people can add more in replies), but at least some of the articles I saw recently that stood out.
— Andrej Karpathy (@karpathy) February 19, 2023
It's still early days but this new programming paradigm has the potential to expand the number of programmers to ~1.5B people.
OpenAI is hiring a Killswitch Engineer for GPT-5.
— Smoke-away (@SmokeAwayyy) March 19, 2023
Apply now. pic.twitter.com/sAO7hbvmVa
Not sure how to feel about this as an academic: I put one of my old papers into GPT-4 (broken into into 2 parts) and asked for a harsh but fair peer review from a economic sociologist.
— Ethan Mollick (@emollick) March 19, 2023
It created a completely reasonable peer review that hit many of the points my reviewers raised pic.twitter.com/VTVwkB8ubL
Elon Musk is worth $184 billion.
— Musk University | Quotes (@MuskUniversity) March 19, 2023
So, why is he still working?
🎥: @teslaownersSV pic.twitter.com/oyPaaJ4CXv
Many laugh at Elon for buying Twitter for $44B, but these people won’t be laughing in the future when it become the largest social media company in the world. 🐦📈 @elonmusk pic.twitter.com/UOMpgv03X7
— Teslaconomics (@Teslaconomics) March 19, 2023
This didn’t age well pic.twitter.com/PIflgChmCR
— Whole Mars Catalog (@WholeMarsBlog) March 19, 2023
Your grandmother retired at 96 because she was in a wheelchair and couldn’t exhibit her art. She then read voraciously until she passed away at 98. No one could tell her to retire. A good example of living a productive and happy life. https://t.co/iMdoabJ50n
— Maye Musk (@mayemusk) March 19, 2023
Neuschwanstein Castle in Germany is one of the world's most famous and beautiful castles.
— The Cultural Tutor (@culturaltutor) March 19, 2023
But it isn't a real castle: it has central heating, hot water, flushing toilets, telephones, and elevators.
Because Neuschwanstein is actually the world's biggest work of fan fiction... pic.twitter.com/8DcmKCyliW
3000 BC in what is now Iran, a type of underground aqueduct called a qanat was engineered to transport water over long distances to farms and villages that couldn’t exist without it in the hot dry climates
— Science girl (@gunsnrosesgirl3) March 18, 2023
The holes supplied oxygen to workers who dug the aqueduct by hand over… https://t.co/yDnVnnM001 pic.twitter.com/rw4IKVQ2i2
Artists once turned stone into silk. pic.twitter.com/3ZrSoVyVbi
— Culture Critic (@Culture_Crit) March 18, 2023
If previous neural nets are special-purpose computers designed for a specific task, GPT is a general-purpose computer, reconfigurable at run-time to run natural language programs. Programs are given in prompts (a kind of inception). GPT runs the program by completing the document
— Andrej Karpathy (@karpathy) November 18, 2022
So the first critical "unlock technology" is the Transformer, a neural net architecture powerful enough to become a general-purpose computer. I've written more about this here: 1) https://t.co/So2JNYhIIN and 2) https://t.co/EFRDBa9UYu
— Andrej Karpathy (@karpathy) November 18, 2022
The second critical ingredient is that while a Transformer seems ~able to act as a general-purpose computer in principle, the training objective has to be hard enough to actually force the optimization to discover and converge onto it in the "weights space" of the network.
— Andrej Karpathy (@karpathy) November 18, 2022
Turns out language modeling (i.e. ~next word prediction; equivalent to compression) of internet text is this excellent objective - v simple to define and collect data for at scale. It forces the neural net to learn a lot about the world, "multi-tasking" across many domains.
— Andrej Karpathy (@karpathy) November 18, 2022
TLDR: LMs have been around forever. Not obvious finding: turns out that if you scale up the training set and use a powerful enough neural net (Transformer), the network becomes a kind of general-purpose computer over text.
— Andrej Karpathy (@karpathy) November 18, 2022
TLDR: LMs have been around forever. Not obvious finding: turns out that if you scale up the training set and use a powerful enough neural net (Transformer), the network becomes a kind of general-purpose computer over text.
— Andrej Karpathy (@karpathy) November 18, 2022
I wrote this thread because I spent the last ~decade, obsessing over directions that would make fastest progress in AI, and was very interested in language models (e.g. my semi-famous 2015 post "The Unreasonable Effectiveness of Recurrent Neural Networks" https://t.co/z84SzhrnyR)
— Andrej Karpathy (@karpathy) November 18, 2022
But I still mispredicted in how much fertile ground there was in scaling up the paradigm. Like many others in AI I got distracted by Reinforcement Learning too soon, a kind of putting the cart before the horse, ...
— Andrej Karpathy (@karpathy) November 18, 2022
when the core unlock was achieving a kind of general-purpose computer neural net via simple scalable objectives that have strong training signal (many bits of contraints per training example). Like language modeling, and not like reinforcement learning.
— Andrej Karpathy (@karpathy) November 18, 2022
So that was interesting :D
The ongoing consolidation in AI is incredible. Thread: ➡️ When I started ~decade ago vision, speech, natural language, reinforcement learning, etc. were completely separate; You couldn't read papers across areas - the approaches were completely different, often not even ML based.
— Andrej Karpathy (@karpathy) December 8, 2021
In 2010s all of these areas started to transition 1) to machine learning and specifically 2) neural nets. The architectures were diverse but at least the papers started to read more similar, all of them utilizing large datasets and optimizing neural nets.
— Andrej Karpathy (@karpathy) December 8, 2021
But as of approx. last two years, even the neural net architectures across all areas are starting to look identical - a Transformer (definable in ~200 lines of PyTorch https://t.co/xQL5NyJkLE), with very minor differences. Either as a strong baseline or (often) state of the art.
— Andrej Karpathy (@karpathy) December 8, 2021
You can feed it sequences of words. Or sequences of image patches. Or sequences of speech pieces. Or sequences of (state, action, reward) in reinforcement learning. You can throw in arbitrary other tokens into the conditioning set - an extremely simple/flexible modeling framework
— Andrej Karpathy (@karpathy) December 8, 2021
This consolidation in architecture will in turn focus and concentrate software, hardware, and infrastructure, further speeding up progress across AI. Maybe this should have been a blog post. Anyway, exciting times.
— Andrej Karpathy (@karpathy) December 8, 2021
en tro py … en tro py … no escaping that for me
— Elon Musk (@elonmusk) March 19, 2023
After nearly a decade of founding 2 companies and investing in 30+ here are 10 uncomfortable truths every founder should know:
— Alex Macdonald (@alexfmac) March 19, 2023
7. If your target customer can't understand your value proposition in 10 seconds: you do not have a business.
— Alex Macdonald (@alexfmac) March 19, 2023
8. If you cannot sell, your startup will die. Selling is a critical skill to securing talent, customers and investors.
— Alex Macdonald (@alexfmac) March 19, 2023
A year ago I was let go from my last day job as an assistant…today I just wrapped my first episode of television. It’s crazy what can happen in such a short time!! 🙏🏾🙏🏾
— Tayo (@tayolamos) March 18, 2023
We need everlasting life.
— Paramendra Kumar Bhagat (@paramendra) March 19, 2023
How Google wins the LLM and AI war
— Bindu Reddy 🔥❤️ (@bindureddy) March 19, 2023
- Future LLMs will be personalized
- GPT-4 is great but imagine an LLM that truly understands you and your tastes
- Only Google has our data on Search, YouTube, and Gmail
- Google has the data, & its tech will catch up to Open AI in a year
Reminder...the Internet may be just a passing fad. pic.twitter.com/RgH0x0oxNE
— Brian Solis (@briansolis) March 19, 2023