Suggestions
Ajay Agrawal
Professor at Rotman School of Management, University of Toronto
Ajay Agrawal is the founder and academic director of the Creative Destruction Lab (CDL), a seed-stage program for science-based companies.12 He is also the Geoffrey Taber Chair in Entrepreneurship and Innovation and Professor of Strategic Management at the University of Toronto's Rotman School of Management.13
Professor Agrawal founded CDL in 2012 at the University of Toronto.2 The program has since expanded to multiple locations across Canada, the United States, Europe, and Australia.4 CDL's mission is to enhance the commercialization of science for the betterment of humankind.1
In addition to his work with CDL, Agrawal is:
- A Research Associate at the National Bureau of Economic Research in Cambridge, MA13
- A Faculty Affiliate at the Vector Institute for Artificial Intelligence in Toronto13
- Co-founder of NEXT Canada (previously The Next 36)2
- Co-author of books on artificial intelligence, including "Prediction Machines: The Simple Economics of Artificial Intelligence"23
Agrawal has been recognized for his contributions to entrepreneurship and innovation. In 2022, he was named to the Order of Canada.5 He also serves as an advisor to the U.S. and Japanese governments on matters related to science, innovation, and technology.3
Under Agrawal's leadership, CDL has become one of the largest programs in the world for seed-stage, science-based companies, with a focus on areas such as artificial intelligence, quantum computing, and other emerging technologies.45
Highlights
Some assert that “Only American leadership is a realistic path to developing a nonproliferation regime to avert the risks of self-destruction super-intelligence will unfold.”
We'll debate this on Tuesday night next week [8.00-9.15pm ET].
Creative Destruction Lab Reading Group: Situational Awareness, by Leopold Aschenbrenner. https://lnkd.in/ezgP9hBe
Self nominate to join: https://lnkd.in/efEv9XDW
Background:
“...only American leadership is a realistic path to developing a nonproliferation regime to avert the risks of self-destruction superintelligence will unfold. Our generation too easily takes for granted that we live in peace and freedom. And those who herald the age of AGI in SF too often ignore the elephant in the room: superintelligence is a matter of national security, and the United States must win. Whoever leads on superintelligence will have a decisive military advantage.
Superintelligence is not just any other technology—hypersonic missiles, stealth, and so on—where US and liberal democracies’ leadership is highly desirable, but not strictly necessary. The military balance of power can be kept if the US falls behind on one or a couple such technologies; these technologies matter a great deal, but can be outweighed by advantages in other areas. The advent of superintelligence will put us in a situation unseen since the advent of the atomic era: those who have it will wield complete dominance over those who don’t.” p. 127
“It seems clear that within a matter of years, pre-superintelligence militaries would become hopelessly outclassed. The military advantage would be decisive even against nuclear deterrents. To be even clearer: it seems likely the advantage conferred by superintelligence would be decisive enough even to preemptively take out an adversary’s nuclear deterrent.” p. 129
“If there is a rapid intelligence explosion, it’s plausible a lead of mere months could be decisive: months could mean the difference between roughly human-level AI systems and substantially superhuman AI systems.” p. 130
“At stake in the AGI race will not just be the advantage in some far-flung proxy war, but whether freedom and democracy can survive for the next century and beyond. The course of human history is as brutal as it is clear. Twice in the 20th century tyranny threatened the globe; we must be under no delusion that this threat is banished forever. For many of my young friends, freedom and democracy feel like a given—but they are not. By far the most common political system in history is authoritarianism.” p.135
“The safety challenges of superintelligence would become extremely difficult to manage if you are in a neck-and-neck arms race. A 2 year vs. a 2 month lead could easily make all the difference. If we have only a 2 month lead, we have no margin at all for safety.” p.138
“There is more than a 2% chance that within the next 5 years, by 2029, we will experience an intelligence explosion as models will be able to do all the work of an AI researcher/engineer.”
We'll discuss this on Tuesday night next week [8.00-9.15pm ET].
CDL Reading Group: Situational Awareness, by Leopold Aschenbrenner. https://lnkd.in/ezgP9hBe
Self nominate to join: https://lnkd.in/efEv9XDW
Background:
“I make the following claim: it is strikingly plausible that by 2027, models will be able to do the work of an AI researcher/engineer. That doesn’t require believing in sci-fi; it just requires believing in straight lines on a graph.” p. 8
“In any case, we are racing through the OOMs, and it requires no esoteric beliefs, merely trend extrapolation of straight lines, to take the possibility of AGI - true AGI - by 2027 extremely seriously.” p. 41
“AI progress won’t stop at human-level. Hundreds of millions of AGIs could automate AI research, compressing a decade of algorithmic progress (5+ OOMs) into 1 year. We would rapidly go from human-level to vastly superhuman AI systems.” p. 46
“The jump to superintelligence would be wild enough at the current rapid but continuous rate of AI progress (if we could make the jump to AGI in 4 years from GPT-4, what might another 4 or 8 years after that bring?). But it could be much faster than that, if AGI automates AI research itself. Once we get AGI, we won’t just have one AGI. … we’ll likely be able to run many millions of them (perhaps 100 million human-equivalents, and soon after at 10x+ human speed). Even if they can’t yet walk around the office or make coffee, they will be able to do ML research on a computer. Rather than a few hundred researchers and engineers at a leading AI lab, we’d have more than 100,000x that - furiously working on algorithmic breakthroughs, day and night. Yes, recursive self-improvement, but no sci-fi required; they would need only to accelerate the existing trendlines of algorithmic progress (currently at ~0.5 OOMs/year).” p. 47
“We don’t need to automate everything - just AI research… And the job of an AI researcher is fairly straightforward, in the grand scheme of things: read ML literature and come up with new questions or ideas, implement experiments to test those ideas, interpret the results, and repeat. This all seems squarely in the domain where simple extrapolation of current AI capabilities could easily take us to or beyond the levels of the best humans by the end of 2027… And automating AI research is all it takes to kick off extraordinary feedback loops.” p.49
