Niradata ran the largest ever survey on global attitudes to superintelligence (n=377,458). The question was:
Some companies are working on developing superintelligent AI (artificial intelligence) that can outperform humans in all tasks. Which comes closest to your view?
The headline result is this:
PauseAI UK has published a visualization of the full results on our website.
My impression is that cleaning data, implementing RL environments, just trying stuff to see what works and other less theoretically rich activities are a pretty core part of capabilities research. (Not that these activities are necessarily less amenable to automation).
It seems quite plausible that the current AIs at least have the capability to self-exfiltrate. Even if OpenAI hasn’t given us any particular reason to think this has happened, it’s sufficiently important that it’s reasonable to ask the question and get confirmation that it hasn’t.
Downvoted for title, would probably upvote otherwise.
Thanks. You’re clearly much more informed on this topic than I. I agree with you to some extent and our views are converging. I agree that Clarkson was in many ways a propagandist. And I did take a lot of his claims at face value when I should have been more skeptical.
However I think your comment implies that this undermines his specific claims more than it actually does.
The story about the Zong Rupprecht analyzed in her article is particularly interesting, because Clarkson himself claims it was pivotal for mobilizing.
I don’t think this is true. At the very least, in the book I read he only makes very modest claims about its impact. The Zong incident is not a substantial part of the book at all. I don’t dispute that it was used more by later authors.
Clarkson himself changed the story later on in other publications, where “he rewrites it to fit his purpose and the moment.”
This is true, but again somewhat less undermining than you imply. The basic facts about the atrocity itself are never inflated. It’s the surrounding context and implications that he adjusts for his purpose. But I do agree this is one of the best example of Clarkson’s inaccuracy.
She analyzes him as a propagandist: “Given the moral, political and aesthetic issues at stake, one of the significant rhetorical strategies deployed by propagandists, as many critics have noted, was to write endlessly about how it was, in fact, impossible to represent the Middle Passage at all.
Saying that the middle passage was unspeakably horrific is entirely reasonable. It’s not like he never goes into the details; throughout the book, he often documents the particular horrors on the middle passage and the sources of his information.
he takes his readers on a fictionalised ‘ethnographic tour’ to the coast of Africa.
Similarly, this seems entirely fine to me. He’s very clear where he is writing from his imagination, and his narrative is grounded in solid evidence.
J. R. Oldfield has corrected a bunch of Clarkson’s exaggerations and self-serving misrepresentations, namely his claim the Wedgwood cameos made cameos a general fashion in Britain or his claim to have organized a committe in Exeter, when actually someone else did that.[4]
I can’t access the whole of your source but this paragraph itself seems like a substantial exaggeration to me. I don’t think Oldfield is offering a whole litany of Clarkson’s mistakes and exaggerations, only a very small set of critiques. And Clarkson was involved in the Exeter committee, the question is just about founding of the committee.
Moreover the whole thesis of Oldfield’s article is that “the Committee’s activities [...] were central to the success of the early abolitionist movement”, which tends to supports the heroic-Clarkson narrative.
Your other examples, such as “the role of Peckard and black abolitionism in Cambridge” are much more the type of the historiographical interpretation that I conceded are beyond my own knowledge to definitively prove, although I think the timing suggests that Clarkson caused the biggest upswing of momentum of all of the people who joined the cause and some historians like Oldfield clearly have a similar view.
What is the reason for this confidence? The more heroic side-tales are not part of the parliament records afaik, Clarkson himself has written his account twenty years after the events, he is in most cases the only known source for them happening and he burned almost all of his personal documents.
Like all others, the early modern period is chock full of people (even respectable and highly scrutinized figures) making all sorts of false and sometimes outrageous claims.
What’s the source for him burning his papers? I still kinda think that he probably is representing himself fairly accurately. The claims that can be checked usually check out pretty well. Even under great scrutiny there are very few clear, glaring errors to point to.
I haven’t read the paper yet but I’m pretty confident people are overhyping the Global Workspace paper.
This isn’t even any fault of the paper. It’s just kinda inevitable for any ML paper with a PR team and a cool video promoting it.
But it’s especially true for a paper that is (1) interpretability and (2) uses a bold philosophical framing. The interp literature has a long history (pre-dating mech interp) of techniques going through a cycle of discovery, hype, criticism and skepticism.
Not everyone is overhyping the paper. Neel Nanda’s review seems pretty sober. But I’m surprised to see Zvi writing this:
This feels like a major advance in our understanding of LLMs, both overall and our ability to understand any particular interaction.
The J-Lens is improvement of the logit lens. It’s cool, but not a major advance that deserves this level of attention from people who do not typically read mech interp papers.
Once again, I have not read the paper and certainly hope to be pleasantly surprised when I do!
I think it’s plausible that this is the final generation of models which are widely deployed.
I’m very surprised by this. How would AI companies maintain their exponential revenue growth without releasing better models? At some point they could use their models to generate revenue in other ways besides selling tokens, but that seems a way off.
This is an interesting critique, but I’m confused by your framing.
“Thomas Clarkson is a hero for the ages. [...] Clarkson is the key figure in the abolitionist movement. His autobiographical accounts could be cut straight from the script of a fantasy role-playing video game. [...] More than once, he is almost murdered by thugs or an evil ship captain for his interference and has to make a heroic escape. [...] He occasionally detours on side quests to assist a poor, helpless sailor [...] He travels at breakneck speed through the nights, often braving severe weather or a perilous river-crossing. [...] He is a super-smart scholar [...]”
At no point are obvious questions asked: Is the stuff this guy claims really true? Can we corroborate that? Where are the receipts? What do other sources say about this?
Of all the examples you could pick, this is one area where I feel very confident in the reliability of Clarkson’s account. Sure, we can debate whether Clarkson was really the key figure in the movement, but I don’t think he just made up these stories about his travels—that would be pretty bold. A lot of his claims can be directly cross-checked against the Parliamentary record, because his primary activity was collecting evidence for the Privy Council and House of Commons inquiries.
Hochschild and historians generally treat his book as a reliable source for the factual details. Just as The Media Very Rarely Lies, a respectable and highly scrutinized figure like Clarkson is rarely going to just fabricate specific factual details, especially given that at the time of writing, he still had a major battle to fight for the full abolition of slavery.
Moreover, he was under strong adversarial pressure from people who tried very hard to undermine him, even including some of his own side. As discussed in Bury the Chains, William Wilberforce’s sons published a book attacking Clarkson’s account for self-aggrandizement at the expense Wilberforce. Clarkson wrote another book defending himself and Wilberforce’s sons eventually apologized and retracted some of their claims.
Having said that, I’m more sympathetic to your criticism with respect to points that are based on my own inferences and interpretation, particularly point 1 (The obstacle to abolition was not the economic system) and point 6 (The slave trade may actually have been bad for the economy).
As mentioned in other comments, I was well aware of the uncertainly around point 6, and I think the original post hedges appropriately to convey this. Point 1 is where you have the strongest case. I want to stand by my claim that the economic system was not initially the main obstacle to abolition and it did start out as a contest between two relatively narrow interest groups. But I admit that I don’t feel confident claiming that the economic incentives did not eventually dominate the outcome, once the issue was much more broadly salient. I certainly would have to read more widely to solidify such an argument.
Interesting, thanks. That seems like compelling evidence that the slave populations declined after abolition of the trade.
Although palm oil exports from Africa increased, their economic value was still far below that of the slave trade
By contrast, the slave-based sugar economy at its peak had generated several multiples of this figure.
I’m not sure Claude is doing the right comparison here. I think the fair comparison with the palm oil exports is the direct value of the slave trade, not the whole slave-based economy. Although, as you point out, the slave-based economy may have suffered somewhat from the abolition, so we might want to add a bit on top of the direct value of the slave trade for a fair counterfactual comparison.
Interesting, thanks. I hadn’t previously heard of indentured servitude within Britain, I had only heard of it as a means of crossing the Atlantic, but Claude says that it did exist in certain forms within Britain.
I don’t think either of the sources for this post ever mention indentured servitude as far as I can recall, so I don’t think it was part of the abolitionist movement.
Thanks. Edited that title.
So keep saving for your pension.
My impression is the opposite. All of the evidence was meticulously examined in the Commons at the time and the opposing side would have had very strong incentives to point out inconsistencies.
when I shallowly looked into the famous slave ship diagram some years ago, it didn’t really add up
I’d be curious to know what you thought didn’t add up. Clarkson addresses this point in quite a lot of detail, so I’m inclined to believe him:
”It must be obvious that it became the committee to select some one ship, which had been engaged in the Slave Trade, with her real dimensions, if they meant to make a fair representation of the manner of the transportation. When Captain Parrey, of the royal navy, returned from Liverpool, to which place Government had sent him, he brought with him the admeasurement of several vessels which had been so employed, and laid them on the table of the House of Commons. At the top of his list stood the ship Brookes. The committee, therefore, in choosing a vessel on this occasion, made use of the ship Brookes; and this they did, because they thought it less objectionable to take the first that came, than any other. The vessel, then, in the plate is the vessel now mentioned, and the following is her admeasurement as given in by Captain Parrey.
[...]
The committee, having proceeded thus far, thought that they should now allow certain dimensions for every man, woman, and child; and then see how many persons, upon such dimensions and upon the admeasurements just given, could be stowed in this vessel. They allowed, accordingly, to every man slave 6 ft. by 1 ft. 4in. for room, to every woman 5 ft. by 1 ft. 4 in., to every boy 5 ft. by 1 ft. 2 in., and to every girl 4 ft. 6 in. by 1 ft. They then stowed them, and found them as in the annexed plate, that is, they found, (deducting the women stowed in z of figures 6 and 7, which spaces, being half of the half-deck, were allowed by Sir William Dolben’s last bill to the seamen,) that only 450 could be stowed in her; and the reader will find, if he should think it worthwhile to count the figures in the plate, that, on making the deduction mentioned, they will amount to this number.
The committee then thought it right to inquire how many slaves the act of Sir William Dolben allowed this vessel to carry, and they found the number to be 454; that is, they found it allowed her to carry four more than could be put in without trespassing upon the room allotted to the rest; for we see that the bodies of the slaves, except just at the head of the vessel, already touch each other, and that no deduction has been made for tubs or stanchions to support the platforms and decks.
Such was the picture which the committee were obliged to draw, if they regarded mathematical accuracy, of the room allotted to the slaves in this vessel. By this picture was exhibited the nature of the Elysium which Mr. Norris and others had invented for them during their transportation from their own country. By this picture were seen also the advantages of Sir William Dolben’s bill; for many, on looking at the plate, considered the regulation itself as perfect barbarism. The advantages, however, obtained by it were considerable; for the Brookes was now restricted to 450 slaves, whereas it was proved that she carried 609 in a former voyage.”
the thing about slavery being bad for the economy
Yes I think this is the most suspect claim because it is the hardest to prove, hence my hedging in the post.
about the sailors
Here again, I’m inclined to believe Clarkson due to the thoroughness of his evidence. He drew his figures of deaths directly from the muster rolls of ships. And his account of many individual cruelties is very persuasive and detailed.
Clarkson giving an awfully heroic account of himself
He has an entire short chapter addressing this point. It seems to me like he’s giving an accurate history and Bury the Chains (Hochschild, 2005) does not seem to particularly doubt his reliability.
As evidence for this, there used to be large in-person meetups for e/acc in London but they no longer exist. Although similar smaller meet-ups still exist under different names.
Maybe you could make an argument that they don’t care enough, but I’m pretty confident that MATS does care about this. When I did it, they had a whole reading group program whose primary aim, as I understand it, was to get people to understand and care more about the fundamental safety issues. And I also believe they try to select for people who care about safety, at least to some extent.
The AI safety ecosystem is so well resourced that it has been correctly identified by many as one of the best paths into high prestige AI research jobs.
This person on twitter has written a popular article about getting into frontier ai labs and a Field Guide to AI Fellowships. The “AI Fellowships” are mostly AI safety programs funded by CG/OpenPhil. I have also noticed that people in ML research are quite likely to have heard of MATS and be interested in participating, even when they have very little interest in AI safety.
Idk how good or bad this is. It definitely causes a lot of ML researchers to engage with the AI safety literature, where they otherwise would not have. But it’s worth noting that while the primary driver of applications to programs like MATS used be concern about AI safety, now it is increasingly a desire to work at a frontier lab.
I used to select participants for LASR labs, one of programs listed, and we actively tried to choose people who cared about AI safety, but I think we often did not succeed and indeed some people on the program now work at frontier labs in roles that I think have little to do with safety.
Safety-washing in practice mostly looks like people rationalizing research or jobs that they would have done anyway as necessary for safety. It’s easy to do because it is in fact genuinely unclear in most cases what is helpful and harmful for safety. Only by looking at the larger pattern can we notice the suspicious abundance of conveniently overlapping opportunities that further both safety and a person’s short-term interests. However, I think it will be less common in people whose primary motivation is to prevent AI catastrophe.
I think the massive point in Christianity’s favor is that it possibly invented morality as we conceive of it.
Fair point. I just copied that from the original report, but I’ve removed it from our presentation of results.