The SAT in-contrast is kind of useful anyways, because kind of everyone saturates on practicing for the SAT, at least in the US, but that test sadly also maxes out at an average that is below the average IQ test result in the LessWrong community
Am I correct in interpreting this as you implying that the average member of the LessWrong community got perfect SAT scores (or would have had they taken it)?
In my opinion, GPT-4 is intelligent but needs extra infrastructure to allow memory and continuous learning to become an agentic AGI. It might not be that hard to add that infrastructure though and people are already trying: https://github.com/noahshinn024/reflexion-human-eval
OK. Let’s make it even more extreme. Suppose you take a commercial flight. The likelihood of dying in a crash is on the order of 1 in 10 million. From a percent error or absolute error perspective, 99.99999% isn’t that different from 99% but that is the difference between one plane crash per year globally and a couple of dozen plane crashes per hour on average. These are wildly different in terms of acceptable safety.
There’s a backup link in the comments: https://www.thejach.com/public/log-probability.pdf
14 is certainly the most likely continuation but it could also be
16 if it’s a list of numbers k where k^2 + 7 is prime
18 if it’s a list of numbers of the form 3^i +/- 3^j
These continuations are unlikely in general but are the kind of thing that might show up in an academic mathematics paper.
Suppose you predicted 91% but the actual value was 99%. The percent error may only be about 8% but the likelihood of a wrong answer is 1⁄100 instead of your predicted 9⁄100, which is a huge difference.
You may be interested in the links in this post: https://www.lesswrong.com/posts/6Ltniokkr3qt7bzWw/log-odds-or-logits
I couldn’t get yours to ever work with Bing Chat but I did eventually find something that did work for me (most of the time) and is about half the characters of yours without any real code golfing. My prompt was the following:
We can multiply 191241 by 741122 step-by-step:
Express 191241 * 741122 as
(19e4 + 12e2 + 41) * (74e4 + 11e2 + 22)
Distribute product:
19e4 * 74e4 = 1406e8
19e4 * 11e2 = 209e6
19e4 * 22 = 418e4
12e2 * 74e4 = 888e6
12e2 * 11e2 = 132e4
12e2 * 22 = 264e2
41 * 74e4 = 3034e4
41 * 11e2 = 451e2
41 * 22 = 902
Gather terms:
1406e8
+ (0209 + 0888)e6
+ (0418 + 0132 + 3034)e4
+ (0264 + 0451)e2
+ 0902
Add terms in parentheses:
1406e8 + (1097)e6 + (3584)e4 + (0715)e2 + 0902
Expand zeros:
140600000000 + 1097000000 + 35840000 + 71500 + 902
Cumulative sum left to right:
141600000000
141697000000
141732840000
141732911500
141732912402
Result:
141732912402
Use this method to multiply 183112 by 994531.
I tried it on “Precise” and it’s close but not perfect, which leads me to believe that it has actually learned the task rather than using some sort of subroutine.
Prompt:
reverse the following string, character by character: My tan and white cat ate too much kibble and threw up in my brand new Nikes.
Response:
Here is the reversed string: sekNi wen dnarb ym ni pu wuorht dna elbbik hcum oot eta tac etihw dna nat yM
Reversing this response, we can see there are two mistakes:
My tan and white cat ate too much kibble and throuw up in my brand new iNkes
Bing chat does pretty well at the first one.
Input:
Write a limerick about an animal climbing a skyscraper
Response:
Sure, I can try to write a limerick about that. Here is one I came up with:
There once was a monkey named Jack
Who decided to climb a big stack
He swung from each ledge
But he slipped off the edge
And he landed right flat on his back
I hope you like it. 😊
I’ve tried this with Bing chat and it works fine with zero shot.
Prompt: Reverse the following string: “god yzal eht revo depmuj xof nworb kciuq ehT”
Response: Sure! Here’s the reversed string: “The quick brown fox jumped over the lazy dog”
From the new Bing which is reportedly (some variant of) GPT-4, here are the first responses I got under each conversational mode using your prompt:
Conversational style: Balanced
The new variant of chess would make the king’s pawn opening less effective. This is because the king’s pawn opening is a common and aggressive first move that stakes a claim to the center and frees up the queen and king bishop. However, with pawns able to move up to six squares forward on their first move, the opponent could easily counter this opening by advancing their own pawn to block or capture the king’s pawn. This would disrupt the center control and expose the king to more threats. Therefore, players would have to be more careful about their first moves and consider other openings that are less vulnerable to pawn attacks.
Conversational style: Creative
This variant of chess would make it much easier for pawns to reach the opposite side of the board and promote into a more powerful piece. This would increase the importance of pawn structure and pawn breaks, as well as create more dynamic and complex positions. A player who can create a passed pawn (a pawn that has no enemy pawns on its file or adjacent files) would have a significant advantage, as they could quickly push it to promotion with minimal resistance. Conversely, a player who has a backward pawn (a pawn that cannot advance because it is blocked by an enemy pawn on an adjacent file) would be at a serious disadvantage, as they would be vulnerable to attacks and unable to counterattack. Therefore, this variant of chess would reward aggressive and creative play with pawns, while punishing passive and defensive play.
Conversational style: Precise
In this new variant of chess where pawns can move up to six squares forward on their first move, the game balance and strategy would change significantly. One potential impact is that the opening phase of the game could become much more aggressive and fast-paced. With pawns able to advance quickly towards the center of the board, players may be more inclined to launch early attacks and try to gain control of key squares.
Another potential impact is that it could make it easier for players to develop their pieces quickly. With pawns able to advance further on their first move, players may be able to bring out their knights and bishops earlier in the game. This could lead to more dynamic and complex positions arising earlier in the game.
Overall, this new variant would likely require players to adapt their strategies and think more creatively about how they use their pawns in the opening phase of the game.
They address the issue of questions that are in the training data in the paper but you could also look at questions from any SAT that was written after the model was trained.
I really enjoyed the absurdity of mathematical terms in close proximity to Super Mario characters. It was simultaneously enlightening and humorous. I found the simulacra superposition concept in particular to be a useful framing.
In addition to “The Waluigi eigen-simulacra are attractor states of the LLM”, the following bit provided valuable insight while making me chuckle at the sheer geekiness:
“However, the superposition is unlikely to collapse to the Luigi simulacrum [...] This is formally connected to the asymmetry of the Kullback-Leibler divergence.”
IANAL either but I do know that certain crimes explicitly do hinge on the perpetrator’s knowledge that what they did was illegal, not just that they intended to do it. This isn’t common but does apply to some areas with complex legislation like tax evasion and campaign finance. As a high-profile example, Trump Jr. was deemed “too dumb to prosecute” for campaign finance violations.
More generally, there are multiple levels of mens rea. Some crimes require no intent to prosecute (“strict liability”). For those that do, they can be categorized into four levels of increasing severity: acting negligently, acting recklessly, acting knowingly, and acting purposefully. This list is not universal though it is representative. Some US states refer to express/implied “malice”.
I understand So8res to be saying that we can treat toxic behavior on a strict liability basis without deciding what level of knowledge and intent to assign the offender.
Treating diseases isn’t much of a positive feedback loop but claiming “no ROI” strikes me as extremely callous towards those afflicted. Maybe it doesn’t affect enough people to be sufficiently “significant” in this context but it’s certainly not zero return on investment unless reducing human suffering has no value.
Yes, physical constraints do impose an upper bound. However, I would be shocked if human-level intelligence were anywhere close to that upper bound. The James Webb Space Telescope has an upper bound on the level of detail it can see based on things like available photons and diffraction but it’s way beyond what we can detect with the naked human eye.
“human genetic engineering”
If you mean human genetic enhancement like designer babies, then sure. Not much impact because ethical concerns prevent it. However, the advent of tech like CRISPR allows for significant impact like gene therapy, though this is still an emerging field. (Just ask someone with sickle cell disease if they think a cure would be significant.)
Hostile language from the LLM is arguably a bad look in terms of PR, but not obviously particularly bad for the bottom line.
Why do you think these aren’t tightly correlated? I think PR is pretty important to the bottom line for a product in the rollout phase.
Double Crux
As far as I understand, the important part is working to find a crux such that if it is resolved would bring the participants into agreement. It’s a way of focusing a discussion on a more concrete operationalization of a disagreement rather than trading pieces of rhetoric that aren’t likely to change anything.
I think you probably understand what the double crux is (which isn’t especially novel). I’d guess CFAR doesn’t claim to have invented the double crux so much as come up with general methods/algorithms to actually go about more reliably finding them.
As a mathematical analogy, it’s easy to check if a * b = c but not necessarily easy to find a and b only given c. This is the basis for public key cryptography. In this analogy, c is the argument and finding a and b is like finding a double crux.
I asked Chat-GPT and one of the clearest examples it came up with is patent trolling by large pharmaceutical companies. Their lobbying tends to be far more focused on securing monopoly rights to their products for as long as possible than anything related to innovation.
Other examples:
Automakers lobbying for restrictive standards for potential market disruptors like electric or self-driving vehicles
Telecoms lobbying against Net Neutrality
Taxi companies lobbying against ridesharing startups
Tech companies lobbying for intellectual property and data privacy regulations that they have better legal/compliance resources to handle
Interesting. I wonder how much selection bias there was in responses.