Great summary! Please if possible could you add the context window of each model, how many tokens can you use for each?
Also, I default to 4.1 for most non complex coding tasks because it’s fast, o3 is too slow, and 4o is too sycophantic (even when you use a good system prompt 4o is so annoying sometimes).
[True Believers] Maybe these people really believe that there’s a 3% chance that Christ will return this year!
If you are a True Believer it’s more likely you believe someone would be sent to hell for gambling on sacred matters. True Believers would never bet that Jesus Christ will return in any specific date. There is one bible passage that Jesus says in Matthew 24:36:
But of that day and hour no one knows, not even the angels of heaven, but My Father only.
If someone truly believes Jesus’ words, then they think the timing of Jesus’ return is unknowable.
The True Believers hypothesis rings false because that would be a frankly ridiculous belief to hold. Sometimes people profess ridiculous things, but very few of them put their money where their mouth is on prediction markets.
I don’t know what the Yes people have in mind, but certainly they are not Christians.
Jesus once said: “Simply let your “yes” mean yes, and your “no” mean no, because anything beyond that comes from evil”.
Will Russia invade only eastern Ukraine or the rest of Ukraine as well?
Zvi, I would love to see you making an analysis of Brazil and I’d compare it to my own thinking and calibrate myself on my own predictions. I strongly believe you are currently the best at what you’re doing and I would be grateful if you spent at least some time reasoning about our situation here and making some predictions.
I would like to see different voting systems on different posts, so we could try them out and report back on how each one allows us to express what we think about those posts.
I don’t like a single 5-star rating system, but I would love multiple 5-stars in different categories. For example, we could choose to give 5 stars for each of these categories you pointed out: clarity, interestingness, validity/correctness, informativeness, friendliness.
If we had a single 5-star rating system, and suppose I read a post that was completely clear and interesting, but with a wrong conclusion, I just don’t know how many stars I would give it. But if I could give 5 stars for clarity and interest, I might give 0~3 stars for correctness (depending on how wrong it was).
Suppose there’s a post with 2-star rating on its clarity and I believe the fair rating should be 3-star; I think I would click on 4~5 stars so that I could steer the rating to 3-star. I am not sure how to fix this behaviour, but the rating could say something like “select the final rating you think this post should have”, and then I would click on 3-star, if the calculations somehow took care of that.
I completely endorse experimenting different voting systems, we can simply not use them if we realize they don’t work well. We should be open to experimentation, and obviously the current system is not perfect and can be improved, and if you are willing to put time and effort into this, I will support and participate and help on discovering if they work better than the current system.
After reading this post, I decided to send a message to Gurkenglas.
This was the best decision I made. I have been developing a mobile software for the last 120 days, and this is my first experience developing something this big. Sharing my screen on Discord, he was able to:
- Completely change my debugging workflow, which is likely to save me hundreds of hours in the long-term
- Teach me a new testing ritual
- Make it easier to identify useless pieces of code
- Refactor hundreds of lines with simple intuitions and good regex
- Teach me dozens of useful keyboard shortcuts and IDE features
- Give me some useful career advice
I wasn’t expecting to level up that fast, but in retrospect it seems obvious that accepting a group invitation from a more experienced player would allow me to level up much faster and learn new tricks. Great learning experience.
Hey younger version of me, start reading LW now, start programming now, study more statistics, go deeper on math.
lol, your first paragraph described exactly the discussion I had with a friend less than one month ago… and you wrote the exact question he made me.
“So you want people to sit at home all day and collect free money?”
On my mind, I had “Yes. I want this. For everyone. Not just money. I want to give everyone all the resources they need to fulfill their lives. I’m trying to discover how”
I wonder why ended up dodging the question; yes, +1 for star-manning. I like the concept.
Please, I love this discussion, I super upvoted and if you have any new thoughts about NFTs I would like to hear you talking about. Did you get any new information or have you updated your thoughts about NFTs? I am completely fascinated on how this is even possible, people caring so much about useless maps, and the territory has become “it is super valuable to have a map that tells which of the two perfect copies of a territory arrived first”, or people are just pretending for the sake of playing the game in which allows you to speculate on the value of useless maps? I say “useless” in the sense that you can not have any extra utility from knowing that you have two exact sequence of bits “00101” and “00101″, but you know that the first one was generated first than the second one, which is a perfect copy of the first one. Even if I have this information, the value provided by those bits doesn’t change, so I wonder if this is just a game in which you pretend to care about which sequence of bits came first, even if you can not be absolutely sure which one came first (it all depends which computers uploaded a copy of this sequence of bits first to the internet, not the computer that really generated the first instance in the first place). This is completely crazy for me, but it makes me imagine that I could get a lot of value from the digital art work that I usually generate while learning new subjects in math, and people would love to pay more for the first copy of my digital art, but the second copy would be worthless, and that’s because the first copy that I uploaded is the first copy that has been put on the blockchain, why would anyone give a f*** about this? But if everyone is playing a game in which “if you get lucky on getting the first copy and everyone pretends that the first copy is more valuable, you can get a lot of money by playing this game and trying to get the best first copies of every digital piece”. LOL HELP ME WHAT ARE WE EVEN TALKING ABOUT, is this what is really happening? Just a new emergent game with strange rules just to have fun and get money and happiness points by randomly assigning points to random digital copies on a specific blockchain? Should we even play this game? I mean, is it net useful to humanity to pretend and play this game?
I’m having fun exploring the subject and I would love anyone to expand on this topic, if anyone has ever explored this craziness we are seeing.
I really like the approach from Tournesol and I was wondering a way to improve the rating system.
Not only I would like to rate a video, but sometimes I would like to dispute the rating given by another user.
Suppose I am a seller on AliExpress. I have been selling a 4.8 star product for a year.
Suddenly I receive a new and rare one star rating. The rating goes like this:
“User123: I waiting the product to arrive, can’t wait to see it. When it arrives I will update the rating”
This is not uncommon. A good proportion of ratings happen without any correlation to the meaning of the rating labels. Sometimes someone miss click, and sometimes the user is completely stupid and does unexpected things. A user may say that some content is unreliable while being factually incorrect.
One thing that could work is that if enough people clicked on ‘dispute button’ to dispute the rating given from some user, the data from that user would be considered “noise” until some debate was settled. Or we could just change the weight of his input. Maybe the user has to provide some explanation for his deviant input and his input weight would go back to normal. Otherwise few trolls could destroy the system by creating bots to steer the rating system to his preference. But it would be harder for him to justify each instance, and a user with a high proportion of ratings being disputed should raise a red flag.
On the UI provided on the white paper I can see 5 horizontal sliders. Maybe we could add optional explanations for the user to clarify why he gives “1 star importance” for this video about “very important topic”. Users that give explanations for their ratings should have higher weights than users that do not provide any explanation. Maybe I could give a like on the rating+explanation from another user, updating my view on the platform and the final weight of his rating.
I believe that many people will take COVID to their relatives during Christmas and the New Year, and I’m seriously thinking about starting some campaign to make people aware that they should at least this time consider wearing a mask for the next weeks and also stop socializing without a mask during this period in order to protect their relatives during the traditional holidays. I started developing a mobile app today to be released by December 12 and another (also related to COVID) to be released by December 20, I don’t know what impact this will have on people’s decisions, but I’m already leaving here publicly registered so that it works as a personal incentive not to pay attention to anything else until the two apps are released.
PM me if you want to join development, I’m using KivyMD to develop the apps. The first app lets you easily record your last social gatherings, saying how many people (with and without masks) were present, when it happened, the duration (short or long), and if were indoors or outdoors. Then it says how risky is going to see your relatives on Christmas or in any other date and what you should do to mitigate the risks. There are also other things, but this is the main idea. The second one is a COVID game, aiming to influence social behavior and public discourse during pandemic, details later.
I haven’t exercised regularly in years, and last week I started thinking about how bad the consequences can be for me. I decided to do something, I wasn’t in the mood, I thought “well, I’ll do 10 push-ups. Maybe it’s not much, but it’s better than nothing”. And I made it. You said “make it ridiculously easy”, and now I just made 15 push-ups. Interesting. This is really easy. And I will do it again. Just a little more on the next time.
Epistemic status: babble all the way down, not pruning. But I believe my approach is better than most of other answers here.
The error from other LWers is not separating the evaluation of lessons to the evaluation of tests.
Students should be allowed to give good teachers a bonus. For each lesson, in any moment of the lesson, the students should have the possibility to rate the teacher’s performance on some metrics. Think on a mobile application that does that. Do you know when you take a ride with an Uber and immediately after finishing the ride you rate it? We should have the same possibility of rating teachers after their lessons (up until some limit, e.g., you had your lesson on Monday, you won’t be able to rate it on the next month, you have until a week to rate this lesson). The teacher should be paid a bonus when he gets good scores. This bonus would be added lesson by lesson to teacher’s account.
Let’s say each week I have three different lessons with professors A, B and C.
Professor A gives me 2 hours lesson/week.
Professor B gives me 4 hours lesson/week.
Professor C gives me 6 hours lesson/week.
For each two hours lesson, the student gains one point to spend. So, I have 6 points to spend on spend on professors A, B, C in any way I choose to.
The professor A, I’ve just watched his lesson and I loved it. I give him 3 points. Professor B is good too, I like him, but I will just give him two points. Professor C is not that good teacher, but he seems to be working hard on these particular difficult topics, I’ll give him one point.
On the end of the month, good teachers will be rewarded by how good their performance were on THE LESSONS. I haven’t spent time thinking on a good function to convert the scores received by the teacher on this week to money, but it doesn’t seem hard to create a fair one.
We should have a separate rating system for the evaluation of the tests applied by the teacher, so we could separate the feelings that appear on our heart when we compare the quality of the lessons to the difficulty of the problems posed by the professor on his test. We know when we go bad on an exam, “that’s the teacher’s fault”. So this separate system would be more strict, asking several questions like “How difficult was this test? How many hours have you studied before doing it? The questions on the test were related to things taught on the lessons? How do you compare the difficult of the test questions to the difficult of the lessons’ questions? Leave a comment about the test on the following Entry box”. Obviously I haven’t pruned these questions, they just arrived at my mind, but certainly there exist a very good set of questions that could let us investigate how well the teachers perform in creating tests and also reward them when we detect it.
Thinking again about the first system, it should also have some questions about the lesson. “How good the professor explains the concepts? How organized he is? Did you learn the concepts? How do you rate the difficulty of the topics this teacher is trying to explain to you? [Leave here what kind of questions you believe would improve this questionnaire]. Leave a comment about the lesson on the following Entry box”.
It shouldn’t be needed for a student to answer these questions to give all his points for a teacher. But we could weight the student points by how many questions he answered. For example, if I gave you 3 points and I said why I’ve done this, this weighs more than a student that gives you 3 points but doesn’t explain why he does that. Justified rating is worth more than unjustified rating.
this advantages teachers with larger classes.
Your reward function can take in consideration the number of students that participated on the lesson, the number of students the rated the professor, and also you could average the scores, I don’t know, come on, you can create a function that is fair for any class size, you just have to think about what function you will use
Where does the money come from?
Diminish all salaries in x%. Now you can redistribute this money more fairly, proportional to performance.
My second-favorite teacher in undergrad was relatively unpopular because he taught very difficult classes, at least some of which were required to graduate.
That’s why it is important to evaluate the LESSONS every week. And when the test comes, this is a different evaluation. This professor was unpopular due to difficult tests, not to bad lessons, right?
Most universitites have already systems where students evaluate their teachers at the end of the year and the scores do figure into administrative decisions of the university
That’s the problem. At the end of the year you are evaluating the “teacher”, which means
If I find the teacher a good professor and I give him +5 points, but I sucked at his tests, and I give −10 points for my bad feelings for doing bad on the test, the final evaluation of the teacher is “tis teacher is bad” == −5
If the system rates week by week, we could detect misuse of the system if we suddenly see bad lessons ratings close to the test application (right after the test, for example).
I don’t think this is how market wages work. If it is known that the average teacher gets a $100 bonus, the school will offer $100 less in base pay than it would otherwise.
Maybe not right now, when the change is introduced. But in the following years, the wages will raise slower than they would otherwise, until the balance is achieved.
It doesn’t seem bad to pay a little less for the average teacher with average lessons, and pay a little more for the above average teacher with above average lessons. It seems like arbitrage. You do good lessons, you earn more. Why not? And if you can now detect which teachers are much worse than average, you can fire them and get even more students that are interested in this school full of good teachers, because the bad ones can’t stay
I don’t think so. For example, follow these instructions:
Say you are a poor guy on a poor country
Say you luckily got a computer when you were a child
Say while you were studying AI, you found LW.
Say bad events happen to you/your family, and now you are in urgent need.
Now you can see, although most LW readers are not like this guy, this guy is among LW readers. My point is that we should financially support this guy, independently if he belongs to LW or not. I would say it is easy to help him and we have a reason to support him, and the fact he reads LW doesn’t change the facts on his life. Again, although not common, we should be prepared to detect and solve this kind of unfortunate situation. At least this is something I would do if I had enough resources to help.
Why would they?
Sometimes non needy people want to help other people in need. If you were looking to maximize happiness points across the world, for example, you would gain more points helping those members in need.
Considering that LW readers are mostly rich Americans
There are non-zero members suffering financially, that’s why I asked that. It would be too easy for some people here to make this number go to zero.
My question was really stupid, actually I was thinking “I would like to spend at least 200 hours on this project, but it seems I won’t get any money from it, maybe I could ask LW members if they want to support it financially”.
A better question is “Can I ask you money to help me to build a software that may help you?”, or “it is inappropriate to ask for money on LW, the platform discourages this”.
Disclaimer: I am still not sure if this is the correct question. Anyway, I am developing some helper tools, and although I won’t monetize them directly, it would be good to get some money from it, because I am not the guy who has enough money to not need any money from the community anymore.
Should LW members support each other financially?
If my stated and/or revealed preferences are that I don’t value joining the elite class very much, is that wrong in either an instrumental or terminal sense?
Considering you haven’t miscalculated the value from joining the elite class, I believe it is wrong to spend energy to be labeled as “elite”. If you lost something you had to protect while you wasted your time with useless pursuits, like trying to “join the elite” by getting some very specific superior pedigree, then you took a very poor instrumental action. It all depends on what you actually want and how joining the elite will help you to achieve that. But it seems obvious that there are several ways of achieving anything you want without having to join the elite, except if your terminal value is being labeled as elite from some specific set of people.
For people who do seem to value it a lot, either for themselves or their kids (e.g., parents obsessed with getting their kids into an elite university), is that wrong in either an instrumental or terminal sense?
That seems wrong if there are less costly and much faster ways to achieve what the parents actually want from their kids without having to make them participate on the “become elite” rituals. Maybe the parents want their kids to be seen as good people, respected among the members of the tribe, without financial troubles. If elite people have these properties, you make your kids to participate on the rituals needed to make them labeled as elite (parents use the “elite” label here as a proxy to status, respect and financial support). But that’s a bad choice when parents discover there are several other cheaper ways of achieving the same ends. And that’s a bad choice when parents discover in the future that the proxies used in the past to filter good people from bad people are not relevant anymore. I believe what parents actually want is not just their kids being seen as good people, but also their kids being good people. Maybe if they become too obsessed with getting elite kids, what if parents discover their elite kids are not actually good people? Due to the weak correlation between being actually good and participating on elite rituals, I believe it is wrong to make your kids to become elite kids. You should focus on making them good, respectable and rich. Otherwise, if the correlation is strong (between participating on what you call elite rituals and becoming good, respectable and rich), you should make your kids participate on these rituals.
You said you “ran Pangram on the previous 4 encyclicals. The first 20 paragraphs on all of them register as 100% human, all with high confidence”, but one of the most obvious features you would add to a commercial AI-generated text detector would be searching the web to find similar results before 2022? (I don’t know if they do this or not) Otherwise I imagine you would have absurd false positives on old texts or well known texts in general that I don’t think users would like. I would be surprised if they weren’t doing that. I was asking GPT5.5 about how Pangram possibly works:
Also, although there may be evidence of Claude assistance in the final wording/style, “Claude, author of the Humanitas” is much stronger than the evidence supports; unless you have a way to differentiate normal human writing polished by AI to “I’m the bishop, the pope asked me my help writing an article saying X and Y, please create a few paragraphs for me”. Maybe that’s exactly your point, but it’s not completely clear what you mean when you say that “Claude wrote” something; do you mean “human drafters wrote the substantive content, argument, and theological framing, then used Claude or another LLM for polishing, grammar, phrasing etc, which produced em-dashes, smoother triads, ‘genuinely’ and things that would be flagged as AI-generated” or do you mean something else? I finished reading your post and I was still confused what you think actually happened, how exactly people used AI (assuming they used it)