I try to practice independent reasoning/critical thinking, to challenge current solutions to be more considerate/complete. I do not reply to DMs for non-personal (with respect to the user who reached out directly) discussions, and will post here instead with reference to the user and my reply.
ZY
Karma: 79
On what evidence do I conclude what I think is know is correct/factual/true and how strong is that evidence? To what extent have I verified that view and just how extensively should I verify the evidence?
For this, aside from traditional paper reading from credible sources, one good approach in my opinion is to actively seek evidence/arguments from, or initiate conversations with people who have a different perspective with me (on both side of the spectrum if the conclusion space is continuous).
I am interested in learning more about this, but not sure what “woo” means; after googling, is it right to interpret as “unconventional beliefs” of some sort?
I personally agree with you on the importance of these problems. But I myself might also be a more general responsible/trustworthy AI person, and I care about other issues outside of AI too, so not sure about a more specific community, or what the definition is for “AI Safety” people.
For funding, I am not very familiar and want to ask for some clarification: by “(especially cyber-and bio-)security”, do you mean generally, or “(especially cyber-and bio-)security” caused by AI specifically?
Does “highest status” here mean highest expertise in a domain generally agreed by people in that domain, and/or education level, and/or privileged schools, and/or from more economically powerful countries etc? It is also good to note that sometimes the “status” is dynamic, and may or may not imply anything causal with their decision making or choice on priorities.
One scenario is “higher status” might correlates with better resources to achieve those statuses, and a possibility is as a result they haven’t experienced or they are not subject to many near-term harms. In other words, it is not really about the difference between “average” and “high status”’s people’s intelligence, but more about what kind of world they are exposed to.
I do think it is good to hear all different perspectives to stay curious/open-minded.
edit: I just saw Dragon nicely listed two potential reasons, with scenario 2 mentioning something similar with my comment here. But something slightly specific in my thinking, is that these choices made by “average” and “high status” people may or may not be conscious, but rather from the experience from their lives and the world they are exposed to.
Could you define what you mean by “correctness” in this context? I think there might be some nuances into this, in terms of what “correct” means, and under what context
Based on the words from this post alone -
I think that would depend on what the situation is; in the scenario of price increases, if the business is a monopoly or have very high market power, and the increase is significant (and may even potentially cause harm), then anger would make sense.
Thanks! I think the term duration is interesting and creative.
Do you think for the short-term ones there might be pre-studies they need to do for the exact topics they need to learn on? Or maybe could design the short-term ones for topics that can be learnt quickly and solved quickly? I am a little worried about the consistency in policy as well (for example even with work, when a person on a project take vacation, and someone need to cover for them, there are a lot of onboarding docs, and prior knowledge to transfer), but could not find a good way just yet. I will think more about these.
Amazingly detailed article covering malevolence, interaction with power, and the other nuances! Have been thinking of exploring similar topics, and found this very helpful. Besides the identified research questions, some of which I highly agree with, one additional question I was wondering is: do self-awareness of one’s own malevolence factors help one to limit the malevolence factors? if so how effective would that be? how would this change when they have power?
Interesting idea, and I think there is a possibility that the responsibility will make the “normal people” make better choices or learn more even though they do not know policy, etc in the first place.
A few questions:
Do you think there is a situation where selected random people do not want to be in office/leadership and want to pursue their own passion/career and thus due to this reason may do a bad job? Is this mandatory?
What are some nuances about population and diversity? (I am not sure yet)
Could you maybe elaborate on “long term academic performance”?
Agree with this, and wanted to add that I am also not completely sure if mechanistic interpretability is a good “commercial bet” yet based on my experience and understanding, with my definition of commercial bet being materialization of revenue or simply revenue generating.
One revenue generating path I can see for LLMs is the company uses them to identify data that are most effective for particular benchmarks, but my current understanding (correct me if I am wrong) is that it is relatively costly to first research a reliable method, and then run interpretability methods for large models for now; additionally, it would be generally very intuitive to researchers on what datasets could be useful to specific benchmarks already. On the other hand, the method would be much useful to look into nuanced and hard to tackle safety problems. In fact there are a lot of previous efforts in using interpretability generally for safety mitigations.
Would agree with most of the posts; To me, humans have some general shared experiences that may activate empathy related to those experiences, but the the numerous small differences in experience make it very hard to know exactly what the others would think/feel, even if in exactly the same situations. We could never really model the entire learning/experience history from another person.
My belief/additional point I want to add/urge is that this should not be interpreted as say empathy is not needed because we don’t get it right anyways (I saw some other comments saying evolution is against empathy) - it is more to recognize what we are not naturally good at empathy(or less well than we thought), and thus create mindsets/systems (such as asking and promoting on gathering more information about the other person) that encourage empathy consciously (when needed).
(maybe I misread, and the second point is addressing another comment)
I think I observe this generally a lot: “as soon as those implications do not personally benefit them”, and even more so when this comes with a cost/conflict of interest.
On rationality on decision making (not the seeking truth part on belief forming I guess) - I thought it is more like being consistent with their own preference and values (if we are constraining to the definition on lesswrong/sequence ish)? I have a hot take that:
If the action space of commit to a belief is a binary choice, then when people do not commit to a belief, the degree they believe in that belief is less than those who do. If we have to make it into binary classification, then it is not really a true belief if they do not commit to that belief.
It could be the action of a belief is a spectrum, and then people in this case for example could eat less meat, matching the degree of belief “eating meat is not moral”.
I think the title could be a bit more specific like—“involving political party in science discussions might not be productive”, or something similar. If using the word “politics”, it would be crucial to define what “politics” here mean or refer to. The reason I say this is “politics” might not be just about actual political party’s power dynamics, but also includes general policy making, strategies, and history that aim to help individuals in the society, and many other aspects. These other types of things included in the word “politics” is crucial to consider when talking about many things (I think it is a little bit similar to precedents in law).
(Otherwise, if this article is about not bringing things all to political party level, I agree. I have observed that many things in the US at least are debated over political party lines, and if a political party debates about A, people reversely attribute some general social topics A to “political values or ideology”, which is false to me.)
I think by winning, he meant: “art of choosing actions that lead to outcomes ranked higher in your preferences”, though I don’t completely agree with this word choice of “winning” which could be ambiguous/causing confusion.
A bit unrelated, but more of a general comment on this—in my belief, I think people generally have unconscious preferences, and knowing/acknowledging these before weighing out preferences are very important, even if some preferences are short term.
I also had similar feelings on the simplicity part, and also how theory/idealized situation and execution could be very different. Also agree on the conflict part (and to me many different type of conflicts). And, I super super strongly support the section on The humans behind the numbers.
(These thoughts still persist after taking intro to EA courses).
I think EA’s big overall intentions are good to me and I am happy/energized by see how passionate people are comparing to no altruism at all at least; but the details/execution are not quite there to me.
I have been having some similar thoughts on the main points here for a while and thanks for this.
I guess to me what needs attention is when people do things along the lines of “benefit themselves and harm other people”. That harm has a pretty strict definition, though I know we may always be able to give borderline examples. This definitely includes the abuse of power in our current society and culture, and any current risks etc. (For example, if we are constraining to just AI with warning on content, https://www.iwf.org.uk/media/q4zll2ya/iwf-ai-csam-report_public-oct23v1.pdf. And this is very sad to see.) On the other hand, with regards to climate change (can also be current too) or AI risks, it probably should also be concerned when corporates or developers neglect known risks or pursue science/development irresponsibly. I think it is not wrong to work on these, but I just don’t believe in “do not solve the other current risks and only work on future risks.”
On some comments that were saying our society is “getting better”—sure, but the baseline is a very low bar (slavery for example). There are still many, many, many examples in different societies of how things are still very systematically messed up.
This is about basic human dignity and respect to other humans, and has nothing to do with politics.
Oxford languages (or really just after googling) says “rational” is “based on or in accordance with reason or logic.”
I think there are a lot of other types of definitions (I think lesswrong mentioned it is related to the process of finding truth). For me, first of all it is useful to break this down into two parts: 1) observation and information analysis, and 2) decision making.
For 1): Truth, but also particularly causality finding. (Very close to the first one you bolded, and I somehow feel many other ones are just derived from this one. I added causality because many true observations are not really causality).
For 2): My controversial opinion is everyone are probably/usually “rationalists”—just sometimes the reasonings are conscious, and other times they are sub/un-conscious. These reasonings/preferences are unique to each person. It would be dangerous in my opinion if someone try to practice “rationality” based on external reasonings/preferences, or reasonings/preferences that are only recognized by the person’s conscious mind (even if a preference is short term). I think a useful practice is to 1. notice what one intuitively want to do vs. what one think they should do (or multiple options they are considering), 2. ask why there is the discrepancy, 3. at least surface the unconscious reasoning, and 4. weigh things (the potential reasonings that leads to conflicting results, for example short term preference vs long term goals) out.
I don’t understand either. If it is meant what it meant, this is a very biased perception and not very rational (truth seeking or causality seeking). There should be better education systems to fix that.