Could you integrate Blackbelt with Anki or another spaced repetition framework? Someone made this set of anki cards on AI alignment sometime ago, which I have found to be a useful resource: https://www.ai-alignment-flashcards.com/decks
krs
Karma: 8
Self-Administered Gell-Mann Amnesia
I think a useful heuristic for updating beliefs is to ask yourself “What would make this belief false?” rather than casting the issue in the framework of confirmation vs. balance. To make this concrete, consider the example of flat earthers vs. scientists. If you believed in a flat earth, there are any number of tests you could do to (e.g. watching ships sail down below the horizon) that would lead you to update towards falsifying your belief. This type of information seeking is neutral with respect to confirming your beliefs. This also allows us to look for more direct evidence around our beliefs rather than appealing to indirect methods such as whether or not a person agrees with us (see hug the query).
Second, I haven’t looked into the work of Weijie Zhong, but I was wondering if there might be a bias variance tradeoff at play here for efficient information seeking (i.e. obtaining only confirmatory evidence seems likely lead to low variance but high bias)?
I assumed that if you are flipping a coin, trials would be independent events and each flip would have a fixed rule (which is what happens when you flip a single coin). Instead, the coin had a different rule for odd and even numbered flips. I think that the language of the website should be amended to reflect this.
My tentative viewpoint on this is that the preference one has over value drift from AI vs. human made value drift comes from an entity’s ability to experience joy and suffering.
In the context of AI safety, many humans who experience mostly positive lives could be killed or made to suffer at the hands of superintelligent AI in the future, and the experience of the AI in terms of pain and suffering is mostly unknown. I’m worried that an AI will optimize for something that does not lead to any subjective increase of wellbeing in the universe at the cost of human happiness.