Esben Kran

Karma: 470

Esben Kran 18 Mar 2022 17:09 UTC
1 point
in reply to: MondSemmel’s comment on: [Beta Feature] Google-Docs-like editing for LessWrong posts
The shortcuts with Ctrl + Alt generally work sub-optimally for EU keyboards, I believe. Seems worth it to test alternative keyboard layouts for usability—something that most major software companies in the U.S. do not seem to do well ;-)

Esben Kran 18 Mar 2022 17:20 UTC
4 points
in reply to: mako yass’s comment on: [Beta Feature] Google-Docs-like editing for LessWrong posts
Agreed, new ways of ordering thoughts online is an awesome opportunity on LessWrong!
Foot notes: I’d say we already have quite a strong Roam-style organization in the hover pop-ups of LessWrong post links but as you say, it would be very nice to have a version of that for foot-notes, given their disparate nature.
Collapsible boxes: Having “fact boxes” as expandable interactive elements seems like a very good idea as well and relatively cheap to implement. I recommend looking at e.g. Hugo XML syntax for these things (XML is a pain, you can probably figure a better writing UX out).
Interactive documents: I’m more for the Obsidian thought representation but the hierarchy in Roam is quite relevant for communicating thoughts to others. I think a strong representation of this is quite relevant and might be an in-line, minimalist, recursive “collapsible box” as described above.

Esben Kran 18 Mar 2022 17:38 UTC
16 points
on: [Beta Feature] Google-Docs-like editing for LessWrong posts
I absolutely love this feature set! I really appreciate the markdown to formatting (Obsidian-style), live collaboration and LaTeX rendering. I think this also creates fantastic opportunities for innovations in communicating thoughts through LessWrong. Here’s a list of my feedback:
Shortcuts: One feature I really enjoy in Google Docs is more deep text editing keyboard shortcuts than the standard set. One of my favourites is the Alt + Shift + Arrow up or down that replaces the current paragraph with the one above or below, effectively moving the paragraph in the text. Obsidian has a keyboard shortcuts editor that is quite nice as well but could have more (see here, page 2). I think shortcuts can make writing as seamless as some code editors (not thinking VIM here, but unintrusive extra editor tooling). Additionally, as I mention elsewhere, some (I just know European) keyboards do not take well to Ctrl + Alt + [] shortcuts like Ctrl + Alt + M, especially on the web.
Collapsible boxes: Having “fact boxes” as expandable interactive elements seems like a very good idea as well and relatively cheap to implement. I recommend looking at e.g. Hugo XML syntax for these things (XML is a pain, you can probably figure a better writing UX out).
Interactive documents: I’m more for the Obsidian thought representation but the hierarchy in Roam is quite relevant for communicating thoughts to others. I think a strong representation of this is quite relevant and might be an in-line, minimalist, recursive “collapsible box” as described above.
Foot notes: I’d say we already have quite a strong Roam-style organization in the hover pop-ups of LessWrong post links and it would be very nice to have a version of that for foot-notes, given their disparate nature. I.e. hover = shows that specific footnote.
Live preview & 100% keyboard editing: See Obsidian’s implementation of this. The markdown to formatting feature is already super awesome is their LaTeX editing that does in-line MathJax with $ wrapping and block with $$ wrapping. The current LessWrong LaTeX editor requires me to use the cursor, AFAIK. Obsidian is also pretty good at minimalist formatting rulesets for inspiration. Check out their vast plugins library for inspiration as well. I’m sure there’s some absolute text editor gold in there.
Collaborative editing extension: This makes or breaks our usage of LessWrong as the editing platform of choice. It would be awesome to have editing group settings, i.e. so I don’t have to share every article with Apart Research members but can have a Google Drive-style folder sharing for blog post edits. Otherwise, we have to maintain a collection of links in a weird format somewhere.
I also echo the other comments’ feedback points. However overall, absolutely marvelous work! Really looking forward to the developments on this!

Esben Kran 18 Mar 2022 23:20 UTC
2 points
in reply to: mako yass’s comment on: [Beta Feature] Google-Docs-like editing for LessWrong posts
I mostly use it from the computer so that missed me but it seems like a very good idea as well!

Esben Kran 28 Mar 2022 1:23 UTC
3 points
in reply to: Ben Pace’s comment on: Your specific attitudes towards AI safety
Updated, thank you. We were unsure if it was best to keep it vague as to not bias the responses, but it is true that you need slight justification and that it is net positive to provide the explanation.

Esben Kran 28 Mar 2022 3:08 UTC
2 points
in reply to: Ben Pace’s comment on: Your specific attitudes towards AI safety
Thank you for the valuable feedback!
We did indeed user test with 3-4 people within our target group and changed the survey quite a bit in response to these. I definitely believe the “basic quality” to be met but I appreciate your reciprocation either way.
Response to your feedback, in order: The career options are multiple-choice, i.e. you may associate yourself with computer science and academia and not industry which is valuable information in this case. It means we need less combinations of occupational features. From our perspective, the number is easier to estimate while time estimates (both in the future and past) generally are less accurate. In a similar vein, it’s fine that LessWrong blog posts each count as one due to their inherently more in-depth nature. I’ll add the “pick the nearest one”. The ranking was expected to feel like that which is in accordance with our priors on the question. There are reasons for most questions’ features as well.
And thank you, I believe the same. Compared to how data-driven we believe we are, there is very little data from within the community, as far as I see it. And we should work harder to positively encourage more action in a similar vein.

Esben Kran 28 Mar 2022 5:20 UTC
2 points
in reply to: TLW’s comment on: Your specific attitudes towards AI safety
Thank you for your perspective on the content! The subjectivity is a design feature since many of the questions are related to the subjective “first exposure”, the subjective concern or subjective probabilities. This also relates to how hard it will be to communicate the same questions for public audiences without introducing a substantial amount of subjectivity. The importance of X-risks is not ranked on a 1-10 scale so that should not be a problem. The 100 forum posts != 100 books is addressed above and is meant to be a highly fuzzy measure. By equating the different modalities, we accept the fuzziness of the responses. AI safety definitional subjectivity, that’s also fine in relation to the above design thoughts. For the rankings, this is also fine and up to the survey recipient themselves. I’ll add that the probabilities should be thought of independently, given the assumptions necessary for it to happen.
Sorry to hear that you did not wish to answer but thank you for presenting your arguments for why.

Esben Kran 28 Mar 2022 13:41 UTC
1 point
in reply to: TLW’s comment on: Your specific attitudes towards AI safety
Haha true, but the feature is luckily not a systematic blockade, more of an ontological one. And sorry for misinterpreting! On another note, I really do appreciate the feedback and With Folded Hands definitely seems within-scope for this sort of answer, great book.

Esben Kran 28 Mar 2022 13:43 UTC
1 point
in reply to: ViktoriaMalyasova’s comment on: Your specific attitudes towards AI safety
Basically, the original intention was “identify” because you might work as a web developer but identify and hobby-work as an academic. But it’s a bit arbitrary so we updated it now.

Esben Kran 28 Mar 2022 13:48 UTC
1 point
in reply to: Vanessa Kosoy’s comment on: Your specific attitudes towards AI safety
It is written as independent probabilities so it’s just your expectation, given that governments develop AGI, that AGI is a net positive. So it’s additive to the “AGI will lead to a net positive future”. You would expect the averages of the “large companies” and “nation state” arguments to average to the “AGI will lead...” question.

Esben Kran 28 Mar 2022 17:04 UTC
2 points
in reply to: Steven Byrnes’s comment on: Your specific attitudes towards AI safety
True, sorry about that! And your interpretation is our interpretation as well however, the survey is updated to a different framing now.

Esben Kran 1 Apr 2022 16:38 UTC
1 point
in reply to: Ben Pace’s comment on: Your specific attitudes towards AI safety
After the fact, I realize that I might have replied too defensively to your feedback as a result of the tone so sorry for that!
I sincerely do thank you for all the points and we have updated the survey with a basis in this feedback, i.e. 1) career = short text answer, 2) knowledge level = which of these learning tasks have you completed and 4) they are now independent rankings.

Esben Kran 1 Apr 2022 16:42 UTC
3 points
in reply to: TLW’s comment on: Your specific attitudes towards AI safety
As I also responded to Ben Pace, I believe I replied too intellectually defensively to both of your comments as a result of the tone and would like to rectify that mistake. So thank you sincerely for the feedback and I agree that we would like not to exclude anyone unnecessarily nor have too much ambiguity in the answers we expect. We have updated the survey as a result and again, excuse my response.

Esben Kran 19 Apr 2022 21:38 UTC
2 points
on: Code Generation as an AI risk setting
Awesome post, love the perspective. I’ve generally thought in these lines as well and it was some of the most convincing arguments for working in AI safety when I switched (“Copilot is already writing 20% of my code, what will happen next?”).
I do agree with other comments that Oracle AI takeover is plausible but will say that a strong code generation tool seems to have better chances and to me seem to arrive parallel to conscious chatbots, i.e. there’s currently more incentive to create code generation tools than chatbots and the chatbots that have virtual assistant-like capabilities seem easier to make as code generation tools (e.g. connecting to Upwork APIs for posting a job).
And as you well mention, converting engineers is much easier with this framing as well and allows us to relate better to the field of AI capabilities, though we might want to just add it to our arsenal of argumentation rather than replace our framing completely ;) Thank you for posting!

Esben Kran 20 Apr 2022 23:30 UTC
4 points
on: [Intro to brain-like-AGI safety] 12. Two paths forward: “Controlled AGI” and “Social-instinct AGI”
Strong upvote for including more brain-related discussions in the alignment problem and for the deep perspectives on how to do it!
Disclaimer: I have not read the earlier posts but I have been researching BCI and social neuroscience.
I think there’s a Steering Subsystem circuit upstream of jealousy and schadenfreude; and
I think there’s a Steering Subsystem circuit upstream of our sense of compassion for our friends.
It is quite crucial to realize that these subsystems probably are very differently organized, are at very different magnitudes, and have very different functions in every different human. Cognitive neuroscience’s view of the modular brain seems (by the research) to be quite faulty and computational/complexity neuroscience generally seem more successful and are more concerned with reverse-engineering the brain, i.e. identifying neural circuitry associated with different evolutionary behaviours. This should inform how we cannot find “the Gaussian brain” and implement it.
You also mention in post #3 that you have the clean slate vs. pre-slate systems (learning vs. steering subsystems) and a less clear distinction here might be helpful instead of modularization. All learning subsystems are inherently organized in ways that evolutionarily seem to fit into that learning scheme (think neural network architectures) which is in and off itself another pre-seeded mechanism. You might have pointed this out in earlier posts as well, sorry if that’s the case!
I think I’m unusually inclined to emphasize the importance of “social learning by watching people”, compared to “social learning by interacting with people”.
In general, it seems babies learn quite a lot better from social interaction than pure watching. And between the two, they definitely learn better if they have something to imitate what they’re seeing upon. There’s definitely a good point in the speed differentials between human and AGI existence and I think exploring the opportunity of building a crude “loving family” simulator might not be a bad idea, i.e. have it grow up at its own speed in an OpenAI Gym simulation with relevant modalities of loving family.
I’m generally pro the “get AGI to grow up in a healthy environment” but definitely with the perspective that this is pretty hard to do with the Jelling Stones and that it seems plausible to simulate this either in an environment or with pure training data. But the point there is that the training data really needs to be thought of as the “loving family” in its most general sense since it indeed has a large influence on the AGI’s outcomes.
But great work, excited to read the rest of the posts in this series! Of course, I’m open for discussion on these points as well.

Esben Kran 21 Apr 2022 23:46 UTC
4 points
in reply to: Steven Byrnes’s comment on: [Intro to brain-like-AGI safety] 12. Two paths forward: “Controlled AGI” and “Social-instinct AGI”
Thank you for the comprehensive response!
To be clear, you & I are strongly disagreeing on this point.
It seems like we are mostly agreeing in the general sense that there are areas of the brain with more individual differentiation and areas with less. The disagreement is probably more in how different this jealousy is exhibited as a result of the neocortical part of the circuit you mention.
And I endorse that criticism because I happen to be a big believer in “cortical uniformity” [...] different neural architectures and hyperparameters in different places and at different stages [...].
Great to hear, then I have nothing to add there! I am quite inclined to believe that the neural architecture and hyperparameter differences are underestimated as a result of Brodmann areas being a thing at all, i.e. I’m a supporter of the broad cortical uniformity argument but against the strict formulation that I feel is way too prescriptive given our current knowledge of the brain’s functions.
Yeah, maybe see Section 2.3.1, “Learning-from-scratch is NOT “blank slate””.
And I will say I’m generally inclined to agree with your classification of the brain stem and hypothalamus.
That sounds great but I don’t understand what you’re proposing. What are the “relevant modalities of loving family”? I thought the important thing was there being an actual human that could give feedback and answer questions based on their actual human values, and these can’t be simulated because of the chicken-and-egg problem.
To be clear, my baseline would also be to follow your methodology but I think there’s a lot of opportunity in the “nurture” approach as well. This is mostly related to the idea of open-ended training (e.g. like AlphaZero) and creating a game-like environment where it’s possible to train for the agent. This can to some degree be seen as a sort of IDA proposal since your environment will need to be very complex (e.g. have other agents that are kind or other “aligned trait”, possibly trained from earlier states).
With this sort of setup, the human-giving-feedback is the designer of the environment itself, leading to a form of scalable human oversight probably iterating over many environments and agents, i.e. the IDA part of the idea. And again, there are a lot of holes in this plan, but I feel like it should not be dismissed outright. This post should also inform this process. So a very broad “loving family” proposal, though the name itself doesn’t seem adequate for this sort of approach ;)

Esben Kran 22 Apr 2022 22:26 UTC
5 points
in reply to: Steven Byrnes’s comment on: [Intro to brain-like-AGI safety] 12. Two paths forward: “Controlled AGI” and “Social-instinct AGI”
[...]Still, both the reward function and the RL algorithm are inputs into the adult’s jealousy-related behavior[...]
I probably just don’t know enough about jealousy networks to comment here but I’d be curious to see the research here (maybe even in an earlier post).
Does anyone believe in “the strict formulation”?
Hopefully not, but as I mention, often a too-strict formulation imh.
[...]first AGI can hang out with younger AGIs[...]
More the reverse. And again, this is probably taking it farther than I would take this idea but it would be pre-AGI training in an environment with symbolic “aligned” models, learning the ropes from this, being used as the “aligned” model in the next generation and so on. IDA with a heavy RL twist and scalable human oversight in the sense that humans would monitor rewards and environment states instead of providing feedback on every single action. Very flawed but possible.
RE: RE:
- Yeah, this is a lot of what the above proposal was also about.
[...] and the toddler gets negative reward for inhibiting the NPCs from accomplishing their goals, and positive reward for helping the NPCs accomplish their goals [...]
As far as I understand from the post, the reward comes only from understanding the reward function before interaction and not after which is the controlling factor for obstructionist behaviour.
- Agreed, and again more as an ingredient in the solution than an ends in itself. BNN OOD management is quite interesting so looking forward to that post!

Esben Kran 31 May 2022 15:40 UTC
2 points
on: Reshaping the AI Industry
This is a wonderful piece and echoes many sentiments I have with the current state of AI safety. Lately, I have also thought more and more about the technical focus’ limitations in the necessary scope to handle the problems of AGI, i.e. the steam engine was an engineering/tinkering feat loong before it was described technically/scientifically and ML research seems much the same. When this is the case, focusing purely on hard technical solutions seems less important than focusing on AI governance or prosaic alignment and not doing this, as echoed in other comments, might indeed be a pitfall of specialists, some of which are also warned of here.

Esben Kran 31 May 2022 15:58 UTC
3 points
on: The Plan
In general a great piece. One thing that I found quite relatable is the point about the preparadigmatic stage of AI safety going into later stages soon. It feels like this is already happening to some degree where there are more and more projects readily available, more prosaic alignment and interpretability projects at large scale, more work done in multiple directions and bigger organizations having explicit safety staff and better funding in general.
With these facts, it seems like there’s bound to be a relatively big phase shift in research and action within the field that I’m quite excited about.

Esben Kran 25 Aug 2022 19:19 UTC
4 points
−4
on: Common misconceptions about OpenAI
Thank you very much for writing this post, Jacob. I think it clears up several of the misconceptions you emphasize.
I generally seem to agree with John that the class of problems OpenAI focuses on might be more capabilities-aligned than optimal but at the same time, having a business model that relies on empirical prosaic alignment of language models generates interesting alignment results and I’m excited for the alignment work that OpenAI will be working on!