lincolnquirk

Karma: 1,536

lincolnquirk 24 Nov 2022 13:58 UTC
3 points
0 ∶ 0
on: LW Beta Feature: Side-Comments
Excited about this!

Points of feedback:
1. I don’t like to have to scroll my screen horizontally to read the comment. (I notice there’s a lot of perfectly good unused white space on the left side; comments would probably fit horizontally if you pushed everything to the left!)
2. Sometimes when you mouse over the side-comment icon, it tries to scroll the page to make the comment readable. This is very surprising and makes me lose my place.
3. Hovering over the icon makes the comment appear briefly. If I then want to scroll in order to read the comment, there seems to be no way to ‘stay hovered’—I have to click and toggle it, to make the comment stick around so I can actually read it. (This plus being forced to scroll the screen makes the hover feature kind of useless.)
Overall, feeling optimistic though, and will probably use this.

lincolnquirk 11 Oct 2022 13:15 UTC
2 points
0 ∶ 0
in reply to: cveres’s comment on: Don’t expect AGI anytime soon
I think your argument is wrong, but interestingly so. I think DL is probably doing symbolic reasoning of a sort, and it sounds like you think it is not (because it makes errors?)

Do you think humans do symbolic reasoning? If so, why do humans make errors? Why do you think a DL system won’t be able to eventually correct its errors in the same way humans do?

My hypothesis is that DL systems are doing a sort of fuzzy finite-depth symbolic reasoning—it has capacity to understand the productions at a surface level and can apply them (subject to contextual clues, in an error-prone way) step by step, but once you ask for sufficient depth it will get confused and fail. Unlike humans, feedforward neural nets can’t think for longer and churn step by step yet; but if someone were to figure out a way to build a looping option into the architecture then I won’t be surprised to see DL systems which can go a lot further on symbolic reasoning than they currently do.

lincolnquirk 4 Oct 2022 19:31 UTC
2 points
0 ∶ 0
on: Will you let your kid play football?
What is Pop Warner in this context? I have googled it and it sounds like he was one of the founders of modern American football, but I don’t understand what it is in contrast to. Is there some other (presumably safer) ruleset?

lincolnquirk 10 Sep 2022 1:07 UTC
36 points
8 ∶ 0
in reply to: habryka’s comment on: My emotional reaction to the current funding situation
(Inside-of-door-posted hotel room prices are called “rack rates” and nobody actually pays those. This is definitely a miscommunication.)

lincolnquirk 19 Aug 2022 0:39 UTC
8 points
3 ∶ 0
on: What does moral progress consist of?
I am guilty of being a zero-to-one, rather than one-to-many, type person. It seems far easier and more interesting to me, to create new forms of progress of any sort, rather than convincing people to adopt better ideas.

I guess the project of convincing people seems hard? Like, if I come up with something awesome that’s new, it seems easier to get it into people’s hands, rather than taking an existing thing which people have already rejected and telling them “hey this is actually cool, let’s look again”.

All that said, I do find this idea-space intriguing partly thanks to this post—it makes me want to think of ways of doing more one-to-many type stuff. I’ve been recently drawn into living in DC and I think the DC effective altruism folks are much more on the one-to-many side of the world.

lincolnquirk 9 Aug 2022 21:00 UTC
12 points
1 ∶ 0
on: Proposal: Consider not using distance-direction-dimension words in abstract discussions
Upvoted for raising something to conscious attention, that I have never previously considered might be worth paying attention to.

(Slightly grumpy that I’m now going to have a new form of cognitive overhead probably 10+ times per day… these are the risks we take reading LW :P)

lincolnquirk 26 Jul 2022 2:33 UTC
3 points
0 ∶ 0
on: How Promising is Theoretical Research on Rationality? Seeking Career Advice
Look, I don’t know you at all. So please do ignore me if what I’m saying doesn’t seem right, or just if you want to, or whatever.

I’m a bit worried that you’re seeking approval, not advice? If this is so, know that I for one approve of your chosen path. You are allowed to spend a few years focusing on things that you are passionate about, which (if it works out) may result in you being happy and productive and possibly making the world better.

If you are in fact seeking advice, you should explain what your goal is. If your goal is to make the maximum impact possible — it’s worth at least hundreds of hours trying to see if you can learn more & motivate yourself along a path which seems like it combines high impact with personal resonance. I wouldn’t discount philosophy along this angle, but (for example) it sounds like you may not know that much about the potential of policy careers; there are plenty that do not require particularly strong mathematical skills (… or even any particularly difficult skills beyond some basic extraversion, resistance to boredom and willingness to spend literal decades grinding away within bureaucracies).

If your goal is to be happy, I think you will be happy doing philosophy, and I think you have a potential to make a huge impact that way. Certainly there are a decent number of full-time philosophers within effective altruism who I have huge respect for (Macaskill, Ord, Bostrom, Greaves, and Trammell jump to mind). Plus, you can save a few hundred hours, which seems pretty important if you might already know the outcome of your experimentation!

lincolnquirk 18 Jul 2022 2:54 UTC
2 points
0 ∶ 0
in reply to: Quintin Pope’s comment on: Why I Think Abrupt AI Takeoff
Thanks! This is very helpful, and yes, I did mean to refer to grokking! Will update the post.

Why I Think Abrupt AI Takeoff

lincolnquirk17 Jul 2022 17:04 UTC

14 points

6 comments1 min readLW link

lincolnquirk 28 Jun 2022 21:18 UTC
16 points
5 ∶ 0
on: What Are You Tracking In Your Head?
Nice post!

One of my fears is that the True List is super long, because most things-being-tracked are products of expertise in a particular field and there are just so many different fields.

Nevertheless:
- In product/ux design, tracking the way things will seem to a naive user who has never seen the product before.
- In navigation, tracking which way north is.
- I have a ton of “tracking” habits when writing code:
  - types of variables (and simulated-in-my-head values for such)
  - refactors that want to be done but don’t quite have enough impetus for yet
  - loose ends, such as allocated-but-not-freed resources, or false symmetry (something that looks like it should be symmetric but isn’t in some critical way), or other potentially-misleading things that need to be explained
  - [there are probably a lot more of these that I am not going to write down now]

lincolnquirk 24 Jun 2022 18:30 UTC
5 points
1 ∶ 0
on: Let’s See You Write That Corrigibility Tag
Here’s my attempt. I haven’t read any of the other comments or the tag yet. I probably spent ~60-90m total on this, spread across a few days.

On kill switches
- low impact somehow but I don’t know how
- Go slow enough so that people can see what you’re doing
- Have a bunch of “safewords” and other kill-switches installed at different places, some hopefully hard-to-reach by the AI. Test them regularly, and consider it a deadly flaw if one stops working.
On the AI accurately knowing what it is doing, and pointing at things in the real world
- watch all the metrics (!)
- Predict all the metrics you watch, and ask humans about any anomalous metrics that you are watching
- group inputs and outputs separately and treat inputs as sacred. Perhaps by having an epistemic module which is incentivized by producing true predictions about the world. Make the epistemic module the one that gets all the juice. Planning module should not have any way to influence it.
On responding predictably
- Require inputs of some kind to get more outputs (something about control theory?) - like a power steering wheel, or an ebike, it helps the human by amplifying the motions you make but not actually Doing Anything on its own
- Have metrics look smooth on an input/output response curve. No sharp edges. let the humans be the one to turn the knobs.
- Try things. Try things on a small scale before going big. (define “big”? maybe look at the metrics!)
On epistemology and ontology
- Ask for feedback both about things you tried; and experiments you want to run
- make the epistemic module have a stable ontology. If it wants to change the ontology, it needs to create a new epistemic module, and leave the old one running. The new module has to be able not only to predict physical things about the world but also what the old model would say about things being pointed-to and tagged as important in the old models ontology.
- have a concept of goal metrics and side effects. Try to predict what the human will say about which metrics are goals and which are side effects. (hmm..this is incoherent because you can come up with many irrelevant metrics… what even is a relevant metric. Maybe the system should be grouping metrics using some kind of natural unsupervised clustering system and then trying to predict the prioritization of metric groups? And then to explain the groups it gives samples of the actual thing it’s measuring. Still difficult to make sense of this. )
On planning
- Bound or penalize plans in complexity and time.
  - “complexity” is hard to define here, and isn’t Kolmogorov I don’t think.
- Harshly penalize plans which lean on predicting what other agents will do. Ban plans which depend on prediction of what your operator will do. (AI can still predict what the operator will think of the plan, but that prediction should not feed into any plan itself.)
- Separate planning from other parts of AI cognition, such that you are never confused about whether you are making a plan or doing other things, such as predicting things about the world, or executing a plan.
- Don’t execute plans at all
- Train the AI to identify the purpose(s) of each step in the plan. Give negative feedback when steps lead to outcomes that are not identified ahead of time.
On transparency
- A part of the AI should be trained to explicate what the rest of the AI is “doing”/focused on at all times. Find a way to make this into a feedback cycle.
- AI should be arranged so it is mostly only doing one thing at a time, and that doesn’t switch its focus too many times each minute.

lincolnquirk 13 Jun 2022 18:13 UTC
12 points
in reply to: GuySrinivasan’s comment on: On A List of Lethalities
I notice that I am extremely surprised by your internship training. Its existence, its lessons and the impact it had on you (not you specifically, just a person who didn’t come in with that mindset) are all things I don’t think I would have predicted. I would be thrilled if you would write as much as you can bring yourself to about this, braindump format is fine, into a top level post!

lincolnquirk 10 Jun 2022 22:58 UTC
5 points
on: Leaving Google, Joining the Nucleic Acid Observatory
Congrats, I’m excited about this!

lincolnquirk 28 Mar 2022 3:35 UTC
15 points
on: Consequentialist veganism
I’ve been turning this over in my head for a while now. (Currently eating mostly vegan fwiw, but I am not sure if this is the right decision.)

I think the main argument against veganism is that it actually incurs quite a large cost. Being vegan is a massive lifestyle change with ripple effects that extend into one’s social life. This argument falls under your “there are higher-impact uses of your (time/energy/money/etc.)”, but what you wrote doesn’t capture the reasons why this is important.

most of us do not have good reason to treat this as a zero-sum game in which each attempt to do good in the world must crowd out another. For one thing, we’re nowhere near putting all available resources into our efforts to do good, so we can simply choose to expand that budget.

I am reminded of Zvi’s Slack post (ctrl+f “afford”). Attention is a very scarce resource. If I am spending all my attention on important things, I cannot also spend attention on creating a whole new diet, finding friends who won’t mock me, learning how to cook all new things, etc. On the other hand, animal welfare offsets are probably quite cheap.

For another, our psychology is complicated, and making a moral effort can just as easily increase our capacity to make further such efforts as deplete it.

Indeed, and this is actually why I have become mostly vegan in recent months; but it is not going to be true for everyone. My current decision to eat mostly vegan except when inconvenient feels somehow indulgent.

I wrote more about how I am trying to be vegan: http://www.lincolnquirk.com/2022/02/15/vegan.html

lincolnquirk 30 Jan 2022 16:50 UTC
−9 points
on: Where can I read about the Roko’s Basilisk drama?
This community has a virtue of taking weird ideas seriously. Roko came up with a weird idea which, the more seriously you took it, the more horrifying it became. This was deemed an info hazard, and censored in some way, I don’t know how. But the people who didn’t take it seriously in the first place weren’t horrified by the idea and thus were confused about why it should have been censored, and thus boosted the Streisand effect.

lincolnquirk 29 Jan 2022 18:09 UTC
2 points
on: Mildly Photochromic Lenses?
I had photochromics for several years. I found them mildly-helpful-and-mostly-unobjectionable in the summer, but ridiculously annoying in the winter (when they both tend to be darker because of low-altitude sun, and the temperature makes them clear up slower once you move inside).

Also, I was relentlessly mocked by the fashion police. :P

Ultimately I moved away from them.

lincolnquirk 21 Jan 2022 21:06 UTC
13 points
on: Consume fiction wisely
I downvoted this. I usually like the concise writing style exhibited in this essay (similar to lsusr, paul graham, both of whom I like) , but I apparently only like it when I think it’s correct. :P

I especially downvoted because I think it is fairly likely to attract low-quality discussion. A differently-written version of a similar but perhaps more nuanced point, with better fleshed-out examples of why given works are net helpful or net harmful, would be a better post. I am sympathetic to the general idea of the post!

lincolnquirk 27 Dec 2021 18:16 UTC
1 point
on: Why did computer science get so galaxy-brained?
I think there’s something about programming that attracts the right sort of people. What could that be? Well, programming has very tight feedback loops, which make it fun. You can “do a lot”: one’s ability to gain power over the universe, if you will, is quite high with programming. I’d guess a combination of these two factors.

lincolnquirk 27 Dec 2021 16:32 UTC
10 points
on: A good rational fiction about IT-inspired magic system?
The Wizard’s Bane series by Rick Cook. The basic idea is great: a Silicon Valley programmer is transported into a magical universe where he has to figure out how to apply programming to a magic system. Caveat lector: the writing is not the best quality, it’s a bit juvenile, but still a light, enjoyable read :)

lincolnquirk 21 Dec 2021 18:06 UTC
4 points
on: Experiences raising children in shared housing
This is great! Thanks for sharing!