Kinrany

Karma: 67

Kinrany 8 May 2023 8:58 UTC
1 point
0
in reply to: Kinrany’s comment on: If alignment problem was unsolvable, would that avoid doom?

it would make the problem more tractable

The problem of creating a strong AI and surviving, that is. We’d still get Hanson’s billions of self-directed EMs.

Kinrany 8 May 2023 8:51 UTC
1 point
0
in reply to: JBlack’s comment on: If alignment problem was unsolvable, would that avoid doom?
Thanks!

It has been explored (multiple times even on this site), and doesn’t avoid doom. It does close off some specific paths that might otherwise lead to doom, but not all or even most of them.

Do you have any specific posts in mind?

To be clear, I’m not suggesting that because of this possibility we can just hope that this is how it plays out and we will get lucky.

If we could find a hard limit like this, it seems like it would make the problem more tractable, however. It doesn’t have to exist simply because we want it to exist. Searching for it still seems like a good idea.

There’s a hundred problems to solve, but it seems like it could avoid the main bad scenario at least: that of AI rapidly self-improving. Improving its hardware wouldn’t be trivial for a human-level AI, and it wouldn’t have options present in other scenarios. And scaling beyond a single machine seems likely to be a significant barrier at least.

It could still create millions of copies of itself. That’s still a problem, but also still a better problem to have than a single AI with no coordination overhead.

[Question] If alignment problem was unsolvable, would that avoid doom?

Kinrany7 May 2023 22:13 UTC

3 points

3 comments1 min readLW link

Kinrany 11 Feb 2023 23:49 UTC
1 point
0
in reply to: evhub’s comment on: EigenKarma: trust at scale
This should be mitigated by pools of mutual trust that naturally form whenever there’s a loop in the trust graph.

Kinrany 21 Jan 2023 11:32 UTC
3 points
0
on: Recursive Middle Manager Hell
When the number of layers grows, the only thing that really works is metrics that cannot be goodhearted. Whenever those metrics exist, money becomes a perfectly good expression of success.

It might work to completely prohibit more than one layer of middle management. Instead, when middle manager Bob wants more people, he and his boss Alice come up with a contract that can’t be gamed too much. Alice spins out Bob’s org subtree into a new organization, and then it becomes Bob’s job to buy the service from the new org as necessary. Alice also publishes the contract, so that entrepreneurs can swoop in and offer a better/cheaper service.

Kinrany 8 Nov 2022 11:34 UTC
1 point
0
in reply to: Dagon’s comment on: Exams-Only Universities
The ability to put up with bullshit is valuable: bullshit cannot be ignored once it is reified into real world objects, documents, habits.

Kinrany 8 Jun 2022 19:31 UTC
4 points
3
in reply to: Daniel Kokotajlo’s comment on: AGI Ruin: A List of Lethalities
Markdown has syntax for quotes: a line with > this on it will look like

this

Kinrany 14 Apr 2022 12:15 UTC
9 points
in reply to: Eliezer Yudkowsky’s comment on: Lies Told To Children
Honestly “fiction” was enough of a spoiler. “As a child, we were always told that every sapient life is precious.” made it a certainty.

Kinrany 13 Apr 2022 5:26 UTC
1 point
on: Playing with DALL·E 2
Suggestion: “sangaku proving the Pythagoras’ theorem”. I wonder if it can do visual explanations.

Kinrany 29 Nov 2021 1:15 UTC
11 points
on: The bonds of family and community: Poverty and cruelty among Russian peasants in the late 19th century
Since Semyonova did not care to look at things from the peasants’ point of view and mixed her research with attempts to convert, I wonder how many of the things she recorded were directly intended to shock her.

Kinrany 4 Oct 2021 21:29 UTC
1 point
in reply to: pjeby’s comment on: The Best Software For Every Need
Hmm, I guess conflict resolution would be garbage, but simultaneous editing is rarely a good experience anyway. Otherwise storing and sharing text files using a file sync service is fairly good compared to other options. Thanks!

Kinrany 27 Sep 2021 14:10 UTC
4 points
in reply to: Tyrathalis’s comment on: The Best Software For Every Need
Thanks!

I wasn’t aware of Etherpad. Other Google Docs equivalents seemed impossible to self-host and extend, so a non-starter.

I agree with your overview:
- Etherpad provides collaborative editing, but integrating it with other services will probably take extra work
- Logseq has better structure, but worse automation
- Emacs can do most things on one computer, but rapid sharing is even harder

Kinrany 15 Sep 2021 22:10 UTC
4 points
in reply to: NunoSempere’s comment on: The Best Software For Every Need
Pieces for a general purpose personal computing system. Ideally:
- Edit data by hand
- Store as plain text
- Self-host, access from any device
- Write formulas to derive data automatically
- Mix and match structured data (markdown, tables, nested lists, whiteboard)
- Search and navigate, like any wiki
- Automate through a web API and webhooks
- Collaborate in real time

Kinrany 14 Aug 2021 0:12 UTC
7 points
on: Slack Has Positive Externalities For Groups
The strict divide between high slack and low slack reminds me of synchronous and asynchronous companies: hybrids seem to work poorly.

Kinrany 20 Jul 2021 10:10 UTC
2 points
in reply to: Mo Putera’s comment on: Compositionality: SQL and Subways
Seven sketches in compositionality explores compositionality (category theory, really) with examples:
- Dish recipes
- Chemistry, resource markets and manufacturing
- Relational database schemas and data migrations
- Projects and teams with conflicting design trade-offs
- Cyber-physical systems, signal flow graphs, circuits

Kinrany 25 Feb 2021 3:20 UTC
1 point
in reply to: asdfrty6’s comment on: Recruiting for esports experiment
I mean, it’s easier to find two people willing to play than ten. So you’ll get more data. With one or two teams it will be hard to draw any conclusions at all.

Kinrany 17 Feb 2021 0:50 UTC
3 points
on: Recruiting for esports experiment
It seems picking a 1v1 game would work better as an experiment.

Kinrany 8 Nov 2020 12:51 UTC
4 points
in reply to: johnswentworth’s comment on: When Money Is Abundant, Knowledge Is The Real Wealth
Caveat: ask each person to name someone they personally worked with.

Hard to get right, but not sure whether it’s harder than knowledge investment.

Wouldn’t have helped Louis XV. We might need infrastructure in place that would incentivize people to make themselves easy to find.

Kinrany 6 Nov 2020 19:28 UTC
2 points
on: When Money Is Abundant, Knowledge Is The Real Wealth
Is it really true that money can’t buy knowledge?

We can ask the most knowledgeable person we know to name the most knowledgeable person they know, and do that until we find the best expert. Or alternatively, ask a bunch of people to name a few, and keep walking this graph for a while.

This won’t let us buy knowledge that doesn’t exist, but seems good enough for learning from experts, given enough money and modern communication technology that Louis XV didn’t have.

Kinrany 6 Nov 2020 19:15 UTC
3 points
in reply to: DirectedEvolution’s comment on: When Money Is Abundant, Knowledge Is The Real Wealth
I suspect being good at finding better scientists is very close to having a complete theory of scientific advancement and being able to automate the research itself.