RHollerith

Karma: 2,530

Richard Hollerith. 15 miles north of San Francisco. hruvulum@gmail.com

My probability that AI research will end all human life is .92. It went up drastically when Eliezer started going public with his pessimistic assessment in April 2022. Till then my confidence in MIRI (and knowing that MIRI has enough funding to employ many researchers) was keeping my probability down to about .4. (I am glad I found out about Eliezer’s assessment.)

Currently I am willing to meet with almost anyone on the subject of AI extinction risk.

Last updated 26 Sep 2023.

RHollerith 14 Jun 2013 2:23 UTC
48 points
in reply to: Scott Alexander’s comment on: After critical event W happens, they still won’t believe you

I know that there were times when it was very controversial whether computers would ever be able to beat humans in chess

Douglas Hofstadter being one on the wrong side: well, to be exact, he predicted (in his book GEB) that any computer that could play superhuman chess would necessarily have certain human qualities, e.g., if you ask it to play chess, it might reply, “I’m bored of chess; let’s talk about poetry!” which IMHO is just as wrong as predicting that computers would never beat the best human players.
What links here?
- gwern's comment on The first AI probably won’t be very smart by Jonathan Paulson (18 Jan 2014 4:37 UTC; 4 points)

RHollerith 21 Sep 2010 3:44 UTC
32 points
on: Less Wrong Should Confront Wrongness Wherever it Appears

Less Wrong has the highest accuracy and signal to noise ratio of any blog I’ve seen, other than those that limit themselves to narrow specialties

This is an argument for being very protective of the s/n ratio here, and to “confront wrongness wherever it appears” is the opposite of being protective.

I recommend our delaying the ambitious worthwhile goal you advocate till we have better mechanisms for protecting the quality of a conversation.

RHollerith 23 Sep 2010 8:57 UTC
28 points
in reply to: waitingforgodel’s comment on: Open Thread: July 2010, Part 2
No one asked or forced Roko to leave Less Wrong. Wounded by Eliezer’s public reprimand, Roko deleted all his comments and said that he is leaving.

(I for one wish he would come back. He was a valuable contributor.)

RHollerith 28 Apr 2023 20:30 UTC
26 points
5
on: Transcript and Brief Response to Twitter Conversation between Yann LeCunn and Eliezer Yudkowsky
Not knowing anything about LeCun other than this twitter conversation, I’ve formed a tentative hypothesis: LeCun is not putting a lot of effort into the debate: if you resort to accusing your opponent of scaring teenagers, it is usually because you feel you do not need to try very hard and you feel that almost any argument would do the job. Not trying very hard and not putting much thought behind one’s words is in my experience common behavior when one perceives oneself as being higher in status than one’s debate opponents. If a total ban on training any AI larger than GPT-4 starts getting more political traction, I expect that LeCun would increase his estimate of the status of his debating opponents and would increase the amount of mental effort he puts into arguing his position, which would in turn make it more likely that he would come around to the realization that AI research is dangerous.

I’ve seen people here express the intuition that it is important for us not to alienate people like LeCun by contradicting them too strongly and that it is important not to alienate them by calling on governments to impose a ban (rather than restricting our efforts to persuading them to stop voluntarily). Well, I weakly hold the opposite intuition for the reasons I just gave.

RHollerith 1 Mar 2023 2:45 UTC
24 points
10
on: Enemies vs Malefactors

I suggest minting a new word, for people who have the effects of malicious behavior, whether it’s intentional or not.

I’ve long used “destructive” for that.

RHollerith 11 May 2011 14:24 UTC
24 points
in reply to: lukeprog’s comment on: Scholarship: How to Do It Efficiently
Oh, what fun! I get to correct the new guy that everyone admires :)

Do not take the suggestion in parent because it would take time away from the time-tested way of testing one’s competence with technical material: doing calculations and proving theorems. About half of the work that goes into the production of a technical textbook should go into the creation of exercises that ask the reader to prove theorems and do calculations. There is in fact a well-regarded series of textbook supplements called Schaum’s Outlines that are nothing but exercises.

Although the most effective learners will tend to spend a lot of their learning time proving theorems and doing calculations of their own choosing, it is important for the student of a technical subject to own textbooks with lots of exercises created by masters of the craft because (especially in the beginning) the student will sometimes lack the knowledge and (vitally) the motivation (specifically, the curiosity) required to choose the theorems and calculations from the space of all possible theorems and calculations in the subject.

RHollerith 7 Nov 2012 1:28 UTC
23 points
on: 2012 Less Wrong Census/Survey
Took it.

RHollerith 15 Sep 2010 3:41 UTC
22 points
in reply to: CronoDAS’s comment on: Self-Improvement or Shiny Distraction: Why Less Wrong is anti-Instrumental Rationality
Another quote from Paul Graham:

The way to work for long periods on something is to be interested in it. Few to no people have the discipline to make themselves work on something that bores them for many hours straight without paging it out. Probably none of the people whose work I admire do. Their trick is to work on stuff they like.

The emphasis is mine, and note that Graham knows a lot of extremely successful people.

Patri links to Paul Graham, but IIRC those links advise one to remove distractions and temptations from one’s office and from one’s life so that one does not have to exert willpower to resist the distractions and temptations. ADDED. The thinking behind that, which is supported by psychology experiments, is that simply successfully resisting a temptation (such as refraining from eating from a plate of fresh cookies left in a waiting room by a psychology researcher) depletes a person’s daily reserve of willpower so that the reserve is unavailable for other things (such as keeping oneself at a tedious task).

In his essays, Graham probably never advised building willpower by forcing yourself to do things you do not like. (I’ve read most of his essays.)

Some people will not be able significantly to increase their ability to exert willpower that way. If you can keep on building up your willpower that way, then congratulations! you are probably headed for great things. Just make sure that you are not just fooling yourself. The rest of us are best advised to learn some tricks, like removing reminders of temptations from our awareness so that we deplete less of our precious reserves of willpower resisting the temptations.

RHollerith 13 Oct 2011 16:58 UTC
21 points
on: Studying business. Rational organizations.
Do you know the difference (in purpose) between managerial accounting and financial accounting? What about the difference between return on equity and rate of increase in stock price (assuming that the company is publicly traded)? Is the U.S. budget deficit more like an expense or a liability? How about the U.S. debt? In other words, learn basic accounting. (On Hacker News most participants have not.) Whole Foods and Apple are both very successful business. One has very high “variable cost” (or as the economists say, “marginal cost” or “cost at the margin”) and consequently their marginal income or “profit margin” is much lower. Which one is it? If you own stock in some company, do you care more about its marginal income’s being high or its return on equity’s being high?

Each of these questions has an unambiguous answer that all accountants and informed businessmen will agree on.

RHollerith 25 May 2010 5:44 UTC
21 points
in reply to: ata’s comment on: Dissolving the Question

Apologies if replying to super-old comments is frowned upon. I’m reading the whole blog from the beginning and occasionally finding that I have things to say.

I have been reading LW since the beginning and have not seen anyone object to replies to super-old comments (and there were 18-month-old comments on LW when LW began because all Eliezer’s Overcoming-Bias posts were moved to LW).

Moreover, a lot of reader will see a reply to a super-old comment. We know that because people have made comments in really old comment sections to the effect of “if you see this, please reply so we can get a sense of how many people see comments like this”.

Moreover, discouraging replies to super-old comments discourages reading of super-old comments. But reading super-old comments improves the “coherence” of the community by increasing the expected amount of knowledge an arbitrary pair of LW participants has in common. (And the super-old stuff is really good.)

So, reply away, I say.
What links here?
- RHollerith's comment on Logical or Connectionist AI? by Eliezer Yudkowsky (29 May 2010 21:39 UTC; 5 points)

RHollerith 29 May 2022 15:59 UTC
19 points
−11
on: Will working here advance AGI? Help us not destroy the world!
IMO most of the jobs where the employer says the job is about AI alignment or AI safety, e.g., the alignment team at OpenAI, is actually contributing to AI danger.

More generally, I humbly suggest erring on the side of safety: avoid contributing to not just AI—except for MIRI and maybe 1 or 2 other outfits—but also anything AI needs, such as hardware, operating systems, compilers, version-control tools, issue-trackers and development methodologies—but only if doing so does not put you at a significant personal disadvantage relative to people with your skills who do not care about AI safety.

If for example, you have invested 3 years of your working life in getting good a compiler development specifically, and you are not yet bored with compiler development, then the right move is to continue to work on compiler development if that is the most efficient way for you to earn money because having a higher income makes a person more effective and influential and we want the people concerned enough about about AI safety to read this thread of conversation to be effective and influential.

215 years ago in France, if your father was not part of the nobility, there were few opportunities for a young man to achieve status and material security. For most young men, the best way to achieve those things was to join Napoleon’s Army, but of course Napoleon’s Army was spreading death and destruction across Europe. If you know or suspect that AI research (and research and development of any computing infrastructure that supports it) is today’s version of Napoleon’s Army and you care about the global consequences of your actions, the right move is not to join Napoleon’s Army in the hopes that you will be able to use your insider position to limit the death and destruction. The right move is to choose not to join Napoleon’s Army (and to avoid spending any of your time and energy acquiring skills and knowledge whose main market is Napoleon’s Army—and to consider joining the movement to get Napoleon’s Army banned by the governments of the developed world if you think that movement has a chance of succeeding).

In contrast to 215 years ago in France, choosing not to join today’s version of Napoleon’s Army is not even much of a personal sacrifice because of the diversity and general abundance of today’s economy.

RHollerith 11 May 2012 4:04 UTC
19 points
on: Thoughts on the Singularity Institute (SI)

I feel that [SI] ought to be able to get more impressive endorsements than it has.

SI seems to have passed up opportunities to test itself and its own rationality by e.g. aiming for objectively impressive accomplishments.

Holden, do you believe that charitable organizations should set out deliberately to impress donors and high-status potential endorsers? I would have thought that a donor like you would try to ignore the results of any attempts at that and to concentrate instead on how much the organization has actually improved the world because to do otherwise is to incentivize organizations whose real goal is to accumulate status and money for their own sake.

For example, Eliezer’s attempts to teach rationality or “technical epistemology” or whatever you want to call it through online writings seem to me to have actually improved the world in a non-negligible way and seem to have been designed to do that rather than designed merely to impress.

ADDED. The above is probably not as clear as it should be, so let me say it in different words: I suspect it is a good idea for donors to ignore certain forms of evidence (“impressiveness”, affiliation with high-status folk) of a charity’s effectiveness to discourage charities from gaming donors in ways that seems to me already too common, and I was a little surprised to see that you do not seem to ignore those forms of evidence.

RHollerith 11 Jan 2011 21:16 UTC
19 points
on: Back to the Basics of Rationality

But videos in the familiar PowerPoint style or the Khan Academy style are easier to do well. All it requires is some free presentation software, free screen capture software and a $70 high-quality USB mic like the Blue Snowball.

Here we see the author breaking a goal into small pieces, which tends to be an excellent move in many kinds of conversations. Nice post.

RHollerith 25 Sep 2017 1:23 UTC
18 points
in reply to: tut’s comment on: LW 2.0 Open Beta Live
I can corroborate that the scrolling is painful on sufficiently old hardware (and two of the not-home not-work places I most like to hang out in these days have hardware that is sufficiently old).

Scrolling for example is painful near the bottom (in the comments) of the recent article on the Cambrian explosion on a Core 2 Duo running Windows Vista, in Chrome. In particular, it takes whole seconds for the text to appear. (Till then the view port is blank / white.)

But even when I’m using reasonably fast hardware, my reaction to any signs that the text on a web page is not being produced “the old fashioned way” (and the new site certainly has such signs) is to ask myself if I really need to continue using the site.

Even a site’s use of a font I don’t recognize I provokes that reaction in me.

Why? Well, it is a sign that I will run into further irritants. Some actions will work slightly differently from the way I am used to with the result that I have to stop concentrating on the reason I came onto the web site to figure out scrolling or searching in the page or making sure the right pane on the screen has “keyboard focus” or how to change the size of the text. Or I will have to figure out how to undo the effects of some action I took accidentally.

If you’re reading this and cannot relate, then maybe that is because I have cataracts, so a large text size is more important to me than it is to you. Or maybe it is because I have a 57-year-old brain and some chronic health issues so that it is harder for me to retain what is in my working memory when things jump around on a page in ways that my brain cannot predict.

Or maybe it is because I prefer the kinesthetic sensory modality which makes me care more about subtleties in the computer’s response to various “attempted manipulations” (e.g., attempting to scroll or to use the pointing device to select an extent of text) of the web page.

The new LW site is not doing anything that many many other web sites are not also doing, so this is a comment about modern web sites more than it is a comment about the new version of LW.

I realize that this comment is rough on the creators of the next version of LW since it is negative feedback, but not actionable negative feedback (since they’ve already implemented a particular design). I considered refraining from publishing it, but went ahead because writing this comment, then observing how many points it ends up with is by far the easiest way for me to find out how many LWers share my frustrations (and knowing that is even more useful to me than knowing how many in the general population of internet users share my frustrations). I won’t make a habit of complaining about it.

RHollerith 4 Aug 2012 23:27 UTC
18 points
on: What are the optimal biases to overcome?
Gerald Weinberg is a celebrated author of computer and management books. And for many years he was a management consultant. Often he would get a consulting gig at an organization he had consulted for in the past. The unhealthy organizations, he observed, had the same (crushing) worst problem during his second gig that they had during his first gig, whereas the better organizations tended to have lots of little problems, which he took as a sign that the organization was able to recognize their worst problems and slowly or quickly shrink them.

I am not sure because I do not have access to the book, but that is probably from the chapter or section “Rudy’s Law of Rutabagas” from Weinberg’s book The Secrets of Consulting: A Guide to Giving and Getting Advice Successfully.

What Weinberg did after he stopping doing management consulting, by the way, is to run workshops on improving what we would call individual and team rationality, and he maintained that people learned the skills he taught a lot better in the right kind of interpersonal situations (e.g., workshops) than they do from written materials.

Hope that helps someone.

RHollerith 10 Sep 2023 16:21 UTC
17 points
7
on: AI presidents discuss AI alignment agendas
The statements strike me as more credible and more interesting than they would be delivering in the speaking style of the people who usually talk on the topic, but then it is no surprise that winners of presidential elections have compelling vocal skills.

RHollerith 23 Apr 2011 5:10 UTC
16 points
in reply to: Arkanj3l’s comment on: How to Beat Procrastination
When Richard Stallman wants to view a web page, he sends the URL in an email to a server operated by him or one of his friends, and the server mails him back the page. He says that he does it that way “for personal reasons”, and my guess is that it is an anti-akrasia measure.

RHollerith 14 Oct 2010 15:31 UTC
16 points
on: 1993 AT&T “You Will” ads
Weary of bankrolling the internet (then called ARPANET) the U.S. government in 1977 offered AT&T the whole thing for a paltry sum (a million dollars IIRC). AT&T declined the offer. Ten years later, the government offered again, and AT&T declined again.

When the internet started to become bigger than a research project of the Department of Defense and then the National Science Foundation, it did so when organizations who wanted to become part of it leased bare copper wires from AT&T (specifically AT&T’s Long Lines department IIRC) -- wires that run between their offices and places that already had access to the internet. AT&T wanted to disallow the use of their wires for this purpose, leaving the organizations with no way to connect to the internet except the extremely expensive way of purchasing rights of way and stringing their own copper, but they were forced to lease the copper by the Courts as part of anti-trust jurisprudence.

They also bitterly opposed the computer modem until forced by the Courts to allow modems to be connected to their network.

So, the organization that spent those advertising dollars crowing about their role in bringing about a connected world had a long history of opposition to the line of development (packet switching, an invention of the 1950s) that proved most efficient in bringing the connected world about.

One Medical? Expansion of MIRI?

RHollerith18 Mar 2014 14:38 UTC

15 points

8 comments1 min readLW link

RHollerith 31 May 2022 0:53 UTC
15 points
on: The Reverse Basilisk
You are not the first person to have thought of that.

The conclusion of my thinking on it is that it probably doesn’t influence the superintelligence to be friendlier to the humans much for the following reason.

Reality has a terrible consistency to it and a superintelligence sees and knows that terrible consistency in greater detail than you and I do. So, for the creator of the superintelligence to successfully fool the superintelligence (by feeding it false sensory data) requires quite a bit of optimization power, i.e., intelligence. That intelligence is needed to ensure that the sensory data is consistent enough in all its details so that the superintelligence is fooled or more precisely remains uncertain as to whether its sensory data represents basement reality.

But any creator intelligent enough to create a false reality good enough to keep a superintelligence guessing has better ways to ascertain or ensure the alignment of the superintelligence. I strongly suspect that running simulations is something humans do because they’re just barely intelligent enough to program computers at all—and simulations make up a smaller fraction of the computations humans run than they did in the early decades of human computing because humans have gotten a little better at programming computers over the decades.

But let us taboo the word “simulation”. The word is more of a hinderance than a help. The crisp way of seeing it is that a civilization or an intelligence significantly better at computers than we are would not need to run a program to learn everything its wants to learn about the program—would not need to give the program any computational resources or ability to affect reality.

If I had a printout of the source code for a superintelligence in front of me, I could stare at it all day at no risk to myself or anyone around me. Of course I might be too dumb to tell what it would do, but an entity much better at analyzing source code than I am is similarly safe.

Some would reply here that it is my brain’s lack of computational resources that makes it safe for me to stare at the source code and to try to understand it. If an entity with vastly greater compute resources did the same thing, the source code might co-opt the staring entity’s mental processes (similar to how a virus co-opts a host, maybe). Some might say that the entity would analyze the source code in such detail—detail that you or I would be incapable of—that an inner computation would arise inside the entity that would tend not to be aligned with the entity’s values.

Well, I spent a lot of time in my youth writing programs to analyze other programs. (E.g., Boyer and Moore have an approach to theorem proving that essentially views theorems as programs.) And I can see that that is not going to happen. I feel that that will fail to convince some of my readers, but that saying something more convincing would make this comment too long. So, I’ll just say that I am happy to explain more if there is interest.

Anyway, the superintelligence wondering for the first time (maybe because it saw your comment!) whether its sensory data has been faked by its creator will tend to undergo the same line of reasoning we have just undergone here, which will likely convince it that it doesn’t have to assign much probability to the possibility that its sensory data is being faked (because, again, if the creator is smart enough to keep the superintelligence guessing as to the fidelity of its sensory data, then the creator is smart enough to find out everything it wants to know about the superintelligence before ever giving the superintelligence any computational resources).

So, the possibility is not much of restraint on the superintelligence’s treating the humans callously.

Nice try, though. Not everyone thinks of that argument.
What links here?
- RHollerith's comment on AGI Safety FAQ / all-dumb-questions-allowed thread by Aryeh Englander (7 Jun 2022 17:38 UTC; 3 points)

RHollerith

One Med­i­cal? Ex­pan­sion of MIRI?

One Medical? Expansion of MIRI?