Jsevillamol

Karma: 1,713

Jsevillamol 22 Jun 2021 18:26 UTC
44 points
on: The Point of Trade
Other magical powers of trade:
1. Economies of scale. It is basically as easy for me to produce 20 sheets of paper as to produce 1 ; after paying the set up costs the marginal costs are much smaller in comparison. So all in all I would rather specialize in paper-making, have somebody else specialize in pencil-making, then trade.
2. Investment. often I need A LOT of capital to get something started, more than I could reasonably accumulate over a lifetime. So I would rather trade the starting capital for IOUs I will get from profit.
3. Insurance. I may have a particularly bad harvest this year and a very good one the next one, while my neighbour might have the opposite problem. All in all I would rather we pool our harvests each year, so that we can have food both years. So we are “trading” part of our harvest for insurance.
What links here?
- Celer's comment on The Point of Trade by Eliezer Yudkowsky (20 Jul 2021 3:06 UTC; 3 points)

Jsevillamol 14 Oct 2022 20:00 UTC
19 points
2
on: Counterarguments to the basic AI x-risk case
Here are my quick takes from skimming the post.

In short, the arguments I think are best are A1, B4, C3, C4, C5, C8, C9 and D. I don’t find any of them devastating.
A1. Different calls to ‘goal-directedness’ don’t necessarily mean the same concept
I am not sure I parse this one.I am reading it as “AI systems might be more like imitators than optimizers” from the example, which I find moderately persuasive
A2. Ambiguously strong forces for goal-directedness need to meet an ambiguously high bar to cause a risk
I am not sure I understand this one either.I am reading it as “there might be no incentive for generality” which I dont find persuasive—I think there is a strong incentive
B1. Small differences in utility functions may not be catastrophic
I dont find this persuasive. I think the evidence from optimization theory setting variables to extreme values is suggestive enough to suggest this is not the default
B2. Differences between AI and human values may be small
B3. Maybe value isn’t fragile
The only example we have of general intelligence (humans) seems to have strayed pretty far from evolutionary incentives, so I find this unpersuasive
B4. [AI might only care about]Short-term goals
I find that somewhat persuasive, or at least not obviously wrong, similar to A1. There is a huge incentive for instilling long term thinking though.
C1. Human success isn’t from individual intelligence
I dont find this persuasive. Im not convinced there is a meaningful difference between “a single AGI” and “a society of AGIs”. A single AGI could be running a billion independent threads of thought and outspeed humans.
C2. AI agents may not be radically superior to combinations of humans and non-agentic machines
I dont find this persuasive. Seems unlikely that human-in-the-loop is going to have any advantages over pure machines.
C3. Trust
I find this plausible but not convincing
C4. Headroom
Plausible but not convincing. I dont find any of the particular examples of lack of headroom convincing, and I think the prior should be that there is a lot of headroom
C5. Intelligence may not be an overwhelming advantage
I find this moderately persuasive though not entirely convincing
C6. Unclear that many goals realistically incentivise taking over the universe
I find this unconvincing. I think there are many reasons to expect that taking over the universe is a convergent goal.
C7. Quantity of new cognitive labor is an empirical question, not addressed
I dont find this superpersuasive. In particular I think there is a good chance that once we have AGI we will be in a hardware overhang and be able to execute tons of AGI-equivalents
C8. Speed of intelligence growth is ambiguous
I find this plausible
C9. Key concepts are vague
Granted but not a refutation in itself
D1. The argument overall proves too much about corporations
I find this somewhat persuasive

Jsevillamol 23 Feb 2022 3:01 UTC
19 points
on: [Beta Feature] Google-Docs-like editing for LessWrong posts
I am pretty pumped about this. Google docs + latex support is huge game for me.

Jsevillamol 12 Oct 2021 8:00 UTC
19 points
in reply to: Jack Stone’s comment on: NVIDIA and Microsoft releases 530B parameter transformer model, Megatron-Turing NLG
Here is your estimate in the context of other big AI models:
The top 3 training compute in the graph are Megatron-Turing, GPT-3 and AlphaZero
An interactive visualization is available here.

Jsevillamol 4 Apr 2023 6:38 UTC
17 points
5
in reply to: dxu’s comment on: Communicating effectively under Knightian norms
Note that Richard is not treating knightian uncertainty as special and unquantifiable, but instead is giving examples of how to treat it like any other uncertainty, that he is explicitly quantifying and incorporating in his predictions.

I’d prefer calling Richard’s “model error” to separate the two, but I’m also okay appropriating the term as Richard did to point to something coherent.

Jsevillamol 25 Apr 2024 6:26 UTC
15 points
6
on: Bayesian inference without priors
I’ve been tempted to do this sometime, but I fear the prior is performing one very important role you are not making explicit: defining the universe of possible hypothesis you consider.

In turn, defining that universe of probabilities defines how bayesian updates look like. Here is a problem that arises when you ignore this: https://www.lesswrong.com/posts/R28ppqby8zftndDAM/a-bayesian-aggregation-paradox

Jsevillamol 14 Aug 2023 7:01 UTC
14 points
0
in reply to: Daniel Kokotajlo’s comment on: AGI is easier than robotaxis
Here is some data through Matthew Barnett and Jess Riedl

Number of cumulative miles driven by Cruise’s autonomous cars is growing as an exponential at roughly 1 OOM per year.

https://twitter.com/MatthewJBar/status/1690102362394992640

Jsevillamol 27 Dec 2022 1:15 UTC
14 points
−1
on: Slightly against aligning with neo-luddites
I agree with the sentiment that indiscriminate regulation is unlikely to have good effects.

I think the step that is missing is analysing the specific policies No-AI Art Activist are likely to advocate for, and whether it is a good idea to support it.

My current sense is that data helpful for alignment is unlikely to be public right now, and so harder copyright would not impede alignment efforts. The kind of data that I could see being useful are things like scores and direct feedback. Maybe at most things like Amazon reviews could end up being useful for toy settings.

Another aspect that the article does not touch on is that copyright enforcement could have an adverse effect. Currently there is basically no one trying to commercialize training dataset curation because enforcing copyright use is a nightmare. It is in fact a common good. I’d expect there would be more incentives to create large curated datasets if this was not the case.

Lastly, here are some examples of “no AI art” legislation I expect the movement is likely to support:
1. Removing copyright protection of AI generated images
2. Enforcing AI training data to be strictly opt-in
3. Forcing AI content to be labelled as such
Besides regulation, I also expect activists to 4) pressure companies to deboost AI made content in social medial sites.

My general impression is that 3) is slightly good for AI safety. People in the AI Safety community have advocated for it in the past, convincingly.

I’m more agnostic on 1), 2) and 4).

1 and 4 will make AI generation less profitable, but also it’s somewhat confused—it’s a weird double standard to apply to AI content over human made content.

2 makes training more annoying, but could lead to commercialization of datasets and more collective effort being put into building them. I also think there is a possibly a coherent moral case for it, which I’m still trying to make my mind about, regardless of the AI safety consequences.

All in all, I am confused, though I wholeheartedly agree that we should be analysing and deciding to support specific policies rather than eg the anti AI art movement as a whole.

Jsevillamol 12 Dec 2021 21:09 UTC
14 points
on: Moore’s Law, AI, and the pace of progress
Re: quantum computing.
I am bearish it will be a big deal in relation to AI.
This is because:
- The exponential speedups are very hard to achieve in practice.
- The quadratic speedups are mostly lost when you parallelize.
- At the current rate of progress and barring a breakthrough it seems it will be a couple of decades until we have useful quantum computing.

Jsevillamol 30 Mar 2022 15:48 UTC
LW: 13 AF: 7
AF
on: DeepMind: Generally capable agents emerge from open-ended play
Marius Hobbhahn has estimated the number of parameters here. His final estimate is 3.5e6 parameters.
Anson Ho has estimated the training compute (his reasoning at the end of this answer). His final estimate is 7.8e22 FLOPs.
Below I made a visualization of the parameters vs training compute of n=108 important ML system, so you can see how DeepMind’s syste (labelled GOAT in the graph) compares to other systems.
[Final calculation]
(8 TPUs)(4.20e14 FLOP/s)(0.1 utilisation rate)(32 agents)(7.3e6 s/agent) = 7.8e22 FLOPs
==========================
NOTES BELOW
[Hardware]
- “Each agent is trained using 8 TPUv3s and consumes approximately 50,000 agent steps (observations) per second.”
- TPUv3 (half precision): 4.2e14 FLOP/s
- Number of TPUs: 8
- Utilisation rate: 0.1
[Timesteps]
- Figure 16 shows steps per generation and agent. In total there are 1.5e10 + 4.0e10 + 2.5e10 + 1.1e11 + 2e11 = 3.9e11 steps per agent.
- 3.9e11 / 5e4 = 8e6 s → ~93 days
- 100 million steps is equivalent to 30 minutes of wall-clock time in our setup. (pg 29, fig 27)
- 1e8 steps → 0.5h
- 3.9e11 steps → 1950h → 7.0e6 s → ~82 days
- Both of these seem like overestimates, because:
“Finally, on the largest timescale (days), generational training iteratively improves population performance by bootstrapping off previous generations, whilst also iteratively updating the validation normalised percentile metric itself.” (pg 16)
- Suggests that the above is an overestimate of the number of days needed, else they would have said (months) or (weeks)?
- Final choice (guesstimate): 85 days = 7.3e6 s
[Population size]
- 8 agents? (pg 21) → this is describing the case where they’re not using PBT, so ignore this number
- The original PBT paper uses 32 agents for one task https://arxiv.org/pdf/1711.09846.pdf (in general it uses between 10 and 80)
- (Guesstimate) Average population size: 32

Jsevillamol 11 Apr 2019 11:05 UTC
12 points
on: Alignment Newsletter One Year Retrospective
Some back of the envelope calculations trying to make sense out of the number of subscribers.
The EA survey gets about ~2500 responses per year from self identified EAs and I expect it represents between 10% and 70% of the EA community, so a fair estimate is that the EA community is about 1e4 people.
They ask about top priorities. About 16% of respondents consider AI risk a top priority.
Assuming representativeness, that means about 2e3 EAs who consider AI risk a priority.
Of those I would expect about half to be considering actively pursuing a career in the field, for 1e3 people.
This checks out with the newsletter number of subscribers.

Jsevillamol 6 May 2018 7:34 UTC
11 points
on: Predicting Future Morality
It seems obvious to me that after clean meat is developed we will see the vegan movement rise, and as life extension tech is perfected we will see fewer people advocating against immortality.

Jsevillamol 15 Mar 2022 21:37 UTC
LW: 10 AF: 3
AF
in reply to: Jsevillamol’s comment on: Compute Trends Across Three eras of Machine Learning
Following up on this: we have updated appendix F of our paper with an analysis of different choices of the threshold that separates large-scale and regular-scale systems. Results are similar independently of the threshold choice.

Jsevillamol 11 Aug 2021 16:36 UTC
10 points
in reply to: Kaj_Sotala’s comment on: Outline of Galef’s “Scout Mindset”
Some discussion from the OP here

Jsevillamol 7 Aug 2019 10:36 UTC
9 points
on: Advocating for factual advocacy
UPDATE AFTER A YEAR: Since most people believe that lives in the developing world are cheaper to save than what they actually are I think that pretty much invalidates my argument.
My current best hypothesis is that the Drowning Child argument derives its strength from creating a cheap opportunity to buy status.

Jsevillamol 14 Oct 2022 17:41 UTC
8 points
13
on: Counterarguments to the basic AI x-risk case
Eight examples, no cherry-picking:
Nit: Having a wall of images makes this post unnecessarily harder to read.
I’d recommend making a 4x2 collage with the photos so they don’t take that much space.

Jsevillamol 18 Feb 2022 15:15 UTC
8 points
AF
in reply to: StellaAthena’s comment on: Compute Trends Across Three eras of Machine Learning
Great questions! I think it is reasonable to be suspicious of the large-scale distinction.
I do stand by it—I think the companies discontinuously increased their training budgets around 2016 for some flagship models.^[1] If you mix these models with the regular trend, you might believe that the trend was doubling very fast up until 2017 and then slowed down. It is not an entirely unreasonable interpretation, but it explains worse the discontinuous jumps around 2016. Appendix E discusses this in-depth.
The way we selected the large-scale models is half intuition and half convenience. We compare the compute of each model to the log compute of nearby papers (within 2 years), and we call it large scale if its log compute exceeds 0.72 standard deviations of the mean of that sample.
I think there is a reasonable case for including NASv3, Libratus, Megatron-LM, T5-3B, OpenAI Five, Turing NLG, iGPT-XL, GShard (dense), Switch, DALL-E, Pangu-α, ProtT5-XXL and HyperClova on either side of this division.
Arguably we should have been more transparent about the effects of choosing a different threshold—we will try to look more into this in the next update of the paper.
1. ^
  See appendix F for a surface discussion

Jsevillamol 21 Jan 2022 0:15 UTC
LW: 8 AF: 7
AF
in reply to: A Ray’s comment on: Estimating training compute of Deep Learning models
Thank you Alex! You make some great points.
It seems like you probably could have gotten certainty about compute for at least a handful of the models studied in question
We thought so too—but in practice it has been surprisingly hard. Profilers are surprisingly buggy. Our colleague Marious looked into this more in depth here.
Maybe we are just going the wrong way about it. If someone here figures out how to directly measure compute in eg a pytorch or TF model it would be a huge boon to us.
I think two more contemporary techniques are worth considering here: structured sparsity in weights (‘blocksparse’), and mixture-of-experts gating (‘switch transformer’)
Great suggestions! I think those would be a great future caveats to look into.
I’d be curious to hear the authors’ expectations of how this research changes in the face of more custom ML hardware.
My naive impression is that our conclusions do not change much. You would just need to plug into the effective performance ( $peak performance \times utilization$ ) in the second formula.
Probably the trickiest part might be figuring out the utilization rate for the custom hardware—though this is a general problem with the second method.
In general I think it’d be good to integrate a bunch of the performance benchmarks that are publicly available (since hardware providers are usually pretty eager to show off stats that make their hardware look good) into calibrations for this method.
I think that would be nice! We started a public spreadsheet with some info on different hardware. This might be of help to someone who wants to dig deeper into the topic!

Jsevillamol 21 Dec 2021 23:18 UTC
8 points
on: Perpetual Dickensian Poverty?
I agree that purchasing power is a better way of measuring absolute poverty!

Interestingly, I could imagine the author of the Tweet being right about using labor value, if what they intended to point at is inequality.

Jsevillamol 6 Aug 2021 12:20 UTC
8 points
in reply to: Isnasene’s comment on: Analysis of World Records in Speedrunning [LINKPOST]
Those are good suggestions!
Here is what happens when we align the start dates and plot the improvements relative to the time of the first run.
Relative improvement vs days since first run for most popular categories
I am slightly nervous about using the first run as the reference, since early data in a category is quite unrealiable and basically reflects the time of the first person to thought to submit a run. But I think it should not create any problems.
Interestingly, plotting the relative improvement reveals some S-curve patterns, with phases of increasing returns followed by phases of diminishing returns.
I did not manage either to beat the baseline by extrapolating the relative improvement times. Interestingly, using a grid to count non-improvements as observations made the extrapolation worse, so this time the best fit was achieved with log linear regression over the last 8 weeks of data in each category.
Log linear extrapolation of relative improvements
As before, the code to replicate my analysis is available here.
Haven’t had time yet to include logistic models or do analysis of the derivative of the improvements—if you feel so inclined feel free to reuse my code to perform the analysis yourself and if you share them here we can comment on the results!
PS: there is a sentence missing an ending in your comment