I have a technical question that’s orthogonal to the discussion of morality.
You write:
But the nearest-neighbors estimator doesn’t assume a particular shape of underlying curve—not even that the curve is a polynomial. Technically, it doesn’t even assume continuity. It just says that we think that the true values at nearby positions are likely to be similar.
I’ve never studied this algorithm before. But the description “true values at nearby positions are likely to be similar” reads to me like an informal description of continuity, except for the word “likely”. Except for that word, I could see giving your description word-for-word as a “preview” for students of the more precise epsilon-delta definition of continuity.
If I understand the algorithm correctly, if you thought it likely that the underlying phenomenon had discontinuities, and you didn’t want your model to smooth them out, you wouldn’t use this algorithm, would you? Taking averages is sure to smooth out discontinuities in the true values, isn’t it?
if you thought it likely that the underlying phenomenon has discontinuities, and you didn’t want your model to smooth them out
This is a change point problem. See the example in section 3.1 of the PyMC manual:
Consider the following dataset, which is a time series of recorded coal mining disasters in the UK from 1851 to 1962 [. . . .] Occurrences of disasters in the time series is thought to be derived from a Poisson process with a large rate parameter in the early part of the time series, and from one with a smaller rate in the later part. We are interested in locating the change point in the series, which perhaps is related to changes in mining safety regulations.
Taking averages is sure to smooth out discontinuities in the true values, isn’t it?
Yes. If the true change points are unknown, then even if every possible underlying phenomenon has discontinuities, the average of credible underlying phenomena (the posterior mean) can still be continuous. See this plot in the Introduction to Bayesian Thinking blog post “A Poisson Change-Point Model” (discontinuous mining safety regulations) and Figure 5 on page 8 of “Bayesian change-point analyses in ecology” by Brian Beckage et al. (discontinuous border between canopy and gap conditions).
Yes, and in the limit of obtaining more data indefinitely, the in-between regions will shrink indefinitely (at least if you’re using k-nearest-neighbors and not smooth kernels).
I have a technical question that’s orthogonal to the discussion of morality.
You write:
I’ve never studied this algorithm before. But the description “true values at nearby positions are likely to be similar” reads to me like an informal description of continuity, except for the word “likely”. Except for that word, I could see giving your description word-for-word as a “preview” for students of the more precise epsilon-delta definition of continuity.
If I understand the algorithm correctly, if you thought it likely that the underlying phenomenon had discontinuities, and you didn’t want your model to smooth them out, you wouldn’t use this algorithm, would you? Taking averages is sure to smooth out discontinuities in the true values, isn’t it?
typo fix
This is a change point problem. See the example in section 3.1 of the PyMC manual:
Yes. If the true change points are unknown, then even if every possible underlying phenomenon has discontinuities, the average of credible underlying phenomena (the posterior mean) can still be continuous. See this plot in the Introduction to Bayesian Thinking blog post “A Poisson Change-Point Model” (discontinuous mining safety regulations) and Figure 5 on page 8 of “Bayesian change-point analyses in ecology” by Brian Beckage et al. (discontinuous border between canopy and gap conditions).
Thanks, Steve. So, can I unpack Eliezer’s condition
as saying of the true values that there might be switchpoints, but most points aren’t switchpoints?
Yes, and in the limit of obtaining more data indefinitely, the in-between regions will shrink indefinitely (at least if you’re using k-nearest-neighbors and not smooth kernels).
Yes, switchpoints or large smooth jumps.