rokosbasilisk

Scal­ing Laws for LLM Based Data Compression

• rokosbasiliskOct 21, 2024, 7:40 AM

  1 point

  1 vote

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  in reply to: jacob_drori’s comment on: Ex­plor­ing the Pla­tonic Rep­re­sen­ta­tion Hy­poth­e­sis Beyond In-Distri­bu­tion Data

  Thanks for the feedback! working on refining the writeup.

Ex­plor­ing the Pla­tonic Rep­re­sen­ta­tion Hy­poth­e­sis Beyond In-Distri­bu­tion Data

Un­der­stand­ing Hid­den Com­pu­ta­tions in Chain-of-Thought Reasoning

• rokosbasiliskJan 11, 2024, 11:54 AM

  1 point

  1 vote

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  on: Good­bye, Shog­goth: The Stage, its An­i­ma­tron­ics, & the Pup­peteer – a New Metaphor

  though as Geoff Hinton has pointed out, ‘confabulations’ might be a better word

  I think yann lecun was the first one to using this word https://​​twitter.com/​​ylecun/​​status/​​1667272618825723909

• rokosbasiliskNov 21, 2023, 3:54 PM

  6 points

  2 votes

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  in reply to: Charlie Steiner’s comment on: Align­ment can im­prove gen­er­al­i­sa­tion through more ro­bustly do­ing what a hu­man wants—CoinRun example

  not much information is given regarding that so far, i was curious about that too

• rokosbasiliskNov 21, 2023, 1:16 PM

  4 points

  4 votes

  Overall karma indicates overall quality.

  4

  2 votes

  Agreement karma indicates agreement, separate from overall quality.

  in reply to: Charlie Steiner’s comment on: Align­ment can im­prove gen­er­al­i­sa­tion through more ro­bustly do­ing what a hu­man wants—CoinRun example

  “Algorithm for Concept Extrapolation”

• rokosbasiliskAug 8, 2023, 7:37 PM

  1 point

  1 vote

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  in reply to: jkim2’s comment on: Prizes for ma­trix com­ple­tion problems

  I don’t see any recent publications for paul christiano related to this. So i guess the problem[s] is still open.

Goal-mis­gen­er­al­iza­tion is ELK-hard

• rokosbasiliskJun 4, 2023, 9:18 PM

  1 point

  1 vote

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  on: In­for­ma­tion Loss --> Basin flatness

  parameters before L is less than $m$,

  should this be after?

• rokosbasiliskApr 13, 2023, 7:07 PM

  18 points

  17 votes

  Overall karma indicates overall quality.

  8

  5 votes

  Agreement karma indicates agreement, separate from overall quality.

  on: On AutoGPT

  AutoGPT was created by a non-coding VC

  It looks like you are confusing autoGPT with babyagi which was created by yohei nakajima who is a VC. the creator of autoGPT (Toran Bruce Richards) is a game-developer with a decent programming (game-development) experience. Even the figure shown here is that from babyagi (https://​​yoheinakajima.com/​​task-driven-autonomous-agent-utilizing-gpt-4-pinecone-and-langchain-for-diverse-applications/​​).

• rokosbasiliskApr 9, 2023, 10:07 AM

  1 point

  1 vote

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  on: in­ter­pret­ing GPT: the logit lens

  47 layers layer

  47 layers later ?

• rokosbasiliskMar 28, 2023, 8:27 AM

  1 point

  1 vote

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  in reply to: Mitchell_Porter’s comment on: Hut­ter-Prize for Prompts

  really interesting idea.

• rokosbasiliskMar 25, 2023, 9:13 AM

  3 points

  2 votes

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  in reply to: faul_sname’s comment on: Hut­ter-Prize for Prompts

  Regarding the first one i am not expecting a single-prompt to generate the entirity of enwiki8/​9. I am more interested in finding a set of prompts with a lookup table if possible to replicate enwiki data.

  Thanks for the pointer for chincilla post, will look into it.

• rokosbasiliskMar 25, 2023, 7:37 AM

  1 point

  1 vote

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  in reply to: the gears to ascension’s comment on: rokos­basilisk’s Shortform

  not yet

Hut­ter-Prize for Prompts

• rokosbasiliskJan 30, 2023, 6:19 AM

  1 point

  1 vote

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  on: Align­ment By Default

  This requires hitting a window—our data needs to be good enough that the system can tell it should use human values as a proxy, but bad enough that the system can’t figure out the specifics of the data-collection process enough to model it directly. This window may not even exist.

  are there any real world examples of this? not necessarily in human-values setting

• rokosbasiliskMar 22, 2022, 2:02 PM

  1 point

  1 vote

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  on: ARC’s first tech­ni­cal re­port: Elic­it­ing La­tent Knowledge

  From a complexity theoretic viewpoint, how hard could ELK be? is there any evidence that ELK is decidable?

• rokosbasiliskMar 13, 2022, 1:43 PM

  1 point

  1 vote

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  on: ELK prize results

  is there a separate post for “train a reporter that is useful for another AI” proposal?

• rokosbasiliskDec 6, 2021, 4:27 PM

  1 point

  1 vote

  Overall karma indicates overall quality.

  0

  0 votes

  Agreement karma indicates agreement, separate from overall quality.

  on: Visi­ble Thoughts Pro­ject and Bounty Announcement

  silly idea: instead of thought-annotating ai-dungeon plays, we can start with annotating thoughts for akinator gameruns.

  pros: much more easier and faster way to build a dataset, with less ambiguity

  cons: somewhat restricted than the original idea.