Paul Colognese

• Paul Colognese 8 Mar 2024 4:05 UTC

  1 point

  0

  in reply to: Jonas Hallgren’s comment on: Ex­plain­ing the AI Align­ment Prob­lem to Ti­be­tan Bud­dhist Monks

  Interesting! I’m working on a project exploring something similar but from a different framing. I’ll give this view some thought, thanks!

Ex­plain­ing the AI Align­ment Prob­lem to Ti­be­tan Bud­dhist Monks

• Paul Colognese 5 Mar 2024 6:08 UTC

  1 point

  0

  in reply to: lukehmiles’s comment on: Ano­ma­lous Con­cept De­tec­tion for De­tect­ing Hid­den Cognition

  Thanks, should be fixed now.

Ano­ma­lous Con­cept De­tec­tion for De­tect­ing Hid­den Cognition

Hid­den Cog­ni­tion De­tec­tion Meth­ods and Bench­marks

Notes on In­ter­nal Ob­jec­tives in Toy Models of Agents

• Paul Colognese 14 Nov 2023 13:01 UTC

  1 point

  0

  in reply to: Charbel-Raphaël’s comment on: Char­bel-Raphaël and Lu­cius dis­cuss Interpretability

  Thanks, that’s the kind of answer I was looking for

• Paul Colognese 6 Nov 2023 14:43 UTC

  2 points

  1

  on: Char­bel-Raphaël and Lu­cius dis­cuss Interpretability

  Interesting discussion; thanks for posting!
  
  I’m curious about what elementary units in NNs could be.

  the elementary units are not the neurons, but some other thing.

  I tend to model NNs as computational graphs where activation spaces/​layers are the nodes and weights/​tensors are the edges of the graph. Under this framing, my initial intuition is that elementary units are either going to be contained in the activation spaces or the weights.
  
  There does seem to be empirical evidence that features of the dataset are represented as linear directions in activation space.
  
  I’d be interested in any thoughts regarding what other forms elementary units in NNs could take. In particular, I’d be surprised if they aren’t represented in subspaces of activation spaces.

In­ter­nal Tar­get In­for­ma­tion for AI Oversight

• Paul Colognese 5 Oct 2023 19:32 UTC

  2 points

  0

  in reply to: Nora Belrose’s comment on: High-level in­ter­pretabil­ity: de­tect­ing an AI’s objectives

  Thanks for pointing this out. I’ll look into it and modify the post accordingly.

[Question] Po­ten­tial al­ign­ment tar­gets for a sovereign su­per­in­tel­li­gent AI

• Paul Colognese 29 Sep 2023 11:11 UTC

  1 point

  0

  in reply to: Jonas Hallgren’s comment on: High-level in­ter­pretabil­ity: de­tect­ing an AI’s objectives

  With ideal objective detection methods, the inner alignment problem is solved (or partially solved in the case of non-ideal objective detection methods), and governance would be needed to regulate which objectives are allowed to be instilled in an AI (i.e., government does something like outer alignment regulation).

  Ideal objective oversight essentially allows an overseer instill whatever objectives it wants the AI to have. Therefore, if the overseer includes the government, the government can influence whatever target outcomes the AI pursues.

  So practically, this means that the governance policies would require the government to have access to the objective detection method results, directly or indirectly through the AI labs.

High-level in­ter­pretabil­ity: de­tect­ing an AI’s objectives

[Linkpost] Fron­tier AI Task­force: first progress report

• Paul Colognese 24 May 2023 22:01 UTC

  1 point

  0

  in reply to: Seth Herd’s comment on: Aligned AI via mon­i­tor­ing ob­jec­tives in Au­toGPT-like systems

  Thanks for the reponse, it’s useful to hear that we can to the same conclusions. I quoted your post in the first paragraph.
  
  Thanks for bringing Fabien’s post to my attention! I’ll reference it.
  
  Looking forward to your upcoming post.

Aligned AI via mon­i­tor­ing ob­jec­tives in Au­toGPT-like systems

• Paul Colognese 20 Apr 2023 6:10 UTC

  3 points

  0

  in reply to: Evan R. Murphy’s comment on: Towards a solu­tion to the al­ign­ment prob­lem via ob­jec­tive de­tec­tion and eval­u­a­tion

  Interesting! Quick thought: I feel as though it over-compressed the post, compared to the summary I used. Perhaps you can tweak things to generate multiple summaries in varying degrees of length.

• Paul Colognese 16 Apr 2023 6:06 UTC

  3 points

  0

  in reply to: Charlie Steiner’s comment on: Towards a solu­tion to the al­ign­ment prob­lem via ob­jec­tive de­tec­tion and eval­u­a­tion

  Thanks for the feedback! I guess the intention of this post was to lay down the broad framing/​motivation for upcoming work that will involve looking at the more concrete details.

  I do resonate with the feeling that the post as a whole feels a bit empty as it stands and the effort could have been better spent elsewhere.

Towards a solu­tion to the al­ign­ment prob­lem via ob­jec­tive de­tec­tion and eval­u­a­tion

De­ci­sion Trans­former Interpretability