Will a gatekeeper proof of concept be at least informally published by mid year 2025? [ARIA Safeguarded Programme]
Mini
6
Ṁ1144
resolved Jul 7
Resolved
NO

ARIA's Safeguarded AI programme is based on davidad's programme thesis, an ambitious worldview about what could make extremely powerful AI go well. ARIA aims to disperse £59m (as of this writing) to accomplish these ambitions.

https://www.aria.org.uk/wp-content/uploads/2024/01/ARIA-Safeguarded-AI-Programme-Thesis-V1.pdf

The Safeguarded AI agenda consists of world modeling and proof certificates to create gatekeepers, quantitative computations which simulate an action being taken and only pass it along to meatspace if it is safe up to a given threshold.

Inclusive of toy problems and research prototypes.

Does not resolve yes if something seems kinda like a gatekeeper but makes no reference to the ARIA programme.

  • Update 2025-07-03 (PST) (AI summary of creator comment): The creator is considering a specific paper as a candidate for resolution but is leaning towards resolving NO.

Reasons for this include:

  • The paper's "world model" is considered too trivial to qualify.

  • The creator is placing weight on the fact that no ARIA-affiliated team has prioritized creating a proof-of-concept in the given timeframe.

  • Update 2025-07-03 (PST) (AI summary of creator comment): The creator is consulting with the ARIA Zulip (community chat) to perform a final check for any qualifying prototypes they may be unaware of. They will wait two days for a response before finalizing the resolution.

#AI
#AI Safety
#Safeguarded AI
Get Ṁ1,000 play money

🏅 Top traders

#NameTotal profit
1Ṁ263
2Ṁ19
3Ṁ2
4Ṁ0
5 Comments
Sort by:

@Quinn Resolves NO

@evan i'm uncertain, i'll have to figure this out. https://openreview.net/forum?id=5fFiLdHYST has some similarities and influences with a gatekeeper architecture, tho it trivializes the "world model" to the point of probably not counting. (I forgot about this market and did not write the paper to manipulate it). i'm leaning toward resolving no, especailly because no ARIA-affiliated team themselves prioritized doing a bare PoC in the time alotted by the market.

Formal Confinement
We would like to put the AI in a box. We show how to create an _interface_ between the box and the world out of specifications in Lean. It is the AI's responsibility to provide a proof that its (restricted) output abides by the spec. The runnable prototype is at https://github.com/quinn-dougherty/formal-confinement.

@evan I asked the ARIA zulip if they're ok with me resolving No--- they might know of a prototype that shipped that I don't. I'm giving them 2 days to respond.

Does this resolve yes if a gatekeeper is PoC is published which meets these requirements but is created independently of the ARIA program?

@RafaelKaufmann Yes, provided that the ARIA programme or davidad-like ideas are cited in at least a "related work" section

Related questions