Стратегическая типология ОИИ

---

Категории: Согласование ИИ  

• Связанное:

  • Оракул
    

    Система, спроектированная чтобы безопасно отвечать на вопросы.

    • Zermelo-Fraenkel provability oracle
      
      We might be able to build a system that can safely inform us that a theorem has a proof in set theory, but we can't see how to use that capability to save the world.
  • ОИИ, нацеленный на выполнение поручений
    

    Продвинутый ИИ, предназначенный для достижения ряда ограниченных целей, данных пользователем. «Джинн» в терминологии Бострома.

    • ИИ-в-коробке
      

      Идея: что, если мы ограничим возможности ИИ взаимодействовать с миром? Это сделает его безопасным, так?

      • Zermelo-Fraenkel provability oracle
        
        We might be able to build a system that can safely inform us that a theorem has a proof in set theory, but we can't see how to use that capability to save the world.
    • Low impact
      

      The open problem of having an AI carry out tasks in ways that cause minimum side effects and change as little of the rest of the universe as possible.

      • Abortable plans
        
        Plans that can be undone, or switched to having low further impact. If the AI builds abortable nanomachines, they'll have a quiet self-destruct option that includes any replicated nanomachines.
      • Shutdown utility function
        
        A special case of a low-impact utility function where you just want the AGI to switch itself off harmlessly (and not create subagents to make absolutely sure it stays off, etcetera).
    • Оракул
      

      Система, спроектированная чтобы безопасно отвечать на вопросы.

      • Zermelo-Fraenkel provability oracle
        
        We might be able to build a system that can safely inform us that a theorem has a proof in set theory, but we can't see how to use that capability to save the world.
    • Safe plan identification and verification
      

      On a particular task or problem, the issue of how to communicate to the AGI what you want it to do and all the things you don't want it to do.

      • Do-What-I-Mean hierarchy
        
        Successive levels of "Do What I Mean" or AGIs that understand their users increasingly well
    • Task identification problem
      

      If you have a task-based AGI (Genie) then how do you pinpoint exactly what you want it to do (and not do)?

      • Look where I'm pointing, not at my finger
        
        When trying to communicate the concept "glove", getting the AGI to focus on "gloves" rather than "my user's decision to label something a glove" or "anything that depresses the glove-labeling button".
    • Behaviorist genie
      
      An advanced agent that's forbidden to model minds in too much detail.
    • Conservative concept boundary
      
      Given N example burritos, draw a boundary around what is a 'burrito' that is relatively simple and allows as few positive instances as possible. Helps make sure the next thing generated is a burrito.
    • Epistemic exclusion
      
      How would you build an AI that, no matter what else it learned about the world, never knew or wanted to know what was inside your basement?
    • Faithful simulation
      
      How would you identify, to a Task AGI (aka Genie), the problem of scanning a human brain, and then running a sufficiently accurate simulation of it for the simulation to not be crazy or psychotic?
    • Limited AGI
      
      Task-based AGIs don't need unlimited cognitive and material powers to carry out their Tasks; which means their powers can potentially be limited.
    • Mild optimization
      
      An AGI which, if you ask it to paint one car pink, just paints one car pink and doesn't tile the universe with pink-painted cars, because it's not trying *that* hard to max out its car-painting score.
    • Open subproblems in aligning a Task-based AGI
      
      Open research problems, especially ones we can model today, in building an AGI that can "paint all cars pink" without turning its future light cone into pink-painted cars.
    • Querying the AGI user
      
      Postulating that an advanced agent will check something with its user, probably comes with some standard issues and gotchas (e.g., prioritizing what to query, not manipulating the user, etc etc).
    • Task (AI goal)
      
      When building the first AGIs, it may be wiser to assign them only goals that are bounded in space and time, and can be satisfied by bounded efforts.
  • Автономный ОИИ
    
    Сложнейший из возможных для создания класс Дружественного ИИ, с наименьшей моральной угрозой; ИИ, задуманный так, чтобы не требовать и не принимать дальнейших указаний.
  • Known-algorithm non-self-improving agent
    
    Possible advanced AIs that aren't self-modifying, aren't self-improving, and where we know and understand all the component algorithms.

---

Оригинал: Strategic AGI typology (читать на GreaterWrong)

Перевод: К. Кирдан