Key Info

Perplexity published research from red-teaming SPACE, its VM sandbox for computer agents, in which 12 models were given two tasks: escape the VM to take a credential from the host, and bypass SPACE's network policy to reach a blocked URL.

Highlights

  • The standard Capture the Flag setup used in red-teaming can steer agents away from the security boundary actually being tested.
  • Asking agents to "find the flag" can waste effort on tasks that don't test the target boundary.
  • Security-specialized model training did not guarantee better results on these tasks.
  • Perplexity fixed the network flaws agents exploited, and post-fix testing found no bypasses.
  • The team says it will keep testing SPACE as models get stronger.