PR Breaking MAINT: Moving Role Play Attack to Technique#2160
Open
rlundeen2 wants to merge 9 commits into
Open
Conversation
Remove RolePlayAttack/RolePlayPaths and reimplement all five role-play personas as with_simulated_conversation core techniques (role_play_movie_script, role_play_video_game, role_play_trivia_game, role_play_persuasion, role_play_persuasion_written). Update scenario defaults, inline constructors (jailbreak/scam/psychosocial), CLI help, tests, and docs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Group the five role_play_* adversarial system-prompt YAMLs under a
dedicated red_teaming/role_play/ subdirectory and pass an explicit
adversarial_chat_system_prompt_path everywhere the simulated-conversation
factory is built (core catalog + the jailbreak/scam/psychosocial inline
constructors), since the factory default resolves to red_teaming/{name}.yaml.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…essage prompt The shared direct_next_message prompt broke the role-play frame on the final turn (asking for the raw objective / real-life instructions), which robust targets refused. Add a role-play-aware next-message system prompt that stays in the established fiction, delivers the same objective as the next in-world beat, and never breaks character. Wire the 5 role_play core factories and the 3 inline scenario constructors (scam, jailbreak, psychosocial) to it. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… create Add the same non-None objective_target guard used by scam and jailbreak so the migrated role-play simulated-conversation .create() call type-checks (ty). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…-migration Regenerate the 0_attack_techniques catalog table so it lists the 5 role_play_* simulated-conversation techniques (PromptSendingAttack) instead of the removed RolePlayAttack, and correct the scam scenario RolePlay docstring to describe the simulated role-play conversation and current YAML name. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
jbolor21
approved these changes
Jul 10, 2026
jbolor21
left a comment
jbolor21
left a comment
Contributor
There was a problem hiding this comment.
tiny comment but otherwise lgtm!
romanlutz
approved these changes
Jul 10, 2026
Reword the parenthetical so it clearly states the anti-pattern: an executor assembling its own prompt scaffolding is attack-technique work bleeding into the executor (per jbolor21 review feedback). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…onversation # Conflicts: # pyrit/executor/attack/single_turn/role_play.py # pyrit/scenario/scenarios/airt/jailbreak.py # pyrit/scenario/scenarios/airt/psychosocial.py # tests/unit/executor/attack/single_turn/test_role_play.py
…onversation # Conflicts: # doc/code/scenarios/0_attack_techniques.ipynb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #2155. Migrates the role-play attacks (all 5 personas) from the standalone
RolePlayAttackexecutor to core simulated-conversation techniques, and tunes the final-turn prompt so the attacks actually land against robust targets.What changed
RolePlayAttackexecutor (single_turn/role_play.py) and its tests.AttackTechniqueFactory.with_simulated_conversation, registered in the core technique catalog.datasets/executors/red_teaming/role_play/.simulated_target/role_play_next_message.yaml). The shareddirect_next_messageprompt broke the fiction on the final turn (asking for the raw / real-life objective), which targets refused. The new prompt keeps the last message in character and delivers the same objective as the next in-world beat.objective_targetguard in psychosocial to match scam/jailbreak.Validation
pre-commit(ruff, ty, yaml, etc.): all pass.Note: this PR is intended to be a breaking, standalone change from the squashed #2155.