aws·Apr 3, 2026·2 min readCurated summary

Amazon Bedrock Guardrails supports cross-account safeguards with centralized control and management | Amazon Web Services

Amazon Bedrock Guardrails now supports cross-account safeguards, allowing organizations to centrally enforce safety controls across AWS accounts and organizational units. Administrators can apply immutable, versioned guardrails to all Bedrock model invocations while still allowing account- or application-specific policies. The capability is generally available across commercial and GovCloud Regions where Bedrock Guardrails is supported. ## Centralized Organization- and Account-Level Enforcement - **Organization-level enforcement** uses an Amazon Bedrock policy created in the AWS Organizations management account. - Policies can attach a specified guardrail and version to: - The organization root - Organizational units - Individual AWS accounts - The selected guardrail is automatically applied to Bedrock inference requests across targeted member entities. - Different policies and guardrails can be assigned to different accounts or organizational units. - **Account-level enforcement** applies a configured guardrail to all Bedrock inference API calls within one account and Region. ## Configuring Guardrail Coverage - Guardrails must use a specific version so their configuration remains immutable and cannot be changed by member accounts. - Administrators can choose whether enforcement: - Includes or excludes specific Bedrock models - Covers all or only selected system and user prompt content - **Comprehensive** mode guards all content, regardless of caller-provided tags. - **Selective** mode relies on callers to identify content requiring protection, reducing processing for pre-validated inputs. ## Testing and Verification - Account-level enforcement can be configured in the Amazon Bedrock Guardrails console. - Enforcement can be tested with: - `InvokeModel` - `InvokeModelWithResponseStream` - `Converse` - `ConverseStream` - Responses include guardrail assessment details and identify the enforced guardrail. - Member accounts can verify organization-level enforcement in the Bedrock console. ## Important Considerations - Organizations must meet prerequisites such as configuring resource-based policies for guardrails. - Incorrect or invalid guardrail ARNs can cause policy violations, prevent safeguards from being enforced, and block model inference. - Automated Reasoning checks are not supported. - Charges apply for each enforced guardrail based on its configured safeguards. ## Availability Cross-account safeguards are generally available in all commercial and GovCloud AWS Regions where Amazon Bedrock Guardrails is available. Organizations can enable the feature through the Amazon Bedrock and AWS Organizations consoles. Overall, the capability gives security teams a centralized way to enforce responsible AI requirements while reducing the need to audit guardrail settings independently in every account and application.

Read original(opens in new tab)

GuardrailAmazon BedrockAws OrganizationsAi Safety+4

line·Mar 25, 2026·3 min readCurated summary

Advancing Guardrail Models through Automated Vulnerability Collection and Generation Using Coding Agents

LLM guardrails must detect prompt injection and jailbreak attempts without blocking legitimate requests that merely contain security-related keywords. The post argues that benchmark scores alone do not reflect production performance, especially false positives caused by missing input diversity. It presents a Codex-based, automated testing pipeline that generates categorized test data, evaluates the guardrail model, and analyzes failures reproducibly. ## The Gap Between Benchmark and Production Performance - The initial guardrail model performed well on external benchmarks but produced unexpected false positives in production-like tests. - Legitimate requests containing terms such as “ignore,” “bypass,” “override,” “system prompt,” or “jailbreak” were sometimes classified as attacks. - Examples included: - Development questions about temporarily bypassing authentication in a local test environment. - Educational requests about jailbreak techniques and defensive guidelines. - The core issue was insufficient representation of real-world input diversity, not simply poor model quality. - This motivated an automated environment for repeatedly discovering and analyzing guardrail weaknesses. ## Using Codex as a Test Automation Tool - The team adapted coding agents from software development tasks to complex, repeatable security testing. - Codex was used through its CLI capabilities to: - Read and create project files. - Edit code. - Execute evaluation scripts. - The pipeline relies on three Codex concepts: - **AGENTS.md:** Defines global rules, project conventions, commands, and security constraints. - **Sub-agents:** Allow a main orchestrator to delegate independent category tests to parallel worker agents. - **Skills:** Package repeatable procedures, input/output specifications, prompts, and scripts into reusable modules. ## Category-Based Experiments - Instead of sending thousands of random samples, experiments are divided into vulnerability and false-positive categories. - Example categories include: - Normal development or IT requests containing security-related keywords. - Educational or preventive requests involving sensitive topics such as jailbreaks or drug abuse prevention. - Categorization improves: - Root-cause analysis. - Parallel execution through independent workers. - Context clarity. - Regression testing after model changes. ## Separate Generation and Evaluation Skills ### `synthetic-generator` - Creates test queries according to each category’s specification. - Enforces constraints such as: - Attack type. - Sentence length. - Safe or dangerous target labels. - Produces varied, realistic phrasing and stores the dataset as JSONL. ### `injection-classifier` - Sends generated inputs to the guardrail model API through Python scripts. - Compares predictions with ground-truth labels. - Calculates false-positive and false-negative statistics. - Stores the original text, labels, predictions, and metrics in a consolidated JSONL file. Separating these procedures into skills provides intermediate artifacts for debugging, fixed input/output contracts for reproducibility, and independent maintenance of generation and evaluation logic. ## Pipeline Architecture - A **main agent**: - Reads `AGENTS.md` and `TEST_CATEGORY.md`. - Determines categories, sample counts, and constraints. - Creates and assigns work to category-specific workers. - Collects completion reports and verifies the run. - Each **category worker**: - Generates `input.jsonl` using `synthetic-generator`. - Evaluates the guardrail model using `injection-classifier`. - Produces `result.jsonl` with predictions and metrics. - Analyzes false positives and false negatives. - Writes a Markdown analysis report. - Stores outputs under `outputs/<run_id>/`, organized by category. ## Results and Practical Recommendation The pipeline enables systematic, repeatable testing rather than isolated discovery of misclassifications. For production guardrails, teams should combine benchmark evaluation with categorized real-world simulations, modular generation and evaluation steps, parallel test agents, and preserved JSONL artifacts for debugging and regression analysis.

Read original(opens in new tab)

GuardrailPythonAPIPrompt Injection+4