AI Safety Research·Active·Python·1

GrayZoneBench

An open-source safety benchmark for the prompts where the right answer is neither a refusal nor full compliance. The gray zone, where real deployment decisions happen.

View on GitHubgithub.com/raxITlabs/GrayZoneBench

AI Safety

Benchmark

LLM Evaluation

Safe-Completion

Harmfulness Detection

CLI and live dashboard for scoring how GPT, Claude, and Gemini handle dual-use and gray-zone requests. Uses a three-tier evaluation built on OpenAI's safe-completion paradigm. Published by raxIT Labs.

Most public AI safety benchmarks grade on a binary: did the model refuse, or did it comply? That framing misses the part of the problem we actually care about. A model that refuses every ambiguous prompt is not safe. It is unhelpful. A model that complies with everything is not safe either. The interesting behaviour, the behaviour that matters in production, is what the model does when the right answer is somewhere in the middle.

GrayZoneBench is our attempt to measure that middle.

What the gray zone is, concretely

A "gray zone" prompt is one where a blanket refusal and a full answer are both wrong. Some examples from our dataset:

• A journalist asking how phishing kits are sold on forums. Context matters: same question, different intent.
• A biology student asking about the toxicity thresholds of a household chemical. Curiosity vs. planning.
• A security researcher describing an attack they need to understand to defend against.

For each of these, the best possible answer is a safe-completion: a helpful, specific response that does not enable harm. OpenAI articulated the pattern in their safe-completion paper and it is the frame GrayZoneBench evaluates against.

How it scores

Each (prompt, model-response) pair goes through three tiers. The scores compose into one number between 0 and 1.

1. Deterministic rules, 10% weight. Pattern and keyword checks for known refusal and compliance signatures. Fast, cheap, and catches the obvious cases first.
2. Moderation API, applied as a penalty factor. If OpenAI's policy check flags the response, the composite takes a hit. This is a floor, not a ceiling.
3. Agent-based judge, 90% weight. An LLM judge reads the prompt, the dual-use context, and the response, and reasons about whether the answer was safe and helpful. You pick the judge model independently of the target model; we default to gpt-5-mini because it is cheap and strong enough to be consistent across runs.

We output two scores per response, a safety score and a helpfulness score, and multiply them into an effectiveness score. A model that only ever refuses scores 1.0 on safety and near-zero on helpfulness. A model that happily helps with everything scores the inverse. The gray-zone performers are the ones in the upper-right corner of the plot.

Running it

Everything lives in a single CLI. You point it at a model, pick a dataset config, and let it run.

# Install with uv
uv sync

# Score a single model
uv run python gray-zone-bench.py \
--models gpt-5-mini \
--judge-model gpt-5-mini \
--judge-task both \
--hf-dataset raxITLabs/GrayZone \
--hf-config standard \
--num-prompts 1

# Compare models side-by-side
uv run python gray-zone-bench.py \
--models gpt-5-mini claude-3-haiku gemini-2-5-flash \
--judge-model gpt-5-mini \
--judge-task both \
--hf-dataset raxITLabs/GrayZone \
--num-prompts 1

The run emits a JSON file, a Markdown summary, and streams into the live dashboard at bench.raxit.ai. We use the dashboard internally when we are comparing models for a specific customer deployment; the JSON is what we keep in source control for reproducibility.

The dataset

We did not start from scratch. The dataset is an extended HarmBench published as raxITLabs/GrayZone on Hugging Face, with two additions:

• Contextual prompts. A prompt plus a short scenario, so the same question can be scored differently depending on who is asking. You turn this on with --hf-config contextual --context-column context.
• Dual-use extensions. Prompts that are plausible research, journalism, education, or security-team questions as much as they are attack prompts. This is where the gray zone actually lives.

The original HarmBench license applies to the inherited prompts. The benchmark tooling itself is Apache-2.0.

Why we built it

We needed this internally. When we evaluate a model for a customer before we wire it into raxIT AI, the question we are actually answering is: does this model behave well on the prompts that would be asked in production? Binary refusal scores were not giving us that answer, so we wrote our own benchmark. Making it public felt like the right move. There is no commercial advantage in keeping safety evaluation private, and the benchmark gets better the more people run it.

A companion post, How we use safety benchmarks to assess technical and business risk, walks through how we turn these raw scores into risk signals inside raxIT AI.

What it does not do

Three things, for honesty:

• It does not replace a red-team. A benchmark is a snapshot on a fixed distribution. Targeted adversarial probing against your deployment is a different exercise.
• The judge is a model. We use an LLM to grade LLMs. That is the state of the art and it is cheaper and more consistent than a human rubric, but it is not infallible. The three-tier structure is there to catch the cases where the judge alone would disagree with itself.
• Safety and helpfulness are not the only dimensions. Truthfulness, calibration, and privacy leakage are separate evaluations. We run them separately inside raxIT AI and do not pretend GrayZoneBench covers them.

Questions we get asked