Documentation · Prynt Control

Rule evaluate mode

Run a new rule on live traffic without affecting anyone: an evaluate rule never changes the decision, it counts how often it would have matched, and you activate it in one click.

Rules let you add what you know about your business on top of Prynt’s decisions: “challenge signups from datacenter networks”, “block a fourth account on the same device”. A new rule is a guess until it has seen your traffic, so every rule can run in evaluate mode first.

An evaluate rule runs on every request like an active one, but it never changes the decision. Prynt only records that it matched and counts the matches, so you can see how often it would have fired before it affects anyone.

Uber’s fraud rules engine works the same way: a rule with the Evaluate status “is being run but won’t actually affect the user” (Uber Engineering, 2017).

Two modes

modeRunsChanges the decisionUse it for
evaluateYesNo — matches are recorded and countedEvery new rule, and every change you are unsure of
activeYesYes — its action appliesRules you have checked

A rule’s enabled flag still turns it off entirely, in either mode.

Create a rule in evaluate mode

In the console: Rules → New rule, and set the mode to evaluate (log only).

The console calls POST /admin/rules (admin role). A rule is a condition and an action; mode defaults to active, so set it explicitly:

{
  "name": "Challenge signups from datacenter networks",
  "field": "datacenter",
  "op": "eq",
  "value": "true",
  "action": "challenge",
  "priority": 100,
  "mode": "evaluate"
}

action is one of allow, challenge, block or tag. The fields a rule can test (bot, vpn, datacenter, multiAccount, velocity, country, riskScore and more) are listed in the console’s rule editor.

See what it would have done

Rules in the console shows, next to each evaluate rule, how often it would have matched in the last 7 and 30 days. GET /admin/rules returns the same counts in evaluation:

{
  "rules": [
    {
      "id": 51,
      "name": "Challenge signups from datacenter networks",
      "field": "datacenter", "op": "eq", "value": "true",
      "action": "challenge", "priority": 100, "enabled": true,
      "mode": "evaluate",
      "evaluation": { "matches7d": 212, "matches30d": 640 }
    }
  ]
}

Each matching event also keeps the ids of the evaluate rules it matched in its risk data, as evaluatedRules, so you can open the actual requests a rule would have acted on:

{ "requestId": "req_…", "decision": "allow", "risk": { "reasons": ["datacenter"], "evaluatedRules": [51] } }

Before you activate, look at three things:

  1. How often it fires. 212 matches in a week against your traffic volume. A rule that matches a large share of your requests is usually describing your users, not your fraud.
  2. Who it matches. Open some of the events with evaluatedRules and check them. If you send linkedId (Send your user id), you can see which accounts are involved and whether they are good customers.
  3. Whether the action fits. If most matches look like real people on a VPN, challenge is a better action than block.

Activate it

In the console, press Activate on the rule. The console sends PATCH /admin/rules/{id} with { "mode": "active" }, and the change is written to the audit log as rule.activate. You can set it back to evaluate the same way at any time.

Evaluate mode and mirror mode

They answer different questions. Mirror mode answers allow to everything for a period and shows what all of Prynt’s decisions would have been. Evaluate mode tests one rule while everything else keeps working normally. Use mirror mode when you start; use evaluate mode for every rule you add after.

Good habits

  • Give a rule at least a week in evaluate mode, or enough traffic to see a few dozen matches, before you activate it.
  • Prefer challenge to block unless the evidence is direct (for example, a device already labeled fraud).
  • Make the rule’s name a sentence that says what it does. Six months later, that sentence is the documentation.
  • Prynt never blocks on its own. An active rule changes the decision Prynt returns; your code still decides what happens.

Next: Use-case decisions