Measuring Trial Abuse: Metrics to Track Before and After Device Limits

Most teams decide to fight trial abuse on a feeling: the signup chart looks too good, the infrastructure bill looks too high, and support keeps seeing the same names. Feelings are a fine reason to start. They are a bad way to choose a threshold, and a worse way to tell whether the threshold is working.

This post defines a small set of metrics, shows how to compute them from data you already have plus a Prynt event lookup, and explains how to read them during a monitor period, before any signup is refused.

Instrument first, enforce later

Every metric below depends on one thing: for each signup, you stored the device verdict even when you did nothing with it. In practice that means your signup handler does the usual lookup and records the result:

const event = await prynt.getEvent(requestId);    // GET /v1/events/{requestId}
await db.signupAudit.insert({
  userId: user.id,
  requestId,
  visitorId: event.visitorId,
  otherAccounts: event.accountsOnDevice.count,
  decision: event.decision,                          // 'allow' | 'challenge' | 'block'
  riskScore: event.riskScore,                        // 0–100
  wouldBlock: event.decision === 'block' || event.accountsOnDevice.count >= LIMIT,
  createdAt: new Date(),
});
await prynt.updateEvent(requestId, { linkedId: String(user.id) });

wouldBlock is the important column. During the monitor period it is never acted on; it records what enforcement would have done. If you use console rules instead of code, the tag action serves the same purpose: it labels events without changing the decision.

Also record signups with no requestId, with visitorId left null. That count is a metric in its own right.

The core metrics

Repeat-device signup rate

The share of new signups from a device that already holds at least one of your accounts.

SELECT date_trunc('week', created_at) AS week,
       avg((other_accounts >= 1)::int) AS repeat_device_rate
FROM signup_audit
WHERE visitor_id IS NOT NULL
GROUP BY 1 ORDER BY 1;

This is your baseline for how much repeat signup exists at all. Look at its distribution too, not just the average: the number of devices with five or more accounts usually says more about organized abuse than the rate does.

Concentration

What share of all trials comes from the most prolific devices. A healthy product has a long, flat tail where nearly every device has one account. Abuse shows up as a small set of devices holding a disproportionate number of accounts. Rank devices by account count and look at the top.

Cost per activated trial

Total trial cost divided by trials that reach your activation event. Trial cost is whatever a trial consumes: compute, model tokens, credits, seats, support time. Activation is the action that predicts conversion in your product.

Repeat trials inflate the numerator and usually not the denominator, because people farming trials tend to use exactly the free resources and skip the setup that real activation requires. If this number falls after enforcement while activations hold steady, the limit removed waste rather than customers.

Flagged-to-converted ratio

Of the signups where wouldBlock was true, how many later became paying customers?

SELECT count(*) FILTER (WHERE s.would_block)                      AS flagged,
       count(*) FILTER (WHERE s.would_block AND u.paid_at IS NOT NULL) AS flagged_and_paid
FROM signup_audit s JOIN users u ON u.id = s.user_id
WHERE s.created_at < now() - interval '30 days';   -- give trials time to convert

This is the number that keeps you honest. Every flagged signup that paid is revenue your rule would have refused. Compare its conversion rate with unflagged signups: if flagged signups convert at a comparable rate, your limit is catching real customers, often on shared devices. If they convert at a small fraction of the normal rate, the rule is aimed well.

Appeal rate

After enforcement, the share of refused signups that contact support, and the share of those appeals you uphold. A rising appeal rate with a high uphold rate means false positives. A low appeal rate is not proof of accuracy, because most wrongly refused users simply leave, which is why the flagged-to-converted ratio during monitoring matters more.

Unidentified rate

The share of signups with no usable identification. Prynt honors Global Privacy Control by default and stores no event for those visitors, and script blockers strip agents. A sudden jump in this rate after you enforce suggests abusers have learned to arrive unidentified, which is your cue to decide what missing identification means: flag for review, require email verification, or withhold the trial.

Reading the monitor period

Run log-only for at least one full trial length plus a week, so the converted column has time to fill in. Then answer four questions in order.

1. Is there a problem worth solving? If the repeat-device rate is low and concentration is flat, your trial cost is driven by something else. Stop here.
2. Would the rule hit the right people? Check flagged-to-converted against your baseline conversion. Read a sample of flagged accounts by hand; patterns like sequential emails, identical setup steps and no activation are obvious once you look.
3. What threshold? Recompute wouldBlock at limits of 1, 2 and 3 other accounts. Lower limits catch more abuse and more shared devices. Choose the point where flagged conversion is clearly below normal.
4. Which response? Not every flagged signup needs a refusal. Many teams create the account without a trial and offer a paid plan, which converts some of the flagged traffic instead of losing it.

After enforcement

Keep the same queries running. The useful comparisons are before versus after on cost per activated trial, the activation rate of the remaining trials, and appeal rate. Expect the repeat-device rate to drop, then partially recover as persistent abusers adapt by switching devices or browsers. Watch reason codes in the stored events, such as VPN, RESIDENTIAL_PROXY, INCOGNITO or TAMPERING, to see how they adapt.

If you want a cleaner causal read, hold out a small random slice of signups from enforcement and compare cohorts; A/B testing risk rules covers how to size and run that safely. Report confirmed bad accounts back with POST /v1/outcomes using labels such as abuse or fraud, and the good ones as legit, so the labels feed Prynt’s reputation for your environment.

Related reading

The bot side of the same discipline is in bot detection metrics and KPIs and measuring your bot block rate. For the enforcement mechanics, the docs cover the event fields used above, and the playground shows what a repeat device looks like in a live result.

The metrics are simple. The discipline is collecting them before you need them, so the first enforcement decision you make is based on your own data rather than a guess.