Skip to content
Rota Nacional

Guides ·

How to compare models without exposing real documents

Build a small task set, measure quality and cost, and evaluate the effect of your privacy policy.

1. Select concrete tasks. Define verifiable outputs: extract a field, classify a request or summarize a decision. Write acceptance criteria before running calls. Separate simple tasks from ambiguous cases; do not rely only on a score given by the model itself.

2. Prepare synthetic examples. Reproduce document formats, lengths and difficulty without copying real identities. Include missing fields and hostile instructions in text. Reserve a separate set to check performance on examples not used during tuning.

3. Compare identical inputs and limits. Record the model, available version, policy, context size, latency and consumption. Distinguish incorrect answers, incomplete answers and appropriate refusals. Longer output is not evidence of better performance.

4. Evaluate privacy alongside quality. Compare usefulness with placeholders, removal and blocking. Check whether responses reconstruct identities or repeat unnecessary details. Choose the combination meeting your criteria and repeat evaluation when models, prompts or formats change.

Get new articles

Privacy, AI engineering and security in your inbox.

Rota Nacional

Bring privacy into your workflow.

30 days, no card, with a starting quota. After that, Pix credit from R$ 5,00.

Try free