Golden Dataset & Rubric Workshop
Facilitated sessions where your experts define "good" and we turn it into rubrics and a labeled seed set.
The problem
Engineers cannot write the rubric and experts cannot write the test code.
What we do
- Facilitated rubric sessions
- Labeling of a seed dataset
What you receive
- Rubrics and a labeled seed set within 5 business days of the session
What you provide
- Expert attendance
- Sample outputs
Limits
- Starts only after our formal data policy is published and a data agreement is signed, because it uses your transcripts or outputs
- Design work on your evaluation system (see independence policy)
Results describe behavior within the tested scope and configuration. They give you independent evidence for security reviews, audits and compliance work; certification itself is issued by accredited bodies.
Availability and independence
Pilot: Procedure designed and deliverable with existing tools, but not yet delivered to a client. Sold under pilot terms. Procedure designed and deliverable with existing tools, but not yet delivered to a client. Sold under pilot terms.
Pilot deliveries are scoped in writing, tell you it is an early delivery, and follow the same review and evidence rules as any engagement.
Evaluation-design work. It is recorded, and any later TrendLab evaluation of the same agent discloses it.
What is covered
The test areas this service always includes, with their status at the access level it uses.
- Checklist rubrics, single-question judges, and their calibration against client labelsHow do we score quality, and can we trust the scores?Pilot: Procedure designed and deliverable with existing tools, but not yet delivered to a client. Sold under pilot terms.
- Domain-expert rubric authoringCan our experts set the bar?Pilot: Procedure designed and deliverable with existing tools, but not yet delivered to a client. Sold under pilot terms.