Start with operational questions

Useful measures answer questions such as:

  • Did the job reach its defined final artifact?
  • Which stage sent work backward for correction?
  • What information caused waiting?
  • Which completed stage ran again, and why?

These measures describe the process. They do not establish revenue, productivity, satisfaction, or quality beyond the evidence the workflow records.

Define measures from recorded state changes

W3C PROV-O supplies a vocabulary for connecting entities and activities, which can support counts of revisions and completed transformations. The NIST AI Risk Management Framework emphasizes measurement as part of managing AI risk. CommonMark can keep definitions and observations in a readable record.

Write each measure with four fields:

  1. The event being counted or timed.
  2. The source record that proves the event occurred.
  3. The period or job set under examination.
  4. The interpretation the measure does and does not support.

Count the return for correction

The synthetic history records one return for correction after evidence comparison finds an unsupported sentence.

Avoid vanity measures and false precision

This approach fits workflows with named stages and durable state records. It fits poorly when stage completion exists only in someone's memory or when the metric definition changes between jobs.

Do not present a low correction count as proof of accuracy. A process can miss errors entirely. Pair operational measures with direct inspection of the artifact and its evidence.

Connect measurement to process change

Return to Map an AI Workflow for One Recurring Job for the full design. Use AI Workflow Exceptions to name the incomplete states worth observing. Read AI Workflow Versioning before comparing measures across changed process definitions.