Scorecards

Scorecard Descriptor

Regardless of whether you're using UI editing or GitOps to manage your scorecards, the definitions are backed by JSON files. Each file is a fully compliant OpenAPI 3 spec file, with our own specific extensions.

You can still use Scorecards even if you don't use OpenAPI/Swagger.

We use the OpenAPI spec as a base for scorecard configuration, since it's an open spec with official support for extensions. That lets us extend it to be a scorecard descriptor spec with optional usage of actual OpenAPI fields.

All scorecard descriptors have 5 metadata fields:

• ID: A unique identifier for the scorecard.

• Title: A user-friendly display name.

• Description (Optional): A concise overview of the scorecard for further context.

• Blueprint ID: References the type of entity the scorecard applies to.

• Is Active: Indicates whether the scorecard is active and currently being applied to or not.

Ranks

The "Ranks" section of a Scorecard in Rely defines the performance tiers or levels that an entity can achieve based on predefined criteria. The available ranks are "bronze," "silver," or "gold". Each rank is associated with a set of rules that determine the conditions under which an entity qualifies for that specific rank.

Ranks in the scorecard descriptor should be sorted in ascending order, starting with the lowest rank: bronze, followed by silver, then gold.

Ranks help categorize performance or compliance levels in a structured way, allowing organisations to easily identify and prioritise areas for improvement or recognition.

Rules

Rules utilize the properties outlined in the blueprint to which the scorecard is applied, allowing for the comparison of each entity's defined value against a threshold value.

Each rule is defined by:

• ID: A unique identifier for the rule.

• Title: A user-friendly display name.

• Description (optional): A concise overview of the rule for further context.

• Conditions: A list of criteria that dictate how the data properties of the entity are evaluated. These conditions include:

  1. A property within the blueprint

  2. An operator (e.g., equal to, less than)

  3. And the value to compare against.

Condition Operators

Operator	Name	Description
eq	Equals	Tests whether the field's value is equal to the specified value.
lt	Less Than	Checks if the field's value is less than the specified value.
lte	Less Than or Equal	Determines if the field's value is less than or equal to the specified value.
gt	Greater Than	Evaluates if the field's value is greater than the specified value.
gte	Greater Than or Equal	Assesses if the field's value is greater than or equal to the specified value.
ne	Not Equal	Verifies that the field's value is not equal to the specified value.
contains	Contains	Checks if the field's value includes the specified substring or list element.
notcontains	Not Contains	Ensures the field's value does not include the specified substring or list element.

Example

{
  "id": "service-dora-performance",
  "title": "DORA Performance",
  "description": "Measures DevOps team performance using DORA metrics, focusing on Deployment Frequency, Lead Time for Changes, Mean Time to Recovery, and Change Failure Rate.",
  "isActive": true,
  "blueprintId": "service",
  "ranks": [
    {
      "id": "bronze",
      "rules": [
        {
          "id": "no-rollbacks-last-30-days",
          "title": "No rollbacks in the last 30 days",
          "description": "Number of rollbacks in the last 30 days equals 0",
          "conditions": [
            {
              "field": "data.properties.rollbackcountlast30days",
              "operator": "eq",
              "value": 0
            }
          ]
        },
        {
          "id": "incident-deploy-ratio-zero",
          "title": "Ratio of incidents to deploys is zero",
          "description": "Ratio of incidents to deployments in the last 30 days equals 0",
          "conditions": [
            {
              "field": "data.properties.incidentdeployratiolast30days",
              "operator": "eq",
              "value": 0
            }
          ]
        },
        {
          "id": "rollback-deploy-ratio-zero",
          "title": "Ratio of rollbacks to deploys is zero",
          "description": "Ratio of rollbacks to deployments in the last 30 days equals 0",
          "conditions": [
            {
              "field": "data.properties.rollbackdeployratiolast30days",
              "operator": "eq",
              "value": 0
            }
          ]
        }
      ]
    },
    {
      "id": "silver",
      "rules": [
        {
          "id": "less-than-five-bugs-last-30-days",
          "title": "Less than 5 bugs in the last 30 days",
          "description": "Number of bugs in the last 30 days is less than 5",
          "conditions": [
            {
              "field": "data.properties.bugcountlast30days",
              "operator": "lt",
              "value": 5
            }
          ]
        },
        {
          "id": "less-than-five-incidents-last-30-days",
          "title": "Less than 5 incidents in the last 30 days",
          "description": "Number of incidents in the last 30 days is less than 5",
          "conditions": [
            {
              "field": "data.properties.incidentcountlast30days",
              "operator": "lt",
              "value": 5
            }
          ]
        }
      ]
    },
    {
      "id": "gold",
      "rules": [
        {
          "id": "incidents-acked-within-5-mins",
          "title": "Incidents acknowledged within 5 minutes",
          "description": "Mean time to acknowledge (MTTA) incidents is less than or equal to 300 seconds over the last 30 days",
          "conditions": [
            {
              "field": "data.properties.meantimetoacknowledge",
              "operator": "lte",
              "value": 300
            }
          ]
        },
        {
          "id": "incidents-resolved-in-1h",
          "title": "Incidents resolved in less than 1 hour",
          "description": "Number of incidents resolved in more than 1 hour is 0 over the last 30 days",
          "conditions": [
            {
              "field": "data.properties.incidentsresolvedover1h",
              "operator": "lte",
              "value": 0
            }
          ]
        },
        {
          "id": "no-incidents-last-30-days",
          "title": "No incidents in the last 30 days",
          "description": "Total number of incidents in the last 30 days equals 0",
          "conditions": [
            {
              "field": "data.properties.incidentcountlast30days",
              "operator": "eq",
              "value": 0
            }
          ]
        }
      ]
    }
  ],
  "medianRank": "silver"
}

Learn More

Scorecard Actions

Creating a Scorecard

Scorecard Actions

Updating a Scorecard

Scorecard Actions

Evaluating Performance