Monitoring Machine Health

This guide explains how to programmatically monitor the health and performance of machines in a disguise network. It mirrors the indicators visible in Designer’s status bar and Network Status Widget, but accessed through the Designer APIs.

There are two approaches to monitoring:

REST endpoints — for one-off queries or infrequent polling. These are available under the Status API and System API.
Live Update API — for real-time streaming of data via WebSocket. This is the recommended approach for ongoing monitoring. See Live Update for full API documentation.

Frequent polling of REST endpoints is discouraged as it can reduce system performance. Use the Live Update API for any data that needs to be monitored continuously.

Machine health status

The health endpoint returns the status of all machines in the network, including their health indicators and dropped frame counts.

GET /api/session/status/health

const DISGUISE_IP = "10.0.0.1";
const response = await fetch(
  `http://${DISGUISE_IP}/api/session/status/health`
);
const data = await response.json();
for (const machine of data.result) {
  console.log(`${machine.machine.hostname}: ${machine.status.averageFPS} FPS`);
  for (const state of machine.status.states) {
      console.log(`  [${state.severity}] ${state.category}: ${state.name} - ${state.detail}`);
  }
}

import requests
DISGUISE_IP = "10.0.0.1"
response = requests.get(f"http://{DISGUISE_IP}/api/session/status/health")
data = response.json()
for machine in data["result"]:
  hostname = machine["machine"]["hostname"]
  fps = machine["status"]["averageFPS"]
  print(f"{hostname}: {fps} FPS")
  for state in machine["status"]["states"]:
      print(f"  [{state['severity']}] {state['category']}: {state['name']} - {state['detail']}")

Each entry in the states array corresponds to a sub-status of one of the four health categories (Connection, Output, Genlock, Devices). The severity field will be one of: “ok”, “warning”, “error”, “busy”, or “offline”. The category field identifies which health indicator the state belongs to.

FPS

Average FPS should be monitored via the Live Update API rather than polling the health endpoint.

const socket = new WebSocket("ws://director:80/api/session/liveupdate");
socket.onopen = () => {
  const subscribeMessage = { subscribe: {
      object: 'subsystem:MonitoringManager.findLocalMonitor("fps")',
      properties: ['object.seriesAverage("Actual", 1)']
  } };
  socket.send(JSON.stringify(subscribeMessage));
};
socket.onmessage = (event) => {
  const data = JSON.parse(event.data);
  if (data.valuesChanged) {
      for (const update of data.valuesChanged) {
          console.log("FPS:", update.value);
      }
  }
};

import json
from websockets.sync.client import connect
with connect("ws://director:80/api/session/liveupdate") as socket:
  subscribe_message = { "subscribe": {
      "object": 'subsystem:MonitoringManager.findLocalMonitor("fps")',
      "properties": ['object.seriesAverage("Actual", 1)']
  } }
  socket.send(json.dumps(subscribe_message))
  while True:
      message = json.loads(socket.recv())
      if "valuesChanged" in message:
          for update in message["valuesChanged"]:
              print(f"FPS: {update['value']}")

Replace MACHINE_NAME with the hostname of the target machine.

const subscribeMessage = { subscribe: {
  object: 'subsystem:MonitoringManager.findRemoteMonitor("MACHINE_NAME:d3", "fps")',
  properties: ['object.seriesAverage("Actual", 1)']
} };
socket.send(JSON.stringify(subscribeMessage));

subscribe_message = { "subscribe": {
  "object": 'subsystem:MonitoringManager.findRemoteMonitor("MACHINE_NAME:d3", "fps")',
  "properties": ['object.seriesAverage("Actual", 1)']
} }
socket.send(json.dumps(subscribe_message))

Interpreting FPS values

The second argument to seriesAverage is the number of samples to average. Using a value of 1 returns the latest value in the series.

For alerting, a rolling average over 10 data points is recommended to avoid false positives during transient events like content loading:

Warning: average FPS drops by 5–9 fps below expected framerate
Error: average FPS drops by 10 fps or more below expected framerate

Notifications

Retrieve all current notifications across machines in the network:

GET /api/session/status/notifications

const response = await fetch(
  `http://${DISGUISE_IP}/api/session/status/notifications`
);
const data = await response.json();
for (const machine of data.result) {
  for (const notification of machine.notifications) {
      console.log(`[${machine.machine.hostname}] ${notification.summary}: ${notification.detail}`);
  }
}

response = requests.get(f"http://{DISGUISE_IP}/api/session/status/notifications")
data = response.json()
for machine in data["result"]:
  hostname = machine["machine"]["hostname"]
  for notification in machine["notifications"]:
      print(f"[{hostname}] {notification['summary']}: {notification['detail']}")

System performance

The following Live Update subscriptions provide real-time performance metrics. These correspond to the profiling graphs visible in Designer but do not affect the health indicators.

GPU profiler

const subscribeMessage = { subscribe: {
  object: 'subsystem:MonitoringManager.findLocalMonitor("GPUProfiler")',
  properties: ['object.seriesAverage("Total", 1)'],
  configuration: { updateFrequencyMs: 100 }
} };
socket.send(JSON.stringify(subscribeMessage));

subscribe_message = { "subscribe": {
  "object": 'subsystem:MonitoringManager.findLocalMonitor("GPUProfiler")',
  "properties": ['object.seriesAverage("Total", 1)'],
  "configuration": { "updateFrequencyMs": 100 }
} }
socket.send(json.dumps(subscribe_message))

CPU utilisation

const subscribeMessage = { subscribe: {
  object: 'subsystem:MonitoringManager.findLocalMonitor("CPU")',
  properties: ['object.seriesAverage("Total", 1)'],
  configuration: { updateFrequencyMs: 100 }
} };
socket.send(JSON.stringify(subscribeMessage));

subscribe_message = { "subscribe": {
  "object": 'subsystem:MonitoringManager.findLocalMonitor("CPU")',
  "properties": ['object.seriesAverage("Total", 1)'],
  "configuration": { "updateFrequencyMs": 100 }
} }
socket.send(json.dumps(subscribe_message))

GPU memory

const subscribeMessage = { subscribe: {
  object: 'subsystem:MonitoringManager.findLocalMonitor("GPUMemory")',
  properties: ['object.seriesAverage("Usage(MB)", 1)'],
  configuration: { updateFrequencyMs: 100 }
} };
socket.send(JSON.stringify(subscribeMessage));

subscribe_message = { "subscribe": {
  "object": 'subsystem:MonitoringManager.findLocalMonitor("GPUMemory")',
  "properties": ['object.seriesAverage("Usage(MB)", 1)'],
  "configuration": { "updateFrequencyMs": 100 }
} }
socket.send(json.dumps(subscribe_message))

CPU memory

const subscribeMessage = { subscribe: {
  object: 'subsystem:MonitoringManager.findLocalMonitor("ProcessMemory")',
  properties: ['object.seriesAverage("Usage (MB)", 1)'],
  configuration: { updateFrequencyMs: 100 }
} };
socket.send(JSON.stringify(subscribeMessage));

subscribe_message = { "subscribe": {
  "object": 'subsystem:MonitoringManager.findLocalMonitor("ProcessMemory")',
  "properties": ['object.seriesAverage("Usage (MB)", 1)'],
  "configuration": { "updateFrequencyMs": 100 }
} }
socket.send(json.dumps(subscribe_message))

Disk utilisation

Disk utilisation monitoring via the Live Update API is currently in development (DSOF-20732).

When available, healthy disk utilisation is approximately 79% or below. For alerting, a rolling 10-data-point average is recommended:

Warning: 80%–89% utilisation
Error: 90% or above

System information

The following REST endpoints are available via the System API to query machine and hardware configuration. These run against the d3 service and do not require a running Designer session.

System API endpoints use the Service namespace and cannot be polled via the Live Update API.

Detect machines on the network

GET /api/service/system/detectsystems

Returns all machines on the network, their hostnames, IP addresses, software versions, and which processes are running.

const response = await fetch(
  `http://${DISGUISE_IP}/api/service/system/detectsystems`
);
const data = await response.json();
for (const machine of data.result) {
  console.log(`${machine.hostname} (${machine.ipAddress}) - v${machine.version.major}.${machine.version.minor}.${machine.version.hotfix}`);
  console.log(`  Designer: ${machine.isDesignerRunning}, Service: ${machine.isServiceRunning}`);
}

response = requests.get(f"http://{DISGUISE_IP}/api/service/system/detectsystems")
data = response.json()
for machine in data["result"]:
  v = machine["version"]
  print(f"{machine['hostname']} ({machine['ipAddress']}) - v{v['major']}.{v['minor']}.{v['hotfix']}")
  print(f"  Designer: {machine['isDesignerRunning']}, Service: {machine['isServiceRunning']}")

OS information

GET /api/service/system/osinfo

Returns Windows and image version information for all machines on the network.

VFC cards

GET /api/service/system/vfcs

Returns VFC card details including slot, type, firmware version, and port configuration. Returns an empty list on non-disguise hardware.

Session and project information

Project information

GET /api/session/status/project

Returns the current project path and Designer version.

Session configuration

GET /api/session/status/session

Returns session configuration including whether the session is running solo, who the Director is, and lists of Actors and Understudies with their GUI state.

RenderStream monitoring

RenderStream status can be monitored via the Live Update API. Use the RenderStream API endpoints to obtain workload IDs.

Workload receive statuses

Subscribe to receive statuses for all machines in a workload:

const subscribeMessage = { subscribe: {
  object: "subsystem:RenderStreamSystem",
  properties: ["object.getWorkloadReceiveStatuses(<workload ID>)"]
} };
socket.send(JSON.stringify(subscribeMessage));

subscribe_message = { "subscribe": {
  "object": "subsystem:RenderStreamSystem",
  "properties": ["object.getWorkloadReceiveStatuses(<workload ID>)"]
} }
socket.send(json.dumps(subscribe_message))

Workload instances

Get all instances in a workload:

const subscribeMessage = { subscribe: {
  object: "subsystem:RenderStreamSystem",
  properties: ["object.getWorkloadInstances(<workload ID>)"]
} };
socket.send(JSON.stringify(subscribeMessage));

subscribe_message = { "subscribe": {
  "object": "subsystem:RenderStreamSystem",
  "properties": ["object.getWorkloadInstances(<workload ID>)"]
} }
socket.send(json.dumps(subscribe_message))

Workload instance health

Health status is a computed property and must be accessed per-instance:

const subscribeMessage = { subscribe: {
  object: "subsystem:RenderStreamSystem",
  properties: ["object.getWorkloadInstance(<workload ID>, <instance index>).health()"]
} };
socket.send(JSON.stringify(subscribeMessage));

subscribe_message = { "subscribe": {
  "object": "subsystem:RenderStreamSystem",
  "properties": ["object.getWorkloadInstance(<workload ID>, <instance index>).health()"]
} }
socket.send(json.dumps(subscribe_message))

Failover status

Check whether a machine has taken over for a failed machine:

const subscribeMessage = { subscribe: {
  object: "getByUID(<machine UID>)",
  properties: ["object.hasTakenOverFailedMachine()"]
} };
socket.send(JSON.stringify(subscribeMessage));

subscribe_message = { "subscribe": {
  "object": "getByUID(<machine UID>)",
  "properties": ["object.hasTakenOverFailedMachine()"]
} }
socket.send(json.dumps(subscribe_message))

Search documentation