The DOMTERM environment variable is normally set when running under DomTerm. However, using DOMTERM to determine if the current terminal is DomTerm is not reliable: If you ssh, DOMTERM will not be automatically set. On the other hand, if you start some other terminal emulator like xterm from within DomTerm, the xterm will inherit the DOMTERM variable will be misleadingly set.

So it is better to probe the terminal itself. You can use the domterm is-domterm command:

if domterm is-domterm
then
  execute domterm-specific actions
fi

If you’re not sure whether the domterm program will be in the PATH, you can use the following shell code. It works by sending a special request code (“Send [Secondary] Device Attributes”) to the terminal; the terminal sends a response unique to DomTerm. (While modern terminals will respond to this request, some older ones may not. Just in case, the script includes a timeout. Also, it only sends the request if TERM includes the string "xterm", which most terminal emulators do, or if DOMTERM is set.)

probe_domterm() {
  # probe if TERM unset, or contains "xterm", or DOMTERM is set
  case "$TERM/$DOMTERM" in
    /* | *xterm*/* | */?*)
      echo -en "\e[>0c" >/dev/tty
      read -s -t 1 -d c </dev/tty
      case "${REPLY}" in
        ?"[>990;"*";"*) DOMTERM_REPLY="${REPLY}" ;;
        "") DOMTERM_REPLY="-timeout)" ;;
        *) DOMTERM_REPLY="-non-match"
      esac
      ;;
    *) DOMTERM_REPLY="-not-tried"
  esac
}

is_domterm() {
  test -z "$DOMTERM_REPLY" && probe_domterm
  case "$DOMTERM_REPLY" in
    -*) return -1;;
    *) return 0;;
  esac
}

if is_domterm
then
  execute domterm-specific actions
fi

The is_domterm function only tests if the current terminal is domterm; it does not check if standard input/output have been re-directed. To check that, you can use the test builtin with the -t option:

if test -t 0 -a -t 1 -a -t 2 && is_domterm
then
  execute domterm-specific actions
fi