Check for the ssh env vars. For example, I use PS1="${SSH_CONNECTION:+\u@\h:}\W\$" to hide the hostname when not on SSH, you could do something similar with the control codes for color.
PS1="${SSH_CONNECTION:+\u@\h:}\W\$"
ESL by Hastie and Tibshirani https://hastie.su.domains/ElemStatLearn/
Check for the ssh env vars. For example, I use
PS1="${SSH_CONNECTION:+\u@\h:}\W\$"to hide the hostname when not on SSH, you could do something similar with the control codes for color.