Yes. When the Heat Tape is plugged into a working outlet the plug will light up.
Note:
If you are using a Thermostatic switch like the ThermaCord with Heat Tape Pro, the plug on your heat tape will only be lit up when the ambient temperature is low enough and your ThermaCord has switched on power to your Heat tape. (for Pipe Freeze Pro, the thermostatic switch is integrated and comes after the plug, so your plug should be illuminated)