If they are advised it should be a prohibited area, that means it's too dangerous to stand in, so they should just agree, sign, and then they are not liable. I imagine the TT insurance would then cover them comprehensively. If, on the other hand they decide to fill their garden with people, they must take responsibility if they have been warned.
It's the same at the Irish road races, and you often see people in their own or their friends gardens, where you would not feel safe at all.