If he makes any sign of wanting to breed the chickens (which will likely be soon as you have already mentioned he displays mating behavior), remove him immediately. Can't take the chance of your hens' insides being ripped apart.
As stated above, geese would rather protect themselves than a totally different species that does not really matter to them. They will sound an alarm and try to protect themselves (which chickens also do), but that is really the most they will do. And, more importantly than that, everyone...