For some, Brexit means nailing your colours to a new flagpole