Swat wants the royal couple to visit its White Palace