Harlequin Shrimp is the only way to really get rid of them. At some point you will need to start feeding the harlequin when it runs out...or trade it to another reefer.

For what it's worth they also siphon out of the tank easily too. I get lots of them that way just off my glass when I do a water change.