Ground pumice, fine sandpaper, or rottenstone in ever decreasing granule size until you reach the desired finish level. It takes scratching to remove scratches. If there's a finish (clear coat, anodize etc...) then nothing but re-coating will help, polishing will only make the holes in the finish bigger.

-Zeke