If it is the factory tower ... expect the rattles. I have the 2003 year sportster 4tec with the factory tower (assume that is what you have), and am living through it.
jesseconsopolus, is on the mark though ... check the mount points and make sure they are solid.
I think where a lot of the movement ,that allows the noise, comes from is the mount point. If you take a look at the tower connector to the mount point, the connector allows the tower arm to move in the connector (there is a groove that it slides in) -- not much but probably 1/2 to 3/4 inch of movement. On each side of the tower, and all the mount points, that can add up and allow significant movement, sway, and noise from the tower.
The other issue I have been having with this tower (and 2004 Sportster is the same design), is two of the welds giving way (have a thread on hear about it) on a recurring basis - which I think is also due to this movement as we haven't really been using the tower for skiing or wake boarding.
Other than that it is a great boat, and tonnes of fun on the water.
As Jesse said ... turn the tunes up a bit louder and enjoy (but watch the welds).
shamski