Neither of the following bands are among the weirdest stuff I listened to, but I don't see them get mentioned around here:
Anything by Gary Wilson is pretty weird. He sings about all these women that got away from him in the 1970s in Endicott NJ. His album from the 70's "You Think You Really Know Me" is great, as well, as his album he released a few years ago "Mary Had Brown Hair."
http://www.myspace.com/garywilson64
listen to the song "Gary saw Linda kissing Frank Roma"
Also, Grand Buffet. Cool rap group from Pittsburgh. Two guys who perform with a CD player playing prerecorded music. Their albums are pretty weird.
Here's a video of them live:
http://www.youtube.com/watch?v=9Fst_dJ48_8