This is probably pretty old but Rolling Stone has a list of the greatest 500 albums of all time. It seems rather too focused on the 60's with 4 beatles albums in the top 10. There is also not a huge diversity of artists in the Top 100 with many artists appearing several times.
Allofmp3.com my favourite download service, largely because it has absolutely no device crippling DRM associated with the music, has conveniently listed all 500 albums.
I am now going through my music collection and filling in some of the gaps in both genre and era with some of those greatest albums.
Check it out at Rolling Stone and get them at allofmp3.com if you are so inclined