Casey seemed to like "Stuck In the Middle With You" by Stealers Wheel, as on 6/23/73 he noted that Gerry Rafferty had left the group and was debating whether to return; Casey said "I hope he does, because they have a great sound together!" Gerry did come back and they had one more top 40 hit together ("Star" in 1974), and then left again, this time for a successful solo career.
And in the same show he said that Charlie Rich's 1960 hit "Lonely Weekends" was one of his favourites.
As people have mentioned in the 70's Show thread, Casey praised Jackson 5's "Lookin' Through The Windows" on 7/29/72, saying that if it was by a new band it would go straight to the top ten. (As it was it reached #16, good for most bands but a disappointment for the Jacksons.)
On 8/15/87, Casey seemed to speak positively about Suzanne Vega's "Luka" and U2's "I Still Haven't Found What I'm Looking For" ("A song of searching, a song of hope"), though I'm not sure if he was really personally praising the songs or reading off the script. If he did listen to the songs, though, he may well have appreciated the anti-child abuse message of the former and the hopeful, meaningful lyrics of the latter.
Not technically a song, but on 10/23/71 Casey said that Les Crane did a great job with his reading of "Desiderata" which became a hit. Casey evidently liked the poem as well, as he said in this show that had a poster of it hanging up in his living room.