Hiya Mike,
It can be done with Windows, but it depends on how you set up and save the file and what version of software you're using. I'm using Illustrator 9 and can view illustrator EPS thumbnails. I'm not sure, but I assume it would work for AI files too.
Most vector clip art is normally saved as an older version of illustrator without the thumbnail option. So, if you resave the file and you select the "include document thumbnail" option, you can view it in explorer's thumbnail view.
I just need to figure out an action that can resave all the clip art I have
Edit...
Just for kicks and giggles, I tried EPS and AI files and they both work. You have to save them as Illustrator version 7 or newer and include document thumbnail.
Checkers