It was quite a task when I started to organize my sheet music. I invited my older students over and we made a few days of fun out of it. I have 2 lateral file cabinets that I bought as well as some 2 drawer plastic storage things (from Target). Per the cabinets, one holds all my chamber music and the other holds all my solo music. I organized by instrumentation for the chamber (duet, trio, quartet, etc... (by composer)) and for the tall cabinet, they hold my music organized by composer and categorized within composer by concerto, sonata, others. For compilation books, I leave them all in a plastic drawer by difficulty (top drawer for students and bottom for myself) and also keep all my teaching materials separated in the plastic drawers by instrument and difficulty. All in all, I have about 4 of those plastic bin/drawer things (8 drawers in all) but everything is organized in hanging folders with the plastic tab that shows the name of the piece and composer. It's the easiest way to find anything. I also have quite a bit of piano and vocal music that I have separated into crates that sit stacked up (open-faced) in a closet in my music room. The crates are a really easy way to keep your music (though many originals don't fit in there very well, which is why I invested in the cabinets). I keep a bucket thing I bought on clearance after Easter on top of on of the plastic storage things to drop any music that I've pulled and have yet to put back. I go through the bucket at the end of the week and put back the music to where it's supposed to go.
I keep seasonal (read: Christmas) and wedding music books separated on a bookshelf in the basement. I find that since I don't use them as often, I don't like to have them cluttering the rest of my space. Of course, when wedding or Christmas season approaches, they are put back into the rotation as necessary. =)
As I was organizing, I made sure to put everything I owned into a spreadsheet so when I lent music to someone, I could keep track of it. I also make sure to keep the list with me so that I don't accidentally make double purchases of music I don't need since it can get quite expensive!
Anyway, good luck to you and let me know what you come up with!!! I'm always up for new ideas and new ways to organize all my clutter!