You know what I think it would be great, if you do a browser plugin that reads your image format. So, if you have the images hosted somewhere and you open them from your browser it shows it like Facebook does: picture big at left, data at the right, you could even tag people 
And then, when you select an images folder it takes it as an album, so it pre-reads all the header info and sort the images by date, and gives you a sort criteria option where you can resort by other tagged info or even view only pictures that accomplish with some criteria 
That will be a good product, you have your album, your data and you don't need to expose your privacy by posting them in FB or any other site alike.
Can you go into more detail on the browser plugin? It sounds fantastic, I am just thinking in depth. As for the tagging, I had the idea of replicating Facebook's to a degree. I already have a basic facial recognition option in it, which is a nice start I think.
How will you handle lazy users (like me) who won't tag their photos right away? Will you have options for applying multiple tags to multiple photos at once, or some sort of auto tagging feature? I'm not sure that you'd want to tag your own thousands of photos one by one when you first run the program, either. If you're going to import EXIF data or Windows file tags, what will make your program worth using over the standard Windows filtering?
That is a massive goal of mine. High priority. Being able to mass edit photo info is a key component. But, why my program will be worth using over the standard Window's filter, is that you will be able to access information that you cannot easily do in Windows. If you want to find photos specific to an event, place, people, time period, etc you can easily find them. I am also going to have the option to add categories of your own. So, the filtering will much more extensive and personable.
When you have thousands of photos, regardless of how organized they are in folders this will give people the ability to quickly find what they are looking for.