Humanity, introspecting
The music services (Lala, Last.fm, Pandora, etc.) provide related content by genre, but is anyone organizing music based on content? Because if you sort songs by content, there's really only, like, eight songs:
- romance ("Fifteen," by Taylor Swift)
- sex ("3," by Britney Spears)
- anger ("Girls", by Eminem)
- humanism ("One Love," by David Guetta)
- drinking ("Whiskey in the Jar," by the Pogues)
- politics ("Beds are Burning," by Midnight Oil)
- religion ("Vertigo," by U2)
- quirkiness ("Fat," by Weird Al)
You'd actually probably end up with a few more categories (nationalism? philosophy? musicianship?) but you get the idea. And here's the thing: we don't have to come up with the categories ourselves. We'll have the computers do it for us. The Netflix prize demonstrated that genres come out in the wash when you have a large enough data set: check out this automatically generated movie map!
Each of the categories above represents a "moment," if you will, that each of us experiences personally to one degree or another. By analyzing the content of popular music, we could get an aggregate view of humanity's collective mental state. I think that would be a fascinating thing to have. We would look at how the content of music evolves over time; that would be humanity remembering what it used to feel like. We would subset by genre to look for empirical answers to assertions like "rap is angry;" that would be humanity, introspecting.
You could do something similar with movies, but pop songs would be better for at least two reasons. First, a pop song is more singular than a movie; the ratio of moments ("anger") to items (each song/movie) is nearly one. Second, the volume of pop songs per year is higher; you have higher throughput.
So I'm thinking the initial data set would be date-stamped lyrics. Sooner or later you would want to bring the waveforms themselves into play. I suppose given waveforms you could do speech recognition to generate lyrics automatically, but there are lots of lyrics sites out there already. At the very least you want waveforms to help identify emphasis (lyrics would give you repetition). Perhaps you could skip natural language processing altogether and go straight to grouping based on the waveforms of the words of the songs rather than their character representations, but for me at least characters are going to be easier to work with than waveforms.
Where would you start?
