This past week, I was watching one of my favorite TV shows House, and I was surprised when they mentioned The Wisdom of the Crowds(WOC). For the most part, the show got the main points of the theory correctly; i.e., the aggregation of knowledge from many diverse people can perform better than a single expert opinion. In the scene, the patient does not trust his doctors and posts a $25,000 reward to the person who first presents the correct diagnosis. After receiving many different opinions, he aggregates them, takes the diagnosis most often given, and demands treatment. Of course, this fails and everyone is given the impression that the theory is faulty. One problem - this is not a true example of the theory.
The problem is the style of the monetary reward - it encourages quackery opinions to stand-out more than normal. A fundamental principle that is often ignored when discussing WOC is the incentives for offering opinions are often small and are rarely monetary or form transitional wealth. An example of transitional wealth is money: I can give someone a dollar and they can now use that dollar to buy something. Non-transitional wealth only possesses value to the person who created it. An example is pride: I can create pride in myself and I may be able to find pride in the actions of others, but I cannot be proud and give that pride to someone else. Another good example is Last.fm. By labeling songs, I can recover them quicker, but someone else may find my choice in labels confusing, wrong, or meaningless. The wealth is entirely within me.
The reason it is important to monetize opinions and effort in this fashion is that it acts as a noise filter; i.e., The Wisdom of the Crowds does not mean every nut delivers an opinion. For example, if I poll the average person on the street (at least in America), I will find some very wrong answers. Most people do not know what a molecule really is. This might explain why many people have a fear of things with "chemicals," which is a rather ridiculous statement that I have previously ridiculed. Further, only 1/3 of people can state what the purpose of DNA is. In other words, the average opinion is, well, average.
We can actually think of WOC from a statistical sampling perspective. While it's true that on average the group opinion will be better than an individual, it does mean that this is occurs with probability of one. The purpose of the WOC is to remove bias, but not expertise. Or from a Bayesian perspective:
P(correct answer AND crowd) = P(answer|crowd)P(crowd)
If I have a bunch of creationists, I am not going to get the correct answer (evolution). Same for medical diagnosis. If I tell people that they'll get $25,000 if they guess the correct answer, then their monetary gain is high. Unless their monetary cost is high as well, they'll offer any crap-based solution they could get from a little bit of time on Wikipedia. Worse, in this particular example, it's the cranks and quacks who are more likely to respond. Who do you think will respond quicker to that reward, a guy going around selling reflexology out of his Winnibego or the Head of Infectious Diseases at Johns Hopkins?
However, I must admit that my ears perked when I heard the phenomena cited on a primetime TV show.
I recently was amazed at the amount of crap that Bill Maher stated in this video:
First off, it is not true that you go to jail in the USA if you practice alternative medicine. Rather, alternative medicine is protected. For example, anything you consume can be protected as a "dietary supplement" under the Dietary Supplement Health and Education Act (DSHEA), which was pushed by the bipartisan team of Orrin Hatch (R) and Tom Harkin (D) (bipartisan does not mean good). The DSHEA states that it is up to the dietary supplement manufacturer to ensure the product is safe. There is no need to prove efficacy or safety in a controlled setting prior to marketing. Further, while it is true the manufacturer must ensure that any material on the label is true, any claim can be stated by using the catch-all phrase:
"These statements have not been evaluated by the Food and Drug Administration. This product is not intended to diagnose, treat, cure, or prevent _______."
In other words, "we made all this shit up and didn't even bother to test it. Here, have seconds." The FDA only steps in if they can find evidence that it is harmful after people have been affected; e.g., Zicam. In fact, Zicam is a great example because it is even mislabeled as a homeopathic treatment, but is not one since the concentrations in Zicam are way too strong - strong enough to have any effect (whether good or bad) disqualifies it as being homeopathic. So, not only are alternative medicines not regulated as medicines, there is not a regulation on themselves.
Second, Bill Maher's statement that we have not improved in the cancer fight is a bold face lie. Cancer rates have been steadily declining for years. But why research when you can just make crap up?
Third, his statement that we have such a big health care system because "we are so sick" is absurd. I am going to refrain from the public/private debate because this blog is about science and engineering and not politics. But Maher's assertion about the size of our health care system is not correlated with any health markers. Rather, it is simply a matter of diminishing returns and efficiency. In fact, if our problem was simply tied to health, we would not be arguing over whether people are covered by insurance. Instead, we would be fighting about ways to make people healthier; e.g., tax on sugars, increased physical education in schools, anti-smoking legislation, etc. (Note: I am not giving any opinion on whether we should do any of these; I am just stating that we would probably be debating these topics.)Regardless of your stand on health insurance, this is largely absent from the debate, except as a way to raise funds for a form of public health. The reason is that our health is not tied to the increase in health care costs.
Next, Maher goes into the conspiracy theory that "Big Pharma" is only interested in treating symptoms is wrong on two fronts. First, several Western medicines and treatments are specifically designed to treat the underlying disease and not symptoms. For example, antibiotics are specifically designed to kill the bacteria causing the infection. You cannot get more offensive than that. If you have heart surgery, an improperly or non-functioning heart valve is either repaired or replaced. This is not to say that all Western medicines treat the underlying case. In some cases, there is no proven cure; therefore, we do the next best thing, which is to manage symptoms. This does not mean that research there is no research into fixing the underlying cause. For example, in 2007 we saw the first vaccine against bird-flu and in 2006 we saw Gardasil. Second, when did alternative medicines start treating diseases? For example, homeopathy was developed before Germ Theory and specifically treats symptoms. Like cures Like? That is treating symptoms. If I have a cold, homeopathy says that the cure is onion because both cause my eyes to tear and agitate my mucus membranes. These are not the cause for the cold - they are the symptoms. Not one homeopathic practitioner ever links onion to the death of viral rhinopharyngitis.
I have no idea why Maher gets a pass from the skeptical and scientific community. It's likely due to his non-religious beliefs. However, this is a gross mischaracterization. The truth is that Maher is religious, but not in the Westernized "only one God in the sky" fashion. He's religious in that he believes in unproven methodologies. Jeffrey Tobin is right: there is no such thing as alternative medicine; it's either medicine, unproven, or disproven. Acupuncture? It's not alternative; it's not medicine because it does not work any better than a placebo. Anytime you put faith in unproven methodologies, you are making a non-scientific statement about the world. Saying there is a "life force" called Qi is nothing more than a religious statement.
Even if Maher were correct about cancer rates not declining (and he clearly is wrong), it does nothing to validate his point. This is simply a straw-man argument. Just because we do not have a solution does not mean you get to make up whatever crap you want to fill in the holes. It does not work for Intelligent Design and it does not work for medicine. Of course, science does not know everything. If it did, we would not be running experiments because, we would already know the answer. But that does not mean that we do not know something. Is there a cure for all cancer? No, but that does not mean we have not found ways to cure some cancers, detect and treat earlier, and increase the life expectancy if diagnosed.
So, to sum up: Bill Maher has more in common with Creationists than he would probably like.
There has be a lotofpress about a recent article in Ear & Hearing, but I found a good bit of it misinformed, sometimes grossly. For example, here are some disturbing misinterpretations:
1) "After years of Mom telling you to turn off the music to protect your ears, there’s finally scientific evidence that music can be good for your hearing."
2) "Music training can help reorder nervous system"
3) "The findings strongly support the potential therapeutic and rehabilitation use of music training to address auditory processing and communication disorders throughout the life span."
In actuality, the paper discusses none of these. I'll take these in turn:
1) This is actually two different statements: that there is benefit to listening to music loudly and that just passively listening to music can help. The most irresponsible is the former. While I'll admit that the statement does not directly say that increasing volume is beneficial, their slight attempt at humor implies it. The study did not discuss any possible benefit about listening to music loudly. As for the later ("passive listening to music can help") is definitely not within the paper. In fact, the study compared people with extensive musical training versus those with no musical training. The amount of time listening to music passively was not a variable within the study. Further, the so called "Mozart effect" has repeatedly been shown to be false (for a some references click here; for a funny discussion, one can check Penn & Teller's Bullshit!), despite the rather sillyproductmarket. The distinction between passively learning and actively learning is significant because it is true that research indicates the musical training may increase cognitive abilities in non-musical areas (see the "article" link above), but this cannot be said for passive listening.
2) The paper does not specifically address anything about the brain being rewired. In fact, since all musicians started practicing an instrument before the age of seven, it is likely that the brain wasn't being "rewired," but rather being "developed" in a different environment. This gets into (3) because...
3) The authors do not say anything about the potential for therapeutic products. Again, since the musicians had extensive training and started before the age of seven this study cannot answer the question about whether the effect can occur if someone begins training at a later age.
So just what did the study say?
I encourage you to read it first hand, but this is effectively what it says:
1) When presented with noise from the same source, people who have received life-long musical training performed better in remembering and processing oral sentences in four-talker babble.
2) When the noise is presented from coming from the same source (i.e., loud speaker), there was no difference.
3) The processing part is important because it signifies that working memory is the determining factor.
The authors then hypothesize that because musicians are better able to pick up on low-level auditory cues, they have more resources for working memory, and, therefore, able to process oral information more efficiently.
This is yet another call to caution whenever research is being presented by the media. It's important to ask to read the final article, including the headline, before you agree to give your consent.
I'm a huge fan of last.fm and the work they do, but I've always been a little curious as to their role with CBS. Apparently, CBS has decided to use the online radio station for something that it is not: a terrestrial, non-personalized radio station. While it might be "neat" to use this tool to see what's popular among the specific demographic of the online station, by no means is this a great tool for recommendation - which is at the heart and core of last.fm's mission. What would such a station play anyway? Depends a lot on the type of "chart" they are using, but suffice it to stay, most plays will fairly bland in terms of their novelty factor. For example, let's look at the current Top Ten artists: (Note: This was written around September 8th and published later, so the current artists are different, but concepts are the same).
1) Radiohead 2) Beatles 3) Coldplay 4) Muse 5) Michael Jackson 6) The Killers 7) Metallica 8) Kings of Leon 9) Red Hot Chili Peppers 10) Artic Monkeys
Besides the obvious British bias, we can see that the most common artists are not exactly "strange." They are all (at least) decent, respected bands. Some have been around a while (Radiohead, Coldplay, Metallica, Red Hot Chili Peppers) and some are staples of any music list (Beatles, Michael Jackson). However, I wouldn't exactly say that these bands are strange to radio play (yes, to some degree, Kings of Leon and Artic Monkeys are not as common, but I would hardly say that they get almost no radio play, at least in the US). (As an update, on today, September 18th, the Beatles passed Radiohead, which came out September 9th. While this does show the power of Last.fm's "buzz meter," it is hardly unforseen. Further, this is an example of already popular, known bands simply trading places.)
Also, these two bands are really in the list due to recent releases, as evident from the current Top 10 tracks:
1) Kings of Leon - "Sex on Fire" 2) Kings of Leon - "Use Somebody" 3) MGMT - "Kids" 4) Arctic Monkeys - "Crying Lightning" 5) Coldplay - "Viva La Vida" 6) Lady GaGa - "Poker Face" 7) Muse - "Supermassive Black Hole" 8) MGMT - "Time to Pretend" 9) Black Eyed Peas - "I Gotta Feeling" 10) MGMT - "Electric Feel"
Yeah, these are all very rare indeed. Hint: If the artist is being asked to play at the MTV Music awards, then it is a safe bet that the artist is not a rare find. What if I contiue farther? Well, #14 is "Wonderwall" by Oasis and thank God because I haven't heard that about a bajillion times.
About the only good list would be the "Hyped List," but this is more than likely just following new releases. Truth is that a station that would be that eclectic probably won't fair well since users would be hearing a high degree of music they don't like (i.e., people probably will not like bouncing from niche to niche).
The truth is that this violates the very nature of last.fm - personalized music discovery. Will the radio station fail? No, of course not. It will actually be interesting to see how well the record companies try to follow the Wisdom of the Crowds. This is, after all, capitalism. It was always believed that record companies tried to make people like certain bands, but in truth, the record companies just knew what most people either liked or, at least, could listen to without running from the room screaming. Truth be told, Nickelback and Creed suck, but most people can listen to them because it isn't so awful that I can't either tune it out or stop myself from sticking pencils in my ear . It's just boring rock template music. People didn't buy millions of Spice Girls albums because they had a bunch of money in their pocket, found themselves in a record store, and didn't know what else to buy. They bought it because they wanted to (and you know who you are and should be ashamed of yourself - if my sister had a webpage, I'd link to it here).
I fear this radio station will just be yet another radio stations which focuses on the rather narrow "head" of the long tail and never really gets into the niche markets. The reason? Radio execs focus on what's popular and what will be popular. They are still successful in this, but what they lack is the ability to tailor distribution channels to individual needs and wants. It is not the price of songs that is destroying the music business. If that were true, then as online marketers slashed prices, sales would increase. But this hasn't happened because, despite what Napster fans would have you believe, it was never about prices, but about finding the music that you, and only you, wanted. This is the market that sites like Last.fm are designed to infiltrate - but a radio station broadcast to millions cannot capture this ability. On the other hand, one can hope that I will be pleasantly surprised.
Phew! It's been a while! Guess I've been busy getting a MIREX submission for chord detection completed. Anyway, news of the day: recently Terry McBride sat down with Paste Magazine for a quick "Where are we going?" interview. It's a very quick, but good view of how the music industry is finally dealing with their new role.
Wow. Imagine if you were stranded on a desert island, without a calculator, and you needed to quickly multiply two large numbers. I mean, when you are stranded on a desert island, life is fairly hectic. When you need something multiplied, you need it now!
Well, thank God, Albert Clay has found the code to mentally multiply two numbers. Actually, I should say "cracked" the code, since that is what the headline said. Funny, I do not remember anyone encrypting math. Was I given a key in third grade when I learned my multiplication table? Damnit! What did I do with that?
I think the opening line from the original article in The New & Observer was even more laughable:
"On a yellow scratch pad, Albert Clay works out a math problem that can stump a calculator -- and all of the ciphering occurs inside his white-haired head."
Wow, that's impressive if true. What was the problem? Fermant's Last Thereom? Maybe the Riemann Hypothesis? How to love? Surely, this must be difficult!
54,321 x 12,345 = ?
(Collective groan). Really? That's not special! What kind of crappy ass calculator couldn't do this? Apparently, the pocket calculator could not display enough digits. Again, not special. If you have a 10 digit display, then you can enter 10,000,000,000 x 10 and the calculator will be unable to display the result unless it can do scientific notation. And a pocket calculator!?!? Was his abacus in the repair shop? Ok, well what is the trick?
"Suffice it to say you multiply the digits on the right, cross-multiply and add the digits in the center, then multiply the digits on the left."
That's not a code! That's math! This is not a short-cut. It's just doing the problem in a different way so that you can remember the steps easier. It's almost as if you can't just create information from nothing. Damn you Shannon! But wait, this old fart will sell you his "secret" for only $33. Why? For what purpose? The only value this has is as a party trick for a really, really dull person at a really, really dull party.
"Dude, this party is so lame!"
"Not to worry braugh, I scored us some paper and #2 pencils. We can long-division the shit out of this party!"
Christ, I wish I could say that this was a just some small town paper in rural America, but this is my hometown newspaper and served the area known as The Research Triangle.
I love how every year, a survey comes out saying that college graduates in math and science fields earn more than other majors. What's more appalling is that some think this is news. The article here states that it is a matter of supply and demand, but this is not entirely true. However, this article inspired me to share a couple of stories from my own family and how the educational system in America is failing.
It is true that there is a "braindrain," which is largely the fault of the poor education in math and science being delivered by the public schools in The United States. Personally, my family has two stories: one from public school and the other is sadly, from private school. The public school story involves my little sister. Her calculus teacher was working through the details of a problem in class, and he must have been proud of his work because he stepped back from the board and stated, "You know, I should have taken calculus; I would have been pretty good at it!" The idea that someone can teach a subject by following along in the book or staying just a lesson ahead of the students is appalling. In order to teach a subject, you need to understand the subject on a level deeper than the book; otherwise, you will not be able to explain the subject in a different way than the book. One thing I've learned in teaching is to present the same subject material in several different ways because what works for one student may not work for another.
Another problem, which I find even worse, is that the teacher says he should have taken it solely because he would have good at it. If you only take on challenges because you are good at it, then you are not really challenging yourself. Imagine the impact of John F. Kennedy's speech if he said,
"And they may well ask why climb a small hill? Why, 35 years ago, fly from New York to Boston? Why does Rice play local high schools? Because these things are easy and we'd be really good at it!" (Original quote may be read here.)
Not exactly inspiring. Thankfully, JFK did not have such defeatist attitude, but believed we do things "not because they are easy, but because they are hard, because that goal [of going to the moon] will serve to organize and measure the best of our energies and skills." I get chills every time I hear this speech. No matter how many times I hear it, I am always inspired.
The other story, and maybe more shocking, involves my little brother. At a parent-teacher conference, his teacher told my mother that my little brother needed to stop working ahead of the class because "it wasn't fair" to the rest of the students. I am still amazed that mother did not bitch slap the teacher for saying that a student needed to stop learning.
There are more reasons that math and science majors earn more than liberal arts degrees than just supply and demand. I hope to discuss other reasons in a future post.
Note: The intent of this article is not meant to persuade one for or against a government, single-payer health care system. Rather, it is to address the specific application of tax dollars to medical methodologies that do not hold up to the rigors of scientific testing.
When I started this blog, I made a promise not to discuss politics unless it crosses the lines of promoting woo through legislation (note: I do not care if a politician believes in crap (for this blog), but I do care if they want to force taxpayers to pay or believe crap). Dennis Kucinich has given me just such a chance with a recent blog post on the Daily Kos. Specifically, this is to address the following comment:
"One amendment brings into standard coverage for the first time complementary and alternative medicine, (integrative medicine)."
This should make anyone who has an understanding of science shudder. What is wrong with this amendment? First, in order for something to be alternative to medicine means that it either has not been shown to be medically (i.e., scientifically) effective or has been disproven to be effective under numerous rigorous test. For instance, here are some examples:
Homeopathy. This is may be my favorite medically-based pseudoscience. Want to cure a cold? Take an a small amount of onion (because both onion and a cold cause you to tear up and affect the mucus membranes in your nose), dilute it with water to the point that you would need a container the size of the solar system to have a realistic chance of retaining just a single molecule of the onion, then ingest or drink a small amount of the diluted solution. How do the practicioners of homeopathy justify their treatment when there is zero probability that I have ingested a single molecule of the onion? Water has memory, which is strengthend by shaking the "mixture" at each stage of dilution. No joke. They really want to throw away everything we know of chemistry. Further, the more you dilute something, supposedly it gets more powerful. Take that physics!
Essentially, this legislation not only allows for any treatment to be used, but the government will now pay for it! While we are at it, why not allow for people to get energy tax benefits because they claim their car is a perpetual motion machine? Is it not bad enough that the government has spent $2.5 billion researching CAM (complimentary and alternative medicine) only to find it does not work? The only thing worse than spending this amount of money is to then ignore the findings and pay for people to get ineffective treatment!
Anyway, I encourage people to write to their congressman or congress woman to voice their concerns about any legislation that uses tax dollars to administer disproven medical methodologies.
At the recent UMAP 2009 conference, a paper raised the possibility that we are reaching the possible performance limits of recommendation systems (RS). If true, this would change the landscape for research and development in RS. In fact, some blogs discussed this paper before it was even presented! However, after reading the paper, I'm a bit inclined to disagree at the hype over this paper. It's true, the paper does point to the performance limit for RS based on the current system of obtaining recommendation data. However, it does not mean that no one can build a better RS.
First, I want to discuss what the potential impact could mean for RS if indeed, we reach a true limit of performance. As an example, assume that for a particular task (e.g., music recommendation), people have a self-agreement 0f 90%. That is, a person will agree with themselves 90 times if they rank 100 songs one day and then rank the same 100 songs two weeks later. Assume that tastes do not change, which the authors argue is the case in their setting (they make three measurements at different points in time). What does this mean? Some possible explanations:
(1) The user doesn't know if or how much he likes the movie. (2) The user doesn't understand or can't specify the degree to what he likes the movie into discrete, deterministic categories.
(1) is, by default, the wrong option since the user's judgment is the correct answer automatically; however, (2) makes some sense. While people may have an understanding if they really like or hate something, there is a rather large ambiguous area in the middle. How many people can consistently listen to a song and say, "I like that song 40%"? What does that even mean? Does the user like it 40% of the time he hears it? Does it mean that it would be in the 40th percentile of songs if the user were to rank every song he has listened to? If he ranked every song he's heard several times, would the average rank be the 40% percentile? The authors of the paper demonstrate this when they show that the inter-subject disagreement occurs 34% between rankings 2 and 3 and 25% of the time between 3 and 4 on a 5 point scale. In other words, people aren't able to rank movies accurately if they do not have a strong opinion. Usually, it assumed that the fault lies with the user; that is, a person is confused about what the categories imply. I disagree. I believe that it's a probabilistic rating because yes, opinions constantly change. People are not machines. They have emotions. Emotional states have an impact on how we both interpret and want to interpret our environment.
Second, how can we measure the success of a RS when 100% is theoretically impossible? What does this even mean? This issue has come up several times in terms of genre recognition. Until the reprint of Scanning the Dial and the accompanying criticism directed at the MIR community, some authors have validated their algorithms by stating that it is more accurate than humans. As pointed out in a couplepapers, this is nonsensical since genres are ill-defined. Ultimately, our categorical dimensions of music is largely subjective and built over a life-time of (often conflicting) feedback from society. Still, we can ask, what if a RS comes out with a better accuracy than the documented limit? Does it know what people will like more than humans? Of course not. It shows an error in the choice of evaluation criteria. Ultimately, a RS is measured at a moment in time. If a person likes something on Tuesday, but does not like it on Wednesday, it does not mean the user is confused. It means he liked it on Tuesday, but not on Wednesday. Tastes may change based on mood, evaluation of new information based on the world around us, etc. Future RS may be able to detect this information to update adaptively.
This again brings up the problem with an RS. Every RS is based on the idea that a user will like something similar to what the user liked in the past. Further, almost all RS model a user as a single entity or that the user must maintain separate profiles for different tastes. For example, Last.fm and Pandora cannot build a station or user profile that maintains two separate personalities - it's up to the user to construct this system. Netflix only allows one user profile per account. While my fiancee and I may like some of the same movies, we certainly do not like all of them. Heck, some days I want a good skeptical show like Bullshit, but on another day I may want pure magical fantasy.
Even with transparency, somethings get muddled by small clusters of users who have a very demonstrated behavior. For example, Netflix is currently telling me that I will like Wallace & Gromit because I like This is Spinal Tap. How are these two even connected? The first is in the category "Children & Family Suggestions" and the other is a movie about a fictitious failed hair-band. Granted, both are good, but the only relation here is only based on discrimination (Wallace & Gromit is a UK show and Spinal Tap is an American movie about a British band). Apparently, all British humor is the same to Netflix users. The weirdest might be that I'll like a Talking Heads concert because I like the movie Fargo. Obviously, content analysis would do a better job of filtering the nonsense predictions.
So, in conclusion, research in recommendation systems is not reaching it's limit in performance. Rather, recommendations based on the idea that a user is a simple, static classifier is limited from the start. Smarter, better recommendation systems that can understand complex user behavior such as emotional state, thoughts on quality, and incorporating content analysis will be the building blocks of the next generation of recommendation systems.
Recently, I presented a paper at ICASSP that discussed the importance of incorporating temporal information into the structure of automatic tag recommendation algorithms. Until this paper, all studies and systems designed to overcome the "cold start" problem ignored temporal information for the most part. In fact, I am aware of only three exceptions:
(1) Using derivatives of features, such as MFCCs. This incorporates temporal information on only a small scale (~50-100ms). (2) Averaging features from multiple frames in a given temporal window (ex: averaging 100 ms frames over a duration of 1 second). (3) Extracting song-level features, such as rhythmic features; e.g., estimated beat histograms.
The problem with the above approaches it that they incorporate information on a very small scale and do not incorporate "syntactic structure." One problem I have had with music recommendation research is that many of the systems are based on a rather faulty assumption. Researchers have taken an abstract from a presentation given by Gjerdingen and Perrot in 1999 and essentially taken the results way too far. Specifically, researchers have taken for granted the "bag-of-frames" approach, which essentially says that any small segment of a song is representative of the whole song. In other words, one can listen to 250 ms of a song and that will be representative. This is obviously a faulty assumption and it has been discussed here and here. Originally, this assumption was used in studies on genre classification. Since genre is an ill-defined concept anyway, it is difficult to verify this assumption. However, even if this assumption is true, it does not make sense to translate this to tags, which have a better defined meaning. For example, the Pandora tags of "repetative melodic phrasing" and "extensive vamping" have obvious acoustic semantic structure.
So how does our study contrast with previous approaches?
In the paper, we build a vocabulary of acoustic tokens, which can be seen as acoustic generalizations of phonemes in automatic speech recognition (or musiphones as Doug Turnbull called them - yes, Doug, I consider this your terminology). Not only are the musiphones represented by a temporal model (i.e., a multi-state HMM), but syntax is also considered through the use of unigram and bigram counts. Compared to the baseline (Turnbull, Barrington, Torres, and Lanckriel), our algorithm performance substantially better; especially for tags which are considered to be temporal in nature (i.e., melody, solos, etc.). While this paper mirrors the implementation we proposed for genre detection in 2006, the results are more informative in the 2009 ICASSP paper.
Note: The Gjerdingen and Perrot presentation has finally been published so that people can read the study in its entirety. Two things to note in addition to the papers linked above:
(1) Gjerdingen and Perrot performed a task of discriminating genres (i.e., a closed, forced choice) and not identification (i.e., an open, forced or unforced choice). (2) Only 10 genres were used and most where fairly easy to discriminate.
Yesterday, I mentioned that it is a bit early to state that the success of Pandora and Last.fm is purely based on the strengths of their recommender algorithms. The newest copy of Psychology of Music contains an article by Silva and Silva on how successful various methods of increasing a unknown song compare. I should caution this is a preliminary study and all conclusions should be taken in this light.
Anyway, the authors had students listen and judge an unknown song in various contexts. Their findings were that the students gave the unknown song higher ratings if the students read an article about the artist of the unknown song, if a well-known and positively liked song followed the unknown song, or if a music critic praised the song prior to its playing. The authors also argue that repeated exposure to a song does not increase a song's appeal, which is a stark contrast to the most common method employed by radio stations (ahem, Payola). However, I think the argument is a little weak, since (as the authors note), they repeated the song six times in a row. Their premise behind the "repeated exposure is beneficial" hypothesis was based on the Pavlovian condition, but I disagree. The Pavlovian effect works by associating an emotional neutral action (ringing a bell around a dog) to a positive or negative association by pairing the two together (ringing a bell around a dog and then feeding him). Done correctly, the subject will associate the neutral action positively (dog salivates when the bell is rung, regardless if food is around). For starters, they did not test the premise since they gave repeated exposure without any pairing of a positive or negative stimulus.
However, we can ask what if some these are true? How does this change how we view recommenders like Pandora and Last.fm? First, comparing radio stations is problematic if done in series because if there is a truly positive song, then songs around it would be viewed positively. In other words, the rich get richer. In order to remove the music critic affect, one needs to establish "placebos" so that users cannot infer a playlist is better than another for reasons other than the audio. For example, if users know that their ratings are taken into consideration (Last.fm) or that experts labeled the music (Pandora), the users may rate the playlist higher than it would under neutral conditions. Also, user should know little about the bands, which is difficult to ensure. Finding an unknown jam band might be difficult to accomplish for that frat guy sitting in class with an old set of Bercinstocks, Grateful Dead T-shirt, and messed-up hair.
This is not to say that previous approaches to compare playlists are worthless - it's just that they need to be considered knowing all the above factors.
Wired had a little blurb about Pandora finally turning a profit in 2010. This can be attributed to many things, such as the growing fan base, the iPhone application, and the addition of commercials. What's more interesting is that Pandora is essentially becoming the same thing as radio in doing so with a small quirk. First, the addition of audio commercials was a major step, not just in the profitability, but also in the precedent. If viewers continue to listen, it's very likely ads will increase in frequency. It appears that listeners are not against advertisements in general, but no one has come up with a limit before users turn away en mass.
Also, the iPhone application is essentially the same in functionality as a personal portable radio. The only real difference is that you are streaming radio from the web to your phone. However, you have marginal control over the content. Yes, you pick the artist and you can assign "thumbs up" and "thumbs down," but I would not say that this radio is truly personal. You have no direct control on novelty, artist frequency, repeats, etc.
I think what has worked best for Pandora is the perceived personalization. Essentially, users do not want to be dictated listening habits. There's also an inherit confirmation bias in any recommender system. I'll remember the hits and forget the misses or explain it away. Also, people tend to respond positively when someone says, "you'll like this." It's incredibly hard to come up with a controled test to measure effectiveness. Some researchers have tried, such as comparing lists of recommendations and asking which is man-made or machine-made. However, the goal of a recommendation engine is to build the one with the best recommendations, not to mimick humans. At best, we can compare how two engines perform by comparing the number of skips.
For example, we could compare Pandora and Last.fm in a large-scale listening test. Again, we need controls. Pandora and Last.fm differ in the amount of perceived control. Both Pandora and Last.fm do not give a lot of control on how music is played, but Last.fm allows more feedback. Both Pandora and Last.fm have access to a positive/negative vote for each song and initial seed song selection. However, Last.fm also allows for tagging individual songs in an effort to improve selection. Of course, this is legitiatimate; however, a real test would let users tag songs for Pandora, but then Pandora would just ignore the tags. In addition, users need to be blinded on how the process works. It's not a fair comparison if someone tells you that a system is based on machine interpretations of expert or wisdom-of-the-crowd ratings.
I'm glad that Internet radio stations that thrive on "personalized" recommendations are beginning to make a profit. However, it's a bit early to say that a new dawn of music has emerged. Promising, absolutely; but more research is needed.
A good friend of mine recently had twins. He now has to tell one of his in-laws that they are no longer welcome to their house because they refused to get their kids vaccinated. Why? Because Jenny McCarthy became an expert because... um...
Anyway, here's to you, McCarthy and your ever increasing body count.
I'm busy getting ready for the ISMIR deadline, but it was extended so I can take a few minutes to write. Yesterday, the Naked CIO posted an anti-social networking column. To his credit, he says that he likes increased social connectivity; however, this could be similar to when a Creationist says they are not religious and then spend the next few paragraphs arguing for the existance of a designer. The Naked CIO's main problem is how social networking sites, like Twitter and Facebook, will impact the work place (Note: I'm on Twitter now). Simple: people will learn how to use the technology.
I have a few problems with some specific arguments he makes. He starts by stating that email and instant messenging have blurred the line between formal communication between discussions on work-related matters and regular social interaction. One problem with this argument is his hidden premise that formality is more productive and healthy to the work environment. In reality, there are plenty of reasons to believe this is not true. Email, local wikis, and instant messenging have allowed for a lot of wasteful formal structures to be eliminated. It is no longer necessary to have a formal board meeting on every topic. I'm reminded of my first job out of college as a great example. The organizer started the meeting by saying that we had a meeting next month to plan a project and he wanted to get together to plan for the meeting. I'm not kidding. After 5 minutes I got up and left. He sent someone after to me when I didn't return and asked what I was doing. I had set up a project mailing list with an archive. Second, it is not fair to say that email and other technologies are the reason for decreased formality. Formality may have degraded for the simple reason that it is not necessary for increased productivity and can even become a negative by wasting time and making people remain silent because it would be rude to speak. As an example, overall attire has progressively moved from a coat and tie environment to khakis and polo shirts as the social norm. Why? Comfort.
He then goes through some scenarios that he feels may impact businesses. Here are the problems and my solutions:
1) Is a tweet from a company executive an official statement? Response: Depends. It's called a disclaimer. Columnists use this all the time so that the media outlet is not held responsible for opinion.
2) What if a friend mentions the possibility of layoffs on a social networking site and that negatively impacts the company, even if there are no layoffs coming? Response: Yes, new technologies make our jobs easier, even the job of being stupid. Social networking sites make it easier for young women to become targets of pediophiles, but it does not change the problem that teenagers are irresponsible and nieve. Most social networking sites have an option to send a private message versus posting to the known universe. The non-digital anology would be something like this: imagine you are sitting at a coffee shop near Wall Street with a friend. You are talking about potential layoffs with your company and you know this is sensitive information. You can either whisper it quitely or yell at him through a bullhorn. Yes, there might be an occasional idiot that unknowningly betrays his company, but there are idiots everywhere. We should not stop progressing as a culture because idiots cannot take care of themselves.
3) How do we resolve the issue of privacy when people can "Google" one another. Is it right for a company to do this type of background check on a potential employee? Reply: Is it right for a potential employee to lie or withhold information that may negatively impact his ability to work? Why should companies assume that everyone who applies for a job is trustworthy? People lie on resumes. People lie in interviews. Here's an example: an interviewer asks a candidate "What's your biggest weakness?" The candidate pauses, glances to the side, and replies, "Probably that I work to hard." Seriously? That's your weakness? The Naked CIO uses his brother as an example and questions what would happen if a company saw that his brother is a musician. Well, if he tours a lot and will miss work, then I fault your brother for withholding that information. The company should know if he's going to be missing work to tour or at least, be tired all day because he's playing music at a local club until 2 AM. Companies should know if you have another job. Heck, it's probably illegal not to tell them if you have to sign a disclosure agreement and your other job or hobby is in a related area. If your brother is an amature and maybe only has a couple gigs a month, your company will not care. Hell, it's better than what 90% of men do, which is sit on the couch and drink beer while watching football.
I'll never understand these nostalogic "those kids today" thoughts. Society evolves as technology evolves. You either learn to use it or end up complaining about the good ole' days.
I recently got an email about the frequency of washing my dog from the kennel where I board my dog, Marley. Needless to say, the brought in some naturalist veterinarian, to recommend shampoos and gave the standard crap advice. I'll forgo my rant on the obvious appeal to nature logical fallacy and go to my response to their call to avoid "chemicals." Here is the letter:
Dear Wag-A-Lot,
In your recent e-letter, you mentioned an article about the proper frequency we should wash our dog. In addition, you mentioned that we should use natural shampoos and "avoid artificially produced shampoos that contain chemicals as their major ingredients." I have had a hard time finding a shampoo that fits your description. The problem is that every natural or artificial shampoo has some form of chemical as a major ingredient. For example, most shampoos, such as the ones you mentioned (Spa and Earthbath), contain water, which is a chemical. Specifically, this is a molecule and all molecules are chemicals, so I originally filtered the list to only elements. However, I then did a little more research and found out that elements are also chemicals! In fact, since chemistry is the study of matter and its interactions with other matter, a chemical substance is any piece of matter which can be described by an empirical formula which expresses the relative numbers of each type of atom in it. Since finding this out, I have tried various forms of energy to clean my dog. First, I tried radiant energy in the form of light. I tried several types of lights from strobe lights to flashlights at various intensities, but this just seemed to make my dog excited and not any cleaner. I could try other forms such as X-rays, but I am not sure of places that offer X-ray dog cleaning. Do you know of any? Next, I tried potential energy by balancing my dog on top of a tall ledge. Not only did my dog stay filthy, but he also seemed rather uncomfortable. Fortunately, my dog was in a perfect setup to try the next form of energy - kinetic energy. Let's just say that the fall made my dog leery of going near me and he was actually dirtier.
Magnetism was my next energy type to try. After giving my dog a few treats so he would trust me again, I waved kitchen magnets over him. Maybe they are not strong enough to clean effectively. Can you suggest some effective magnets that might help clean my dog? He's getting very filthy and annoyed. In the meantime, maybe you can suggest some improvements on some other approaches. Heat - how long and hot do I need to cook my dog? Also, I never thought of hot dogs as being particularly clean. Electricity - I'm not a fan of shocking my dog, but if you guys know of an effective voltage and amperage I'll try it. I am an electrical engineer by trade and I feel I can administer this effectively on my own. Nuclear - are there nuclear power companies that offer dog cleaning? The ones I've contacted thought I was crazy.
I realize that I could also maybe try dark matter and dark energy, but if I'm not mistaken, scientists are still trying to detect these and there have been no attempts to harness them or put them into a dog shampoo. Please, let me know of any advice you can give me. My neighbors are complaining about my dogs stench and his howling over my treatments.
Well, at least Last.fm allows a brief free trial. One can only hope it's temporary and international copyright issues get resolved. It's a good thing I have only one more day! But this leaves an interesting question: if I were to stay here, how would I get my multimedia needs fulfilled?
I even went to a couple brick and mortar store in Taipei. My choices in the developed parts of the city were the Christinas and the Justins of Western music or I could get the Tawainese version. In the undeveloped area, I could find music I might want to listen to - on cassette tapes! I haven't been able to play a cassette since 1998! So I ask, what legal means can I get music? Maybe we need to let the Far East actually be able to get the music they want before we criticize them for downloading music illegaly...
Note: Evolution is always a sensitive issue in the US. I am clearly on the science side of things. This may be offensive, but to be honest, it's offensive that a non-scientist would dictate science to a scientist. This would be like a bunch of lawyers dictating what medical procedures should be performed... wait, strike that.
"I don't see how we can say there is no disagreement about evolution. There is disagreement," [Board member Ken Mercer] said."
Actually Ken Mercer's right. There is a disagreement between people of science and those who do not understand science. But maybe it's best to have scientists determine how to study science and not ministers, priests, etc. Notice that Stephen Hawking is not telling Pope Benedict how to teach Christ's message. The truth is that there is no disagreement over whether evolution occurred, but rather a disagreement on how evolution occurred. That is, the specific historical path. One that drives me crazy is when people will use the "evolution is just a theory" argument. This is nonsense. Everything in science is a theory. This just shows a clear misunderstanding of what it theory means in a scientific context. Creationists would have you believe that a scientific theory is the same as a hypothesis, which is not true. A hypothesis is a guess for which there is little to no experimental validation (yet). To be a scientific theory, you must have performed experiments, done analysis, verified predictions, etc.
Plus, let's be honest, this debate in Texas has nothing to do with providing children with a better, more "open" version of science. If so, where are the works debating the round-earth theory (yes, this is a theory and there exist those who dispute it). Creationists would make great lawyers because they have a way of combating logic with nonsensical crap to the point your brain wants to explode. For example, I'm often told that there is not one thing that I can do to disprove creationism and prove evolution. Well, duh. Science doesn't work that way. However, I can easily point to many observations that suggest that evolution explains the observations we've made. I can also easily point out that creationism is NOT a science. Here goes: design a test to disprove God. Can't do it? Then it's not science and does not belong in science class.
Today was a much more interesting day in terms of technical content (at least, to me). In the morning I went to Content-based Audio Processing session (at least the first half). All the presentations were good, but I liked "Interpolating Hidden Markov Model and its Application to Automatic Instrument Recognition" by Tumoas Virtanen and Toni Heittola the best. Essentially, the motivation is that hidden Markov models (HMMs) make an assumption that an audio element is decomposed into discrete states. In reality, there would be a smoother transition between states and while one can add states to the model to reduce the error, it requires a lot more data since there are more parameters. Their suggestion is to create auxillary states by using interpolation methods on the HMM parameters. They demonstrated a 5% absolute improvement over the baseline (no interpolation). However, their test database was isolated instrument recognition and I would have liked to see how their approach behaved in the presence of different noises and levels. Using discrete states can help with noise reduction by vector quantizing the state space. It's possibe that the presence of noise could take the state trajectory down an erroneous path. For the afternoon, I went to the poster sessions and there's plenty I could say, but I'd miss the rest of the conference. Instead I took pictures of the posters and they are presented below. This is a plug for Emiru Tsunoo's paper titled "Rhythm Map: Extraction of Unit Rhythmic Patterns and Alanysis of Rhythmic Structure from Music Acoustic Signals." I've read the paper and it's very good work. I like the search for new and better features.
Emiru presenting his poster. He was crowded the entire time.
Dr. Sagayama explaining RhythmMap to Malcolm Slaney of Yahoo! and Stanford.
The ICASSP technical sessions started today. They had a cool intro with an authentic Taiwanese singer/drum pair. This time I thought ahead and brought my camera.
There was a demo for a voice-activated music player with an emotion-based display by Chungwa Telecom. It's always cool to see new music retrieval products; however, it there did not appear to be a lot of novelty in either the processing or visualization. Essentially, you spoke the name of the artist and brought up some of their songs and locations in the arousal-valence emotional plane. There was no discovery involved. Anyway, today was a popular day for sight seeing among the speech and music part of the conference. Tomorrow should be a better day for technical content.
From Japan to Taipei! I have a back catalog of pictures to post from my visit in Japan, but I figured that I should cover ICASSP in "real time." Yesterday was the opening ceremony, which was actually quite nice. I forgot my camera, so I'll have to track down pictures of the opening musical act, which contained live authentic Tawainese music. I'll try to post everyday, but no promises. The ISMIR deadline is soon afterwards and I may submit a couple of papers, which means that ICASSP is a working conference. However, I will take a minute to say that I liked the proceedings.
My fiancee bought me a new laptop before I left for Japan and Taipei (ICASSP), but rather than pay $100-150 for MS Office, I decided to try Open Office because I think it's easier to have the application on my machine versus having to synchronize Google Docs if I want to work on a flight or something. I'm a huge fan of Open Office so far since it seems to have all the power of MS Office with no real cost. However, ICASSP doesn't feel that way. First, you are supposed to use their laptops. Fair enough, but why this:
The ICASSP laptops are equipped with: --Windows XP sp3 --WirelessLAN connection, USB port, DVD Player --Office 2003 and Office 2007 --Adobe Reader, Flash Player, Media Player (Microsoft/real/Quicktime)
Ugh! Apparently if you use free software, you're screwed. I've tried to take my slides and save it as MS Powerpoint, but that didn't work because all the animations and MS Visio drawings I made didn't go through the transition. I have MS Office, but it's on my computer back at Georgia Tech, so I have to remote connect and fix my slides. It's an incredibly slow process! Anyway, for shame on ICASSP for no embracing open source.
Since it was Saturday, I decided to do some sight-seeing and went to the Sensoji Temple, which is pretty close to the university. I'm not going to bore you with all the history because quite frankly, you can read a lot more about it some place else.
The gate is almost on top of the street. It's kind of amazing how close historic and cultural sites can be to modern society.
There are statues on all sides of the temple.
From the gate to the temple, the whole street is lined with various shops.
Some sell food.
I'm thinking about buying this for Jenny.
Oh, hell ya... octopus balls! I've been craving some octopus balls since I got here. Now if only I could find the crab juice stand...
The actual temple is being renovated, so I could not get any good pictures, but this was a pretty cool one.
Leading into the temples are these smoke pits. People wave the smoke over to them before the enter.
The inside of the temple was actually a little disappointing because we could only go in half of it (there was a religious service going on), but there were some pretty good painting on the ceiling. Take that Michelangelo.
Well, all in all, a good day... I'm getting hungry. Good thing my iPod is loaded because I'll bet there's a line at the octopus balls stand.
From time to time, I mention cool research being done at Georgia Tech. Today, I'm taking the pleasure of talking about my fiancee. Her name is Jenny Matthews and she's mentioned towards the bottom under a section titled "New Insights into Tornadoes." Here's an abstract from one of her papers. I guess I miss her today!
It's amazing how being a "day ahead" the rest of the world can completely catch you off-guard. This morning, I found articles about the Echo Nest's new "goodness" measure API. I'm in Japan, so it's April 2, so it took me a couple minutes to realize this whole thing was a joke (I actually got started to get angry). Anyway, fun with the system (higher score is better):
Twisted Sister 0.81 N-sync: 0.74 The Raconteurs 0.42 Bob Dylan 0.31 Nickelback: 0.13 Rush: 0.09 (Them's fighten' words) Coldplay 0.02 Warrant 0.01 (Actually, this makes sense)
Posts
