Sam Harris is not OK with the mosque because he believes that Islam is unlike other religions and that previous Western liberal tolerance towards Islam betrayed weakness.
I disagree with him on several counts. The first is the issue of Islam being a violent religion in passage. That, it undeniably is. There are quotes in the Koran that invoke violence. But at the end of the day - what does that prove? Nothing, really. There are violent sections of the Christian bible. And furthermore, people do not need religion to make them evil - they can do that on their own. They also don't necessarily become evil because of religion. In other words, even though Islam does advocate violent action in certain parts, it does not follow that Muslims are destined to become violent. No doubt someday they will mostly come to ignore their religious tenants just like we have here in the West.
The most important point of disagreement is the context of our tolerance. There is a fundamental difference between European tolerance and American tolerance of Islam. Europeans were tolerant from a position of Muslims on that continent not being well integrated into the society at all, an extremely anemic European growth rate, and a very robust Muslim birth rate from a relatively high starting base population. In other words, Europeans are tolerant of Islam out of weakness.
In the United States, we can be tolerant of Islam for different reasons - we can be tolerant because of our strength. We do not have to fear Islam because we know that it poses no threat to America. Why? Well for starters, American Muslims are far more integrated than European ones. This is a testament to our melting pot society, which has an almost limitless capacity to absorb and welcome newcomers. Muslims succeed in America, and they feel welcome in America. Furthermore, they represent a very tiny proportion of the population, barely 1%.
Americans should not worry about the mosque near ground zero because we are strong enough such that we don't need to. Allowing the mosque reflects our confidence and strength, even though a similar situation in Europe might have reflected weakness. This partially explains the European tendency lately to overreact in bits of anti-Islamic hysteria (the Swiss, the French); they are compensating! Nobody in the world will question that our allowing the mosque is from a position of strength - except maybe all of the unpatriotic Americans who are expressing their vote of no confidence in their country by whining about this issue.