studies suggest smokers die of lung cancer more often than non-smokers.
studies suggest people who are chronic drinkers or have hep c are at risk of developing liver cancer.
studies suggest people who were exposed to radiation from the chernobyl disaster had a high rate of cancer death.

of course we don't know the hows and whys, but i'd say there are enough observable patterns being made by intelligent people?

my educated guess on the way colon cancer being tied to diet is that your poo gets hard from a meat-heavy diet and causes constipation, leading to toxins staying in that area for longer than people who have regular poos.