What happens when a man of middle eastern descent is detained on allegations that he is a terrorist? Think about what happened to Boumediene, among (hundreds of) others.
Now, what happens when Americans are arrested and charged with kidnapping (or potentially child trafficking)? Complaints of Human Rights Violations.
The concern should be about fair treatment for all who are detained, not just those who share your flag. Innocence or guilt are exclusive to no country's citizenry.