To the best of my knowledge (which I admittedly don't completely trust on this subject because a lot of stuff did get whitewashed, and so my "knowledge" may just be dead wrong), not many of the original European immigrants were people of colour. Unless you call people brought in by the chattel slave trade "immigrants," but that's yet another story.
Aloe, remember, even by the 1900's America was racist about those "sneaking Irish" immigrants: Everyone who's white getting a "free pass" is actually a pretty recent development... To be honest I don't know whether to call it an improvement or a degredation.
Plenty of them did have debts, though. I mean, if you're not having financial trouble, would you move across the ocean (when that takes several months and has a nontrivial fatality rate, no less) just to find better prospects? Especially in an era where most people didn't move around much.