Forcibly migrating hundreds of thousands of people to work in cotton fields was a more efficient way of quickly increasing the labour force rather than having people migrate freely. Industrialized slavery increases the labour force at a much faster speed than letting the free market solve the problem.
I don’t see much reason to believe that slavery is faster than voluntary migration. Maybe a political stance that it’s easier to ignore the masses when bringing in “competition” to their labor or something, but at the time landowners were the only votes to begin with.
Another factor is the labeling itself. Blood libel/witch hunts/etc stuff like that are all conspiracy theories but I don’t think many people are thinking of those as examples.