And those seeds of ill will that were spread by Melkor, and Sauron after him, live on in the hearts of Men, even unto this day.
This is a legacy of what is known as Morgoth's Ring. Unlike Sauron who concentrated his power into one small, but very powerful ring, Morgoth disseminated his own power over the whole of Middle Earth. In order to be able to control the hearts and minds of men he corrupted the very flesh they were made from by corrupting every single piece of matter in Middle Earth. This is also why elves grow weary in Middle Earth and begin to fade.
The Black Numenorians, by the way, were not just centered in Umbar either. They had settlements all around the coasts of southern Middle Earth, and possibly into the East too. Umbar was mainly a problem for Gondor, however, because of the Kin-Strife civil war. When the exciled Gondorian King, Eldacar returned with an army and slew the usurper Castamir, Castamir's son fled to Umbar with the entire Gondorian fleet. These folk then either allied themselves with or siezed Umbar from the Black Numenorians and became the Corsairs of Umbar. They were not actually Black Numenorians themselves, but pure blooded Gondorians who had been unhappy that Eldarcar's father had taken a Rohirrim bride and diluted the bloodline. Their hatred for Gondor did run deep though, and they persistantly raided Gondor's coasts between TA 1448 and 1810. In TA 1810 King Telumehtar Umbardacil managed to take Umbar and had the last descendants of Castamir killed. Shortly after this Umbar fell to the men of Harad, who themselves quickly became the new Corsairs of Umbar. It is these men, therefore, who Aragorn was fighting against; Haradrim/Gondorian hybrids not Black Numenorians.