Modernity is of course a relative term. But, for a while now many scholars have identified the German extermination of the Herero and Nama people in Namibia during 1904-1907 as the first modern genocide. However, it appears that all of the modern state technology and ideology necessary to commit genocide had been in place and used considerably earlier by another state. The Russian genocide against the Circassians during 1860-1864 appears to have all the elements at least in an embryonic form of more modern genocides such as the Holocaust and the Soviet mass deportations in the 1940s.
A good argument can be made that the US committed genocide against certain Native American nations earlier than 1860. I am thinking in particular of the forced relocation of the Cherokee and other peoples by Andrew Jackson. However, I think these earlier cases had not yet acquired all of the elements of modernity that are shared by genocides in the 20th century. They certainly had many of the same features as more recent cases of genocide, but the first fully formed case of an ideologically driven attempt to destroy a group of people as a distinct entity appears to be the Russian extermination of the Circassians. The emphasis on ideology rather than land I think is a key factor in modern genocides. The articulation of totalized exclusionary concepts of the state that did not allow for the survival of certain separate cultural groups within its territory had not yet fully evolved until the 1860s.
So while the forced relocation of the Cherokee certainly constituted a genocide in being a deliberate act that the US government knew would lead to large numbers of deaths, it lacked the modern ideological drive of dissolving them as a distinct group. The survivors were allowed to reorganize their national life in what became Oklahoma. The initial removal did not aim to liquidate the Cherokee as a coherent national group, but rather to violently displace them to a marginal territory of the US. Modern genocides driven by ideology in contrast have sought to eliminate the victim groups as national entities through various means. The Tsarist expulsion of the Circassians sought to completely annihilate the idea of a Circassian nation. There was no place within the Russian Empire for the Circassians, no matter how marginalized. The Tsarist government succeeded in permanently detaching the Circassians from their ancestral lands and deporting them to the Ottoman Empire. While Circassian communities continue to exist in Turkey, Jordan, Syria, and Palestine the dispersal prevented them from forming any large contiguous, coherent, and compact settlement from which to reclaim any political autonomy. It is only among the descendants of the small number of Circassians that avoided expulsion from the Caucasus that any territorial autonomy could be reestablished after the overthrow of the Tsarist regime.