The confidential report to the Security Council, obtained by the New York Times and the Washington Post, said that Prince deployed a force of foreign mercenaries and weapons to warlord Khalifa Haftar, who has fought to overthrow the UN-backed Libyan government, in 2019.
The $80 million operation included plans to form a hit squad to track and kill Libyan commanders opposed to Haftar - including some who were also European Union citizens, the New York Times said.
Prince, a former Navy SEAL and the brother of Trump's education secretary Betsy Devos, drew infamy as the head of the Blackwater private security firm, whose contractors were accused of killing unarmed Iraqi civilians in Baghdad in 2007.
Four who were convicted were pardoned by Trump last year.
Read also: Blackwater founder bids again for privatised Afghan war
The accusation exposes Prince to possible UN sanctions, including a travel ban, the Times said.
Prince did not cooperate with the UN inquiry and his lawyer declined to comment to the New York Times, it added.
A request for comment to the Hong Kong-based Frontier Services Group, for which Prince is a board member and deputy chairman, went unanswered.
Oil-rich Libya has been torn by civil war since a NATO-backed uprising led to the toppling and killing of dictator Muammar Gaddafi in 2011.
The country has in recent years been split between a Government of National Accord (GNA) in Tripoli, and an eastern-based administration, backed by Haftar, who has faced charges of war crimes.
Then-President Trump in 2019 praised the warlord for his role in "fighting terrorism" in Libya.
A new interim executive for the country was chosen on February 5 by the Libyan Political Dialogue Forum in Switzerland, comprising 75 participants selected by the UN to represent a broad cross-section of society.
Haftar has pledged his support for the initiative.
Follow us on Facebook, Twitter and Instagram to stay connected