But according to Africa's Visa Openness Index - which measures the extent to which each country in Africa is open to visitors from other African countries - most countries are making progress towards simplifying entry processes and dropping restrictions to some other nations.
In 2022, Kenya was ranked 31st on the index out of 54 states.
President Ruto told an audience in Congo-Brazzaville that it was bad for business.
"When people cannot travel, businesspeople cannot travel, entrepreneurs cannot travel, we all become net losers.
"Let me say this: As Kenya, by the end of this year, no African will be required to have a visa to come to Kenya," he said to loud cheers from the conference delegates.
"Our children form this continent should not be locked in borders in Europe and also be locked in borders in Africa."
He was speaking at a summit aimed at protecting some of the world's largest rainforests.
The AU launched its African passport in 2016. The idea behind the passport is for all African citizens to be able to travel throughout the continent without visas - but it is still not widely available.
This is in part because of concerns about security, smuggling and the impact on the local employment markets.
If dropping all visa restrictions is currently a step too far, the Visa Openness Index report recommends a number of other measures. These include lowering fees, making visa on arrival standard for African visitors and implementing a secure e-visa system.