The election was expected to take place in October 2019. But at the ceremony in Kiev Mr Zelensky said "I am dissolving the Verkhovna Rada (parliament)".
He said ending the conflict with Russian-backed rebels in the east would be his top priority.
A political novice, he campaigned strongly against official corruption.
"People must come to power who will serve the public," he said on Monday.
Russian President Vladimir Putin said he would not congratulate Mr Zelensky on his inauguration, but would wait for "the first successes in settling the internal conflict in south-eastern Ukraine, and in normalising Russian-Ukrainian relations".
Meanwhile, Russia's Foreign Minister Sergei Lavrov called on Ukraine's new leader to implement the Minsk peace accord agreed in 2015, and also suggested an "all for all" prisoner swap between both sides.
Mr Zelensky a former television star, acted the part of Ukrainian president in a popular comedy series. He scored a landslide victory in last month's presidential election.
"We must become Icelanders in football, Israelis in defending our native land, Japanese in technology," he said at the swearing-in.
He said Ukrainians must also "become Swiss in our ability to live happily with each other, despite any differences".
"Our first task is to achieve a ceasefire in Donbas," he said, referring to the eastern region controlled by Russian-backed rebels.
In the election Mr Zelensky ousted incumbent Petro Poroshenko, who had been in power since 2014.
Mr Zelensky has given few details of his plans since winning by a landslide on 21 April. He has left it to a team of advisers to try and reassure people that he knows what he's doing.
At the ceremony, Mr Zelensky was given golden symbols of office, including a sceptre, which he held aloft in a victory salute.
A choir in folk costume sang patriotic hymns to welcome him.
Russian state TV said no Russian official had been invited to the inauguration.
Mr Zelensky made his inaugural address in Ukrainian but at one point, referring to the conflict in the east, he switched to Russian, saying: "I'm convinced that for this dialogue to start, we must see the return of all Ukrainian prisoners" – apparently a message to the Russian government.
Fighting in the east has claimed about 13,000 lives since Moscow annexed Ukraine's Crimea peninsula in 2014.
