Firefighters said the blaze is thought to have started in the emergency room of the hospital in Miryang on Friday morning.
Officials said there are around 79 people injured, eight of those in a critical condition.
At least 41 people died in the fire, according to the Yonhap news agency.
Sejong Hospital also has a nursing home for the elderly, and there were 194 people in the building when the fire broke out, police said.
During the early stages of the rescue operation, fire chief Choi Man-Wu said: "Two nurses said they had seen fire suddenly erupting in the emergency room.
"The victims came both from the hospital and the nursing home. Some died on their way to another hospital."
Most of those who died were on the first and second floors of the six-storey building.
According to AFP news agency, Mr Man-Wu said all patients had been brought out of the building but there had been a delay getting patients out of the intensive care unit, where firefighters had to wait for medical staff to supervise the process.
Survivors were carried out wrapped in blankets, and images show firefighters tackling the dark smoke as the fire engulfed the building.
At a news conference, Mr Man-Wu said: "About 90 patients at the care hospital cannot move by themselves.
"Therefore, we primarily rescued all 94 patients [at the care hospital] and completed their transfers [to another hospital].
"Yet, there were about 100 patients at Sejong Hospital - in front of the care hospital - and among those hundred patients, some of them were in the intensive care unit on the third floor. The 15 patients in the intensive care unit were wearing oxygen masks."
The fire broke out at 7.35am local time (10.30pm GMT) and was extinguished by 10.26am (1.30am GMT), officials said.
South Korean President Moon Jae-In held an emergency meeting after the fire to discuss response measures.
The cause of the blaze is under investigation.
It comes just a month after 29 people were killed in a fire at a fitness club in Jecheon, which was blamed on insufficient emergency exits and flammable finishing materials.