At least 43 others are missing, officials told the Kyodo news agency.
Police quoted by the agency say that more people could be unaccounted for, buried by mudslides and not yet reported as missing.
About 3,000 rescue personnel have been trying to remove mud and debris hampering the search.
Torrential rains have led to the evacuation of up to 100,000 people.
Experts say the chances of survival for people trapped without food or water in such a disaster decreases significantly after the first 72 hours, which passed early on Saturday.
On Friday afternoon all searches in the area were called off when the shape of nearby hillsides appeared to change, raising fears that more landslips could be on the way.
"Operations in (two districts) were halted as hills there were becoming misshapen," a Hiroshima police spokesman is quoted as saying by the AFP news agency.
Dozens of homes in a residential area close to a mountain on the outskirts of Hiroshima were buried.
Among those killed was one 53-year old rescue worker who died when a second landslide struck after he had already pulled several people to safety.
Reports said he was killed while holding a toddler he was trying to rescue. A father was handing his small son to the rescue worker only to see both engulfed as a fresh mudslide swept down the mountain.
"There was a really strange smell, a very raw, earthy smell. When we opened a window to see what was going on, the entire hillside just came down, with a crackling noise, a thundering noise," Reuters news agency quotes one woman who survived as telling local television.
She and her husband fled moments before mud gushed through their house, leaving boulders where they had been sleeping, Reuters says.
Correspondents add that a number of children are thought to have perished in the disaster.
Much of central and southern Japan is mountainous, with many homes nestled into steep slopes.