Building collapses are common in Africa’s most populous nation, where millions live in dilapidated structures and construction standards are often flouted.
The church collapsed Tuesday evening in the Okpanam suburb of Asaba, the capital of Delta State, local police spokesman Bright Edafe said.
Eleven of those rescued were still being treated in hospital, he added.
Local media said the building came down as an evening service was being held.
Building standards have been in the spotlight since a high-rise building under construction collapsed in Lagos in November, killing at least 45 people.
Bad workmanship, low-quality materials and corruption to bypass official oversight are often blamed for Nigerian building disasters.
Since 2005, at least 152 buildings have collapsed in Lagos, a city of about 20 million people, according to a South African university researcher.
One of those incidents that sparked widespread anger was in 2014 when dozens of people died in a church collapse in Lagos.