Between 50 and 60 homes in a five-block area sustained damage, officials said.
Pastor Lester Adams said he met with a family that was shattered by the explosion. The mother had part of her ankle missing and her feet were crushed, he said. Her daughter had cuts and her son had to get six "staples" in the back of his head.
"They went to check and see what was going on. They went out in front yard and (the) blast came from the back," he told CNN affiliate WOAI. "If they'd stayed in the house they would have probably been killed because their house collapsed."
Brad Smith lives 50 miles away and felt his house shake.
"We didn't know exactly what it was," he said. "The forecast said a line of thunderstorms was going to come through. My wife and I looked up and wondered, 'Did it get here six hours early?' "
As of Thursday afternoon, authorities had not determined what led to the deadly explosion. Cawthon said his sheriff's office; the federal Bureau of Alcohol, Tobacco, Firearms and Explosives; and the state fire marshal's office are working "to determine the exact cause of the situation."
Swanton said earlier there were no indications of criminal activity but that wasn't being ruled out yet.
Texas Attorney General Greg Abbott told reporters "it's way premature" to determine whether any criminal charges could be sought in relation to the deadly explosion.
A U.S. intelligence official told CNN there is no indication so far that the blast is related to terrorism.
While state authorities are leading the investigation, the federal government is assisting.
With help from heavy rains early Thursday, firefighters managed to quell most of the flames in the area, authorities said.
The rain and heavy winds also helped dissipate chemicals that may have been released.
Swanton emphasized that there was no cause for alarm about the air. There was no "chemical escape" that is "out of control," he said.
Anhydrous ammonia, a gas used in making fertilizer, can cause severe burns if it combines with water in the body. Exposure to high concentrations can lead to death.
The West Fertilizer Co. said it had 54,000 pounds of the chemical, The Dallas Morning News reported.
But doctors reported that the injuries they were treating, in general, came from the blast, not chemical exposure. Many people had cuts and puncture wounds.
The Federal Aviation Administration instituted a flight restriction over the town.
Authorities closed schools for the rest of the week and said after that, they will probably depend on temporary buildings and schools in other districts for a while.
West is about 75 miles south of Dallas and 120 miles north of Austin. The town's chamber of commerce touts it as "the Czech point of central Texas."
Czech immigrants arrived in the town in the 1880s, and the community still maintains strong ties to its Central European roots, with businesses named "Little Czech Bakery" and "The Czech Inn."
In 2006, the Environmental Protection Agency fined the company that ran the fertilizer plant $2,300 and told the owners to correct problems, an EPA spokesman told CNN.
David Gray said the company certified that it had fixed the deficiencies, which included a failure to file a risk management program plan on time.