Interior Secretary Ken Salazar said the British company had missed "deadline after deadline" in its efforts to seal a blown-out oil well.
But he said BP had agreed to pay clean-up costs beyond the current US $75m (£52m) liability limit.
Mr Salazar is due to visit the disaster site on Monday with other officials.
The oil leak began more than a month ago, when a drilling rig operated on behalf of BP exploded, killing 11 people.
Millions of barrels of oil have spewed into the ocean since then from the well's ruptured riser pipe, 1,524m (5,000ft) beneath the surface
The spill has reached Louisiana and is threatening Florida and Cuba.
Limited options
"If we find they're not doing what they're supposed to be doing, we'll push them out of the way appropriately," Mr Salazar told reporters after visiting BP's US headquarters.
The BBC's Madeleine Morris in Washington says Mr Salazar's comment is the strongest language yet used by a US official towards BP.
Although it is within the government's power to push BP aside, our correspondent says BP is the only organisation with the knowledge to deal with a situation like this at such a depth.
Washington is coming under increasing criticism for its response to the disaster, as well as pressure to assume control from BP.
Most of the information about the extent of the spill as well as measures to plug the leak have come from BP itself, leading to allegations the American government has been too reliant on the company for dealing with the spill, our correspondent adds.
Mr Salazar said the government had sent a team of scientific "all-stars" to BP's headquarters in Houston.
"They have pushed BP in every way that they can to kill the well and they have pushed BP in every way that they can to stop the pollution," he said.
"If there is a way to kill this well, they will find it. If there is a way to stop this pollution from spreading, they will find it."
He said the best option at the moment was to proceed with a "top kill" which involves injecting heavy drilling fluids into the ruptured pipe.
Mr Salazar added that there was "no question" that BP was doing its utmost to resolve the situation "because this is an existential crisis for one of the world's largest companies".
"Do I have confidence that they know exactly what they're doing? No, not completely."
Short-term options
Speaking on Saturday, President Barack Obama blamed the spill on "a breakdown of responsibility" at BP.
BP's efforts suffered a new setback when it reported that its containment method on the ocean floor was capturing much less of the leaking oil than three days ago.
Its managing director, Bob Dudley, said it would press ahead with a series of short-term options while working on a relief well not due to be completed before August.
"We will keep trying, we will not wait until August," he told CNN.
"There is no certainty at these kinds of depth."
The chief of the US coast guard, Admiral Thad Allen, acknowledged on Sunday that the government was forced to rely on BP and the private oil sector to try to plug the spill.
"They have the eyes and ears that are down there. They are necessarily the modality by which this is going to get solved," he said.
