Relations between two longtime allies hit a new low Thursday as American officials scrambled to react to Philippine President Rodrigo Duterte’s announcement in Beijing that he was pursuing a “separation” from the United States and moving closer to China and Russia, asserting that Mr. Duterte’s remarks were “inexplicably at odds” with the close alliance between Washington and Manila.

It was just the latest provocative declaration from the outspoken new Philippine leader, at a time when the Obama administration has been trying to rally countries in the region to stand up to a major Chinese push for control of the South China Sea.

U.S. officials appeared divided on the impact of Mr. Duterte’s remarks, which he made during a high-profile visit to China. One official suggested that the move may be positive, but another said Mr. Duterte’s stance contradicted his own government’s efforts to build stronger diplomatic and security ties with Washington. But the confusion was evident.

“We are going to be seeking an explanation of exactly what the president meant when he talked about separation,” said State Department spokesman John Kirby. “It’s not clear to us exactly what that means in all its ramifications.”

The White House tried to downplay the development. Spokesman Eric Schultz told reporters that Filipino officials had issued no request to alter bilateral cooperation with the U.S. and even suggested that the development may be positive for American interests in Asia.

“We don’t consider this a zero-sum game. We believe that it’s in our national security interest when our partners and allies in the region have strong relationships with China,” he said in remarks that drew ire from some on Capitol Hill.

“Good foreign policy should be grounded in reality, but the White House just said that the Philippines aligning with China and dismissing the United States is in our national security interests,” said Sen. Ben Sasse, Nebraska Republican. “If that’s the case, I’d like to know how the White House defines ‘interests’ and ‘allies.’”

Mr. Duterte, a longtime provincial mayor with a populist law-and-order agenda, has been called the “Filipino Donald Trump” because he won the presidency in May with his penchant for politically incorrect remarks.

Although he stopped short of saying he would formally revoke a 70-year-old treaty alliance with Washington, his comments in a speech sparked fears that he may well be bent on eventually revoking the U.S. military’s access to five key bases in the Philippines.

“In this venue, I announce my separation from the United States,” Mr. Duterte told an audience of some 200 Chinese and Philippine business people in the Chinese capital. “Both in military, not maybe social, but economics also. America has lost,” he said to applause.

He made the statement a day after a rally outside the U.S. Embassy in Manila turned violent when a Philippine police van rammed into protesters, leaving several bloodied.

The tensions have come with remarkable swiftness, Max Boot, a senior fellow at the Council on Foreign Relations, said in a blog post Thursday for the journal Foreign Policy, and largely reflect the change ushered in by Mr. Duterte’s election.

“The Philippines has seen a vertigo-inducing change in its foreign policy orientation since Rodrigo Duterte became president this summer,” Mr. Boot wrote. “This crude populist is now transforming the Philippines‘ relationship with the United States in a fundamental and worrying manner.”

Growing Chinese might

Mr. Duterte’s effort to engage China reflects Beijing’s growing economic and military might in the region.

Earlier this year, a tribunal in The Hague handed Manila a major victory over Beijing in a legal dispute over sovereignty rights in the South China Sea. The case was brought by Mr. Duterte’s predecessor.

The Obama administration has been scrambling to get a clear read on Mr. Duterte’s posturing since the 71-year-old maverick and previous longtime provincial mayor took office on June 30.

He made global headlines in September by referring to President Obama as a “son of a whore” for criticizing his harsh anti-drug campaign and subsequently announced that he may end Philippine navy participation in joint patrols with U.S. forces in the South China Sea.

Mr. Obama canceled a planned bilateral meeting at a regional summit in Laos in response to Mr. Duterte’s outburst.

“This massive geopolitical shift is entirely Duterte’s doing. It cannot be explained any other way,” Mr. Boot wrote. “It is a product of his peculiar psychology.”

Behind the scenes, the clashes raised red flags in the Obama administration, which spent years pursuing deeper ties with Manila as part of a “rebalance” of American resources to Asia in the face of a rising China.

Analysts say the initiative has largely centered on using smaller nations, including the Philippines and Vietnam, to stand up to growing Chinese military and economic dominance in the region.

But Mr. Duterte said in September that he was seriously considering buying weapons from China and from Russia, despite the Philippines‘ longtime reliance on Washington for such purchases and for other security needs.

This week in Beijing, Mr. Duterte’s trade secretary said $13.5 billion in deals would be signed during the trip.

Mr. Duterte told his Chinese hosts, “I’ve realigned myself in your ideological flow and maybe I will also go to Russia to talk to [President Vladimir] Putin and tell him that there are three of us against the world — China, Philippines and Russia. It’s the only way.”

China pulled out all the stops to welcome Mr. Duterte. His speech Thursday was during a forum in China’s Great Hall of the People attended by Chinese Vice Premier Zhang Gaoli. During the official greeting ceremony earlier in the day, a marching band played, complete with a baton-twirling master outside the Great Hall — an honor not extended to most other leaders.

President Xi Jinping called the Philippine president’s visit a milestone in their relations and said the two countries were brothers that could “appropriately handle disputes,” although he made no explicit mention of the South China Sea.

In Washington, Mr. Kirby told reporters that U.S. officials were baffled by Mr. Duterte’s comments about a “separation” from the U.S. and would seek an explanation when Daniel Russel, the top U.S. diplomat for East Asian and Pacific Affairs, visits Manila on a scheduled trip this weekend.

Mr. Kirby called the separation comment “inexplicably at odds with the very close relationship that we have with the Filipino people, as well as the government there, on many different levels.”

“It isn’t just the United States who is baffled by this rhetoric,” he said. “We have heard from many of our friends and partners in the region who are likewise confused about where this is going.”

In the interim, Mr. Kirby said, “we remain rock-solid in our commitment in the mutual defense treaty that we have with the Philippines.”

Mr. Schultz told reporters at the White House that “we continue to be the Philippines‘ strongest economic partner outpac[ing] any other country in terms of direct investment into that country.”

• Dave Boyer contributed to this article, which is based in part on wire service reports.