President Barack Obama, opening the biggest international meeting hosted by the US since 1945, greeted leaders from nearly 50 countries.
Officials said more should be done to prevent theft or smuggling.
Meanwhile France's leader stressed his country could not give up its own nuclear weapons.
The US welcomed a Ukrainian pledge to eliminate its stockpile of highly enriched uranium by 2012.
The two-day summit is taking place without representatives of Iran and North Korea, neither of whom were invited by the US because of the disputes over their nuclear programmes.
Israeli Prime Minister Benjamin Netanyahu dropped plans to attend the summit, reportedly because of concern that Muslim states planned to press for Israel to open its own nuclear facilities to international inspection.
'Proliferators not welcome'
The head of the International Atomic Energy Agency, Ikia Amano, said that nuclear powers needed to do more to protect nuclear materials.
"The problem is that nuclear material and radioactive material are not well protected and member states need to better protect these materials against the theft or smuggling," he told the BBC.
"On average every two days we receive one new information on an incident involving theft or smuggling of nuclear material."
British Foreign Secretary David Miliband said that nuclear nations like Pakistan were vulnerable.
"The message from this summit is that any country can be treated as a normal country on nuclear matters if it behaves like a normal country," he said.
"Proliferators are not welcome in the modern world, nuclear proliferators especially, and I think it's a very clear message to the Iranians and others that there is an international desire to use civilian nuclear power for beneficial purposes, but not to allow it to leech into a military weapons programme that could be so dangerous, especially in a region like the Middle East."
A senior American counter-terrorism expert, John Brennan, warned that al-Qaeda had been seeking material for a nuclear bomb for more than 15 years.
"There have been numerous reports over the past eight or nine years of attempts to obtain various types of purported material," he told reporters.
"We know al-Qaida has been involved a number of times. We know they have been scammed a number of times."
President Obama and his Chinese counterpart Hu Jintao reportedly agreed at pre-summit talks to step up pressure on Iran over its atomic plans.
"The Chinese very clearly share our concern about the Iranian nuclear programme," said Jeff Bader, Mr Obama's senior director for Asia on the National Security Council.
"The two presidents agreed the two delegations should work together on a sanctions resolution in New York."
