Technologies

Facebook’s AI research could spur smarter AR glasses and robots

Rummaging through drawers to find your keys could become a thing of the past.

Facebook envisions a future in which you’ll learn to play the drums or whip up a new recipe while wearing augmented reality glasses or other devices powered by artificial intelligence. To make that future a reality, the social network needs its AI systems to see through your eyes.

“This is the world where we’d have wearable devices that could benefit you and me in our daily life through providing information at the right moment or helping us fetch memories,” said Kristen Grauman, a lead research scientist at Facebook. The technology could eventually be used to analyze our activities, she said, to help us find misplaced items, like our keys.

That future is still a ways off, as evidenced by Facebook’s Ray-Ban branded smart glasses, which debuted in September without AR effects. Part of the challenge is training AI systems to better understand photos and videos people capture from their perspective so that the AI can help people remember important information.

Facebook said it teamed up with 13 universities and labs that recruited 750 people to capture more than 2,200 hours of first-person video over two years. The participants, who lived in the UK, Italy, India, Japan, Saudi Arabia, Singapore, the US, Rwanda and Colombia, shot videos of themselves engaging in everyday activities such as playing sports, shopping, gazing at their pets or gardening. They used a variety of wearable devices, including GoPro cameras, Vuzix Blade smart glasses and ZShades video recording sunglasses.

Starting next month, Facebook researchers will be able to request access to this trove of data, which the social network said is the world’s largest collection of first-person unscripted videos. The new project, called Ego4D, provides a glimpse into how a tech company could improve technologies like AR, virtual reality and robotics so they play a bigger role in our daily lives.

The company’s work comes during a tumultuous period for Facebook. The social network has faced scrutiny from lawmakers, advocacy groups and the public after The Wall Street Journal published a series of stories about how the company’s internal research showed it knew about the platform’s harms even as it downplayed them publicly. Frances Haugen, a former Facebook product manager turned whistleblower, testified before Congress last week about the contents of thousands of pages of confidential documents she took before leaving the company in May. She’s scheduled to testify in the UK and meet with Facebook’s semi-independent oversight board in the near future.

Even before Haugen’s revelations, Facebook’s smart glasses sparked concerns from critics who worry the device could be used to secretly record people. During its research into first-person video, the social network said it addressed privacy concerns. Camera wearers could view and delete their videos, and the company blurred the faces of bystanders and license plates that were captured.

Fueling more AI research

As part of the new project, Facebook said, it created five benchmark challenges for researchers. The benchmarks include episodic memory, so you know what happened when; forecasting, so computers know what you’re likely to do next; and hand and object manipulation, to understand what a person is doing in a video. The last two benchmarks are understanding who said what, and when, in a video, and who the partners are in the interaction.

“This sets up a bar just to get it started,” Grauman said. “This usually is quite powerful because now you’ll have a systematic way to evaluate data.”

Helping AI understand first-person video can be challenging because computers typically learn from images that are shot from the third-person perspective of a spectator. Challenges such as motion blur and footage from different angles come into play when you record yourself kicking a soccer ball or riding a roller coaster.

Facebook said it’s looking at expanding the project to other countries. The company said diversifying the video footage is important because if AR glasses are helping a person cook curry or do laundry, the AI assistant needs to understand that those activities can look different in various regions of the world.

Facebook said the video dataset includes a diverse range of activities shot in 73 locations across nine countries. The participants included people of different ages, genders and professions.

The COVID-19 pandemic also created limitations for the research. For example, more footage in the data set is of stay-at-home activities such as cooking or crafting rather than public events.

Some of the universities that partnered with Facebook include the University of Bristol in the UK, Georgia Tech in the US, the University of Tokyo in Japan and Universidad de los Andes in Colombia.

Technologies

Trump Maintains US‑Iran Negotiations Continue Amid Tehran’s Denials of Duplicity

President Trump insists that US‑Iran talks are ongoing despite Tehran’s denial of any negotiation plans, while warning that only a deal or total surrender will allow passage through the Strait of Hormuz. Conflicting statements from both sides have heightened uncertainty as the conflict enters its sixth month.

On Monday, President Donald Trump asserted that negotiations between the United States and Iran are still taking place, even after Tehran stated it has no intention of engaging in direct talks with Washington.

In a fiery Truth Social post, Trump labeled Iran’s leaders “unbelievably duplicitous,” claiming they are lying about ongoing peace talks “whether Iran wants to admit it or not.” He repeated his assertion that the United States completely controls the Strait of Hormuz, despite maritime traffic through the crucial route lingering at only a small fraction of pre‑conflict levels.

He wrote, “Nothing reaches Iran unless we allow it, and nothing will pass unless a deal—or total surrender—is achieved.”

Earlier that day, Iranian Foreign Ministry spokesperson Esmail Baghaei told reporters there is no imminent plan for U.S.–Iran negotiations, contradicting Trump’s earlier comment that talks would resume Monday afternoon. Baghaei added that Iran’s only current discussions are with Oman concerning the Strait of Hormuz.

The conflicting statements have heightened uncertainty over the peace‑talk process and the broader conflict, now in its sixth month.

Trump’s assertion about new negotiations came a day after he said on Truth Social that he had agreed to cancel a massive strike against Iran “subject to being able to rapidly make a DEAL.” He said in the same post that Iran and other Middle Eastern countries had asked him to hold off on that attack because “the perimeters of a deal has been agreed to.”

Trump has claimed dozens of times throughout the more‑than‑five‑month‑long war that a deal is at hand. No permanent deal has been signed, and a temporary ceasefire reached in June has fallen apart.

Trump has also repeatedly threatened to launch devastating strikes against Iran before backing off. After the latest example, oil prices on Monday fell and stocks surged.

BMI, a research unit of Fitch Solutions, said in a note Monday that a broader diplomatic understanding on reopening the Strait of Hormuz is still achievable this quarter, while raising the probability of its escalation scenario to 35% from 25%, citing mounting military, diplomatic and economic signs of rising U.S.-Iran tensions.

“Diplomatic progress is likely to be punctuated by periodic military flare-ups, while miscalculation by either side could trigger a renewed escalation,” BMI analysts wrote in a note. The firm said the key issue to watch is the future governance of the strait, as the Iran-Oman talks — potentially backed by Gulf states, China and the U.S. — point to efforts to build a post-conflict shipping framework.

Shipping risks persist even as diplomacy appears to be advancing. The United Kingdom Maritime Trade Operations Centre said it received a report of an incident 20 nautical miles (23 miles) northeast of Khasab, Oman — at the mouth of the strait — with a tanker’s master reporting an explosion in close proximity to the vessel at about 20:37 UTC Sunday (4:37 pm ET). The vessel and crew were safe and authorities are investigating, UKMTO said, advising ships to transit with caution.

The proposal Trump announced over the weekend calls for the U.S. and Iran to return to negotiations and continue ironing out some of the thorny issues that had derailed diplomatic efforts, according to The Associated Press, citing a regional official involved in the mediation efforts.

The official said the proposal also includes a reopening of the Hormuz Strait and halting attacks across the region, including by Iranian-backed militias in Iraq on the Arab Gulf countries and Jordan.

The U.S., for its part, will end its naval blockade on Iran and allow Tehran to export its oil, the official said, adding that no deal has been reached, although the mediation efforts remained underway.

Trump’s weekend reversal has lowered the temperature after days of escalating attacks across the Gulf. Kuwait said Saturday that Iranian forces launched a wave of drones within its airspace, with its military destroying multiple aircraft after Iran targeted critical infrastructure in the country’s north.

A parallel track with Muscat is also advancing. Iranian diplomats said Tehran was close to reaching a new arrangement with Oman to manage shipping through the Strait of Hormuz, a deal critical to preventing the war from escalating further, according to the Financial Times.

Iranian officials said negotiations over future management of the Hormuz Strait with Oman, which sits on the opposite shore of the waterway, are in their final stages. The agreed shipping route would be different from those used before, according to Iran’s Foreign Ministry spokesperson, Baghaei, adding that the new route was separate from the issue of the strait’s reopening or continued closure.

Technologies

Oil Prices Slide as Trump Halts Planned Iran Strike

Oil prices dropped after Trump cancelled a planned strike on Iran, with WTI down about 5% and Brent near 5% lower. He said the move followed requests from Tehran and regional allies for a pause while a deal shaping the Strait of Hormuz and Iran’s nuclear program is negotiated.

Oil prices fell on Monday after President Donald Trump announced he had cancelled a planned strike on Iran. WTI futures dropped roughly 5% to $80.34 a barrel, while Brent slipped about 4.7% to $83.77 a barrel.

Trump said early Sunday he called off the strike after Iran and several Middle Eastern nations asked him to hold off, noting that the outlines of a deal had been agreed upon. He added that the prospective accord would entail the immediate, full opening of the Strait of Hormuz and an end to Iran’s nuclear ambitions, according to his Truth Social post.

The president had been considering another round of strikes as diplomatic hopes waned since the conflict began on February 28. He said the U.S. and Iran would meet for talks on Monday, but Iran denied any scheduled negotiations with Washington, citing PressTV. Iran’s foreign‑ministry spokesperson Esmaeil Baghaei clarified that Tehran was only discussing shipping routes through the Strait of Hormuz with Oman. In a follow‑up Truth Social message, Trump insisted that, regardless of Iran’s acknowledgment, the United States is indeed discussing a solution to a long‑standing problem created by Iran.

Technologies

Oil Prices Drop as Trump Cancels Planned Attack on Iran

Oil prices fell sharply after President Trump announced the cancellation of a planned strike on Iran, citing a new deal that would open the Hormuz Strait and end Iran’s nuclear threat.

Oil prices fell sharply on Monday after President Donald Trump announced that he had called off a planned strike on Iran.

West Texas Intermediate futures, the U.S. benchmark, slipped roughly 5% to close at $80.34 per barrel, while Brent crude, the international benchmark, declined 4.7% to settle at $83.77 a barrel.

Trump made the announcement early Sunday, saying he had canceled the strike following requests from Tehran and other Middle Eastern countries.

In a Truth Social post, he wrote: “We have just been asked by Iran, and other Middle Eastern Countries, to hold off any attack in that the perimeters of a deal has been agreed to.”

The president indicated that the proposed agreement would include the immediate, complete, and total opening of the Hormuz Strait, as well as an end to Iran’s nuclear threat.

Trump had been weighing additional strikes amid diminishing prospects for a diplomatic resolution to the conflict that began on Feb. 28. He stated that the U.S. and Iran would hold negotiations on Monday.

Tehran denied that talks were planned with Washington, according to state news outlet PressTV.

Iran’s Foreign Ministry spokesman Esmaeil Baghaei said Tehran was only holding talks with Oman regarding the routes ships can use through the Strait of Hormuz.

In a subsequent Truth Social post, Trump added that whether “Iran wants to admit it or not, we are, in fact, talking of a solution to a problem that they have caused for decades.”