Technologies

Everything I Learned Testing Photoshop’s New Generative AI Tool

Adobe’s Firefly AI feature brings new fun and fakery to photos. It’s a huge change for image editing, though far from perfect.

Adobe is building generative AI abilities into its flagship image-editing software with a new Photoshop beta release Tuesday. The move promises to release a new torrent of creativity even as it gives us all a new reason to pause and wonder if that sensational, scary or inspirational photo you see on the internet is actually real.

In my tests, detailed below, I found the tool impressive but imperfect. Adding it directly to Photoshop is a big deal, letting creators experiment within the software tool they’re likely already using without excursions to Midjourney, Stability AI’s Stable Diffusion or other outside generative AI tools.

With Adobe’s Firefly family of generative AI technologies arriving in Photoshop, you’ll be able to let the AI fill a selected part of the image with whatever it thinks most fitting – for example, replacing road cracks with smooth pavement. You can also specify the imagery you’d like with a text prompt, such as adding a double yellow line to the road.

Firefly in Photoshop also can also expand an image, adding new scenery beyond the frame based on what’s already in the frame or what you suggest with text. Want more sky and mountains in your landscape photo? A bigger crowd at the rock concert? Photoshop will oblige, without today’s difficulties of finding source material and splicing it in.

Photoshop’s Firefly, which is scheduled to emerge from beta testing in the second half of 2023, can be powerful. In Adobe’s live demo, the tool was often able to match a photo’s tones, blend in AI-generated imagery seamlessly, infer the geometric details of perspective even in reflections and extrapolate the position of the sun from shadows and sky haze.

Such technologies have been emerging over the last year as Stable Diffusion, Midjourney and OpenAI’s Dall-Ecaptured the imaginations of artists and creative pros. Now it’s built directly into the software they’re most likely to already be using, streamlining what can be a cumbersome editing process.

«It really puts the power and control of generative AI into the hands of the creator,» said Maria Yap, Adobe’s vice president of digital imaging. «You can just really have some fun. You can explore some ideas. You can ideate. You can create without ever necessarily getting into the deep tools of the product, very quickly.»

Now you’d better brace yourself for that future.

Photoshop’s Firefly AI imperfect but useful

In my testing, I frequently ran into problems, many of them likely stemming from the limited range of the training imagery. When I tried to insert a fish on a bicycle to an image, Firefly only added the bicycle. I couldn’t get Firefly to add a kraken to emerge from San Francisco Bay. A musk ox looked like a panda-moose hybrid.

Less fanciful material also presents problems. Text looks like an alien race’s script. Shadows, lighting, perspective and geometry weren’t always right.

People are hard, too. On close inspection, their faces were distorted in weird ways. Humans added into shots could be positioned too high in the frame or in otherwise unconvincingly blended in.

Still, Firefly is remarkable for what it can accomplish, particularly with landscape shots. I could add mountains, oceans, skies and hills to landscapes. A white delivery van in a night scene was appropriately yellowish to match the sodium vapor streetlights in the scene. If you don’t like the trio of results Firefly presents, you can click the «generate» button to get another batch.

Given the pace of AI developments, I expect Firefly in Photoshop will improve.

It’s hard and expensive to retrain big AI models, requiring a data center packed with expensive hardware to churn through data, sometimes taking weeks for the largest models. But Adobe plans relatively frequent updates to Firefly. «Expect [about] monthly updates for general improvements and retraining every few months in all likelihood,» Adobe product chief Scott Belsky tweeted Tuesday.

Automating image manipulation

For years, «Photoshop» hasn’t just referred to Adobe’s software. It’s also used as a verb signifying photo manipulations like slimming supermodels’ waists or hiding missile launch failures. AI tools automate not just fun and flights of fancy, but also fake images like an alleged explosion at the Pentagon or a convincingly real photo of the pope in a puffy jacket, to pick two recent examples.

With AI, expect editing techniques far more subtle than the extra smoke easily recognized as digitally added to photos of an Israeli attack on Lebanon in 2006.

It’s a reflection of the double-edged sword that is generative AI. The technology is undeniably useful in many situations but also blurs the line between what is true and what is merely plausible.

For its part, Adobe tries to curtail problems. It doesn’t permit prompts to create images of many political figures and blocks you for «safety issues» if you try to create an image of black smoke in front of the White House. And its AI usage guidelines prohibit imagery involving violence, pornography and «misleading, fraudulent, or deceptive content that could lead to real-world harm,» among other categories. «We disable accounts that engage in behavior that is deceptive or harmful.»

Firefly also is designed to skip over styling prompts like that have provoked serious complaints from artists displeased to see their type of art reproduced by a data center. And it supports the Content Authenticity Initiative‘s content credentials technology that can be used to label an image as having been generated by AI.

Generative AI for photos

Adobe’s Firefly family of generative AI tools began with a website that turns a text prompt like «modern chair made up of old tires» into an image. It’s added a couple other options since, and Creative Cloud subscribers will also be able to try a lightweight version of the Photoshop interface on the Firefly site.

When OpenAI’s Dall-E brought that technology to anyone who signed up for it in 2022, it helped push generative artificial intelligence from a technological curiosity toward mainstream awareness. Now there’s plenty of worry along with the excitement as even AI creators fret about what the technology will bring now and in the more distant future.

Generative AI is a relatively new form of artificial intelligence technology. AI models can be trained to recognize patterns in vast amounts of data – in this case labeled images from Adobe’s stock art business and other licensed sources – and then to create new imagery based on that source data.

Generative AI has surged to mainstream awareness with language models used in tools like OpenAI’s ChatGPT chatbot, Google’s Gmail and Google Docs, and Microsoft’s Bing search engine. When it comes to generating images, Adobe employs an AI image generation technique called diffusion that’s also behind Dall-E, Stable Diffusion, Midjourney and Google’s Imagen.

Adobe calls Firefly for Photoshop a «co-pilot» technology, positioning it as a creative aid, not a replacement for humans. Yap acknowledges that some creators are nervous about being replaced by AI. Adobe prefers to see it as a technology that can amplify and speed up the creative process, spreading creative tools to a broader population.

«I think the democratization we’ve been going through, and having more creativity, is a positive thing for all of us,» Yap said. «This is the future of Photoshop.»

Editors’ note: CNET is using an AI engine to create some personal finance explainers that are edited and fact-checked by our editors. For more, see this post.

Technologies

iOS 17 Cheat Sheet: Your Questions on the iPhone Update Answered

Here’s what you need to know about new features and upcoming updates for your iPhone.

Apple’s iOS 17 was released in September, shortly after the company held its Wonderlust event, where the tech giant announced the new iPhone 15 lineup, the Apple Watch Series 9 and the Apple Watch Ultra 2. We put together this cheat sheet to help you learn about and use the new features in iOS 17. It’ll also help you keep track of the subsequent iOS 17 updates.

iOS 17 updates

Using iOS 17

Getting started with iOS 17

Make sure to check back periodically for more iOS 17 tips and how to use new features as Apple releases more updates.

17 Hidden iOS 17 Features You Should Definitely Know About

+15 More

See all photos

Technologies

Get Ready for a Striking Aurora That Could Also Disrupt Radio Communications

Don’t expect the storm to cause a lingering problem, though.

A geomagnetic storm is threatening radio communications Monday night, but that doesn’t mean you should be concerned. In fact, it may be an opportunity to see a colorful aurora in the night sky.

The National Oceanic and Atmospheric Administration has issued a geomagnetic storm watch after witnessing a coronal mass ejection from the sun on Saturday. The watch, which was issued over the weekend and will expire after Monday, said the onset of the storm passing over Earth on Sunday night represented a «moderate» threat to communications. As the storm continues to pass through, it could deliver a «strong» threat on Monday night that could cause radio communications to be temporarily disrupted during the worst of it.

Even so, NOAA said, «the general public should not be concerned.»

A coronal mass ejection occurs when magnetic field and plasma mass are violently expelled from the sun’s corona, or the outermost portion of the sun’s atmosphere. In the vast majority of cases, the ejection occurs with no real threat to Earth. However, in the event the ejection happens in the planet’s direction, a geomagnetic storm occurs, and the Earth’s magnetic field is temporarily affected.

In most cases, geomagnetic storms cause little to no disruption on Earth, with radio communications and satellites affected most often. In extreme cases, a geomagnetic storm can cause significant and potentially life-threatening power outages — a prospect that, luckily, the planet hasn’t faced.

Switching poles

Every 11 years, the sun’s magnetic poles switch, with the north pole and south pole swapping positions. During those cycles, the sun’s activity ramps up as it gets closer to pole-switching time. The height of its activity is called solar maximum, and scientists believe we either may be entering the solar maximum or may be already in it.

During periods of heightened solar activity, sunspots increase on the sun and there’s an increase in coronal mass ejections, among other phenomena. According to NOAA, solar maximum could extend into October of this year before the sun’s activity calms and it works towards its less-active phase, solar minimum.

Even when geomagnetic storms hit Earth and disrupt communications, the effects are usually short-lived. Those most affected, including power grid operators and pilots and air traffic controllers communicating over long distances, have fail-safe technologies and backup communications to ensure operational continuity.

But geomagnetic storms aren’t only about radios. In most cases, they also present unique opportunities to see auroras in the night sky. When the storms hit, the plasma they carry creates a jaw-dropping aurora, illuminating the night sky with brilliant colors. Those auroras can be especially pronounced during the most intense phases of the storm, making for nice stargazing.

If you’re interested in seeing the aurora, you’ll need to be ready. The NOAA said the «brunt of the storm has passed» and even if it lingers into Tuesday, there won’t be much to see after Monday night.

Technologies

Last Total Solar Eclipse for 20 Years Is Coming: How to See and Photograph It

It’s your last chance until 2044.

Get your eclipse glasses ready, Skygazers: the Great American Eclipse is on its way. On April 8, there’ll be a total eclipse over North America, the last one until 2044.

A total solar eclipse happens when the moon passes between the Earth and the sun, blocking the sun and turning an otherwise sunny day to darkness for a short period of time. Depending on the angle at which you’re viewing the eclipse, you may see the sun completely shrouded by the moon (called totality) or some variation of it. The more off-angle you are and the further you are from the path of the eclipse, the less likely you’ll be to see the totality.

The 2024 total solar eclipse will happen on Monday, April 8. The Great American Eclipse will reach the Mexican Pacific coast at 11:07 a.m. PT (2:07 p.m. ET), and then traverse the US in a northeasterly direction from Texas to Maine, and on into easternmost Canada. If you want a good look at it, but don’t live in the path of totality, you shouldn’t wait much longer to book accommodation and travel to a spot on the path.

Or how about booking a seat in the sky? Delta Airlines made headlines for offering a flight that allows you to see the entire path of totality. Its first eclipse flight, from Austin, Texas, to Detroit sold out quickly. But as of Monday, Delta has added a second flight from Dallas to Detroit, which also covers the path of totality. The airline also has five flights that will offer prime eclipse viewing.

Not everyone can get on one of those elusive eclipse-viewing flights. Here’s a look at other options to nab a chance to see this rare sight and what to know about it.

Total solar eclipse path

The eclipse will cross over the Pacific coast of Mexico and head northeast over mainland Mexico. The eclipse will then make its way over San Antonio at approximately 2:30 p.m. ET on April 8 and move through Texas, over the southeastern part of Oklahoma and northern Arkansas by 2:50 p.m. ET.

By 3 p.m. ET, the eclipse will be over southern Illinois, and just 5 minutes later, will be traveling over Indianapolis. Folks in northwestern Ohio will be treated to the eclipse by 3:15 p.m. ET, and it will then travel over Lake Erie and Buffalo, New York, by 3:20 p.m. ET. Over the next 10 minutes, the eclipse will be seen over northern New York state, then over Vermont. By 3:35 p.m. ET, the eclipse will work its way into Canada and off the Eastern coast of North America.

Best places to watch the Great American Eclipse

When evaluating the best places to watch this year’s total eclipse, you’ll first want to determine where you’ll have the best angle to see the totality. The farther off-angle you are — in other words, the farther north or south of the eclipse’s path — the less of an impact you can expect.

Therefore, if you want to have the best chance of experiencing the eclipse, you’ll want to be in its path. As of this writing, most of the cities in the eclipse’s path have some hotel availability, but recent reports have suggested that rooms are booking up. And as more rooms are booked, prices are going up.

So if you want to be in the eclipse’s path, and need a hotel to do it, move fast. And Delta’s eclipse-viewing flight from Dallas to Detroit has just four seats left at the time of publication.

Eclipse eye safety and photography

As with any solar eclipse, it’s critical you keep eye safety in mind.

During the eclipse, and especially during the periods before and after totality, don’t look directly at the sun without special eye protection. Also, be sure not to look at the sun through a camera (including the camera on your phone), binoculars, a telescope or any other viewing device. This could cause serious eye injury. Sunglasses aren’t enough to protect your eyes from damage.

If you want to view the eclipse, you’ll instead need solar viewing glasses that comply with the ISO 12312-2 safety standard. Anything that doesn’t meet that standard or greater won’t be dark enough to protect your eyes. Want to get them for free? If you’ve got a Warby Parker eyeglasses store nearby, the company is giving away free, ISO-certified solar eclipse glasses at all of its stores from April 1 until the eclipse, while supplies last.

If you don’t have eclipse viewing glasses handy, you can instead use indirect methods for viewing the eclipse, like a pinhole projector.

Read more: A Photographer’s Adventure With the Eclipse

In the event you want to take pictures of the eclipse, attach a certified solar filter to your camera. Doing so will protect your eyes and allow you to take photos while you view the eclipse through your lens.

There’s also a new app to help you both protect your eyes and take better photos of the eclipse on your phone. Solar Snap, designed by a former Hubble Space Telescope astronomer, comes with a Solar Snap camera filter that attaches to the back of an iPhone or Android phone, along with solar eclipse glasses for protecting your eyesight during the event. After you attach the filter to your phone, you can use the free Solar Snap Eclipse app to zoom in on the eclipse, adjust exposure and other camera settings, and ultimately take better shots of the eclipse.

2024 eclipse compared to 2017

The last total solar eclipse occurred in 2017, and many Americans had a great view. Although there are plenty of similarities between the 2017 total solar eclipse and the one coming April 8, there are a handful of differences. Mainly, the 2024 eclipse is going to cover more land and last longer.

The 2017 eclipse started over the northwest US and moved southeast. Additionally, that eclipse’s path was up to 71 miles wide, compared with a maximum width of 122 miles for this year’s eclipse. Perhaps most importantly, the moon completely covered the sun for just 2 minutes, 40 seconds in 2017. This year, maximum totality will last for nearly four-and-a-half minutes.