Visual 1st Perspectives


July 24, 2024

Sketch-to-Image: the next frontier of hybrid photo/GenAI images

GenAI images these days can be created through many different prompts, ranging from good-"old" text-to-image prompts, to photos-to-image, video-to-image, audio-to-image or even multimodal-to-image prompts. More recently, we're also seeing solutions coming to market that use sketches to generate images (at Visual 1st last year we saw early demos of PromeAI and Corbu).


The newest kid on the block is Samsung’s Sketch to Image feature, for now exclusively available on their new and shiny Galaxy Z Fold 6 and Z Flip 6 phones, which enables the user to submit a rough sketch that turns into a photorealistic object that blends into an existing photo.

Image credit: Allison Johnson, for The Verge

Image credit: Allison Johnson, for The Verge

In other words, Sketch to Image turns a photo into a hybrid (photo/GenAI) image. And according to the reviews I’ve read, it does a remarkably good job merging these image types. For instance, the AI-generated objects are properly lit for the photo scene and they cast an appropriate shadow. If the part of the photo where the object is being placed is out of focus (such as the poppies on the left in the image above), the AI-generated object (the bee above) will be appropriately blurred as well. 

Image credit: Future / Lance Ulanoff, for rechradar

Like for all GenAI imagery at this point, don’t expect everything to turn out 100% perfect just yet (zoom in on the doggie in this otherwise great hybrid image and notice that it actually features a paw coming out of its jaw 🙂 – but hey, that’s life on the cutting edge of things! 


For now, Sketch to Image appears to work great for smaller objects, such as a sketch-triggered photorealistic car added to a photo of a busy road, and the end result is surprisingly realistic in the eyes of the casual observer.


While these types of image additions to camera-captured photos are nothing new – the power user could have created them in Photoshop or the likes as well; or the casual user could have created them in dedicated GenAI image web apps by using text-prompts or the app’s UI – Samsung’s Sketch to Image takes the instant gratification and ease of use of hybrid image creation to a whole new level.


Not only can you create these hybrid images on a camera-equipped device that you always have with you, of from which you can immediately share your images with anyone anywhere, or even turn them into print products by uploading them into a print ordering app, but you can also create that perfect image in a fraction of a second by just adding a doodle to the photo that you just captured.


You took that great photo of these pretty flowers but missed a cute bumble bee that would have made the image perfect for sharing or printing? No worries, 30 seconds and a rough sketch later, and you’ve got it!


If you’re a developer of, say, photo capture, editing, sharing, UGC stock, ecommerce, printing or other photo solutions: It’s time to strategize how you could benefit from these “perfect” and so easily created images that will be popping up all over the place.


But do I hear the sound of groaning and jaws clenching? Yes – it’s the exact sound I heard all the time when Photoshop started getting traction! And yes, that photo of the flowers with that cute bumble bee ain’t the same as when a nature photographer spent hours waiting for that special moment when a real bumble bee finally decided to descend on those pretty flowers.


And yes, for the foreseeable folks will go overboard using apps like Sketch to Image – I can imagine the doodling on that device that’s always there when we have some time to kill is addictive – but at some point we’ll probably get tired of it. Or find an equilibrium and figure out when it makes or doesn’t make sense to embellish our photos with AI objects this way. Just how most folks have learned to decide when to alter their photos in Photoshop


But in the meantime: be aware, soon we’ll see a lot of awfully cute images popping up and we’ll keep wondering how they were created.

In Samsung’s case hybrid images do include an “AI Generated Content” watermark that you can’t remove – but could easily crop out. Samsung is also currently not listed as a C2PA member, so the company has its work carved out to avoid being seen as a deepfake enabler.


But the ins and outs of image authentication is a whole other discussion I’ll keep for another time. 


Best,


Hans Hartman 

And a few more things ...

Meta. Imagine… GenAI prompts for turning that boring selfie into ... whatever. Now in beta in Meta AI (Meta’s AI-powered assistant across Facebook, Instagram, Messenger and the web): Imagine Yourself. This feature creates images based on your selfie and a prompt like “Imagine me surfing” or “Imagine me on a beach vacation.” Meta AI also launched AI editing tools that let you add or remove, change or edit image objects with prompts like “Change the cat to a corgi.” 


Meta. OpenAI, here we come. Meta is releasing Llama 3.1, the largest-ever open-source AI model, which reportedly outperforms GPT-4o and Anthropic’s Claude 3.5 Sonnet on several benchmarks. The largest version has 405 billion parameters and was trained with over 16,000 of Nvidia’s ultra-expensive H100 GPUs. And yes, Meta IA was trained on Llama.


Microsoft. Canva, here we come. Canva is increasingly stepping on Adobe’s toes. Now it’s Microsoft’s turn to step on Canva’s. Microsoft’s AI-powered Designer app is officially coming out of preview and is now available as an iOS or Android app, a web app, and a Windows app. Designer lets you generate images and designs with text prompts to create things like stickers, greeting cards, invitations, collages and more.

Conference:

Oct. 16 (PM) – 17 (AM + PM)


Pre-conference networking:

Oct. 16 (AM)

Dead Pixels Society Meetup

Women in Imaging Luncheon


Where: Fort Mason, San Francisco


Buy $100-off Super Early Bird ticket! 


Program & speakers to date

Speaker bios to today

Tinder. How do I look at my best? Choices, choices… To help you find that perfect match, Tinder launches an AI-powered curation feature to select that gorgeous photo of yourself that’s hiding somewhere in your camera roll. [if all fails, imagine what Meta's Imagine Yourself can do for you!]


Taopix. New release. Taopix announces a new release of Taopix Online, its white label solution for selling personalized gifts, calendars and photobooks online. With more than 80 new features and improvements, the new release includes the ability to create UI-theming unique to the retailer. Other features include a new cropping tool, improved mobile experiences and HiDPI Editor previews.


IMG.LY. AI for design and workflow automation. Visual 1st Platinum sponsor IMG.LY is integrating LLMs with its CreativeEditor SDK for natural language design alteration and workflow instructions. Just tell IMG.LY’s AI Assistant something along the lines of “Change his design to fit into a portrait Instagram story. Move elements and resize if necessary” and the redesign is done before you know it.


Pixii. Rangefinder going full-frame. Past Visual 1st presenter Pixii is now accepting pre-orders for its first full-frame rangefinder camera, the Pixii Max. The camera features a full-frame 24.5MP BSI CMOS sensor to deliver sharp, low-noise images with enhanced dynamic range. With a 64-bit processor, 128GB of internal storage, and a Leica M-compatible lens mount, the Pixii Max merges state-of-the-art digital imaging with a traditional, analog shooting experience.


Click & Lens Protocol. Partnership. Click, a new mobile app to fight misinformation by making it easy to capture and publish authenticated content, powered by the Nodle Network, is partnering with Lens Protocol, an open social platform that allows for content ownership. The partnership will enable a new “Share to Lens” integration that enables Click photos & videos, aka "Deep Reals" with immutable digital proofs of authenticity, to be shared across the Lens decentralized social web. Click will be demoed in a Show & Tell session at our upcoming Visual 1st conference.

Join us Oct. 16-17 in San Francisco for our 12th annual edition of Visual 1st !


Platinum Conference Sponsors to date:


Gold Conference Sponsors to date:


Silver Conference Sponsors to date:

Partner Sponsors to date:


Archive & Subscribe Share your news with us | Connect on LinkedIn