Vroom hits the brakes on its online used car business to go full throttle on auto financing and AI

Vroom hits the brakes on its online used car business to go full throttle on auto financing and AI Kirsten Korosec 16 hours

Vroom is shutting down its online used car marketplace and shifting all of its resources and capital into two business units focused on auto financing and AI-powered analytics. About 800 employees, or 90% of its workforce, will lose their jobs as a result, according to a regulatory filing.

The company said it is suspending all used car transactions through vroom.com and plans to sell off its used vehicle inventory to wholesalers. The company said it has not determined how much this wind-down will cost, partly because of uncertainty on what its used vehicle inventory will sell for, according to the regulatory filing.

Vroom, which went public in 2020, said it now plans to concentrate on two other businesses its owns and operates: United Auto Credit Corporation (UACC) and CarStory. UACC, an automotive finance company, and CarStory, which developed an AI-powered analytics and digital services platform for automotive retailers, will continue to operate.

Vroom was part of a wave of U.S. startups that launched about a decade ago all aiming to disrupt used car sales and the traditional dealership model. Most of these startups — a batch that included Shift Technologies, Beepi and Fair.com — have shut down or were acquired. Vroom survived and emerged as competitor to Carvana, which went public in 2017.

The company had one of the more successful IPOs of 2020, its share price more than doubling on its first day of trading and raising $468 million, above earlier plans for $356 million. Its share price hit a high of $65.01 in August 2020, only to come tumbling down more than 60% over the following year. The drop in valuation continued into 2022, pushing Vroom shares to under $2. Vroom shares closed at $0.53 on Monday and then dropped to $0.25 after the company announced its plans to shutter the e-commerce business. Shares are now hovering at about $0.32.

It was during those high-flying stock days that Vroom acquired Vast Holdings, which included CarStory, for $120 million in cash and stock.

At the time, the acquisition was couched as complementary to Vroom’s e-commerce business. The Austin,Texas–based company used machine learning — a form of AI in which computer systems can analyze, process and learn from data — to analyze millions of vehicle listings a day. The software it developed could then be used to provide predictive data to Vroom’s e-commerce platform. Vroom has since sold those services to third parties as well.

In October 2021,Vroom acquired United Auto Credit Corporation for $300 million in cash.

AI won’t replace your job yet because it’s still too expensive, according to MIT

Money and technology creative image

As artificial intelligence (AI) developments continue to boom, the technology becomes more capable, causing people to fear more for the security of their roles. However, a new Massachusetts Institute of Technology (MIT) study shows that the cost of deploying the technology likely makes it more economical for your employer to keep you, at least for now.

When people think of jobs being replaced by AI, most people only consider how good the technology is at performing human tasks. For that reason, many of the studies and headlines you see focus their attention on what tasks AI could automate, only exacerbating feelings of job insecurity.

Also: Why Agile doesn't work for most IT pros: The bigger you are, the harder you fall

However, the MIT study, Beyond AI Exposure, shifts from that typical approach and accounts for an overlooked factor — the cost.

In the study, the five researchers first surveyed workers to understand what performance would be required of an automated system. They then modeled the cost of building such a model, and finally, they saw if AI adoption is economically attractive. It's worth noting that the study looks explicitly at tasks that can be automated by computer vision, meaning visual tasks such as inspecting for quality.

An economically grounded estimate of task automation is a good indicator of what the future for roles will look like since, ultimately, implementing these elaborate AI models has a high cost, and companies are only interested in investing in technology that offers a return on investment.

To explain why considering cost is so essential, the study uses the example of a small bakery that might be considering using computer vision to automate visually checking its ingredients for quality.

Also: How tech professionals can survive and thrive at work in the time of AI

Since this task itself only makes up 6% of a baker's duties, a small bakery with five bakers making $48,000 yearly can save up to $14,000 in automating the tasks. The cost of deploying the computer vision is much higher than the money they would save, so it would not be financially sound to automate the task even though the technology is available.

The 45-page paper goes through the framework and methods leveraged to conduct the study, as well as the results, which should help put your job concerns at ease.

"We find that only 23% of worker compensation "exposed" to AI computer vision would be cost-effective for firms to automate because of the large upfront costs of AI systems," said the study.

Furthermore, 77% of vision tasks are not worth automating due to the cost-effectiveness of deploying a single system that can only be used at a firm level.

Also: Despite all the AI hype, success depends on just one thing

The study acknowledges that as the technology continues to be developed, the cost involved with deploying the technology will decrease. However, even with cost decreases of 20% per year, it would still take decades for it to become economical for most firms to use computer vision.

Although the study doesn't account for other AI uses, such as text generation, the researchers envision their framework being used to investigate other areas beyond computer vision.

Artificial Intelligence

Google Chrome gains AI features, including a writing helper, theme creator, and tab organizer

Google Chrome gains AI features, including a writing helper, theme creator, and tab organizer Sarah Perez @sarahintampa / 13 hours

Google’s Chrome web browser is getting an infusion of AI technology in the latest release. The company announced today it’s soon adding a trio of new AI-powered features to Chrome for Mac and Windows, including a way to smartly organize your tabs, customize your theme, and get help when writing things on the web — like forum posts, online reviews, and more.

The latter is similar to a feature already available to Google’s experimental AI search experience, SGE (Search Generative Experience), which allows users to get help drafting things like emails in different tones, like more formal or more casual, or in different lengths.

With the built-in writing helper in Chrome, Google suggests users could write business reviews, “craft a friendly RSVP to a party,” or make a more formal inquiry about a vacation rental, among other things, including writing posts in public spaces, like online forum sites.

Image Credits: Google

The still-experimental feature will be accessible in next month’s Chrome release by right-clicking on a text box or field on the web and then choosing “help me write.” To get started, you’ll first write a few words and then Google’s AI will jump in to help.

In addition to the writing assistant, AI can also be used to help organize tab groups and personalize your browser.

Chrome’s Tab Groups feature allows users who keep many tabs open to manage them by organizing them into groups. However, curating them can be a manual process, the company explains. With the new Tab Organizer, Chrome will automatically suggest and create groups based on the tabs you already have open. This feature will be available by right-clicking on a tab and selecting “Organize Similar Tabs.” Chrome will also suggest names and emojis for the tab groups it creates to make them easier to find. This feature is intended to assist when users are online shopping, researching, trip planning, or doing other tasks that tend to leave a lot of open tabs.

Image Credits: Google

A final addition mirrors the new generative AI wallpaper experience that recently arrived on Android 14 and Pixel devices. Now Google will use the same text-to-image diffusion model to allow users to generate custom themes for their Chrome browser. The feature allows you to generate these themes by subject, mood, visual style, and color by selecting the new “Create with AI” option after opening the “Customize Chrome” side panel and clicking “Change theme.” Before, Chrome offered a variety of colorful but simple themes to choose from alongside those from artists, but this feature will allow users to expand beyond the built-in choices to create a theme that better matches their own current vibe.

Image Credits: Google

Though a busy theme could be distracting, the feature at least allows users who don’t have an Android phone to test-drive Google’s generative AI for personalization, even if they end up returning to a more basic theme for day-to-day use.

While the drafting feature won’t arrive until next month’s Chrome release, Google says that the other features, like the tab organizer and AI theme creator, will roll out over the next few days in the U.S. on both Mac and Windows with the current Chrome release (M121). To access these features, you’ll sign into Chrome, select “Settings” from the three-dot menu, and then navigate to the “Experimental AI” page. Because the features are experimental, they won’t ship to enterprise and educational customers at this time, the company notes.

The features join other AI-powered and machine learning (ML) tools already available in Chrome, like its ability to caption audio and video, protect users from malicious sites via Android’s Safe Browsing feature in Chrome, silence permission prompts, and summarize web pages via the “SGE while browsing” feature.

Google says that Chrome will be updated with more AI and ML features in the coming year, including through integrations with its new AI model, Gemini, which will be used to help make web browsing easier.

UK Tech Trends & Predictions for 2024: AI Boosts Productivity, Hiring Increases, Fintech Matures, More 5G Rollouts

Harnessing AI to boost productivity will top the U.K.’s tech agenda in 2024, as the country’s IT industry and startups try to regain lost momentum, while tech job hunters will have to adapt to a changing market, according to predictions from industry experts. Here are tech predictions for the U.K. in 2024.

AI will bring the U.K. opportunities and risks

The U.K.’s AI strategy is clear: The government sees artificial intelligence as a key way to boost productivity and the sluggish economy, even as it acknowledges the technology may bring new challenges as well.

“We should look at AI more as a co-pilot rather than something that necessarily is going to replace someone’s job. AI is a tool that can help almost everybody do their jobs faster, better, quicker, and that’s how we’re seeing it being deployed,” said U.K. Prime Minister Rishi Sunak at an AI summit in November 2023 at Bletchley Park, home of the war-time codebreakers.

The U.K. also wants to be a leading voice on AI safety: The Bletchley Park Declaration, which came out of the same summit, saw nations agree to work to understand the threats of AI and the work to mitigate them.

SEE: Impact of AI on Jobs in the UK: 10-30% of Jobs Could be Automated with AI (TechRepublic)

The first big test for the U.K.’s AI strategy will come in 2024, when politics and AI are likely to collide. The U.K.’s cybersecurity agency, the National Cyber Security Centre, has warned that AI will make the spread of fake news easier in the run up to the next general election, likely to take place in the second half of 2024.

“Large language models (LLMs) will almost certainly be used to generate fabricated content; that hyper-realistic bots will make the spread of disinformation easier; and that deepfake campaigns are likely to become more advanced in the run up to the next nationwide vote,” the NCSC said. “The NCSC also assesses that democratic event, such as elections, almost certainly represent attractive targets for malicious actors and so organisations and individuals need to be prepared for threats, old and new.”

Many U.K. companies will rethink tech investments

The U.K. government has the ambition of creating half the tech unicorns (i.e., companies worth more than $1billion) in Europe by 2030 and have a similar level of venture capital investment as a share of Gross Domestic Product as the U.S.

Last year turned out to be a tough one for the U.K. software and IT services, with the market seeing little-to-no growth and 2024 could be challenging for the U.K., according to tech analyst firm TechMarketView. “This toughened environment, combined with other challenging factors such as increasing cyber threats, growing competitive pressure, and rising employee and customer expectations, is forcing many organisations to rethink tech investments,” it said in a post on its annual predictions.

“In 2024, buyers will put much greater focus on ensuring they are ready to adopt and leverage – at pace – the value of emerging technologies,” the analyst said. That will mean investing in AI, which may in turn play a pivotal role in increasing the pace of wider digital transformation strategies, the analysts said, as companies are making a bigger effort to finally rethink their legacy systems in 2024.

Fintech will mature

In terms of the technology that is booming in the U.K., financial tech (fintech) companies have been the standout success in the last decade or so, building on the historical status of London’s financial district. The fintech sector is made up of more than 1,600 companies, making it the largest in Europe.

Law firm Shoosmiths said fintech is likely to see market consolidation in 2024, with the smaller and less profitable companies being bought out. “Fintech will to an extent shift its focus from ‘newness’ and disruption to maturity and stability,” Shoosmiths said in its predictions for the year.

Hiring in the U.K. will increase

Tech recruitment had a tough year in 2023, with companies being generally more cautious than before; in an unpredictable business environment, they were reluctant to commit to big hiring decisions, Bev White, chief executive officer at recruitment firm Nash Squared told TechRepublic over email.

“However, the signs are that confidence is starting to grow back somewhat as macro conditions begin to modestly improve. The collective wisdom is that this growth in confidence will continue and will translate into more activity and increased hiring as 2024 goes on as businesses ramp up the fulfilment of their ever-present technology resourcing needs,” she said.

SEE: Crucial Skills Gaps in the UK Include AI and Strategic Thinking, According to Red Hat (TechRepublic)

Goodbye 3G networks, hello 5G?

This year will see a major technology transition, with Vodafone, EE and the Three networks all due to switch off their 3G networks. Data from telecoms regulator Ofcom shows that, as of late 2023, there were still around 2.4 million devices reliant on 2G or 3G networks, although 3G only makes up 3% of network data traffic. Consumers still using 3G devices will need to upgrade; businesses should check whether any of their wireless devices, like alarms or payment terminals, will need to be replaced or updated.

The majority of mobile traffic is carried over 4G, while Ofcom said the availability of 5G continues to “grow rapidly” and rollout will continue across 2024: For example, London’s tube network is due to be getting 5G connectivity this year.

Wireless and broadband connectivity is going from nice-to-have to vital: tech analyst CCS Insight predicts that in one major European market by 2025, a “connectivity performance rating” could be required before the sale of any property. This would rate the indoor and outdoor mobile coverage by each network operator and the current and expected status of full-fibre broadband, it said, reflecting how the level of connectivity has an impact on property prices.

Artisse AI raises $6.7M for its ‘more realistic’ AI photography app

Artisse AI raises $6.7M for its ‘more realistic’ AI photography app Sarah Perez @sarahintampa / 9 hours

Artisse, one of the many AI photo creation apps that let users generate photos of themselves using uploaded selfies combined with prompts, has raised $6.7 million in seed funding, following AI selfies’ viral moment. Similar to others on the market, Artisse users upload photos of themselves to train its AI on their images, then use a text or image prompt to generate new photos of themselves across various settings, postures, and styles. But unlike the competition, Artisse is focused on making its images more photo-realistic, so they could stand in for professional photography if required.

Under the hood, Artisse is using its own propriety model, alongside best practices and elements from open source models and tools. Because of the hyper-realistic images the app produces, Artisse became a top photo app on the Google Play Store at various times across markets including the U.S., U.K., Japan, South Korea, Canada, and Australia.

The app has been downloaded over 200,000 times to date, and its images reached an estimated 43 million people across social media, the company claims. To date, users have created over 5 million photos, its website notes.

Image Credits: Artisse

Originally bootstrapped, Artisse was founded by William Wu, who previously worked in investment and strategy with roles at McKinsey & Co. and Oaktree Capital. The founder told TechCrunch back in September that he was inspired to build an AI app after seeing how many people had “perfect” photos uploaded to their Instagram or dating profiles. However, he realized that to have those results, you’d need time and expertise with personal photography. His idea was to make that same type of photography more accessible to anyone with a smartphone

Training Artisse therefore takes longer than competitors — around 30 to 40 minutes, though the AI images take minutes to produce. Wu said this system allows it to produce more realistic images than some others on the market, as a result.

This is what Wu hopes will be Artisse’s differentiating feature.

“Midjourny clearly does well when it comes to landscapes and design work, but when it comes to people — the way to think about it, is there’s a lot of different factors and you need to build individual training sets for each of those factors.”

That means Artisse’s model takes into consideration factors like race, facial structures, skin color, lighting, camera type, camera angle, the way the body is shot, the scenery, and more.

Plus, adds Wu, “There’s an incredible amount of work that’s required in terms of data collection, data tagging, knowing what makes a good camera photo versus not.”

Artisse’s AI was trained on public domain photography, Wu notes.

“A lot of this is not about volume, it’s actually a lot about the quality of the image,” he says.

Like many apps in this space, Artisse has to overcome struggles in areas like the diversity of body shapes and skin tones, especially if users upload a reference photo where the person in the image is thinner. Another viral app Remini faced complaints in this area from women who said the app made them skinnier or with larger chests.

Artisse aims to stand out from apps like Remini and Lensa by producing photos that could be used in real life.

However, the startup’s AI model is flexible enough that users could do things with their photos that wouldn’t be appropriate, like changing their race, for example.

But Wu says he’s not encouraging that nor is this how people are generally using the product.

Image Credits: Artisse

Instead, Artisse’s users tend to leverage the app to post photos of themselves on social media — particularly those they wouldn’t be able to capture otherwise — like shots where they’re posed next to a fancy car or wearing some high-fashion look. Models and influencers are among Artisse’s early adopters along with some businesses using AI photography for their ads.

The app initially monetized by offering 25 photos for free, then charging around 20 cents per photo afterward. That attracted a casual audience who dabbled with the tech — Artisse said around 60-70% of users have been “light” users who try out the app one time. Of the 200K downloads, around 4,000 have converted to subscribers, which is the app’s new monetization model.

There are currently three tiers, priced at $7, $15, and $40 per month, where you receive anywhere from 25 to 370 photos.

Artisse claims to have tripled revenue to $1 million ARR in December 2023 and is on track to $2.5 million ARR as of this month.

“Revenue is growing pretty fast, payback period is relatively low,” Wu notes. “I see AI photography as a new category that should be probably of a similar size, if not bigger than, photo editing apps,” he says.

The startup’s $6.7 million seed funding round was led by The London Fund, a firm that makes strategic investments in high-growth companies with several consumer businesses in their portfolio.

The investment, which was inbound, made sense because the fund has an influencer marketing arm and could help with marketing the app, Wu explains. The round is still open to others.

Going forward, the 22-person team is looking to leverage its AI tech in other ways beyond consumer photos. It’s currently exploring virtual fitting room tech for online shopping, where you can model clothes on yourself in different fits and poses, as well as a group photo feature that could one day let you “pose” with a friend or even celebrity you’re a fan of (with permission). Shopping from AI photos and turning them into physical prints are other ideas being explored.

Artisse’s AI app is available on both iOS and Android.

Artisse can generate AI photos of you from prompts, templates or even a reference pic

Remini tops the App Store for its viral ‘AI headshots’ but its body edits go too far, some say

KDnuggets News, January 24: 5 Free University Courses to Learn Data Science • Convert Unstructured Data into Structured Insights with LLMs

Editor's Picks

5 Free University Courses to Learn Data Science
Bala Priya C 3 Crucial Challenges in Conversational AI Development and How to Avoid Them
Suman Saurav

This Week's Nuggets

5 FREE Courses on AI with Microsoft for 2024
Nisha Arya How Semantic Vector Search Transforms Customer Support Interactions
Taranjeet Singh 5 Ways of Converting Unstructured Data into Structured Insights with LLMs
Josep Ferrer 5 Super Helpful SQL Cheat Sheets You Can't Miss!
Bala Priya C Enroll in a Data Science Undergraduate Program For Free
Abid Ali Awan Exploring the Zephyr 7B: A Comprehensive Guide to the Latest Large Language Model
Ahmad Anis 3 Interesting Uses of Python's Context Managers
Bala Priya C AI Prompt Engineers are Making $300k/y
Nisha Arya Top 16 Technical Data Sources for Advanced Data Science Projects
Nate Rosidi 7 Steps to Landing Your First Data Science Job
Bala Priya C

From Our Partners

6 Reasons Why a Universal Semantic Layer is Beneficial to Your Data Stack
Cube Powering Up with Predictive GenAI
Pecan AI

More On This Topic

  • 5 Ways of Converting Unstructured Data into Structured Insights with LLMs
  • From Unstructured to Structured Data with LLMs
  • How to Use ChatGPT to Convert Text into a PowerPoint Presentation
  • 5 Free University Courses to Learn Data Science
  • Easily Integrate LLMs into Your Scikit-learn Workflow with Scikit-LLM
  • 5 Free University Courses to Learn Python

DSC Weekly 23 January 2024

Announcements

  • Generative AI signifies a pivotal shift in today’s technological landscape. It promises profound insights, streamlined operations, and assistance with data-driven decisions on an unprecedented scale. However, GenAI also brings forth ethical and regulatory considerations that require attention for modern businesses seeking to capitalize on the still-evolving technology. Register for the Enterprise Strategy Group’s upcoming GenAI Summit​ to engage with thought leaders as they navigate the intricate dilemmas, evolving regulatory landscape, and responsible AI practices that maximize the benefits of GenAI technology and mitigate inherent risks and biases.​
  • Sustainable IT is the backbone of the green economy. As businesses become increasingly reliant on artificial intelligence and other resource-intensive technologies, actionable measures need to be taken to reduce the environmental and social impact of technological innovation. Tune into the upcoming Achieving IT Sustainability Goals summit to hear leading experts discuss strategies for tracking carbon emissions metrics, enhancing direct current energy efficiency, optimizing data usage, and other methods to help design a sustainable IT strategy that delivers tangible results.

Top Stories

  • GenAI: Beware the Productivity Trap; It’s About Cultural Empowerment – Part 3
    January 20, 2024
    by Bill Schmarzo
    2024 promises to be a breakout year for Generative AI (GenAI) and AI. However, there are two challenges that organizations will face in 2024 to “leverage AI to get value from their data.” Challenge #1: Too much focus is on “implementing AI” and not enough on gaining organizational alignment regarding where and how value will be created using AI.
  • The future of cloud computing in business operations
    January 18, 2024
    by Manoj Kumar
    The digital era has witnessed the remarkable evolution of cloud computing, transforming it into a cornerstone of modern business operations. This technology, which began as a simple concept of centralized data storage, has now evolved into a complex and dynamic ecosystem, enabling businesses to operate more efficiently and effectively than ever before.
  • The AI radiologists replacement saga: Don’t be misled by the scaremongering – science v.s. science fiction
    January 22, 2024
    by Ajit Jaokar
    Seven years ago, an unexpected nationwide shortage of radiologists was triggered by a single statement from Professor Geoffrey Hinton. The statement was:“I think if you work as a radiologist, you are like the Wilie E Coyote in the cartoon.
Education_DSC_160x600-2

In-Depth

  • How (and when?) to hire a data scientist
    January 23, 2024
    by Saqib Jan
    Ten years ago, data was something an analyst reviewed and handed over to people who were going to use it. Now, businesses run on data, with automated processes, machine learning models, and hundreds, sometimes thousands, of people in the organization using data daily.
  • The impact of emerging technologies on data excellence
    January 23, 2024
    by Ovais Naseem
    Data is the lifeblood of our digital world. We crave it, analyze it, and base decisions on it. But a hidden truth lurks beneath the glossy surface of charts and graphs: our data is often a muddy mess. Inconsistent, riddled with errors, and prone to manipulation, it can lead to faulty insights, misguided decisions, and even financial losses.
  • Choosing the right machine learning algorithm for business success
    January 23, 2024
    by John Lee
    Machine learning can be overwhelming with its variety of tasks. Most tasks can be solved with a few ML algorithms. You need to be aware of which algorithms to select, when to apply them, what parameters to take into consideration, and how to test them.
  • Unlocking team productivity: Integrating data analytics into your Slack workflow
    January 22, 2024
    by Karen Anthony
    In a technology of rapid digital transformation, leveraging records analytics and collaborative tools may be a sport changer. One such integration that is proving to be impactful is that of data analytics with Slack. This effective merger provides teams with the capability to engage and make selections based totally on actual-time insights.
  • How to build a robust data science portfolio from scratch
    January 19, 2024
    by Rob Turner
    It’s always wise to craft a killer data science portfolio if you want to get noticed in this increasingly competitive and in-demand niche. Of course, achieving this is easier said than done, particularly if you’re getting started with nothing more than a dream of eventual career success.
  • DSC Weekly 16 January 2024
    January 16, 2024
    by Scott Thompson
    Read more of the top articles from the Data Science Central community.
  • Graph databases: Unveiling the hidden connections in unstructured data
    January 16, 2024
    by Ovais Naseem
    Traditional relational databases struggle with unstructured data – the text, images, videos, and social media feeds that flood our modern world. But graph databases, with their unique structure, offer a powerful tool for taming this chaos and extracting valuable insights. Here’s how they bring a game-changing perspective to unstructured data analytics.

Paint3D : Lighting-Less Diffusion Model for Image Generation

The rapid development of AI Generative models, especially deep generative AI models, has significantly advanced capabilities in natural language generation, 3D generation, image generation, and speech synthesis. These models have revolutionized 3D production across various industries. However, many face a challenge: their complex wiring and generated meshes often aren't compatible with traditional rendering pipelines like Physically Based Rendering (PBR). Diffusion-based models, notably without lighting textures, demonstrate impressive diverse 3D asset generation, enhancing 3D frameworks in filmmaking, gaming, and AR/VR.

This article introduces Paint3D, a novel framework for producing diverse, high-resolution 2K UV texture maps for untextured 3D meshes, conditioned on visual or textual inputs. Paint3D's main challenge is generating high-quality textures without embedded illumination, enabling user re-editing or re-lighting within modern graphics pipelines. It employs a pre-trained 2D diffusion model for multi-view texture fusion, generating initial coarse texture maps. However, these maps often show illumination artifacts and incomplete areas due to the 2D model's limitations in disabling lighting effects and fully representing 3D shapes. We will delve into Paint3D's workings, architecture, and comparisons with other deep generative frameworks. Let's begin.

Paint3D : An Introduction

The capabilities of Deep Generative AI models in natural language generation, 3D generation, and image synthesis tasks is well-known and implemented in real-life applications, revolutionizing the 3D generation industry. Despite their remarkable capabilities, modern deep generative AI frameworks generate meshes that are characterized by complex wiring and chaotic lighting textures that are often incompatible with conventional rendering pipelines including PBR or Physically based Rendering. Like deep generative AI models, texture synthesis has also advanced rapidly especially in utilizing 2D diffusion models. Texture synthesis models employ pre-trained depth-to-image diffusion models effectively to use text conditions to generate high-quality textures. However, these approaches face troubles with pre-illuminated textures that can significantly impact the final 3D environment renderings and introduce lighting errors when the lights are changed within the common workflows as demonstrated in the following image.

As it can be observed, the texture map with free illumination works in sync with the traditional rendering pipelines delivering accurate results whereas the texture map with pre-illumination includes inappropriate shadows when relighting is applied. On the other hand, texture generation frameworks trained on 3D data offer an alternative approach in which the framework generates the textures by comprehending a specific 3D object’s entire geometry. Although they might deliver better results, texture generation frameworks trained on 3D data lack generalization capabilities that hinders their capability to apply the model to 3D objects outside their training data.

Current texture generation models face two critical challenges: using image guidance or diverse prompts to achieve a broader degree of generalization across different objects, and the second challenge being the elimination of coupled illumination on the results obtained from pre-training. The pre-illuminated textures can potentially interfere with the final outcomes of the textured objects within rendering engines, and since the pre-trained 2D diffusion models provide 2D results only in the view domain, they lack comprehensive understanding of shapes that leads to them being unable to maintain view consistency for 3D objects.

Owing to the challenges mentioned above, the Paint3D framework attempts to develop a dual-stage texture diffusion model for 3D objects that generalizes to different pre-trained generative models and preserve view consistency while learning lightning-less texture generation.

Paint3D is a dual-stage coarse to fine texture generation model that aims to leverage the strong prompt guidance and image generation capabilities of pre-trained generative AI models to texture 3D objects. In the first stage, the Paint3D framework first samples multi-view images from a pre-trained depth aware 2D image diffusion model progressively to enable the generalization of high-quality and rich texture results from diverse prompts. The model then generates an initial texture map by back projecting these images onto the 3D mesh surface. In the second stage, the model focuses on generating lighting-less textures by implementing approaches employed by diffusion models specialized in the removal of lighting influences and shape-aware refinement of incomplete regions. Throughout the process, the Paint3D framework is consistently able to generate high-quality 2K textures semantically, and eliminates intrinsic illumination effects.

To sum it up, Paint3D is a novel coarse to fine generative AI model that aims to produce diverse, lighting-less and high-resolution 2K UV texture maps for untextured 3D meshes to achieve state of the art performance in texturing 3D objects with different conditional inputs including text & images, and offers significant advantage for synthesis and graphics editing tasks.

Methodology and Architecture

The Paint3D framework generates and refines texture maps progressively to generate diverse and high quality texture maps for 3D models using desired conditional inputs including images and prompts, as demonstrated in the following image.

In the coarse stage, the Paint3D model uses pre-trained 2D image diffusion models to sample multi-view images, and then creates the initial texture maps back-projecting these images onto the surface of the mesh. In the second stage i.e. the refinement stage, the Paint3D model uses a diffusion process in the UV space to enhance coarse texture maps, thus achieving high-quality, inpainting, and lighting-less function that ensures the visual appeal and completeness of the final texture.

Stage 1: Progressive Coarse Texture Generation

In the progressive coarse texture generation stage, the Paint3D model generates a coarse UV texture map for the 3D meshes that use a pre-trained depth-aware 2D diffusion model. To be more specific, the model first uses different camera views to render the depth map, then uses depth conditions to sample images from the image diffusion model, and then back-projects these images onto the mesh surface. The framework performs the rendering, sampling, and back-projection approaches alternately to improve the consistency of the texture meshes, which ultimately helps in the progressive generation of the texture map.

The model starts generating the texture of the visible region with the camera views focusing on the 3D mesh, and renders the 3D mesh to a depth map from the first view. The model then samples a texture image for an appearance condition and a depth condition. The model then back-projects the image onto the 3D mesh. For the viewpoints, the Paint3D model executes a similar approach but with a slight change by performing the texture sampling process using an image painting approach. Furthermore, the model takes the textured regions from previous viewpoints into account, allowing the rendering process to not only output a depth image, but also a partially colored RGB image with an uncolored mask in the current view.

The model then uses a depth-aware image inpainting model with an inpainting encoder to fill the uncolored area within the RGB image. The model then generates the texture map from the view by back-projecting the inpainted image into the 3D mesh under the current view, allowing the model to generate the texture map progressively, and arriving at the entire coarse structure map. Finally, the model extends the texture sampling process to a scene or object with multiple views. To be more specific, the model utilizes a pair of cameras to capture two depth maps during the initial texture sampling from symmetric viewpoints. The model then combines two depth maps and composes a depth grid. The model replaces the single depth image with the depth grid to perform multi-view depth-aware texture sampling.

Stage 2: Texture Refinement in UV Space

Although the appearance of coarse texture maps is logical, it does face some challenges like texture holes caused during the rendering process by self-occlusion or lightning shadows owing to the involvement of 2D image diffusion models. The Paint3D model aims to perform a diffusion process in the UV space on the basis of a coarse texture map, trying to mitigate the issues and enhance the visual appeal of the texture map even further during texture refinement. However, refining the mainstream image diffusion model with the texture maps in the UV space introduces texture discontinuity since the texture map is generated by the UV mapping of the texture of the 3D surface that cuts the continuous texture into a series of individual fragments in the UV space. As a result of the fragmentation, the model finds it difficult to learn the 3D adjacency relationships amongst the fragments that leads to texture discontinuity issues.

The model refines the texture map in the UV space by performing the diffusion process under the guidance of texture fragments’ adjacency information. It is important to note that in the UV space, it is the position map that represents the 3D adjacency information of texture fragments, with the model treating each non-background element as a 3D point coordinate. During the diffusion process, the model fuses the 3D adjacency information by adding an individual position map encoder to the pretrained image diffusion model. The new encoder resembles the design of the ControlNet framework and has the same architecture as the encoder implemented in the image diffusion model with the zero-convolution layer connecting the two. Furthermore, the texture diffusion model is trained on a dataset comprising texture and position maps, and the model learns to predict the noise added to the noisy latent. The model then optimizes the position encoder and freezes the trained denoiser for its image diffusion task.

The model then simultaneously uses the position of conditional encoder and other encoders to perform refinement tasks in the UV space. In this respect, the model has two refinement capabilities: UVHD or UV High Definition and UV inpainting. The UVHD method is structured to enhance the visual appeal and aesthetics of the texture map. To achieve UVHD, the model uses an image enhance encoder and a position encoder with the diffusion model. The model uses the UV inpainting method to fill the texture holes within the UV plane that is capable of avoiding self-occlusion issues generated during rendering. In the refinement stage, the Paint3D model first performs UV inpainting and then performs UVHD to generate the final refined texture map. By integrating the two refinement methods, the Paint3D framework is able to produce complete, diverse, high-resolution, and lighting-less UV texture maps.

Paint3D : Experiments and Results

The Paint3D model employs the Stable Diffusion text2image model to assist it with texture generation tasks while it employs the image encoder component to handle image conditions. To further enhance its grip on conditional controls like image inpainting, depth, and image high definition, the Paint3D framework employs ControlNet domain encoders. The model is implemented on the PyTorch framework with rendering and texture projections implemented on Kaolin.

Text to Textures Comparison

To analyze its performance, we start by evaluating Paint3D’s texture generation effect when conditioned using textual prompts, and compare it against state of the art frameworks including Text2Tex, TEXTure, and LatentPaint. As it can be observed in the following image, the Paint3D framework not only excels at generating high-quality texture details, but it also synthesizes an illumination-free texture map reasonably well.

In comparison, the Latent-Paint framework is prone to generating blurry textures that results in suboptimal visual effects. On the other hand, although the TEXTure framework generates clear textures, it lacks smoothness and exhibits noticeable splicing and seams. Finally, the Text2Tex framework generates smooth textures remarkably well, but it fails to replicate the performance for generating fine textures with intricate detailing.

The following image compares the Paint3D framework with state of the art frameworks quantitatively.

As it can be observed, the Paint3D framework outperforms all the existing models, and by a significant margin with nearly 30% improvement in the FID baseline and approximately 40% improvement in the KID baseline. The improvement in the FID and KID baseline scores demonstrate Paint3D’s ability to generate high-quality textures across diverse objects and categories.

Image to Texture Comparison

To generate Paint3D’s generative capabilities using visual prompts, we use the TEXTure model as the baseline. As mentioned earlier, the Paint3D model employs an image encoder sourced from the text2image model from Stable Diffusion. As it can be seen in the following image, the Paint3D framework synthesizes exquisite textures remarkably well, and is still able to maintain high fidelity w.r.t the image condition.

On the other hand, the TEXTure framework is able to generate a texture similar to Paint3D, but it falls short to represent the texture details in the image condition accurately. Furthermore, as demonstrated in the following image, the Paint3D framework delivers better FID and KID baseline scores when compared to the TEXTure framework with the former decreasing from 40.83 to 26.86 whereas the latter showing a drop from 9.76 to 4.94.

Final Thoughts

In this article, we have talked about Paint3D, a coarse-to-fine novel framework capable of producing lighting-less, diverse, and high-resolution 2K UV texture maps for untextured 3D meshes conditioned either on visual or textual inputs. The main highlight of the Paint3D framework is that it is capable of generating lighting-less high-resolution 2K UV textures that are semantically consistent without being conditioned on image or text inputs. Owing to its coarse-to-fine approach, the Paint3D framework produce lighting-less, diverse, and high-resolution texture maps, and delivers better performance than current state of the art frameworks.

Powering Up with Predictive GenAI

Sponsored Content

Salted caramel. Yin and yang. Rock and roll. Predictive AI and generative AI. They’re all combinations that make such a bigger impact when they’re put together than alone. And today, we’re here to explore that last combination: Pecan’s Predictive GenAI. What is it, and how will it reshape your AI experience?

Let's explore the capabilities of Predictive GenAI and discover how it enhances the accessibility, efficiency, and impact of using predictive analytics in your business.

Defining Predictive GenAI
Defining Predictive GenAI

‎What Predictive GenAI means

Pecan’s Predictive GenAI combines predictive AI and generative AI. Predictive GenAI uses generative AI methods to support the predictive modeling process, including defining the needed model, generating an initial framework, and offering guidance in fine-tuning the model.

Predictive modeling refers to traditional machine learning methods that acquire information from training data and then use that analysis to make predictions with new data.

Consider a customer churn prediction model as an example. By analyzing historical customer data, machine learning algorithms can pinpoint patterns of behavior that often customer churn. Then, the model can examine current customer data to detect those same patterns, identifying individuals at risk of churning in the future.

Pecan AI has successfully helped many customers tackle that modeling task over the years. (These pioneers are getting a competitive edge, jumping ahead of the 61% of CEOs who aren’t yet even starting to explore predictive AI, according to McKinsey.)

While predicting customer churn is a commonly discussed use case for predictive analytics, even this example of predictive AI presents challenges. Even seasoned professionals may find it perplexing to choose the optimal approach — and something seemingly as basic as defining "churn" for a particular business can spark disagreements. The process of gathering requirements, establishing a shared understanding with the business, and manually coding to meet specific needs can be exceedingly time-consuming, demands specialized technical expertise, and brings a degree of risk.
‎
As you can imagine, tackling predictive tasks in more specific industries or departments can be even more difficult. What if instead of predicting customer churn, you wanted to predict:

  • Which of your city’s fleet of garbage trucks will break down in the next month, given their service records and mileage
  • The time it will take for a meal delivery to arrive at your home based on weather, traffic, and restaurant selection
  • The likely winner of arm-wrestling contests, given information on past competitions and the physical characteristics of contestants

These might be less common predictive tasks, but they could be incredibly useful for those who need to know those answers. However, it’s unlikely that you’ll find a pre-built tool that would help you build predictive models for these questions. As far as we know, no “arm-wrestling contest prediction software solution” is currently available on the market (but, hey, there’s a free startup idea for you).

So, whether you have a commonly shared business challenge like churn or a very niche issue, you face an uphill battle in building a predictive model to help solve it. Your options may feel limited: do you need to hand-code a solution? Hire a team of data scientists to take on the task, and brace yourself for six months to a year of waiting for a model that may or may not deliver on your actual needs?

Nope. Because Predictive GenAI is now here to help you achieve your predictive modeling goals, without needing to dive into coding or statistics yourself. With Predictive GenAI, you’ll arrive at accurate predictions more easily and far faster, a notable improvement on the tedious, slow processes of the past.

‎What Predictive GenAI can do

Whether it’s customer churn, truck maintenance, or anything else you need to solve with a predictive model, our Predictive GenAI is here to help. But what does the “GenAI” component bring into the picture?

As you're probably aware, humans can have in-depth, insightful interactions with generative AI tools using plain natural language. Dialogues with GenAI can produce diverse creative outputs, including text, images, audio, synthetic data, code, and beyond. If you've ever bombarded ChatGPT with silly questions, you've experienced firsthand GenAI's remarkable ability to handle the array of input that humans can provide.

Pecan’s Predictive GenAI transforms humans' input into predictive modeling. Our Predictive GenAI capabilities make it possible to translate your business concern into a predictive model faster than ever — no matter how specific or niche your needs may be.
‎
First, Predictive Chat will gather essential information about your goals and help you define a “predictive question” that your model can answer. Coming up with a question might sound like a relatively minor step in the process, but at Pecan, we’ve found it’s critical. For example, consider the customer churn model mentioned above. “Churn” for one business might mean they never hear from a customer again; for another business, it might mean a reduction of activity (but not a complete end of the relationship).

Defining these details as part of the predictive questions is like specifying the precise amount of ingredients for a recipe. Sure, water is part of your ingredient list for baking bread, but the amount of water makes an enormous difference in your final product.

Predictive Chat leads you through determining all of these critical ingredients to make it as easy as possible to create a successful predictive model. No guesswork required — you’ll know from the start exactly what you’ll be predicting and that it will be relevant to your business’s needs and goals.

Powering Up with Predictive GenAI
Pecan’s Predictive GenAI provides fast, seamless predictive modeling

Then, you’ll move forward to a SQL-based Predictive Notebook experience that guides your next steps. Using GenAI, our Predictive Notebook is pre-filled with the SQL queries, so you don’t have to write them yourself. Just tweak them where needed to make any adjustments you’d like. You won’t have to struggle to parse out which fields are relevant, which tables to join and how, or which aggregations you’ll need.

Combining predictive and generative AI will lead to better business outcomes. Pecan’s Predictive GenAI introduces a powerful way to gain business value by combining these technologies.

How Predictive GenAI will transform your AI efforts

It’s not always easy for data analysts and businesspeople to translate their business concerns into actionable, well-defined predictive modeling tasks. Pecan’s Predictive GenAI now handles precisely that translation — from business need to predictive model. Its generative component can understand and pursue the necessary information about business needs.

With Predictive GenAI, there’s no more wondering: “Could we build a model to predict that?” The hassles of previous AI efforts are in the past. You won’t need to try to learn Python, recruit a data scientist to your team at the cost of $250K or more per year, or seek a consulting firm that might be able to grasp the nuances of your business.

Instead, just sit down for a simple chat and start your journey toward success with Pecan’s Predictive GenAI.

You can get started for free. We can’t wait to see what you’ll predict.

More On This Topic

  • Guide To Finding The Right Predictive Maintenance Machine Learning…
  • 6 Predictive Models Every Beginner Data Scientist Should Master
  • Building Predictive Models: Logistic Regression in Python
  • KDnuggets™ News 22:n01, Jan 5: 3 Tools to Track and Visualize…
  • How Predictive Analytics is Revolutionizing Decision-Making in Tech

Kin.art launches free tool to prevent GenAI models from training on artwork

Kin.art launches free tool to prevent GenAI models from training on artwork Kyle Wiggers 8 hours

It’s a wonder what GenAI, particularly text-to-image AI models like Midjourney and OpenAI’s DALL-E 3, can do. From photorealism to cubism, image-generating models can translate practically any description, short or detailed, into art that might well have emerged from an artist’s easel.

The trouble is, many of these models — if not most — were trained on artwork without artists’ knowledge or permission. And while some vendors have begun compensating artists or offering ways to “opt out” of model training, many haven’t.

In lieu of guidance from the courts and Congress, entrepreneurs and activists are releasing tools designed to enable artists to modify their artwork so that it can’t be used in training GenAI models. One such tool, Nightshade — released this week — makes subtle changes to the pixels of an image to trick models into thinking the image depicts something different from what it actually does. Another, Kin.art, uses image segmentation (i.e., concealing parts of artwork) and tag randomization (swapping an art piece’s image metatags) to interfere with the model training process.

Launched today, Kin.art’s tool was co-developed by Flor Ronsmans De Vry, who co-founded Kin.art, an art commissions management platform, alongside Mai Akiyoshi and Ben Yu a few months ago.

As Ronsmans De Vry explained in an interview, art-generating models are trained on data sets of labeled images to learn the associations between written concepts and images, like how the word “bird” can refer not only to bluebirds but parakeets and bald eagles (in addition to more abstract notions). By “disrupting” either the image or the labels associated with a given piece of art, it becomes that much harder for vendors to use the artwork in model; training, he says.

Kin.art

An artist profile on Kin.art.

“Designing a landscape where traditional art and generative art can coexist has become one of the major challenges the art industry faces,” Ronsmans De Vry told TechCrunch via email. “We believe this starts from an ethical approach to AI training, where the rights of artists are respected.”

Ronsmans De Vry asserts that Kin.art’s training-defeating tool is superior in some ways to existing solutions because it doesn’t require cryptographically modifying images, which can be expensive. But, he adds, it can also be combined with those methods as additional protection.

Kin.art

Kin.art’s model-defeating segmentation method.

“Other tools out there to help protect against AI training try to mitigate the damage after your artwork has already been included in the dataset by poisoning,” Ronsmans De Vry said. “We prevent your artwork from being inserted in the first place.”

Now, Kin.art has a product to sell. While the tool is free, artists have to upload their artwork to Kin.art’s portfolio platform in order to use it. The idea at present, no doubt, is that the tool will funnel artists toward Kin.art’s range of fee-based art commission-finding and -facilitating services, its bread-and-butter business.

But Ronsmans De Vry is positioning the effort as mostly philanthropic, pledging that Kin.art will make the tool available for third parties in the future.

“After battle-testing our solution on our own platform, we plan to offer it as a service to allow any small website and big platform to easily protect their data from unlicensed use,” he said. “Owning and being able to defend your platform’s data in the age of AI is more important than ever … Some platforms are fortunate enough to be able to gate their data by blocking non-users from accessing it, but others need to provide public-facing services and don’t have this luxury. This is where solutions like ours come in.”