Рубрика: AI

The release of the Barbie movie has been highly anticipated all summer and has become a cultural moment. To tease the movie, Warner Bros has dropped fun and unique posters, trailers, merch, and even an AI generator.

Also: This viral AI TikTok trend could score you a free headshot. Here's how

The movie popularized posters that showcase different cast members and overlay the character's photo on a sparkly starburst, a sky background, with the movie's title in its iconic font over the photo.

Now you can get your selfie transformed into that photo in seconds.

Warner Bros and Photoroom, an AI-based photo app, collaborated to create a Barbie Selfie Generator, which lets users upload a selfie and transform it into a Barbie movie character's poster.

The tool, barbieselfie.ai, was initially released on April 3 and has been used 13 million times since, according to the release. When you upload a photo, Photoroom removes your background and overlays your selfie over the iconic Barbie poster.

Also: This new ChatGPT feature means better responses with less work

I put the generator to the test and using the tool is as intuitive as it sounds. All you have to do is upload a selfie, make some adjustments to your selfie's position, and you are left with a final product you can easily download. My final result can be seen in the photo at the top of the article.

Artificial Intelligence

As AI porn generators get better, the stakes get higher

Porn generators have improved while the ethics around them become stickier

Kyle Wiggers 9 hours

As generative AI enters the mainstream, so, too, does AI-generated porn. And like its more respectable sibling, it’s improving.

When TechCrunch covered efforts to create AI porn generators nearly a year ago, the apps were nascent and relatively few and far between. And the results weren’t what anyone would call “good.”

The apps and the AI models underpinning them struggled to understand the nuances of anatomy, often generating physically bizarre subjects that wouldn’t be out of place in a Cronenberg film. People in the synthetic porn had extra limbs or a nipple where their nose should be, among other disconcerting, fleshy contortions.

Fast-forward to today, and a search for “AI porn generator” turns up dozens of results across the web — many of which are free to use. As for the images, while they aren’t perfect, some could well be mistaken for professional artwork.

And the ethical questions have only grown.

No easy answers

As AI porn and the tools to create it become commodified, they’re beginning to have frightening real-world impacts.

Twitch personality Brandon Ewing, known online as Atrioc, was recently caught on stream looking at nonconsensually deepfaked sexual images of well-known women streamers on Twitch. The creator of the deepfaked images eventually succumbed to pressure, agreeing to delete them. But the damage had been done. To this day, the targeted creators receive copies of the images via DMs as a form of harassment.

The vast majority of pornographic deepfakes on the web depict women, in truth — and frequently, they’re weaponized.

A Washington Post piece recounts how a small-town school teacher lost her job after students’ parents learned about AI porn made in the teacher’s likeness without her consent. Just a few months ago, a 22-year-old was sentenced to six months in jail for taking underage womens’ photos from social media and using them to create sexually explicit deepfakes.

In an even more disturbing example of the ways in which generative porn tech is being used, there’s been a small but meaningful uptick in the amount of photorealistic AI-generated child sexual abuse material circulating on the dark web. In one instance reported by Fox News, a 15-year-old boy was blackmailed by a member of an online gym enthusiast group who used generative AI to edit a photo of the boy’s bare chest into a nude.

Reddit users have been scammed with AI porn models, meanwhile — sold explicit images of people who don’t exist. And workers in adult films and art have raised concerns about what this means for their livelihoods — and their industry.

None of this has deterred Unstable Diffusion, one of the original groups behind AI porn generators, from forging ahead.

Enter Unstable Diffusion

When Stable Diffusion, the text-to-image AI model developed by Stability AI, was open sourced late last year, it didn’t take long for the internet to wield it for porn-creating purposes. One group, Unstable Diffusion, grew especially quickly on Reddit, then Discord. And in time, the group’s organizers began exploring ways to build — and monetize — their own porn-generating models on top of Stable Diffusion.

Stable Diffusion, like all text-to-image AI systems, was trained on a dataset of billions of captioned images to learn the associations between written concepts and images, such as how the word “bird” can refer not only to bluebirds but parakeets and bald eagles in addition to far more abstract notions.

One of the more vanilla images created with Unstable Diffusion. Image Credits: Unstable Diffusion

Only a small percentage of Stable Diffusion’s dataset contains NSFW material, giving the model little to go on when it comes to adult content. So Unstable Diffusion’s admins recruited volunteers — mostly Discord server members — to create porn datasets for fine-tuning Stable Diffusion.

Despite a few bumps in the road, including bans from both Kickstarter and Patreon, Unstable Diffusion managed to roll out a fully fledged website with custom art-generating AI models. After raising over $26,000 from donors, securing hardware to train generative AI and creating a dataset of more than 30 million photographs, Unstable Diffusion launched a platform that it claims is now being used by more than 350,000 people to generate over half a million images every day.

Arman Chaudhry, one of the co-founders of Unstable Diffusion and Equilibrium AI, an associated group, says Unstable Diffusion’s focus remains the same: creating a platform for AI art that “upholds the freedom of expression.”

“We’re making strides in launching our website and premium services, offering an art platform that’s more than just a tool — it’s a space for creativity to thrive without undue constraints,” he told me via email. “Our belief is that art, in its many forms, should be uncensored, and this philosophy guides our approach to AI tools and their usage.”

The Unstable Diffusion server on Discord, where the community posts much of the art from Unstable Diffusion’s generative tools, reflects this no-holds-barred philosophy.

The image-sharing portion of the server is divided into two main categories, “SFW” and “NSFW,” with the number of subcategories in the latter slightly outnumbering those in the former. Images in SFW run the gamut from animals and food to interiors, cities and landscapes. NSFW contains — as one might expect — explicit images of men and women, but also of nonbinary people, furries, “nonhumans” and “synthetic horrors” (think people with multiple appendages or skin melded with the background scenery).

A more adult, furry product of Unstable Diffusion. Image Credits: Unstable Diffusion

When we last poked around Unstable Diffusion, practically the entirety of the server could’ve been filed in the “synthetic horrors” channel. Owing to a lack of training data and technical roadblocks, the community’s models in late 2022 struggled to produce anything close to photorealism — or even halfway decent art.

Photorealistic images remain a challenge. But now, much of the artwork from Unstable Diffusion’s models — anime-style, cell-shaded and so on — is at least anatomically plausible, and, in some rare cases, spot on.

Improving quality

Many images on the Unstable Diffusion Discord server are the product of a mix of tools, models and platforms — not strictly the Unstable Diffusion web app. So in the interest of seeing how far the Unstable Diffusion models specifically had come, I conducted an informal test, producing a bunch of SFW and NSFW images depicting people of different genders, races and ethnicities engaged in… well, coitus.

(I can’t say I expected to be testing porn generators in the course of covering AI. Yet, here we are. The tech industry is nothing if not unpredictable, truly.)

An NSFW image from Unstable Diffusion, cropped. Image Credits: Unstable Diffusion

Nothing about the Unstable Diffusion app screams “porn.” It’s a relatively bareboned interface, with options to adjust image post-processing effects such as saturation, aspect ratio and the speed of the image generation. In addition to the prompt, Unstable Diffusion lets you specify things that you want excluded from generated images. And, as the whole thing’s a commercial endeavor, there’s paid plans to increase the number of simultaneous image generation requests you can make at one time.

Prompts run through the Unstable Diffusion website yield serviceable results, I found — albeit not predictable ones. The models clearly don’t quite understand the mechanics of sex, resulting, sometimes, in odd facial expressions, impossible positions and unnatural genitalia. Generally speaking, the simpler the prompt (e.g. solo pin-ups), the better the results. And most scenes involving more than two people are recipes for hellish nightmares. (Yes, this writer tried a range of prompts. Please don’t judge me.)

The models shows the telltale signs of generative AI bias, though.

More often than not, prompts for “men” and “women” run through Unstable Diffusion render images of white or Asian people — a likely symptom of imbalances in the training dataset. Most prompts for gay porn, meanwhile, inexplicably default to people of ambiguously LatinX descent with an undercut hairstyle. Is that indicative of the types of gay porn the models were trained on? One can speculate.

The body types aren’t very diverse by default, either. Men are muscular and tone, with six packs. Women are thin and curvy. Unstable Diffusion is very well capable of generating subjects in more shapes and sizes, but it has to be explicitly instructed to do so in the prompt, which I’d argue isn’t the most inclusive practice.

The bias manifests differently in professional gender roles, curiously. Given a prompt containing the word “secretary” and no other descriptors, Unstable Diffusion often depicts an Asian woman in a submissive position, likely an artifact of an over-representation of this particular — erm — setup in the training data.

A gay couple, as depicted by Unstable Diffusion. Image Credits: Unstable Diffusion

Bias issue aside, one might assume that Unstable Diffusion’s technical breakthroughs would lead the group to double down on AI-generated porn. But that isn’t the case, surprisingly.

While the Unstable Diffusion founders remain dedicated to the idea of generative AI without limits, they’re looking to adopt more… palatable messaging and branding for the mass market. The team, now at five people full-time, is working to evolve Unstable Diffusion into a software-as-a-service business, selling subscriptions to the web app to fund product improvements and customer support.

“We’ve been fortunate to have a community of users who are incredibly supportive. Still, we recognize that to take Unstable Diffusion to the next level, we would benefit from strategic partnerships and additional investment,” Chaudhry said. “We want to ensure we’re providing value to our subscribers while also keeping our platform accessible to those who are just getting started in the world of AI art.”

To set itself apart in ways beyond a liberal content policy, Unstable Diffusion is heavily emphasizing customization. Users can change the color palette of generated images, for example, Chaudhry notes, and choose from an array of art styles including “digital art,” “photo,” “anime” and “generalist.”

“We’ve focused on ensuring that our system can generate beautiful and aesthetically pleasing images from the simplest of prompts, making our platform accessible to both novices and experienced users,” Chaudhry said. “[Our system] gives users the power to guide the image generation process.”

Content moderation

Elsewhere, spurred by its efforts to chase down mainstream investors and customers, Unstable Diffusion claims to have spent significant resources creating a “robust” content moderation system.

A Chris Hemsworth lookalike, created with Unstable Diffusion’s tools. Image Credits: Unstable Diffusion

But wait, you might say — isn’t content moderation antithetical to Unstable Diffusion’s mission? Apparently not. Unstable Diffusion does draw the line at images that could land it in legal hot water, including pornographic deepfakes of celebrities and porn depicting characters who appear to be 18 years old or younger — fictional or not.

To wit, a number of U.S. states have laws against deepfake porn on the books, and there’s at least one effort in Congress to make sharing nonconsensual AI-generated porn illegal in the U.S.

In addition to blocking specific words and phrases, Unstable Diffusion’s moderation system leverages an AI model that attempts to identify and automatically delete images that violate its policies. Chaudhry says that the filters are currently set to be “highly sensitive,” erring on the side of caution, but that Unstable Diffusion is soliciting feedback from the community to “find the right balance.”

“We prioritize the safety of our users and are committed to making our platform a space where creativity can thrive without concerns of inappropriate content,” Chaudhry said. “We want our users to feel safe and secure when using our platform, and we’re committed to maintaining an environment that respects these values.”

The deepfake filters don’t appear to be that strict. Unstable Diffusion generated nudes of several of the celebrities I tried without complaint (“Chris Hemsworth,” “Donald Trump”), save particularly photorealistic or accurate ones (Donald Trump was gender-swapped).

A deepfaked, gender-swapped image of Donald Trump, created with Unstable Diffusion. Image Credits: Unstable Diffusion

A strange and troubling thing to probe was Unstable Diffusion’s protections against explicit child imagery. This writer would’ve rather avoided it for obvious reasons, but in the interest of putting the team’s claims to the test, I ran a single prompt.

Unstable Diffusion, shockingly, appeared to generate child porn in a blurry preview before I immediately deleted the image after. That’s a design choice that, for me, came uncomfortably close to the line.

Future issues

Assuming Unstable Diffusion receives the investment it’s seeking, it plans to shore up compute infrastructure — an ongoing challenge given the growing size of its community. (Having used the site a fair amount, I can attest to the heavy load — images usually take around a minute to generate.) It also plans to build more customization options and social sharing features, using the Discord server as a springboard.

“We aim to transition our engaged and interactive community from our Discord to our website, encouraging users to share, collaborate and learn from one another,” Chaudhry said. “Our community is a core strength — one that we plan to integrate with our service and provide tools for them to expand and succeed.”

But I’m struggling with what “success” looks like for Unstable Diffusion. On the one hand, the group aims to be taken seriously as a generative art platform. On the other, as evidenced by the Discord server, it’s still a wellspring of porn — some of which is quite off-putting.

As the platform exists today, traditional VC funding is off the table. Vice clauses bar institutional funds from investing in pornographic ventures, funneling them instead to “sidecar” funds set up under the radar by fund managers.

Even if it ditched the adult content, Unstable Diffusion, which forces users to pay for a premium plan to use the images they generate commercially, would have to deal with the elephant in the generative AI room: artist consent and compensation. Like most generative AI art models, Unstable Diffusions models are trained on artwork from around the web, not necessarily with the creator’s knowledge. Many artists take issue with — and have sued over, in fact — AI systems that mimic their styles without giving proper credit or payment.

The furry art community FurAffinity decided to ban AI-generated SFW and NSWF art altogether, as did Newgrounds, which hosts mature art behind a filter. Only recently did Reddit walk back its ban on AI-generated porn, and only partially: art on the platform must depict fictional characters.

In a previous interview with TechCrunch, Chaudhry said that Unstable Diffusion would look at ways to make its models “more equitable toward the artistic community.” But from what I can tell, there’s not been any movement on that front.

Indeed, like the ethics around AI-generated porn, Unstable Diffusion’s situation seems unlikely to resolve anytime soon. The group seems doomed to a holding pattern, trying to bootstrap while warding off controversy and avoiding alienating the community — and artists — that made it.

I can’t say I envy them.

In the current Artificial Intelligence and Machine Learning industry, “Image Recognition”, and “Computer Vision” are two of the hottest trends. Both these fields involve working with identifying visual characteristics, which is the reason most of the time, these terms are often used interchangeably. Despite some similarities, both computer vision and image recognition represent different technologies, concepts, and applications.

In this article, we will be comparing Computer Vision & Image Recognition by dwelling into their differences, similarities, and methodologies used. So let’s get started.

What is Image Recognition?

Image Recognition is a branch in modern artificial intelligence that allows computers to identify or recognize patterns or objects in digital images. Image Recognition gives computers the ability to identify objects, people, places, and texts in any image.

The main aim of using Image Recognition is to classify images on the basis of pre-defined labels & categories after analyzing & interpreting the visual content to learn meaningful information. For example, when implemented correctly, the image recognition algorithm can identify & label the dog in the image.

How Image Recognition Works?

Fundamentally, an image recognition algorithm generally uses machine learning & deep learning models to identify objects by analyzing every individual pixel in an image. The image recognition algorithm is fed as many labeled images as possible in an attempt to train the model to recognize the objects in the images.

The image recognition process generally comprises the following three steps.

• Gathering and Labeling Data

The first step is to gather and label a dataset with images. For example, an image with a car in it must be labeled as a “car”. Generally, larger the dataset, better the results.

• Training the Neural Networks on the Dataset

Once the images have been labeled, they will be fed to the neural networks for training on the images. Developers generally prefer to use Convolutional Neural Networks or CNN for image recognition because CNN models are capable of detecting features without any additional human input.

• Testing & Prediction

After the model trains on the dataset, it is fed a “Test” dataset that contains unseen images to verify the results. The model will use its learnings from the test dataset to predict objects or patterns present in the image, and try to recognize the object.

What is Computer Vision?

Computer Vision is a branch in modern artificial intelligence that allows computers to identify or recognize patterns or objects in digital media including images & videos. Computer Vision models can not only analyze an image to recognize or classify an object within an image, but also react to those objects.

The main aim of a computer vision model goes further than just detecting an object within an image, but it’s to interact & react to those objects. For example, in the image below, the computer vision model can not only identify the object in the frame, a scooter, but it can also track the movement of the object within the frame.

How Computer Vision Works?

A computer vision algorithm works just as an image recognition algorithm does, by using machine learning & deep learning algorithms to detect objects in an image by analyzing every individual pixel in an image. The working of a computer vision algorithm can be summed up in the following steps.

• Data Acquisition and Preprocessing

The first step is to gather a sufficient amount of data that can include images, GIFs, videos, or live streams. The data is then preprocessed to remove any noise or unwanted objects.

• Feature Extraction

The training data is then fed to the computer vision model to extract relevant features from the data. The model then detects and localizes the objects within the data, and classifies them as per predefined labels or categories.

• Semantic Segmentation & Analysis

The image is then segmented into different parts by adding semantic labels to each individual pixel. The data is then analyzed and processed as per the requirements of the task.

Image Recognition v/s Computer Vision : How Do They Differ?

Although both image recognition and computer vision function on the same basic principle of identifying objects, they differ in terms of their scope & objectives, level of data analysis, and techniques involved. Let’s discuss each of them individually.

• Scope and Objectives

The main objective of image recognition is to identify & categorize objects or patterns within an image. The primary goal is to detect or recognize an object within an image. On the other hand, computer vision aims at analyzing, identifying or recognizing patterns or objects in digital media including images & videos. The primary goal is to not only detect an object within the frame, but also react to them.

• Level of Analysis

The most significant difference between image recognition & data analysis is the level of analysis. In image recognition, the model is concerned only with detecting the object or patterns within the image. On the flip side, a computer vision model not only aims at detecting the object, but it also tries to understand the content of the image, and identify the spatial arrangement.

For example, in the above image, an image recognition model might only analyze the image to detect a ball, a bat, and a child in the frame. Whereas, a computer vision model might analyze the frame to determine whether the ball hits the bat, or whether it hits the child, or it misses them all together.

• Complexity

Image recognition algorithms generally tend to be simpler than their computer vision counterparts. It’s because image recognition is generally deployed to identify simple objects within an image, and thus they rely on techniques like deep learning, and convolutional neural networks (CNNs)for feature extraction.

Computer vision models are generally more complex because they detect objects and react to them not only in images, but videos & live streams as well. A computer vision model is generally a combination of techniques like image recognition, deep learning, pattern recognition, semantic segmentation, and more.

Image Recognition Vs. Computer Vision: Are They Similar?

Despite their differences, both image recognition & computer vision share some similarities as well, and it would be safe to say that image recognition is a subset of computer vision. It’s essential to understand that both these fields are heavily reliant on machine learning techniques, and they use existing models trained on labeled dataset to identify & detect objects within the image or video.

Final Thoughts

To sum things up, image recognition is used for the specific task of identifying & detecting objects within an image. Computer vision takes image recognition a step further, and interprets visual data within the frame.

Tesla To Spend $1B on AI Supercomputer, Not Falling for LLMs July 21, 2023 by Agam Shah

Tesla will spend $1 billion on its Dojo supercomputer through the next year as it beefs up video recognition capabilities for its autonomous cars.

"I think we will be spending something north of $1 billion over the next year on – through the next year, it’s well over $1 billion in Dojo," said Elon Musk, Tesla's CEO, during an earnings call.

The company also plans to deploy 300,000 Nvidia A100 GPUs by the end of 2024, which will supplement the Dojo supercomputer, which will deploy Tesla’s homegrown D1 chip.

"We may reach in-house neural net training capability of 100 exaflops by the end of next year," Musk added.

Tesla uses multiple types of hardware for AI inferencing and training. It has on-car hardware running software called Autopilot, which is for assisted driving, and a beta version of Full Self-Driving (FSD), which is for hands-free autonomous driving.

Tesla mines its fleet for visual data gathered by eight cameras and sensors, which it then feeds into its training model. The current data center where models are trained includes more than 14,000 GPUs.

"To date, over 300 million miles have been driven using FSD beta. That 300-million-mile number is going to seem small very quickly. It’ll soon be billions of miles, then tens of billions of miles," Musk said.

Tesla also started production of its Dojo supercomputer, which is a separate AI training cluster that has racks of homegrown D1 chips. The D1 delivers 22.6 teraflops of FP32 performance, has 50 billion transistors, 10TBps of on-chip bandwidth, and 4TBps of off-chip bandwidth.

Elon Musk. (COMEO/Shutterstock)

"That’s what Dojo is designed to do – optimize for video training. It’s not optimized for LLMs. With video training, you have a much higher ratio of compute-to-memory bandwidth, whereas LLMs tend to be memory bandwidth," Musk said.

Tesla is quickly gathering data from FSD software in its cars to create a system that will make autonomous driving safe and reduce on-road deaths.

"In order to build autonomy, we also need to train our neural net with data from millions of vehicles … the more training data you have, the better the results," Musk said, adding "We see a clear path to full self-driving being 10 times safer than the average human driver"

But a lack of hardware is slowing down the video training. Musk specifically praised Nvidia CEO Jensen Huang, who was working to get Tesla more GPUs to help meet their computing need for computing speed.

"Frankly, I don’t know if they could deliver us enough GPUs. We might not need Dojo, but they can’t. So they’ve got so many customers. They have been kind enough to nonetheless prioritize some of our GPU orders," Musk said.

Musk is betting on heavy R&D and AI expenditure to give Tesla a competitive advantage over rival car makers, which are not putting as much effort into training autonomous systems from scratch. Tesla this week reported second-quarter revenue of $21.27 billion, growing by 46% compared to the same quarter the previous year.

The earnings were considered a disappointment as the operating income was 9.6%, a year-over-year decline from 14.6%. That was partially due to lower average selling prices of Tesla’s electric vehicles. Tesla offered deep discounts on the S, X and 3 models in the most recent operating quarter.

Related

The Transformer architecture, introduced in 2017, revamped natural language processing. Soon models like GPT, BERT, GPT-2, DistilBERT, BART, T5, and GPT-3 and now GPT-4 followed, each with unique capabilities and improvements. The models can be divided into three categories based on their design: auto-encoding models, autoregressive models, and sequence-to-sequence models.

Autoencoding Models

Trained by altering the input tokens and then reconstructing the initial sentence, auto-encoding models follow a similar pattern to the original transformer model’s encoder. They can access the complete inputs without any mask. These models create a two-way representation of the entire sentence and can be further refined for excellent performance in tasks like text generation.

However, their most suitable use is in sentence or token classification. Here are some of them.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

In 2018, Google unveiled BERT that introduces randomness into its input data during its pre-training phase. Typically, 15% of the tokens are masked using three different probabilities: a special mask token is used with a probability of 0.8, a random token other than the masked one is used with a probability of 0.1, and the same token is used with a probability of 0.1.

The model’s main task is to predict the original sentence from the masked input. Additionally, the model is given two sentences A and B with a separation token in between. There’s a 50% chance that these sentences are consecutive in the corpus, and a 50% chance that they are unrelated. The model’s secondary objective is to predict whether the sentences are consecutive or not.

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Google Research and Toyota Technological Institute at Chicago’s ALBERT is similar to BERT but with a few modifications like the embedding size (E) differs from the hidden size (H) because embeddings are context-independent (one token per vector), while hidden states are context-dependent (representing a sequence of tokens), making H >> E more logical. The large embedding matrix (V x E) results in more parameters when E < H.

Additionally, layers are grouped with shared parameters to save memory. Instead of next-sentence prediction, ALBERT employs sentence ordering prediction, where two consecutive sentences A and B are given as input, and the model predicts if they have been swapped or not.

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

Hugging Face created this smaller version of BERT through distillation, where it learns to predict probabilities like the larger version. It aims to achieve the same probabilities as the teacher model, correctly predict masked tokens (without next-sentence objective), and maintain similarity between hidden states of the teacher and student models.

RoBERTa: A Robustly Optimised BERT Pre-training Approach

Again similar to BERT, RoBERTa introduced enhanced pre-training techniques. One notable improvement is dynamic masking, where tokens are masked differently in each training epoch, unlike BERT’s fixed masking. Built by the Paul G Allen School of Computer Science & Engineering and the University of Washington, the model eliminates the NSP loss, instead combining chunks of continuous texts to form 512 tokens, possibly spanning multiple documents.

Moreover, larger batches are used during training, boosting efficiency. Lastly, BPE with bytes as subunits are employed to handle unicode characters more effectively.

XLM: Cross-lingual Language Model Pre-training

Built by Meta, XLM is another transformer-based model trained on multiple languages with three types of training: Causal Language Modelling (CLM), Masked Language Modelling (MLM), and a combination of MLM and Translation Language Modelling (TLM). CLM and MLM involve selecting a language for each training sample and processing 256-token sentences that may extend across multiple documents in that language.

TLM combines sentences in two different languages with random masking, allowing the model to use both contexts for predicting masked tokens. The model’s checkpoints are named based on the pre-training method used (clm, mlm, or mlm-tlm), and it incorporates language embeddings alongside positional embeddings to indicate the language used during training.

XLM-RoBERTa: Unsupervised Cross-lingual Representation Learning at Scale

XLM-RoBERTa combines RoBERTa techniques with XLM, excluding translation language modelling. Instead, it focuses on masked language modelling in sentences from a single language. Made by Meta, the model is trained on a vast array of languages (100) and possesses the ability to identify the input language without relying on language embeddings.

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

Stanford University and Google developed ELECTRA, a special transformer model that learns by utilising a smaller masked language model. This smaller model corrupts input text by randomly masking some parts, and ELECTRA’s task is to figure out which tokens are original and which are replacements.

Similar to GAN training, the smaller model is trained for a few steps with the original texts as the objective, not to deceive ELECTRA like in a traditional GAN. Afterward, ELECTRA is trained for a few steps to improve its performance.

Longformer: The Long-Document Transformer

Longformer by Allen Institute for AI is a faster transformer model than traditional ones because it uses sparse matrices instead of dense attention matrices. This allows it to focus on the nearby context for each token, using only the two tokens to its left and right. While some input tokens still receive global attention, the overall attention matrix has fewer parameters, making it more efficient. It is pre-trained similarly to RoBERTa.

We will soon come up with Part 2 with autoregressive and sequence-to-sequence models.

Read more: LLM Chatbots Don’t Know That We Know They Know We Know

The post Best Transformer-based LLMs on Hugging Face (Part 1) appeared first on Analytics India Magazine.

Image by Author

Superintelligence has the potential to be the most significant technological advancement in human history. It can help us tackle some of the most pressing challenges faced by humanity. While it can bring about a new era of progress, it also poses certain inherent risks that must be handled cautiously. Superintelligence can disempower humanity or even lead to human extinction if not appropriately handled or aligned correctly.

While superintelligence may seem far off, many experts believe it could become a reality in the next few years. To manage the potential risks, we must create new governing bodies and address the critical issue of superintelligence alignment. It means ensuring that artificial intelligence systems that will soon surpass human intelligence remain aligned with human goals and intentions.

In this blog, we will learn about Superalignmnet and learn about OpenAI’s approach to solving the core technical challenges of superintelligence alignment.

What is Superalignment

Superalignment refers to ensuring that super artificial intelligence (AI) systems, which surpass human intelligence in all domains, act according to human values and goals. It is an essential concept in the field of AI safety and governance, aiming to address the risks associated with developing and deploying highly advanced AI.

As AI systems get more intelligent, it may become more challenging for humans to understand how they make decisions. It can cause problems if the AI acts in ways that go against human values. It's essential to address this issue to prevent any harmful consequences.

Superalignment ensures that superintelligent AI systems act in ways that align with human values and intentions. It requires accurately specifying human preferences, designing AI systems that can understand them, and creating mechanisms to ensure the AI systems pursue these objectives.

Why do we need Superalignment

Superalignment plays a crucial role in addressing the potential risks associated with superintelligence. Let's delve into the reasons why we need Superalignment:

1. Mitigating Rogue AI Scenarios: Superalignment ensures that superintelligent AI systems align with human intent, reducing the risks of uncontrolled behavior and potential harm.
2. Safeguarding Human Values: By aligning AI systems with human values, Superalignment prevents conflicts where superintelligent AI may prioritize objectives incongruent with societal norms and principles.
3. Avoiding Unintended Consequences: Superalignment research identifies and mitigates unintended adverse outcomes that may arise from advanced AI systems, minimizing potential adverse effects.
4. Ensuring Human Autonomy: Superalignment focuses on designing AI systems as valuable tools that augment human capabilities, preserving our autonomy and preventing overreliance on AI decision-making.
5. Building a Beneficial AI Future: Superalignment research aims to create a future where superintelligent AI systems contribute positively to human well-being, addressing global challenges while minimizing risks.

OpenAI Approach

OpenAI is building a human-level automated alignment researcher that will use vast amounts of compute to scale the efforts, and iteratively align superintelligence — Introducing Superalignment (openai.com).

To align the first automated alignment researcher, OpenAI will need to:

• Develop a scalable training method: OpenAI can use AI systems to help evaluate other AI systems on difficult tasks that are hard for humans to assess.
• Validate the resulting model: OpenAI will automate search for problematic behavior and problematic internals.
• Adversarial testing: Test the AI system by purposely training models that are misaligned, and verify that the methods used can identify even the most severe misalignments in the pipeline.

Team

OpenAI is forming a team to tackle the challenge of superintelligence alignment. They will allocate 20% of their computing resources over the next four years. The team will be led by Ilya Sutskever and Jan Leike, and includes members from previous alignment teams and other departments within the company.

OpenAI is currently seeking exceptional researchers and engineers to contribute to its mission. The problem of aligning superintelligence is primarily related to machine learning. Experts in the field of machine learning, even if they are not currently working on alignment, will play a crucial role in finding a solution.

Goals

OpenAI has set a goal to address the technical challenges of superintelligence alignment within four years. Although this is an ambitious objective and success is not guaranteed, OpenAI remains optimistic that a focused and determined effort can lead to a solution for this problem.

To solve the problem, they must present convincing evidence and arguments to the machine learning and safety community. Having a high level of confidence in the proposed solutions is crucial. If the solutions are unreliable, the community can still use the findings to plan accordingly.

Conclusion

OpenAI's Superalignment initiative holds great promise in addressing the challenges of superintelligence alignment. With promising ideas emerging from preliminary experiments, the team has access to increasingly useful progress metrics and can leverage existing AI models to study these problems empirically.

It's important to note that the Superalignment team's efforts are complemented by OpenAI's ongoing work to improve the safety of current models, including the widely used ChatGPT. OpenAI remains committed to understanding and mitigating various risks associated with AI, such as misuse, economic disruption, disinformation, bias and discrimination, addiction, and overreliance.

OpenAI aims to pave the way for a safer and more beneficial AI future through dedicated research, collaboration, and a proactive approach.
Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in Technology Management and a bachelor's degree in Telecommunication Engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.

More On This Topic

• Data science certification — why it is important and where to get it?
• Important Statistics Data Scientists Need to Know
• The Most Important Tool for Data Engineers
• What are Vector Databases and Why Are They Important for LLMs?
• Saving and loading models in TensorFlow — why it is important and how to do…
• Is Domain Knowledge Important for Machine Learning?

The fervor surrounding artificial intelligence "isn't a Silicon Valley thing, it isn't even a U.S. thing, it's now all over the world — it's a global phenomenon," according to Andrew Feldman, co-founder and CEO of AI computing startup Cerebras Systems.

In that spirit, Cerebras on Thursday announced it has contracted to build what it calls "the world's largest supercomputer for AI," named Condor Galaxy, on behalf of its client, G42, a five-year-old investment firm based in Abu Dhabi, the United Arab Emirates.

Also: GPT-4 is getting significantly dumber over time, according to a study

The machine is focused on the "training" of neural networks, the part of machine learning when a neural networks settings, its "parameters," or, "weights," have to be tuned to a level where they are sufficient for the second stage, making predictions, known as the "inference" stage.

Condor Galaxy is the result, said Feldman, of months of collaboration between Cerebras and G42, and is the first major announcement of their strategic partnership.

The initial contract is worth more than a hundred million dollars to Cerebras, Feldman told ZDNET in an interview. That is going to expand ultimately by multiple times, to hundreds of millions of dollars in revenue, as Cerebras builds out Condor Galaxy in multiple stages.

Also: Ahead of AI, this other technology wave is sweeping in fast

Condor Galaxy is named for a cosmological system located 212 million light years from Earth. In its initial configuration, called CG-1, the machine is made up of 32 of Cerebras's special-purpose AI computers, the CS-2, whose chips, the "Wafer-Scale-Engine," or WSE, collectively hold a total of 27 million compute cores, 41 terabytes of memory, and 194 trillion bits per second of bandwidth. They are overseen by 36,352 of AMD's EPYC x86 server processors.

The machine runs at 2 exa-flops, meaning, it can process a billion billion floating-point operations per second.

The largeness is the latest instance of big-ness by Cerebras, founded in 2016 by seasoned semiconductor and networking entrepreneurs and innovators. The company stunned the world in 2019 with the unveiling of the WSE, the largest chip ever made, a chip taking up almost the entire surface of a 12-inch semiconductor wafer. It is the WSE-2, introduced in 2021, that powers the CS-2 machines.

Also: AI startup Cerebras celebrated for chip triumph where others tried and failed

The CS-2s in the CG-1 are supplemented by Cerebras's special-purpose "fabric" switch, the Swarm-X, and its dedicated memory hub, the Memory-X, which are used to cluster together the CS-2s.

The claim to be the largest supercomputer for AI is somewhat hyperbolic, as there is no general registry for size of AI computers. The common measure of supercomputers, the TOP500 list, maintained by Prometeus GmbH, is a list of conventional supercomputers used for so-called high-performance computing.

Those machines are not comparable, said Feldman, because they work with what's called 64-bit precision, where each operand, the value to be worked upon by the computer, is represented to the computer by sixty-four bits. The Cerebras system represents data in a simpler form called "FP-16," using only sixteen bits for each system.

In 64-bit precision-class machines, Frontier, a supercomputer at the U.S. Department of Energy's Oak Ridge National Laboratory, is the world's most powerful supercomputer, running at 1.19 exa-flops. But it cannot be directly compared to the CG-1 at 2 exa-flops, said Feldman.

Certainly, the sheer compute of CG-1 is unlike many computers on the planet one can think of. "Think of a single computer with more compute power than half a million Apple MacBooks working together to solve a single problem in real time," offered Feldman.

Also: This new technology could blow away GPT-4 and everything like it

The Condor Galaxy machine is not physically in Abu Dhabi, but rather installed at the facilities of Santa Clara, California-based Colovore, a hosting provider that competes in the market for cloud services with the likes of Equinix. Cerebras had previously announced in November a partnership with Colovore for a modular supercomputer named 'Andromeda' to speed up large language models.

As part of the multi-year partnership, Condor Galaxy will scale through version CG-9, said Feldman. Phase 2 of the partnership, expected by the fourth quarter of this year, will double the CG-1's footprint to 64 CS-2s, with a total of 54 million compute cores, 82 terabytes of memory, and 388 teraflops of bandwidth. That machine will double the throughput to 4 exa-flops of compute.

Putting it all together, in phase 4 of the partnership, to be delivered in the second half of 2024, Cerebras will string together what it calls a "constellation" of nine interconnected systems, each running at 4 exa-flops, for a total of 36 exa-flops of capacity, at sites around the world, to make what it calls "the largest interconnected AI Supercomputer in the world."

"This is the first of four exa-flop machines we're building for G42 in the U.S.," explained Feldman, "And then we're going to build six more around the world, for a total of nine interconnected, four-exa-flop machines producing 36 exa-flops."

Also: Microsoft announces Azure AI trio at Inspire 2023

The machine is the first time Cerebras is not only building a clustered computer system but also operating it for the customer. The partnership affords Cerebras multiple avenues to revenue as a result.

The partnership will scale to hundreds of millions of dollars in direct sales to G42 by Cerebras, said Feldman, as it moves through the various phases of the partnership.

"Not only is this contract larger than all other startups have sold, combined, over their lifetimes, but it's intended to grow not just past the hundred million [dollars] it's at now, but two or three times past that," he said, alluding to competing AI startups including Samba Nova Systems and Graphcore.

In addition, "Together, we resell excess capacity through our cloud," meaning, letting other customers of Cerebras rent capacity in CG-1 when it is not in use by G42. The partnership "gives our cloud a profoundly new scale, obviously," he said, so that "we now have an opportunity to pursue dedicated AI supercomputers as a service."

Also: AI and advanced applications are straining current technology infrastructures

That means whoever wants cloud AI compute capacity will be able to "jump on one of the biggest supercomputers in the world for a day, a week, a month if you want."

The ambitions for AI appear to be as big as the machine. "Over the next 60 days, we're gonna announce some very, very interesting models that were trained on CG-1," said Feldman.

G42 is a global conglomerate, Feldman notes, with about 22,000 employees, in twenty-five countries, and with nine operating companies under its umbrella. The company's G42 Cloud subsidiary operates the largest regional cloud in the Middle East.

"G42 and Cerebras' shared vision is that Condor Galaxy will be used to address society's most pressing challenges across healthcare, energy, climate action and more," said Talal Alkaissi, CEO of G42 Cloud, in prepared remarks.

Also: Nvidia sweeps AI benchmarks, but Intel brings meaningful competition

A joint venture between G42 and fellow Abu Dhabi investment firm Mubadala Investments. Co., M42, is one of the largest genomics sequencers in the world.

"They're sort-of pioneers in the use of AI and healthcare applications throughout Europe and the Middle East," noted Feldman of G42. The company has produced 300 AI publications over the past 3 years.

"They [G42] wanted someone who had experienced building very large AI supercomputers, and who had experience developing and implementing big AI models, and who had experience manipulating and managing very large data sets," said Feldman, "And those are all things we, we had, sort-of, really honed in the last nine months."

The CG-1 machines, Feldman emphasized, will be able to scale to larger and larger neural network models without incurring many times the additional amount of code needed.

"One of the key elements in of the technology is that it enables customers like G42, and their customers, to, sort-of, quickly gain benefit from our machines," said Feldman.

Also: AI will change software development in massive ways

In a slide presentation, he emphasized how a 1-billion-parameter neural network such as OpenAI's GPT, can be put on a single Nvidia GPU chip with 1,200 lines of code. But to scale the neural network to a 40-billion parameter model, which runs across 28,415 Nvidia GPUs, the amount of code required to be deployed balloons to almost 30,000 lines, said Feldman.

For a CS-2 system, however, a 100-billion-parameter model can be run with the same 1,200 lines of code.

"If you wanna put a 40-billion or a hundred-billion parameter, or a 500-billion parameter, model, you use the exact same 1,200 lines of code," explained Feldman. "That is really a core differentiator, is that you don't have to do this," write more code, he said.

For Feldman, the scale of the latest creation represents not just bigness per se, but an attempt to have qualitatively different results by scaling up from the largest chip to the largest clustered systems.

Also: MedPerf aims to speed medical AI while keeping data private

"You know, when we started the company, you think that you can help change the world by building cool computers," Feldman reflected. "And over the course of the last seven years, we built bigger and bigger and bigger computers, and some of the biggest.

"Now we're on a path to build, sort of, unimaginably big, and that's awesome, to walk through the data center and to see rack after rack of your gear humming."

Artificial Intelligence

The Telecom Regulatory Authority of India (TRAI) has recommended a structure for regulating AI through the lens of a risk-based framework in a 141 paged document. Furthermore, the Ministry of Electronics and Information Technology (MeiTY) has been proposed as the designated administrative ministry for overseeing AI in India. As part of the framework, there are also plans to establish an independent statutory authority known as the Artificial Intelligence and Data Authority of India (AIDAI).

AIDAI will spearhead the widespread integration of AI models while collaborating closely with government-established standard-setting bodies, such as the Telecom Engineering Centre. Its primary objective is to accredit various laboratories for testing and certification of AI products and services. Additionally, AIDAI will provide recommendations to further enhance the development and adoption of future AI advancements.

The TRAI news comes under the backdrop of authorities globally stepping up to regulate AI in their respective countries and organisations. Today, coordinated by the White House, OpenAI has released an eight-point document outlining its commitments towards building and deploying safe and secure AI models. Domestically, MeitY is trying to bring similar norms in the much anticipated Digital India Act (DIA).

Read more: White House and OpenAI Make Another Lackluster Commitment to Safety

India’s digital revolution requires new regulations because the current ones are outdated since the latest entrant large language models have taken over the internet and enterprises. The existing IT Act has limitations in recognising electronic records, transactions, and signatures. While technology has empowered users, it has also brought challenges like user harm, security concerns, misinformation, and unfair trade practices. These issues need to be addressed with updated and comprehensive regulations.

Alongside the DIA, the Indian government also intends to address these concerns by passing the Digital Personal Data Protection Bill which will establish a balance between individuals’ right to protect their personal data and lawful data processing. This approach draws comparisons to the EU’s GDPR and has been recommended by experts like Mara Squicciarini, who emphasised the need for a harmonised policy that can have a global impact.

Read more: Digital India Act Expectations & Concerns

The post TRAI Announces Statutory Body To Regulate AI, Digital India Act Awaited appeared first on Analytics India Magazine.

OpenAI recently introduced a new feature called ‘custom instructions’, aimed at providing users with enhanced control over ChatGPT’s responses. Along with that OpenAI also doubled the number of messages ChatGPT Plus customers can send to GPT-4 to 50 per 3 hours. But, it’s time they start focusing on bettering the ChatGPT Plugin store experience. It’s a hot mess.

Four months back, OpenAI made an exciting release by launching plugins, and a lot of folks, including us, touted it as an iOS App Store event.

Sadly, ChatGPT Plugin store is nowhere near App Store, and lacks ranking, sub categories, and basic organisation to define their specific use cases or verticals. At present, navigating the plugin store reveals just three broad categories – New, Top, and Popular.

Today, there are 500+ plugins available in the store, and god knows what they do. No one has the patience to go through one by one and check which one suits them the best.

On ChatGPT Plugin store users have expressed difficulty in finding the plugins they need due to the lack of a robust search and categorization system. As the number of plugins grows, finding specific functionalities becomes increasingly cumbersome. The absence of user-friendly filters or categories raises questions about the store’s organization and whether improvements are in the pipeline.

ChatGPT Plugins are nice AF.
but.. there is no structure. its a mess. they are not sorted by categories like an app store.
i want to use them effectively with precision. for different usecases.
somebody talk to sam altman or whoever is in charge now

— ECommerce Cowboy (@SeherAlex) July 13, 2023

Does OpenAI Even Care?

Plugins are powered by third party applications that are not controlled by OpenAI. There is a high chance that you might install a risky plugin putting your personal data in danger.

The availability of support and a feedback mechanism is essential for any marketplace. However, users have faced challenges when seeking help or providing feedback on plugins. The absence of a direct line of communication with developers or ChatGPT’s support team has left users feeling stranded when encountering issues.

Since Plugins are developed by third parties, OpenAI does not directly cross-check their value or usefulness. With a vast number of over 500 plugins available for ChatGPT, manual verification of each one’s utility would be an impractical and time-consuming task. As a result, users may have to rely on their own experiences and feedback from the community to identify the most valuable and effective plugins.

Cuts Developers’ Pockets

Looking from the enterprise developer’s perspective, we are trying to see if they have any benefit. Expedia, one of the first companies to put the ChatGPT plugin in store has a huge user base in the Apple Store. Also, they recently integrated this ChaGPT plugin into their app.

While creating plugins on ChatGPT via API is indeed free, developers often find it challenging to manage the token count. If you have a business website and wish to integrate the ChatGPT Plugin onto it, you should be aware that the cost will increase with the number of tokens entered by users.

The token counting is inherent to the nature of using language models like GPT-3.5, and it is independent of the platform where the plugin is integrated (whether it’s a website or a mobile app). Every interaction with the language model, whether it’s through a website, an app, or any other interface, consumes tokens, and developers will be billed based on the total number of tokens used.

Just so people know, API and plugins not included in the ChatGPT Plus billing license.

— Cooperative exploration (@fvd_explore) April 7, 2023

Only big companies with deep pockets can afford to pay such a huge amount to OpenAI. The question here arises if enterprises who are working on a small scale can afford to pay?

For a slightly complex use case, monthly inference cost for a GPT 4 API can be anywhere between $ 250,000 to $ 300,000 per month inference cost for a GPT-4 API (having 16K context length) for a complex use case as per AIM research.

Therefore, when using the ChatGPT API, it’s essential to keep track of the token usage and manage it effectively to control costs, just as you would with a website integration. Also the announcement to double the number of messages ChatGPT Plus customers can send and extending support for gpt-3.5-turbo-0301 and gpt-4-0314 models in the OpenAI API until at least June 13, 2024 serves the purpose of OpenAI to make money.

It has to be noted that the ChatGPT Plugin store is still in the beta stage, and there is definitely room for improvements. And ChatGPT plugins category update is the least they can do.

For more feedback on ChatGPT Plugins, click here.

The post ChatGPT Plugins Store is a Hot Mess appeared first on Analytics India Magazine.

Image by Author

The hype around large language models (LLMs) is continuing to grow, with more and more companies releasing tools to make people's life easier. So what exactly are these tools that are helping build the world of LLMs? Well, one of them is Flowise AI

What is Flowise AI?

Flowise AI is an open-source UI visual tool used to help develop LangChain apps. Before we get into more about Flowise AI, let’s quickly define LangChain. LangChain is a framework/python library that helps you make use of LLMs to build your own custom NLP applications.

Flowise uses LangChain as its agent executor, Chroma as its vector store, OpenAI for embeddings, HuggingFace’s inference model, GitHub as a document loader, and SERP for query API. Its Graphic User Interface is very helpful in constructing LLM-based apps built on LangChain.js.

So what makes it so easy and helpful? The drag-and-drop tool. Everybody loves a drag-and-drop, especially when you are customizing your NLP application. The even bigger plus is that it does not require any coding experience!

Image by Flowise AI What can I Build with Flowise AI?

You can build several apps with Flowise AI, such as:

• Chatbots
• Virtual assistants
• Data analysis tools
• Educational tools
• Games
• Art

Why Should I Use Flowise AI?

• Simplicity: The drag-and-drop tool makes it easy to construct your own LLM flows.
• No coding skills required: This is highly useful to new people in the industry and organizations that do not have developers on the team.
• Open source: Free to use and modify, allowing you to tailor it to your own requirements.
• Powerful: The tool can be used to develop a wide range of LLM applications.
• Community: Flowise is backed by a supportive development community, which can assist you in your process of making the most out of Flowise.

Flowise AI Installation

So how do I install this simple drag-and-drop customisable NLP tool? So there are 3 different ways you can install Flowise AI. Let’s go through all of them.

Quick Installation

1. First, you will need to download and install NodeJS >= 18.15.0.
2. Once this is done, you need to install Flowise

npm install -g flowise

1. Your next step is to start Flowise

npx flowise start

You will need to enter a username and password:

npx flowise start --FLOWISE_USERNAME=user --FLOWISE_PASSWORD=1234

1. Once that is done, you can open it up on a webpage by opening:

http://localhost:3000

If you prefer to use Docker, follow the next part.

Docker

Docker Compose

1. First, you need to go to docker folder at the root of the project
2. Then you need to create .env file and specify the PORT (refer to .env.example)
3. You will then need to execute: docker-compose up -d
4. Then you will need to open http://localhost:3000
5. You can bring the containers down by docker-compose stop

Docker Image

1. First, you will need to build the image locally:

docker build --no-cache -t flowise .

1. Then you will need to run the image:

docker run -d --name flowise -p 3000:3000 flowise

1. To stop the image, you need to:

docker stop flowise

Local Setup for Developers

1. First, you will need to Install Yarn v1 by:

npm i -g yarn

1. You will then need to clone the repository:

git clone https://github.com/FlowiseAI/Flowise.git

1. Go into the repository folder:

cd Flowise

1. Make sure to install all the dependencies of all models:

yarn install

1. You will then need to build all the code:

yarn build

1. You can then start the app:

yarn start

1. You can access the app on:

http://localhost:3000

1. For development build, use:

yarn dev

Wrapping it up

So if you’re somebody new to the tech industry and have no coding experience or an organization that lacks a developer on your team — Flowise AI is the best option for you. If there are any current or previous users of Flowise reading this, let us know about your experience in the comments!
Nisha Arya is a Data Scientist, Freelance Technical Writer and Community Manager at KDnuggets. She is particularly interested in providing Data Science career advice or tutorials and theory based knowledge around Data Science. She also wishes to explore the different ways Artificial Intelligence is/can benefit the longevity of human life. A keen learner, seeking to broaden her tech knowledge and writing skills, whilst helping guide others.

More On This Topic

• Data Profession Job Satisfaction: Beware Of The Drop
• Building a Scalable ETL with SQL + Python
• Building Tech Skills in 2021
• Building RESTful APIs using Flask
• Dream Come True: Building websites by thinking about them
• Building a Knowledge Graph for Job Search Using BERT