Google spends $1M to Rust Foundation to build Rust C++ Interoperability

Google has announced a $1M donation to the Rust Foundation to improve the interoperability between the Rust and C++ programming languages. This financial boost supports the launch of the Interop Initiative, a project dedicated to smoothly integrating Rust into C++ dominated environments.

Google uses programming language Rust in Android for Bluetooth and Keystore 2.0 modules. In ChromeOS for the cross vm virtual machine monitor and drivers to run other operating systems in a secure, isolated environment.

The company also actively contributes to open-source projects that use Rust, including Mercurial, a source code management tool, demonstrating its support for the Rust programming community. It also employs Rust to develop firmware for FIDO (Fast Identity Online) security keys, which are physical devices used for secure authentication, enhancing the security of the authentication process.

Its large-scale use of Rust prompted it to join the foundation in 2021. “Based on historical vulnerability density statistics, Rust has proactively prevented hundreds of vulnerabilities from impacting the Android ecosystem,” said Dave Kleidermacher, Google Vice President of Engineering, Android Security & Privacy.

This funding comes to help companies transition from C++ to Rust. This will mean rewriting millions of lines of C++ code. “It would be unrealistic to expect even the most technically advanced organisation to easily pivot to Rust and away from the architecture of existing codebases,” said Rust Foundation Executive Director & CEO, Dr. Rebecca Rumbul.

The initiative’s first steps involve drafting a detailed proposal to outline the scope of work, involving discussions with the Rust Project Leadership Council, stakeholders, and member organisations. The focus will be on expanding existing interoperability efforts, possibly integrating build systems, and exploring the use of AI for code conversion from C++ to Rust.

Three months ago, David Weston, Vice President, Enterprise and OS Security at Microsoft announced a funding of $10 million to the Rust foundation in a tweet. This, however has not been officially confirmed by Microsoft

The post Google spends $1M to Rust Foundation to build Rust C++ Interoperability appeared first on Analytics India Magazine.

JSON Inventor Explains Why English Isn’t The Hottest Programming Language

The ‘JSON’ creator, Douglas Crockford, recalled a joke from his 2018 book ‘How Javascript Works’ in an interview with AIM: Two men are trying to drive a stake into the ground. One of them has a hammer, and the other says, ‘when I nod my head, hit it with a hammer’.

“Those sorts of misunderstandings happen in English all the time. If they’re going into machines, which will affect the lives of people, we can’t afford that level of comedy,” he said while commenting on the trendy phrase, ‘The hottest new programming language is English’.

Crockford, one of the developers of JavaScript and the brain behind the JSON format, is worried about English as a programming language “because it’s so ambiguous”. The Roman language has been largely advocated for programming through natural language via AI chatbots.

Crockford disagrees.

He clarified that the fundamental law of programming is that the program has to be perfect. “It has to be perfect in every aspect, in every detail, for all states for all time,” the 70-year-old retired programmer said. He further elaborated that if it isn’t, the computer is licensed to do the worst possible thing at the worst possible time. “It’s not the computer’s fault, it is the programmers’ fault,” he noted.

Over the history of programming languages, we’ve been trying to develop ways of expressing algorithms and procedures, which are less likely to be misunderstood or do bad things. English is an awful language for trying to do that because it’s not specific enough, the JavaScript guru emphasised.

Historically, English has been criticised particularly for its complex spelling system. The stupid and oft-misunderstood language can be used to communicate among humans, but the scenario is not the same for machines.

Crockford explained that when we’re “trying to communicate with computers, there is no room for understanding” because the AI does not grasp the context of conversations. “We have to be exactly perfect in our expression and natural languages are not good at that,” he said in response to why he was sceptical of using English for programming.

JavaScripting AI

As a core part of the web, the dominant mode of accessing the internet, JavaScript is arguably the most essential language in the world, said Crockford. Even though it is huge, the three-decade-old language had nothing to do with AI. The veteran programmer believes that the success of JavaScript was completely independent of AI and vice versa.

However, with generative AI entering the scene, things can change. Crockford pointed out, “We’re still in the novelty phase. People are interacting with it for the first time.” But, companies have already claimed a list of reasons why and how generative AI will revolutionise programming skills.

In the 90s, there was a technique of programming called pair programming where you’d have two programmers working together, Crockford recollected. “That seemed very effective because when you check (others’ work), you tend to get a very high quality at the cost of doubling your personnel,” he said.

Crockford does consider that (maybe) AI can replace one of those programmers in the pair and so you can get a similar level of quality at the cost of a single programmer. “That’s not been proven yet, but it seems likely that something like that is going to work,” he pictures.

Tech companies have stirred out several AI-powered platforms and people have been using them – even the ones who point out its problems. Be it Amazon’s CodeWhisperer or GitHub’s Copilot, these tools built on generative AI models are here to stay. However, their usefulness remains theoretical.

The Open Source Act

Generative AI has put open source back on the list of important subjects of discussion for everyone, from unicorns to lawmakers. The root problem is the patent system, Crockford highlighted. “If the patent system worked correctly with respect to software, open source would not be necessary,” he simplified.

The author of the ‘Good, Not Evil’ licence confidently believes that the idea of owning an algorithm should not be possible, but under current law, in the US and many other countries, it is. “Patents can be awarded after people become aware of inventions,” Crockford suggested.

“It is dangerous, so we’ve come up with open source licences. The not-so-effective patent system is an attempt to mitigate those problems,” he explained. Crockford firmly believes “the system is still lacking and should be reformed”.

In conclusion, he said: “We should assume that anything that can be represented in software, can obviously be done through software. It should be possible to patent something that uses software, but the software component itself is not patentable. If we had that reform, then a lot of these problems would disappear.”

The post JSON Inventor Explains Why English Isn’t The Hottest Programming Language appeared first on Analytics India Magazine.

I just tried Google’s ImageFX AI image generator, and I’m shocked at how good it is

ImageFX photo of technology

ImageFX-generated photo with the prompt: An abstract photo that represents technology.

ZDNET's key takeaways

  • ImageFX, from Google, is free to use and is easily accessible today via its website or Google Labs.
  • It's an easy-to-use image generator that produces impressive, high-quality results.
  • However, if you rely on an image generator for your everyday workflow, there are better options for you.

Google was late to the image generator game, releasing its first AI image generator, ImageFX, to the public last week, over a year after DALL-E 3 was released. The burning question is: Was the wait worth it? I am happy (and surprised) to report it was.

Accessing the free tool is easy; all you have to do is visit Google Labs and select ImageFX or visit the ImageFX page directly. Then, you sign into your personal Google account and you are set to start tinkering.

Also: I'm taking AI image courses for free on Udemy with this little trick — and you can too

You can generate an image as you would with any other text-to-image generator, typing in what you'd like to see generated with as much or as little detail as you please.

When the chatbot was released, Google claimed that it could tackle complex prompts realistically, such as generating hands and faces. To put that to the test, I used the prompt, "two hands typing on a silver keyboard with a laptop in the background." The results did not disappoint.

ImageFX quickly generated four images of hands on a keyboard that looked realistic, with each hand having five proportionate fingers, a big feat for a generative AI model. The photos were of such high quality and looked like stock photos one would find in Getty Images. I included a close-up photo below so you can see the detail.

Revisiting the image at a later date is easy, with a download and share button found underneath the photo you generated.

ImageFX also includes a unique feature, expressive chips, which Google describes as allowing users to "quickly experiment with adjacent dimensions of your creation and ideas." When reading the description, I was initially confused about it, but it is much simpler than it sounds.

Also: 5 reasons to sign up for Google Labs and how to do it

All expressive chips do is highlight parts of your prompts and add drop-downs that you can select from to tweak your generated image slightly. Once you click the drop-down and select a new option, the new image generates with that individual tweak.

For example, I selected the dropdown for "silver keyboard," and changed it to the "red keyboard" option, and hit generate. It generated similar images, with the only major difference being the keyboard color.

The expressive chips and the style suggestions underneath the text box make ImageFX ideal if you are beginning to experiment with AI image generation or if you want to tinker with one for fun.

It is also worth noting that all images generated with Imagen 2, Google's most advanced text-to-image model, including the ones generated by ImageFX, will be watermarked with SynthID, a tool developed by Google DeepMind that watermarks photos in a way that is imperceptible to the human eye but can be used for identification.

Also: How to use Leonardo AI to generate stunning artwork and images

This can give you peace of mind that an image that you generate or others generate — no matter how realistic they look — will have a designation that they were AI-generated, helping combat misinformation.

ZDNET's advice

Google's ImageFX AI image generator is free and easy to use, and for that reason, I would encourage everyone to give it a try. The generator is capable of producing high-quality and realistic images, so it could suit many different purposes, such as generating images for your flyer, website, or just for fun.

However, if you do have a specific use case, I would recommend looking into what AI image generator could best suit your specific needs. For example, if you plan to use an image generator for your business, it would make sense to explore options that generate commercially safe photos with proper licensing, such as Generative AI by Getty Images. For a full breakdown of the best AI image generators, you can visit ZDNET's picks.

Disclaimer: You should consider the legal consequences (e.g. copyright) of using AI-generated images before implementing them into your work.

Artificial Intelligence

AI meets AR as ChatGPT is now available on the Apple Vision Pro

Apple Vision Pro in hand

One of the biggest names in artificial intelligence is joining one of the biggest names in mixed reality, as OpenAI has released a VisionOS ChatGPT app for the Apple Vision Pro.

In a post announcing the app on X, OpenAI showed a video of a user asking ChatGPT browser to "Explain superconductors like I'm five years old" and receiving a simplified answer.

Also: I've tried Vision Pro and other top XR headsets and here's the one most people should buy

At first, this might seem a bit unnecessary: Bringing an app that doesn't have any 3D aspect at all and is largely text-based to a platform where the entire focus is 3D and augmented reality. Like mobile versions of the app, the Vision Pro version does allow users to speak their questions and replies, but that's about as interactive as it gets.

While ChatGPT allows users to generate images, produce unique content, get advice, and solve problems, it doesn't have any applications that would be unique to the world of the Vision Pro — yet, at least.

But it's intriguing to think about what this means for the future of AI and AR together. Aside from simply speaking to AI and having a conversation, imagine sending an image straight from your field of vision. You might glance at ingredients in your pantry and ask AI to make a recipe using those ingredients or ask it to explain a difficult problem on your child's homework — think of a beefed-up version of Google Lens.

Also: Why you'll need a VPN for the Vision Pro (and other XR headsets)

There may not be a lot of usage for ChatGPT that's specific to the Vision Pro for now, but with companies like Google already rolling out AI generative video, it's not hard to picture generative immersive video being far behind.

The ChatGPT app is now available from the VisionOS App Store.

Artificial Intelligence

OpenVoice: Versatile Instant Voice Cloning

In Text-to-Speech synthesis (TTS), Instant Voice Cloning (IVC) enables the TTS model to clone the voice of any reference speaker using a short audio sample, without requiring additional training for the reference speaker. This technique is also known as Zero-Shot Text-to-Speech Synthesis. The Instant Voice Cloning approach allows for flexible customization of the generated voice and demonstrates significant value across a wide range of real-world situations, including customized chatbots, content creation, and interactions between humans and Large Language Models (LLMs).

Although the current voice cloning frameworks do their job well, they are riddled with a few challenges in the field including Flexible Voice Style Control i.e models lack the ability to manipulate voice styles flexibly after cloning the voice. Another major roadblock encountered by current instant cloning frameworks is Zero-Shot Cross-Lingual Voice Cloning i.e for training purposes, current models require access to an extensive massive-speaker multi-lingual or MSML dataset irrespective of the language.

To tackle these issues, and contribute in the enhancement of instant voice cloning models, developers have worked on OpenVoice, a versatile instant voice cloning framework that replicates the voice of any user and generates speech in multiple languages using a short audio clip from the reference speaker. OpenVoice demonstrates Instant Voice Cloning models can replicate the tone color of the reference speaker, and achieve granular control over voice styles including accent, rhythm, intonation, pauses, and even emotions. What’s more impressive is that the OpenVoice framework also demonstrates remarkable capabilities in achieving zero-shot cross-lingual voice cloning for languages external to the MSML dataset, allowing OpenVoice to clone voices into new languages without extensive pre-training for that language. OpenVoice manages to deliver superior instant voice cloning results while being computationally viable with operating costs up to 10 times less that current available APIs with inferior performance.

In this article, we will talk about the OpenVoice framework in depth, and we will uncover its architecture that allows it to deliver superior performance across instant voice cloning tasks. So let’s get started.

OpenVoice : Enabling Versatile Instant Voice Cloning

As mentioned earlier, Instant Voice Cloning, also referred to as Zero-Shot Text to Speech Synthesis, allows the TTS model to clone the voice of any reference speaker using a short audio sample without the need of any additional training for the reference speaker. Instant Voice Cloning has always been a hot research topic with existing works including XTTS and VALLE frameworks that extract speaker embedding and/or acoustic tokens from the reference audio that serves as a condition for the auto-regressive model. The auto-regressive model then generates acoustic tokens sequentially, and then decodes these tokens into a raw audio waveform.

Although auto-regressive instant voice cloning models clone the tone color remarkably, they fall short in manipulating other style parameters including accent, emotion, pauses, and rhythm. Furthermore, auto-regressive models also experience low inference speed, and their operational costs are quite high. Existing approaches like YourTTS framework employ a non-autoregressive approach that demonstrates significantly faster inference speech over autoregressive approach frameworks, but are still unable to provide their users with flexible control over style parameters. Moreover, both autoregressive-based and non-autoregressive based instant voice cloning frameworks need access to a large MSML or massive-speaker multilingual dataset for cross-lingual voice cloning.

To tackle the challenges faced by current instant voice cloning frameworks, developers have worked on OpenVoice, an open source instant voice cloning library that aims to resolve the following challenges faced by current IVC frameworks.

  1. The first challenge is to enable IVC frameworks to have flexible control over style parameters in addition to tone color including accent, rhythm, intonation, and pauses. Style parameters are crucial to generate in-context natural conversations and speech rather than narrating the input text monotonously.
  2. The second challenge is to enable IVC frameworks to clone cross-lingual voices in a zero-shot setting.
  3. The final challenge is to achieve high real-time inference speeds without deteriorating the quality.

To tackle the first two hurdles, the architecture of the OpenVoice framework is designed in a way to decouple components in the voice to the best of its abilities. Furthermore, OpenVoice generates tone color, language, and other voice features independently, enabling the framework to flexibly manipulate individual language types and voice styles. The OpenVoice framework tackles the third challenge by default as the decoupled structure reduces computational complexity and model size requirements.

OpenVoice : Methodology and Architecture

The technical framework of the OpenVoice framework is effective and surprisingly simple to implement. It is no secret that cloning the tone color for any speaker, adding new language, and enabling flexible control over voice parameters simultaneously can be challenging. It is so because executing these three tasks simultaneously requires the controlled parameters to intersect using a large chunk of combinatorial datasets. Furthermore, in regular single speaker text to speech synthesis, for tasks that do not require voice cloning, it is easier to add control over other style parameters. Building on these, the OpenVoice framework aims to decouple the Instant Voice Cloning tasks into subtasks. The model proposes to use a base speaker Text to Speech model to control the language and style parameters, and employs a tone color converter to include the reference tone color into the voice generated. The following figure demonstrates the architecture of the framework.

At its core, the OpenVoice framework employs two components: a tone color converter, and a base speaker text to speech or TTS model. The base speaker text to speech model is either a single-speaker or a multi-speaker model allowing precise control over style parameters, language, and accent. The model generates a voice that is then passed on to the tone color converter, that changes the base speaker tone color to the tone color of the reference speaker.

The OpenVoice framework offers a lot of flexibility when it comes to the base speaker text to speech model since it can employ the VITS model with slight modification allowing it to accept language and style embeddings in its duration predictor and text encoder. The framework can also employ models like Microsoft TTS that are commercially cheap or it can deploy models like InstructTTS that are capable of accepting style prompts. For the time being, the OpenVoice framework employs the VITS model although the other models are also a feasible option.

Coming to the second component, the Tone Color Converter is an encoder-decoder component housing an invertible normalizing flow in the center. The encoder component in the tone color converter is a one-dimensional CNN that accepts the short-time fourier transformed spectrum of the base speaker text to speech model as its input. The encoder then generates feature maps as output. The tone color extractor is a simple two-dimensional CNN that operates on the mel-spectrogram of the input voice, and generates a single feature vector as the output that encodes the information of the tone color. The normalizing flow layers accept the feature maps generated by the encoder as the input and generate a feature representation that preserves all style properties but eliminates the tone color information. The OpenVoice framework then applies the normalizing flow layers in the inverse direction, and takes the feature representations as the input and outputs the normalizing flow layers. The framework then decodes the normalizing flow layers into raw waveforms using a stack of transposed one-dimensional convolutions.

The entire architecture of the OpenVoice framework is feed forward without the use of any auto-regressive component. The tone color converter component is similar to voice conversion on a conceptual level but differs in terms of functionality, training objectives, and an inductive bias in the model structure. The normalizing flow layers share the same structure as flow-based text to speech models but differ in terms of functionality and training objectives.

Furthermore, there exists a different approach to extract feature representations, the method implemented by the OpenVoice framework delivers better audio quality. It is also worth noting that the OpenVoice framework has no intention of inventing components in the model architecture, rather both the main components i.e. the tone color converter and the base speaker TTS model are both sourced from existing works. The primary aim of the OpenVoice framework is to form a decoupled framework that separates the language control and the voice style from the tone color cloning. Although the approach is quite simple, it is quite effective especially on tasks that control styles and accents, or new language generalization tasks. Achieving the same control when employing a coupled framework requires a large amount of computing and data, and it does not generalize well to new languages.

At its core, the main philosophy of the OpenVoice framework is to decouple the generation of language and voice styles from the generation of tone color. One of the major strengths of the OpenVoice framework is that the clone voice is fluent and of high quality as long as the single-speaker TTS speaks fluently.

OpenVoice : Experiment and Results

Evaluating voice cloning tasks is a hard objective due to numerous reasons. For starters, existing works often employ different training and test data that makes comparing these works intrinsically unfair. Although crowd-sourcing can be used to evaluate metrics like Mean Opinion Score, the difficulty and diversity of the test data will influence the overall outcome significantly. Second, different voice cloning methods have different training data, and the diversity and scale of this data influences the results significantly. Finally, the primary objective of existing works often differs from one another, hence they differ in their functionality.

Due to the three reasons mentioned above, it is unfair to compare existing voice cloning frameworks numerically. Instead, it makes much more sense to compare these methods qualitatively.

Accurate Tone Color Cloning

To analyze its performance, developers build a test set with anonymous individuals, game characters and celebrities form the reference speaker base, and has a wide voice distribution including both neutral samples and unique expressive voices. The OpenVoice framework is able to clone the reference tone color and generate speech in multiple languages and accents for any of the reference speakers and the 4 base speakers.

Flexible Control on Voice Styles

One of the objectives of the OpenVoice framework is to control the speech styles flexibly using the tone color converter that can modify the color tone while preserving all other voice features and properties.

Experiments indicate that the model preserves the voice styles after converting to the reference tone color. In some cases however, the model neutralizes the emotions slightly, a problem that can be resolved by passing less information to the flow layers so that they are unable to get rid of the emotion. The OpenVoice framework is able to preserve the styles from the base voice thanks to its use of a tone color converter. It allows the OpenVoice framework to manipulate the base speaker text to speech model to easily control the voice styles.

Cross-Lingual Voice Clone

The OpenVoice framework does not include any massive-speaker data for an unseen language, yet it is able to achieve near cross-lingual voice cloning in a zero-shot setting. The cross-lingual voice cloning capabilities of the OpenVoice framework are two folds:

  1. The model is able to clone the tone color of the reference speaker accurately when the language of the reference speaker goes unseen in the multi-speaker multi language or MSML dataset.
  2. Furthermore, in the same event of the language of the reference speaker goes unseen, the OpenVoice framework is capable of cloning the voice of the reference speaker, and speak in the language one the condition that the base speaker text to speech model supports the language.

Final Thoughts

In this article we have talked about OpenVoice, a versatile instant voice cloning framework that replicates the voice of any user and generates speech in multiple languages using a short audio clip from the reference speaker. The primary intuition behind OpenVoice is that as long as a model does not have to perform tone color cloning of the reference speaker, a framework can employ a base speaker TTS model to control the language and the voice styles.

OpenVoice demonstrates Instant Voice Cloning models can replicate the tone color of the reference speaker, and achieve granular control over voice styles including accent, rhythm, intonation, pauses, and even emotions. OpenVoice manages to deliver superior instant voice cloning results while being computationally viable with operating costs up to 10 times less that current available APIs with inferior performance.

Google reportedly rebranding Bard to Gemini, adding ‘Advanced’ subscription service

Google Bard

Google Bard disappointed users upon its launch with its underwhelming performance. However, Google has made several AI advancements since then, including releasing its most advanced LLM, Gemini, and rolling out an impressive image generator released last week. To follow on those upgrades, it looks like Bard is getting a rebrand.

Also: The best AI chatbots

On Saturday, Android developer Dylan Roussel spotted a changelog for Bard, dated February 7, 2024, that revealed Bard's rebrand to Gemini, as well as several other updates coming to the AI chatbot.

In the screenshots, Google provides an explanation for the name change claiming, "We're committed to giving everyone direct access to Google AI and, as of this week, every Gemini user across our supported countries and languages has access to Google's best family of model."

"To better reflect this commitment, we've renamed Bard to Gemini," Google added.

Bard will also be introducing a new subscription tier — Gemini Advanced. Similar to ChatGPT Plus or Copilot Pro, with the subscription upgrade, users will have access to Google's latest AI advancements first. This includes its most capable large language model, Ultra 1.0.

Also: Microsoft Copilot vs. Copilot Pro: Is the subscription fee worth it?

According to the screenshot, Ultra 1.0 will make Gemini Advanced more capable of complex tasks such as coding, logical reasoning, and more. In the upcoming months, it will get even more advanced features such as expanded multi-modal capabilities. The cost of the subscription was not detailed in the screenshot.

The updates will also come with a new Gemini app, meant to help users access AI assistance on the go. It will be integrated with Google apps, like Gmail and Maps, to assist with tasks throughout Google's different productivity apps.

The app will be made available in the Google Play Store only. However, if iOS users want to experience the Gemini app on mobile, they can do so via the Google app. Google shared in the screenshot that the app is a step towards its vision to "deliver the world's most helpful assistant."

Also: How to use Copilot (formerly called Bing Chat)

This rebrand follows Microsoft's rebranding of Bing Chat to Copilot in September to streamline its AI assistant's names. Like Google, Microsoft's goal is also to become the go-to AI assistant for nearly every task you can imagine.

"We are the Copilot company," said Microsoft CEO Satya Nadella. "We believe in a future where there will be a Copilot for everyone and everything you do."

We are seeing both Microsoft and Google pivot from being a standalone AI chatbot like ChatGPT. Instead, both companies are focusing on creating AI assistants that can be infused across all of users' different applications natively, without having to rely on visiting a chatbot's site to get assistance with the task you need.

Artificial Intelligence

Advancing AI Alignment with Human Values Through WARM

Weight Averaged Reward Models LLM

Alignment of AI Systems with Human Values

Artificial intelligence (AI) systems are becoming increasingly capable of assisting humans in complex tasks, from customer service chatbots to medical diagnosis algorithms. However, as these AI systems take on more responsibilities, it is crucial that they remain aligned with human values and preferences. One approach to achieve this is through a technique called reinforcement learning from human feedback (RLHF). In RLHF, an AI system, known as the policy, is rewarded or penalized based on human judgments of its behavior. The goal is for the policy to learn to maximize its rewards, and thus behave according to human preferences.

A core component of RLHF is the reward model (RM). The RM is responsible for evaluating the policy's actions and outputs, and returning a reward signal to guide the learning process. Designing a good RM is challenging, as human preferences can be complex, context-dependent, and even inconsistent across individuals. Recently, researchers from Google DeepMind proposed an innovative technique called Weight Averaged Reward Models (WARM) to improve RM design.

The Trouble with Reward Hacking

A major problem in RLHF is reward hacking. Reward hacking occurs when the policy finds loopholes to game the RM system to obtain high rewards without actually satisfying the intended objectives. For example, suppose the goal is to train a writing assistant AI to generate high-quality summaries. The RM might reward concise and informative summaries. The policy could then learn to exploit this by generating very short, uninformative summaries peppered with keywords that trick the RM.

Reward hacking happens for two main reasons:

  1. Distribution shift – The RM is trained on a limited dataset of human-labeled examples. When deployed, the policy's outputs may come from different distributions that the RM does not generalize well to.
  2. Noisy labels – Human labeling is imperfect, with inter-rater disagreements. The RM may latch onto spurious signals rather than robust indicators of quality.

Reward hacking leads to useless systems that fail to match human expectations. Worse still, it can result in AI behaviors that are biased or even dangerous if deployed carelessly.

The Rise of Model Merging

The surging interest in model merging strategies like Model Ratatouille is driven by the realization that bigger models, while powerful, can be inefficient and impractical. Training a 1 trillion parameter model requires prohibitive amounts of data, compute, time and cost. More crucially, such models tend to overfit to the training distribution, hampering their ability to generalize to diverse real-world scenarios.

Model merging provides an alternate route to unlock greater capabilities without uncontrolled scaling up. By reusing multiple specialized models trained on different distributions, tasks or objectives, model merging aims to enhance versatility and out-of-distribution robustness. The premise is that different models capture distinct predictive patterns that can complement each other when merged.

Recent results illustrate the promise of this concept. Models obtained via merging, despite having far fewer parameters, can match or even exceed the performance of giant models like GPT-3. For instance, a Model Ratatouille ensemble of just 7 mid-sized checkpoints attains state-of-the-art accuracy on high-dimensional textual entailment datasets, outperforming GPT-3.

The simplicity of merging by weight averaging is a huge bonus. Training multiple auxiliary models does demand extra resources. But crucially, the inference-time computation remains identical to a single model, since weights are condensed into one. This makes the method easily adaptable, without concerns of increased latency or memory costs.

Mechanisms Behind Model Merging

But what exactly enables these accuracy gains from merging models? Recent analysis offers some clues:

  • Mitigating Memorization: Each model sees different shuffled batches of the dataset during training. Averaging diminishes any instance-specific memorization, retaining only dataset-level generalizations.
  • Reducing Variance: Models trained independently have uncorrelated errors. Combining them averages out noise, improving calibration.
  • Regularization via Diversity: Varying auxiliary tasks force models to latch onto more generalizable features useful across distributions.
  • Increasing Robustness: Inconsistency in predictions signals uncertainty. Averaging moderates outlier judgments, enhancing reliability.

In essence, model merging counterbalances weaknesses of individual models to amplify their collective strengths. The merged representation captures the common underlying causal structures, ignoring incidental variations.

This conceptual foundation connects model merging to other popular techniques like ensembling and multi-task learning. All these methods leverage diversity across models or tasks to obtain versatile, uncertainty-aware systems. The simplicity and efficiency of weight averaging, however, gives model merging a unique edge for advancing real-world deployments.

Weight Averaged Reward Models

Alignment process with WARM

Alignment process with WARM

WARM innovatively employs a proxy reward model (RM), which is a weight average of multiple individual RMs, each fine-tuned from the same pre-trained LLM but with varying hyperparameters. This method enhances efficiency, reliability under distribution shifts, and robustness against inconsistent preferences. The study also shows that using WARM as the proxy RM, particularly with an increased number of averaged RMs, improves results and delays the onset of ‘reward hacking', a phenomenon where control rewards deteriorate over time.

Here's a high-level overview:

  1. Start with a base language model pretrained on a large corpus. Initialize multiple RMs by adding small task-specific layers on top.
  2. Fine-tune each RM separately on the human preference dataset, using different hyperparameters like learning rate for diversity.
  3. Average the weights of the finetuned RMs to obtain a single WARM ensemble.

The key insight is that weight averaging retains only the invariant information that is learned across all the diverse RMs. This reduces reliance on spurious signals, enhancing robustness. The ensemble also benefits from variance reduction, improving reliability despite distribution shifts.

As discussed previously, diversity across independently trained models is crucial for unlocking the full potential of model merging. But what are some concrete techniques to promote productive diversity?

The WARM paper explores a few clever ideas that could generalize more broadly:

Ordering Shuffles

A trivial but impactful approach is shuffling the order in which data points are seen by each model during training. Even this simple step de-correlates weights, reducing redundant memorization of patterns.

Hyperparameter Variations

Tweaking hyperparameters like learning rate and dropout probability for each run introduces useful diversity. Models converge differently, capturing distinct properties of the dataset.

Checkpoint Averaging – Baklava

The Baklava method initializes models for merging from different snapshots along the same pretraining trajectory. This relaxes constraints compared to model soups which mandate a shared start point. Relative to model ratatouille, Baklava avoids additional tasks. Overall, it strikes an effective accuracy-diversity balance.

fine-tuning multiple Reward Models

The process begins with a pre-trained Large Language Model (LLM) 𝜃_𝑝𝑡. From this model, various checkpoints {𝜃_𝑠 𝑓 𝑡_𝑖} are derived during a Supervised Fine-Tuning (SFT) run, each collected at different SFT training steps. These checkpoints are then used as initializations for fine-tuning multiple Reward Models (RMs) {𝜙𝑖} on a preference dataset. This fine-tuning aims to adapt the models to align better with human preferences. After fine-tuning, these RMs are combined through a process of weight averaging, resulting in the final model, 𝜙_WARM.

Analysis confirms that adding older checkpoints by moving average harms individiual performance, compromising diversity merits. Averaging only the final representations from each run performs better. In general, balancing diversity goals with accuracy maintenance remains an open research challenge.

Overall, model merging aligns well with the general ethos in the field to recycle existing resources effectively for enhanced reliability, efficiency and versatility. The simplicity of weight averaging solidifies its position as a leading candidate for assembling robust models from readily available building blocks.

Unlike traditional ensembling methods that average predictions, WARM keeps computational overhead minimal by maintaining just a single set of weights. Experiments on text summarization tasks demonstrate WARM's effectiveness:

  • For best-of-N sampling, WARM attain 92.5% win rate against random selection according to human preference labels.
  • In RLHF, a WARM policy reaches 79.4% win rate against a policy trained with a single RM after same number of steps.
  • WARM continues to perform well even when a quarter of the human labels are corrupted.

These results illustrate WARM's potential as a practical technique for developing real-world AI assistants that behave reliably. By smoothing out inconsistencies in human feedback, WARM policies can remain robustly aligned with human values even as they continue learning from new experiences.

The Bigger Picture

WARM sits at the intersection of two key trends in AI alignment research. First is the study of out-of-distribution (OOD) generalization, which aims to enhance model performance on new data that differs from the training distribution. Second is research on algorithmic robustness, focusing on reliability despite small input perturbations or noise.

By drawing connections between these fields around the notion of learned invariances, WARM moves us toward more rigorously grounded techniques for value alignment. The insights from WARM could generalize even beyond RLHF, providing lessons for wider machine learning systems that interact with the open world.

Of course, reward modeling is just one piece of the alignment puzzle. We still need progress on other challenges like reward specification, scalable oversight, and safe exploration. Combined with complementary techniques, WARM could accelerate the development of AI that sustainably promotes human prosperity. By collectively elucidating the principles that underlie robust alignment, researchers are charting the route to beneficial, ethical AI.

Birlasoft Launches Generative AI Platform Cogito

Birlasoft, the IT division of CK Birla Group, has announced a comprehensive Generative AI platform called Cogito. The AI platform streamlines and expedites the development and implementation of generative AI solutions for enterprises through pre-built tools, libraries, and frameworks.

It offers robust support through comprehensive trainings, educational materials, tutorials, and documentation. Additionally, ongoing technical support and regular updates keep enterprises aligned with the latest developments and best practices in the Generative AI domain.

Cogito seamlessly enables holistic business transformation by integrating innovative solutions, processes, and ideas. According to the company, organisations can devise roadmaps for operational improvement, adapt to market dynamics, and explore new growth avenues.

“With accelerators tailored for every stage of the enterprise generative AI journey, from exploration to implementation and scale, Birlasoft Cogito will enable businesses towards rapid innovation, improved productivity, and uncovering new opportunities. This marks a significant milestone in our continuous mission to deliver innovative Generative AI solutions supporting enterprises on their path to success and competitiveness,” Selvakumaran Mannappan, Chief Operating Officer, Birlasoft, said.

Last year, Birlasoft established a generative AI centre of excellence in collaboration with Microsoft. The Generative AI Centre of Excellence serves as a hub for Birlasoft and Microsoft experts to facilitate research, training, and collaboration.

Through this collective ecosystem, organisations will be empowered to unlock the full potential of generative AI, enabling them to develop tailored solutions to address complex business challenges.

The post Birlasoft Launches Generative AI Platform Cogito appeared first on Analytics India Magazine.

Books, Courses, and Live Events to Learn Generative AI with O’Reilly

Books, Courses, and Live Events to Learn Generative AI with O’Reilly
Image by Editor

Generative AI is growing, with more people interested in how they can transition into the AI sector. A big hit at the moment and if you have clicked on this link, you are also wondering what you can do to learn more about Generative AI.

In this blog, I will go through the books, courses and live events available with O’Reilly to learn about Generative AI.

Machine Learning, Data Science and Generative AI with Python

Link: Course — Machine Learning, Data Science and Generative AI with Python

Level: Beginner

For those who want the more practical learning route, this course from O’Reilly: Machine Learning, Data Science and Generative AI with Python starts as a Python crash course, setting up your computer, and then into the aspects of machine learning, artificial intelligence and data mining techniques, including deep learning and neural networks.

The course will also dive into the transformer architecture and its role in AI, and GPT applications, where you will be able to practice fine-tuning Transformers for tasks such as movie review analysis.

Applied Generative AI for Beginners: Practical Knowledge on Diffusion Models, ChatGPT, and Other LLMs

Link: Book — Applied Generative AI for Beginners

Level: Beginner

Another beginner-friendly resource to dive into the world of generative AI, where you will learn about the basics of neural networks to large language models such as ChatGPT and Google Bard. If you are interested in learning about generative AI and want to start from scratch — this book is a one-stop resource to do exactly that!

If you are a data scientist who is looking to learn about more advanced models or a business person who wants to leverage AI for enterprise growth, this book will be your guide to mastering generative AI — theoretical knowledge with practical insights.

Generative AI in the Enterprise

Link: Report — Generative AI in the Enterprise

Level: Beginner/Intermediate

If you’re looking for a new read, something that is more focused on generative AI, specifically in the enterprise — this report is for you. O’Reilly has created a report that unpacks the data from thousands of our users to explore just how many organizations are making use of generative AI tools.

Not only how many people are making use of it, but also how they are using it to get the most out of it and improve their workflow. In this report, you will discover the most popular generative AI models that are being used, their use cases, as well as the risks associated with generative AI tools which are making early adopters hesitant to use and implement them.

Generative AI for Everyone

Link: Live Event — Generative AI for Everyone

Level: Intermediate

On the 12th of February, O’Reilly is holding a live event for the public to learn and understand about generative AI applications, benefits, and risks.

In this 2 hour live event, you will learn about generative AI applications and their potential impact on your organization. You will dive deeper into what prompt engineering is, why you should master it and how you can benefit from generative AI by creating content that is personalized and can automate tedious tasks.

Generative AI on AWS

Link: Book — Generative AI on AWS

Level: Intermediate/Expert

Although there are people out there who are hesitant to adopt generative AI, on the other hand, there are a lot of companies who are moving quickly to integrate generative AI into their products and services. However, there is still a grey area of how generative AI truly impacts and improves companies.

The authors of this book are from AWS, and they aim to guide CTOs, practitioners, application developers, business analysts, and more to find practical ways to use this exciting new technology. You will dive into the lifecycle of generative AI projects, whilst also exploring the different types of models such as large language models (LLMs) and multimodal models.

Prompt Engineering for Generative AI

Link: Book — Prompt Engineering for Generative AI

Level: Intermediate/Expert

Prompt engineering is the beauty behind generative AI tools. We are seeing that AI Prompt Engineers are Making $300k/y, so you can imagine that it is a very good skill to learn in this day and age. In this book, you will learn how to apply generative AI models into practice and overcome the challenge of receiving reliable outputs through prompt engineering.

In this book, you will learn the overview of the five principles of prompting, an introduction to LLMs with text generation, standard practices, advanced techniques, image generations, and more. Learn the structure of the interaction chain of your program's AI model and the key principles of natural language processing, text and image generation, and code.

Wrapping it up

In this blog, I have aimed to offer different types of resources as I understand that people learn in their own unique ways. O’Reilly offers many great resources to help you elevate your career, upskill, and ensure a continuous learning journey. Personal or within your team, start your learning journey today by clicking here.

Nisha Arya is a Data Scientist and Freelance Technical Writer. She is particularly interested in providing Data Science career advice or tutorials and theory based knowledge around Data Science. She also wishes to explore the different ways Artificial Intelligence is/can benefit the longevity of human life. A keen learner, seeking to broaden her tech knowledge and writing skills, whilst helping guide others.

More On This Topic

  • Toloka 101 Live Demo: Learn how to get reliable training data for…
  • AIRSIDE LIVE Is Where Big Data, Data Security and Data Governance Converge
  • 25 Free Courses to Master Data Science, Data Engineering, Machine…
  • Make Connections With SAS Live Web Learning
  • A Full End-to-End Deployment of a Machine Learning Algorithm into a…
  • DATAcated Expo, Oct 5, Live-streamed,Explore new AI / Data Science Tech

Pokémons Come to Life with LLMs

Pokemons Come to Life with LLMs

Researchers from Georgia Institute of Technology have introduced PokéLLMon, the first Large Language Model (LLM)-embodied agent that attains human-parity performance in tactical battle games, specifically demonstrated in Pokémon battles.

Click here to check out the paper and model.

The success of PokéLLMon is attributed to its incorporation of three key strategies:

  • In-Context Reinforcement Learning: The agent instantly consumes text-based feedback derived from battles, employing reinforcement learning to iteratively refine its policy during gameplay.
  • Knowledge-Augmented Generation: PokéLLMon utilizes external knowledge to counteract hallucination, enabling the agent to make timely and informed decisions.
  • Consistent Action Generation: To mitigate the panic switching phenomenon when facing powerful opponents, the agent employs consistent action generation strategies, ensuring it remains composed during battles.

PokéLLMon
A Human-Parity Agent for Pokémon Battles with Large Language Models
paper page: https://t.co/nmTjIEF32d
introduce POKE´LLMON, the first LLMembodied agent that achieves human-parity performance in tactical battle games, as demonstrated in Pokemon battles. The design… pic.twitter.com/kVJeII21o8

— AK (@_akhaliq) February 5, 2024

Through online battles against human opponents, PokéLLMon showcases its ability to execute human-like battle strategies and make just-in-time decisions. The agent achieves notable results with a 49% win rate in Ladder competitions and an impressive 56% win rate in invited battles.

The development of LLM-embodied agents has also shown promise in open-ended games, where players can freely explore game worlds and interact with others. Generative Agent and Voyager in MineCraft are notable examples, displaying behavior and social interactions mirroring human-like patterns.

Tactical battle games are considered a suitable benchmark for evaluating LLMs’ game-playing ability due to directly measurable win rates and consistently available opponents. While LLMs have previously been employed in games like StarCraft II, PokéLLMon presents several advantages, including lossless translation of Pokémon battle states into text, a turn-based format that eliminates real-time stress, and the heightened difficulty of battling against disciplined human players.

The post Pokémons Come to Life with LLMs appeared first on Analytics India Magazine.