The AI world needs more data transparency and web3 startup Space and Time says it can help

The AI world needs more data transparency and web3 startup Space and Time says it can help Jacquelyn Melinek 9 hours

As AI proliferates and things on the internet are easier to manipulate, there’s a need more than ever to make sure data and brands are verifiable, said Scott Dykstra, CTO and co-founder of Space and Time, on TechCrunch’s Chain Reaction podcast.

“Not to get too cryptographically religious here, but we saw that during the FTX collapse,” Dykstra said. “We had an organization that had some brand trust, like I had my personal life savings in FTX. I trusted them as a brand.”

But the now-defunct crypto exchange FTX was manipulating its books internally and misleading investors. Dykstra sees that as akin to making a query to a database for financial records, but manipulating it inside their own database.

And this transcends beyond FTX, into other industries, too. “There’s an incentive for financial institutions to want to manipulate their records … so we see it all the time and it becomes more problematic,” Dykstra said.

But what is the best solution to this? Dykstra thinks the answer is through verification of data and zero-knowledge proofs (ZK proofs), which are cryptographic actions used to prove something about a piece of information — without revealing the origin data itself.

“It has a lot to do with whether there’s an incentive for bad actors to want to manipulate things,” Dykstra said. Anytime there’s a higher incentive, where people would want to manipulate data, prices, the books, finances or more, ZK proofs can be used to verify and retrieve the data.

At a high level, ZK proofs work by having two parties, the prover and the verifier, that confirm a statement is true without conveying any information more than whether it’s correct. For example, if I wanted to know whether someone’s credit score was above 700, if there’s one in place, a ZK proof — prover — can confirm that to the verifier, without actually disclosing the exact number.

Space and Time aims to be that verifiable computing layer for web3 by indexing data both off-chain and on-chain, but Dykstra sees it expanding beyond the industry and into others. As it stands, the startup has indexed from major blockchains like Ethereum, Bitcoin, Polygon, Sui, Avalanche, Sei and Aptos and is adding support for more chains to power the future of AI and blockchain technology.

Dykstra’s most recent concern is that AI data isn’t really verifiable. “I’m pretty concerned that we’re not really efficiently ever going to be able to verify that an LLM was executed correctly.”

There are teams today that are working on solving that issue by building ZK proofs for machine learning or large language models (LLMs), but it can take years to try and create that, Dykstra said. This means that the model operator can tamper with the system or LLM to do things that are problematic.

Blockchain tech could be the answer to uncovering deepfakes and validating content

There needs to be a “decentralized, but globally, always available database” that can be created through blockchains, Dykstra said. “Everyone needs to access it, it can’t be a monopoly.”

For example, in a hypothetical scenario, Dykstra said OpenAI itself can’t be the proprietor of a database of a journal, for which journalists are creating content. Instead, it has to be something that’s owned by the community and operated by the community in a way that’s readily available and uncensorable. “It has to be decentralized, it’s going to have to be on-chain, there’s no way around it,” Dykstra said.

This story was inspired by an episode of TechCrunch’s podcast Chain Reaction. Subscribe to Chain Reaction on Apple Podcasts, Spotify or your favorite pod platform to hear more stories and tips from the entrepreneurs building today’s most innovative companies.

Connect with us:

  • On X, formerly known as Twitter, here.
  • Via email: chainreaction@techcrunch.com

Gartner Data & Analytics Summit São Paulo: Mercado Livre’s AI and Data Democratization in Brazil

I had the opportunity to attend the Gartner Data & Analytics Summit in São Paulo, Brazil, from March 25-27. The conference brought together industry leaders, experts, and practitioners to discuss the latest trends, strategies, and best practices in data and analytics. Brazil's growing importance in the AI landscape was evident throughout the event, with many insightful presentations and discussions focusing on AI adoption and innovation.

One of the interesting talks I attended was delivered by Eduardo Cantero Gonçalves, a senior Data Analytics manager at Mercado Livre (MercadoLibre). Mercado Livre is a leading e-commerce and fintech company that has established itself as a dominant player in the Latin American market. With operations spanning 18 countries, including major economies such as Brazil, Argentina, Mexico, and Colombia, Mercado Livre has built a vast online commerce and payments ecosystem. The company's strong market presence and extensive user base have positioned it as a leader in the region.

During his presentation, Gonçalves shared Mercado Livre's remarkable journey in democratizing data and AI across the organization while fostering a strong data-driven culture. As AI continues to transform industries worldwide, Mercado Livre's experience offers valuable lessons for organizations looking to harness the power of AI and build a data-driven culture.

In this article, we will explore the key takeaways from Gonçalves's presentation, focusing on the company's approach to data democratization, empowering non-technical users with low-code AI tools, and cultivating a data-driven mindset throughout the organization.

Mercado Livre's Data Democratization Journey

Mercado Livre's journey towards data democratization has been a transformative process that has reshaped the company's approach to data and AI. Gonçalves emphasized the importance of transitioning from a centralized data environment to a decentralized one, enabling teams across the organization to access and leverage data for decision-making and innovation.

A key aspect of this transition was the development of in-house data tools. By creating their own tools, Mercado Livre was able to tailor solutions to their specific needs and ensure seamless integration with their existing systems. This approach not only provided greater flexibility but also fostered a sense of ownership and collaboration among teams.

One of the most significant milestones in Mercado Livre's data democratization journey was the introduction of machine learning tools designed for both data scientists and business users. Gonçalves highlighted the importance of empowering non-technical users to harness the power of AI and ML without relying heavily on data science teams. By providing low-code tools and intuitive interfaces, Mercado Livre has enabled business users to experiment with AI and ML, driving innovation and efficiency across various departments.

The democratization of data and AI has had a profound impact on Mercado Livre's operations and culture. It has fostered a more collaborative and data-driven environment, where teams can easily access and analyze data to inform their strategies and decision-making processes. This shift has not only improved operational efficiency but has also opened up new opportunities for growth and innovation.

Empowering Non-Technical Users with Low-Code AI Tools

One of the key highlights of Mercado Livre's data democratization journey is their focus on empowering non-technical users with low-code AI tools. During his presentation, Gonçalves emphasized the importance of enabling business users to experiment with AI and machine learning without relying heavily on data science teams.

To achieve this, Mercado Livre developed an in-house tool called “Data Switch,” which serves as a single web portal for users to access all data-related tools, including query builders, dashboards, and machine learning tools. This centralized platform makes it easier for non-technical users to leverage AI and ML capabilities without needing extensive programming knowledge.

Gonçalves specifically mentioned that Mercado Livre introduced low-code machine learning tools to allow business users to run experiments independently. By providing intuitive interfaces and pre-built models, these tools enable domain experts to apply their knowledge and insights to AI-powered solutions. This approach not only democratizes AI but also accelerates innovation by allowing more people within the organization to contribute to AI initiatives.

The impact of empowering non-technical users with low-code AI tools has been significant for Mercado Livre. Gonçalves noted that the company saw a substantial increase in the number of active users, data storage, ETL jobs, and dashboards following the introduction of these tools.

Mercado Livre's success in this area serves as a valuable case study for other organizations looking to democratize AI and empower their workforce. By investing in low-code AI tools and providing the necessary training and support, companies can unlock the potential of their non-technical users and foster a culture of innovation.

Fostering a Data-Driven Culture

In addition to democratizing data and AI tools, Mercado Livre recognized the importance of fostering a data-driven culture throughout the organization. Gonçalves highlighted several key initiatives that the company undertook to cultivate a mindset that embraces data and AI-driven decision-making.

One notable step was the creation of a dedicated Data Culture area within Mercado Livre. This team was tasked with promoting data literacy, providing training, and supporting data-driven initiatives across the organization.

To measure the success of their data culture efforts, Mercado Livre developed a “Data Driven Index” that tracks user engagement with data tools. This index provides a quantitative measure of how well employees are adopting and leveraging data in their daily work. By regularly monitoring this index, the company can identify areas for improvement and adjust their strategies accordingly.

Another key initiative was the “Data Champions” program, which aimed to train advanced users who could then help multiply the data-driven culture throughout the organization. These champions serve as advocates and mentors, promoting best practices and assisting their colleagues in leveraging data and AI tools effectively. By empowering a network of champions, Mercado Livre was able to scale its data culture efforts and drive adoption across the company.

Lessons Learned from Mercado Livre's Experience

Mercado Livre's journey in democratizing data and AI offers valuable lessons for other organizations looking to embark on a similar path. One of the key takeaways from Gonçalves's presentation was the importance of executive sponsorship in promoting a data-driven culture. Having strong support and advocacy from leadership is critical in driving organizational change and ensuring that data and AI initiatives are given the necessary resources and priority.

Another important lesson is the value of collaborating with HR to integrate data-driven culture into employee onboarding and development programs. By making data literacy and AI skills a core part of employee training, organizations can ensure that their workforce is well-equipped to leverage these tools effectively. Mercado Livre's partnership with HR helped them to scale their data culture efforts and make it a fundamental part of their employees' growth and development.

Finally, Gonçalves emphasized the importance of continuously measuring and iterating on data-driven initiatives. By tracking key metrics such as the Data Driven Index and regularly seeking feedback from employees, organizations can identify areas for improvement and make data-informed decisions to optimize their strategies. This iterative approach ensures that data and AI initiatives remain aligned with business objectives and drive meaningful impact.

As organizations navigate the challenges and opportunities of the AI era, Mercado Livre's experience serves as a valuable case study in democratizing data and AI while fostering a data-driven culture. By empowering employees at all levels to leverage these tools and cultivating a mindset that embraces data-driven decision-making, companies can position themselves for success in our AI-driven world.

OpenAI’s Voice Engine Can Recreate Human Voices with Emotions

OpenAI announced its latest model Voice Engine, built to generate natural-sounding speech from text input and a mere 15-second audio sample. Notably, Voice Engine can create emotive and realistic voices using this brief audio input.

We're sharing our learnings from a small-scale preview of Voice Engine, a model which uses text input and a single 15-second audio sample to generate natural-sounding speech that closely resembles the original speaker. https://t.co/yLsfGaVtrZ

— OpenAI (@OpenAI) March 29, 2024

OpenAI said the Voice Engine project commenced in late 2022, initially powering preset voices within OpenAI’s text-to-speech API, ChatGPT Voice, and Read Aloud features. However, due to concerns about potential misuse, the company has not yet released it to the public, similar to its text-to-video generation model, Sora.

“We recognize that generating speech that resembles people’s voices has serious risks, which are especially top of mind in an election year. We are engaging with U.S. and international partners from across government, media, entertainment, education, civil society and beyond to ensure we are incorporating their feedback as we build,” the company wrote in the blog post.

OpenAI has been privately testing Voice Engine with a small group of trusted partners. Early trials have yielded promising applications across various sectors. These include:

Enhancing Education: Age of Learning, an education technology company, utilises Voice Engine to create pre-scripted voice-over content for reading assistance among non-readers and children. The integration of Voice Engine and GPT-4 enables personalised real-time interactions, expanding content accessibility and audience reach.

Global Content Translation: HeyGen, an AI visual storytelling platform, employs Voice Engine for translation of videos and podcasts into multiple languages while preserving the native accent of the original speaker. This innovation facilitates global content dissemination and audience engagement.

Community Health Services: Dimagi leverages Voice Engine and GPT-4 to enhance essential service delivery in remote areas, particularly in healthcare settings. The use of interactive feedback in local languages aids community health workers in providing counseling and support services effectively.

Assistive Communication: Livox, an AI communication app, integrates Voice Engine to offer non-robotic and customisable voices for individuals with speech-related disabilities. This advancement empowers users to express themselves authentically across different languages and communication contexts.

Clinical Applications: The Norman Prince Neurosciences Institute at Lifespan explores Voice Engine’s potential in clinical settings, restoring speech for patients with speech impairments caused by medical conditions. The short audio sample requirement makes Voice Engine a viable tool for speech rehabilitation and patient care.

The post OpenAI’s Voice Engine Can Recreate Human Voices with Emotions appeared first on Analytics India Magazine.

OpenAI built a voice cloning tool, but you can’t use it… yet

OpenAI built a voice cloning tool, but you can’t use it… yet Kyle Wiggers 8 hours

As deepfakes proliferate, OpenAI is refining the tech used to clone voices — but the company insists it’s doing so responsibly.

Today marks the preview debut of OpenAI’s Voice Engine, an expansion of the company’s existing text-to-speech API. Under development for about two years, Voice Engine allows users to upload any 15-second voice sample to generate a synthetic copy of that voice. But there’s no date for public availability yet, giving the company time to respond to how the model is used and abused.

“We want to make sure that everyone feels good about how it’s being deployed — that we understand the landscape of where this tech is dangerous and we have mitigations in place for that,” Jeff Harris, a member of the product staff at OpenAI, told TechCrunch in an interview.

Training the model

The generative AI model powering Voice Engine has been hiding in plain sight for some time, Harris said.

The same model underpins the voice and “read aloud” capabilities in ChatGPT, OpenAI’s AI-powered chatbot, as well as the preset voices available in OpenAI’s text-to-speech API. And Spotify’s been using it since early September to dub podcasts for high-profile hosts like Lex Fridman in different languages.

I asked Harris where the model’s training data came from — a bit of a touchy subject. He would only say that the Voice Engine model was trained on a mix of licensed and publicly available data.

Models like the one powering Voice Engine are trained on an enormous number of examples — in this case, speech recordings — usually sourced from public sites and data sets around the web. Many generative AI vendors see training data as a competitive advantage and thus keep it and info pertaining to it close to the chest. But training data details are also a potential source of IP-related lawsuits, another disincentive to reveal much.

OpenAI is already being sued over allegations the company violated IP law by training its AI on copyrighted content including photos, artwork, code, articles and e-books without providing the creators or owners credit or pay.

OpenAI has licensing agreements in place with some content providers, like Shutterstock and the news publisher Axel Springer, and allows webmasters to block its web crawler from scraping their site for training data. OpenAI also lets artists “opt out” of and remove their work from the data sets that the company uses to train its image-generating models, including its latest DALL-E 3.

But OpenAI offers no such opt-out scheme for its other products. And in a recent statement to the U.K.’s House of Lords, OpenAI suggested that it’s “impossible” to create useful AI models without copyrighted material, asserting that fair use — the legal doctrine that allows for the use of copyrighted works to make a secondary creation as long as it’s transformative — shields it where it concerns model training.

Synthesizing voice

Surprisingly, Voice Engine isn’t trained or fine-tuned on user data. That’s owing in part to the ephemeral way in which the model — a combination of a diffusion process and transformer — generates speech.

“We take a small audio sample and text and generate realistic speech that matches the original speaker,” said Harris. “The audio that’s used is dropped after the request is complete.”

As he explained it, the model is simultaneously analyzing the speech data it pulls from and the text data meant to be read aloud, generating a matching voice without having to build a custom model per speaker.

It’s not novel tech. A number of startups have delivered voice cloning products for years, from ElevenLabs to Replica Studios to Papercup to Deepdub to Respeecher. So have Big Tech incumbents such as Amazon, Google and Microsoft — the last of which is a major OpenAI’s investor incidentally.

Harris claimed that OpenAI’s approach delivers overall higher-quality speech.

We also know it will be priced aggressively. Although OpenAI removed Voice Engine’s pricing from the marketing materials it published today, in documents viewed by TechCrunch, Voice Engine is listed as costing $15 per one million characters, or ~162,500 words. That would fit Dickens’ “Oliver Twist” with a little room to spare. (An “HD” quality option costs twice that, but confusingly, an OpenAI spokesperson told TechCrunch that there’s no difference between HD and non-HD voices. Make of that what you will.)

That translates to around 18 hours of audio, making the price somewhat south of $1 per hour. That’s indeed cheaper than what one of the more popular rival vendors, ElevenLabs, charges — $11 for 100,000 characters per month. But it does come at the expense of some customization.

Voice Engine doesn’t offer controls to adjust the tone, pitch or cadence of a voice. In fact, it doesn’t offer any fine-tuning knobs or dials at the moment, although Harris notes that any expressiveness in the 15-second voice sample will carry on through subsequent generations (for example, if you speak in an excited tone, the resulting synthetic voice will sound consistently excited). We’ll see how the quality of the reading compares with other models when they can be compared directly.

Voice talent as commodity

Voice actor salaries on ZipRecruiter range from $12 to $79 per hour — a lot more expensive than Voice Engine, even on the low end (actors with agents will command a much higher price per project). Were it to catch on, OpenAI’s tool could commoditize voice work. So, where does that leave actors?

The talent industry wouldn’t be caught unawares, exactly — it’s been grappling with the existential threat of generative AI for some time. Voice actors are increasingly being asked to sign away rights to their voices so that clients can use AI to generate synthetic versions that could eventually replace them. Voice work — particularly cheap, entry-level work — is at risk of being eliminated in favor of AI-generated speech.

Now, some AI voice platforms are trying to strike a balance.

Replica Studios last year signed a somewhat contentious deal with SAG-AFTRA to create and license copies of the media artist union members’ voices. The organizations said that the arrangement established fair and ethical terms and conditions to ensure performer consent while negotiating terms for uses of synthetic voices in new works including video games.

The writers’ strike is over; here’s how AI negotiations shook out

ElevenLabs, meanwhile, hosts a marketplace for synthetic voices that allows users to create a voice, verify and share it publicly. When others use a voice, the original creators receive compensation — a set dollar amount per 1,000 characters.

OpenAI will establish no such labor union deals or marketplaces, at least not in the near term, and requires only that users obtain “explicit consent” from the people whose voices are cloned, make “clear disclosures” indicating which voices are AI-generated and agree not to use the voices of minors, deceased people or political figures in their generations.

“How this intersects with the voice actor economy is something that we’re watching closely and really curious about,” Harris said. “I think that there’s going to be a lot of opportunity to sort of scale your reach as a voice actor through this kind of technology. But this is all stuff that we’re going to learn as people actually deploy and play with the tech a little bit.”

Ethics and deepfakes

Voice cloning apps can be — and have been — abused in ways that go well beyond threatening the livelihoods of actors.

The infamous message board 4chan, known for its conspiratorial content, used ElevenLabs’ platform to share hateful messages mimicking celebrities like Emma Watson. The Verge’s James Vincent was able to tap AI tools to maliciously, quickly clone voices, generating samples containing everything from violent threats to racist and transphobic remarks. And over at Vice, reporter Joseph Cox documented generating a voice clone convincing enough to fool a bank’s authentication system.

There are fears bad actors will attempt to sway elections with voice cloning. And they’re not unfounded: In January, a phone campaign employed a deepfaked President Biden to deter New Hampshire citizens from voting — prompting the FCC to move to make future such campaigns illegal.

FCC officially declares AI-voiced robocalls illegal

So aside from banning deepfakes at the policy level, what steps is OpenAI taking, if any, to prevent Voice Engine from being misused? Harris mentioned a few.

First, Voice Engine is only being made available an exceptionally small group of developers — around 10 — to start. OpenAI is prioritizing use cases that are “low risk” and “socially beneficial,” Harris says, like those in healthcare and accessibility, in addition to experimenting with “responsible” synthetic media.

A few early Voice Engine adopters include Age of Learning, an edtech company that’s using the tool to generate voice-overs from previously-cast actors, and HeyGen, a storytelling app leveraging Voice Engine for translation. Livox and Lifespan are using Voice Engine to create voices for people with speech impairments and disabilities, and Dimagi is building a Voice Engine-based tool to give feedback to health workers in their primary languages.

Here’s generated voices from Lifespan:

https://techcrunch.com/wp-content/uploads/2024/03/lifespan_generation_ordering.mp3

https://techcrunch.com/wp-content/uploads/2024/03/lifespan_generation_talking.mp3

And here’s one from Livox:

https://techcrunch.com/wp-content/uploads/2024/03/livox_generation_english.mp3

Second, clones created with Voice Engine are watermarked using a technique OpenAI developed that embeds inaudible identifiers in recordings. (Other vendors including Resemble AI and Microsoft employ similar watermarks.) Harris didn’t promise that there aren’t ways to circumvent the watermark, but described it as “tamper resistant.”

“If there’s an audio clip out there, it’s really easy for us to look at that clip and determine that it was generated by our system and the developer that actually did that generation,” Harris said. “So far, it isn’t open sourced — we have it internally for now. We’re curious about making it publicly available, but obviously, that comes with added risks in terms of exposure and breaking it.”

OpenAI launches a red teaming network to make its models more robust

Third, OpenAI plans to provide members of its red teaming network, a contracted group of experts that help inform the company’s AI model risk assessment and mitigation strategies, access to Voice Engine to suss out malicious uses.

Some experts argue that AI red teaming isn’t exhaustive enough and that it’s incumbent on vendors to develop tools to defend against harms that their AI might cause. OpenAI isn’t going quite that far with Voice Engine — but Harris asserts that the company’s “top principle” is releasing the technology safely.

General release

Depending on how the preview goes and the public reception to Voice Engine, OpenAI might release the tool to its wider developer base, but at present, the company is reluctant to commit to anything concrete.

Harris did give a sneak peek at Voice Engine’s roadmap, though, revealing that OpenAI is testing a security mechanism that has users read randomly generated text as proof that they’re present and aware of how their voice is being used. This could give OpenAI the confidence it needs to bring Voice Engine to more people, Harris said — or it might just be the beginning.

“What’s going to keep pushing us forward in terms of the actual voice matching technology is really going to depend on what we learn from the pilot, the safety issues that are uncovered and the mitigations that we have in place,” he said. “We don’t want people to be confused between artificial voices and actual human voices.”

And on that last point we can agree.

RAFT – A Fine-Tuning and RAG Approach to Domain-Specific Question Answering

As the applications of large language models expand into specialized domains, the need for efficient and effective adaptation techniques becomes increasingly crucial. Enter RAFT (Retrieval Augmented Fine Tuning), a novel approach that combines the strengths of retrieval-augmented generation (RAG) and fine-tuning, tailored specifically for domain-specific question answering tasks.

The Challenge of Domain Adaptation

While LLMs are pre-trained on vast amounts of data, their ability to perform well in specialized domains, such as medical research, legal documentation, or enterprise-specific knowledge bases, is often limited. This limitation arises because the pre-training data may not adequately represent the nuances and intricacies of these specialized domains. To address this challenge, researchers have traditionally employed two main techniques: retrieval-augmented generation (RAG) and fine-tuning.

Retrieval-Augmented Generation (RAG)

RAG

RAG

RAG is a technique that enables LLMs to access and utilize external knowledge sources during inference.

It achieves this by integrating real-time data retrieval into the generative process, thus making the model's outputs more accurate and up-to-date. RAG consists of three core steps: retrieval, where relevant documents are gathered; generation, where the model produces an output based on the retrieved data; and augmentation, which refines the output further.

The retrieval process in RAG starts with a user's query. LLMs analyze the query and fetch pertinent information from external databases, presenting a pool of data from which the model can draw to formulate its responses. The generation phase then synthesizes this input into a coherent narrative or answer. The augmentation step refines the generation by adding context or adjusting for coherence and relevance.

RAG models can be evaluated using a variety of metrics, assessing their ability to provide accurate, relevant, and up-to-date information.

Fine-Tuning

supervised-fine-tuning

supervised-fine-tuning

Fine-tuning, on the other hand, involves adapting a pre-trained LLM to a specific task or domain by further training it on a smaller, task-specific dataset. This approach allows the model to learn patterns and align its outputs with the desired task or domain. While fine-tuning can improve the model's performance, it often fails to effectively incorporate external knowledge sources or account for retrieval imperfections during inference.

The RAFT Approach

RAFT

RAFT

RAFT standing for Retrieval-Aware Fine-Tuning, is an innovative training method tailored for language models to enhance their performance in domain-specific tasks, particularly for open-book exams. RAFT diverges from standard fine-tuning by preparing training data that incorporates questions with a mix of relevant and non-relevant documents, along with chain-of-thought styled answers derived from the relevant texts. This method aims to improve models’ abilities to not only recall information but also reason and derive answers from provided content.

In essence, RAFT fine-tunes language models to be more proficient in tasks that involve reading comprehension and knowledge extraction from a set of documents. By training with both “oracle” documents (which contain the answer) and “distractor” documents (which do not), the model learns to discern and utilize relevant information more effectively.

Training Data Preparation

The training process under RAFT involves a proportion of the data to contain oracle documents that directly relate to the answers, while the remaining data consists only of distractor documents. The fine-tuning encourages the model to learn when to rely on its internal knowledge (akin to memorization) and when to extract information from the context provided.

RAFT's training regimen also emphasizes the generation of reasoning processes, which not only help in forming the answer but also cite sources, similar to how a human would justify their response by referencing material they have read. This approach not only prepares the model for a RAG (Retrieval Augmented Generation) setting where it has to consider top-k retrieved documents but also ensures the model's training is independent of the retriever used, allowing for flexible application across different retrieval systems.

This approach serves multiple purposes:

  1. It trains the model to identify and utilize relevant information from the provided context, mimicking the open-book exam setting.
  2. It enhances the model's ability to disregard irrelevant information, a critical skill for effective RAG.
  3. It exposes the model to scenarios where the answer is not present in the context, encouraging it to rely on its own knowledge when necessary.

Another key aspect of RAFT is the incorporation of chain-of-thought reasoning into the training process. Instead of simply providing the question and answer pairs, RAFT generates detailed reasoning explanations that include verbatim citations from the relevant documents. These explanations, presented in a chain-of-thought format, guide the model through the logical steps required to arrive at the correct answer.

By training the model on these reasoning chains, RAFT encourages the development of strong reasoning abilities and enhances the model's understanding of how to effectively leverage external knowledge sources.

Evaluation and Results

The authors of the RAFT paper conducted extensive evaluations on various datasets, including PubMed (biomedical research), HotpotQA (open-domain question answering), and the Gorilla APIBench (code generation). Their results demonstrated that RAFT consistently outperformed baselines, such as domain-specific fine-tuning with and without RAG, as well as larger models like GPT-3.5 with RAG.

RAFT improves RAG performance

RAFT improves RAG performance

For instance, on the HuggingFace dataset, RAFT achieved an accuracy of 74%, a significant improvement of 31.41% over domain-specific fine-tuning (DSF) and 44.92% over GPT-3.5 with RAG. Similarly, on the HotpotQA dataset, RAFT exhibited a 28.9% accuracy gain compared to DSF.

One of the key advantages of RAFT is its robustness to retrieval imperfections. By training the model with a mix of relevant and irrelevant documents, RAFT enhances the model's ability to discern and prioritize relevant information, even when the retrieval module returns suboptimal results.

The authors demonstrated that fine-tuning with only the oracle documents often leads to inferior performance compared to configurations that include distractor documents. This finding underscores the importance of exposing the model to varying retrieval scenarios during training, ensuring its preparedness for real-world applications.

Practical Applications and Future Directions

The RAFT technique has significant implications for a wide range of practical applications, including:

  1. Question Answering Systems: RAFT can be employed to build highly accurate and domain-specific question answering systems, leveraging both the model's learned knowledge and external knowledge sources.
  2. Enterprise Knowledge Management: Organizations with large knowledge bases can leverage RAFT to develop customized question answering systems, enabling employees to quickly access and utilize relevant information.
  3. Medical and Scientific Research: RAFT can be particularly valuable in domains such as biomedical research, where access to the latest findings and literature is crucial for advancing scientific understanding.
  4. Legal and Financial Services: RAFT can assist professionals in these fields by providing accurate and context-aware responses based on relevant legal documents or financial reports.

As research in this area continues, we can expect further advancements and refinements to the RAFT technique. Potential future directions include:

  1. Exploration of more efficient and effective retrieval modules, tailored for specific domains or document structures.
  2. Integration of multi-modal information, such as images or tables, into the RAFT framework for enhanced context understanding.
  3. Development of specialized reasoning architectures that can better leverage the chain-of-thought explanations generated during training.
  4. Adaptation of RAFT to other natural language tasks beyond question answering, such as summarization, translation, or dialogue systems.

Conclusion

RAFT represents a significant leap forward in the field of domain-specific question answering with language models. By harmoniously blending the strengths of retrieval-augmented generation and fine-tuning, RAFT equips LLMs with the ability to effectively leverage external knowledge sources while also aligning their outputs with domain-specific patterns and preferences.

Through its innovative training data curation, incorporation of chain-of-thought reasoning, and robustness to retrieval imperfections, RAFT offers a powerful solution for organizations and researchers seeking to unlock the full potential of LLMs in specialized domains.

As the demand for domain-specific natural language processing capabilities continues to grow, techniques like RAFT will play a pivotal role in enabling more accurate, context-aware, and adaptive language models, paving the way for a future where human-machine communication becomes truly seamless and domain-agnostic.

Top 4 GCC Summits of 2024 Transforming India’s Global Capability Centers

Top 7 Generative AI Conferences Worldwide

With the recent growth of Global Capability Centers (GCCs) in India, several summits have emerged as pivotal platforms for sharing insights, innovations, and strategies. These gatherings bring together industry leaders, visionaries, and pioneers to discuss the future of GCCs, focusing on technological advancements, operational excellence, and strategic growth.

Here, we explore four leading GCC summits in India that are shaping the future of this dynamic sector.

1. MachineCon GCC Summit 2024

Scheduled for June 28, 2024, at the Hotel Radisson Blu in Bangalore, the MachineCon GCC Summit stands as a premier event in the realm of generative AI and its integration into Global Capability Centers. This summit, now in its 5th edition, focuses on the transformative potential of Generative AI, emphasizing ethical considerations, international cooperation, and strategic market expansions. It serves as a vital congregation for leaders to navigate the complexities of adopting cutting-edge technologies within an ethical framework, aiming to foster global partnerships and venture into new markets effectively.

Learn more about MachineCon GCC Summit here.

2. Shared Services and GCC Week India 2024

Taking place from April 16 to 19, 2024, at the JW Marriott in Bengaluru, the Shared Services and GCC Week India is a leading summit that highlights the role of AI-driven transformation and innovation in powering India’s GCCs. Amidst challenges such as a pandemic and economic fluctuations, this summit underscores the resilience and adaptability of GCCs, focusing on the shift from cost to value arbitrage and the crucial role of digital in gaining a competitive edge. It offers a platform for leaders to explore digital transformation mandates, leveraging India’s talent hub, and innovating at scale.

Learn more here.

3. NASSCOM GCC Conclave 2023

The NASSCOM GCC Conclave, returning to a full in-person format in 2023 at the Sheraton Grand in Bengaluru, emphasizes India’s leadership in the GCC landscape. This conclave delves into continuous digital transformation, collaborative innovation, and the strategic alignment of business strategies to thrive in a hyperconnected world. It provides insights into building seamless enterprises, harnessing technology at the edge, reimagining the future of work, and preparing for emerging risks and opportunities.

Learn more here.

4. EY GCC Conclave 2024 – Delhi

Scheduled for February 9, 2024, at the ITC Maurya in New Delhi, the EY GCC Conclave focuses on the theme ‘Envision | Explore | Elevate’. It addresses the latest trends and topics pertinent to today’s GCCs across various cities in India. With discussions around emerging technologies, the future of work, new GCC setups, and the government’s role in supporting GCCs, the conclave aims to foster dialogue on transformative frontiers, talent cultivation, and driving the ESG agenda within GCCs.

Learn more here.

These summits not only offer a wealth of knowledge and networking opportunities but also highlight the strategic importance of GCCs in driving digital transformation, operational efficiency, and global business success. As the GCC ecosystem continues to evolve, these platforms play a crucial role in shaping the future of global business services in India and beyond, emphasizing innovation, strategic growth, and the relentless pursuit of excellence.

The post Top 4 GCC Summits of 2024 Transforming India’s Global Capability Centers appeared first on Analytics India Magazine.

10 GitHub Repositories to Master MLOps

10 GitHub Repositories to Master MLOps
Image by Author

It is becoming more important to master MLOps (Machine Learning Operations) for those who want to effectively deploy, monitor, and maintain their ML models in production. MLOps is a set of practices that aims to merge ML system development (Dev) and ML system operation (Ops). Luckily, the open-source community has created numerous resources to assist beginners in mastering these concepts and tools.

Here are ten GitHub repositories that are essential for anyone looking to master MLOps:

1. MLOps-Basics

GitHub Link: graviraja/MLOps-Basics

It is a 9-week study plan designed to help you master various concepts and tools related to Model Monitoring, Configurations, Data Versioning, Model Packaging, Docker, GitHub Actions, and AWS Cloud. You will learn how to build an end-to-end MLOps project, and each week will focus on a specific topic to help you achieve this goal.

2. MLOps examples by Microsoft

GitHub Link: microsoft/MLOps

The repository provides MLOps end-to-end examples & solutions. A collection of examples showing different end to end scenarios operationalizing ML workflows with Azure Machine Learning, integrated with GitHub and other Azure services such as Data Factory and DevOps.

3. Made-With-ML

GitHub Link: GokuMohandas/Made-With-ML

If you are looking for MLOps end-to-end examples and solutions, this repository has got you covered. It contains a diverse collection of scenarios that demonstrate how to operationalize ML workflows using Azure Machine Learning. Plus, it is integrated with other Azure services like Data Factory and DevOps, as well as GitHub.

4. Awesome MLOPs

GitHub Link: Pythondeveloper6/Awesome-MLOPS

The repository contains links to various free resources available online for MLOps. These resources include YouTube videos, career roadmaps, LinkedIn accounts to follow, books, blogs, free and paid courses, communities, projects, and tools. You can find almost everything related to MLOps in one place, so instead of searching online for various things, you can just visit the repository and learn.

5. MLOps Guide

GitHub Link: mlops-guide/mlops-guide.github.io

The repository will take you to a static site hosted on GitHub that will help projects and companies build a more reliable MLOps environment. It covers principles of MLOPs, implementation guides, and project workflow.

6. Awesome MLOps Tools

GitHub Link: kelvins/awesome-mlops

The repository contains a list of MLOps tools that can be used for AutoML, CI/CD for Machine Learning, Cron Job Monitoring, Data Catalog, Data Enrichment, Data Exploration, Data Management, Data Processing, Data Validation, Data Visualization, Drift Detection, Feature Engineering, Feature Store, Hyperparameter Tuning, Knowledge Sharing, Machine Learning Platform, Model Fairness and Privacy, Model Interpretability, Model Lifecycle, Model Serving, Model Testing & Validation, Optimization Tools, Simplification Tools, and Visual Analysis and Debugging.

7. MLOps by DTU

GitHub Link: SkafteNicki/dtu_mlops

This is a repository for the DTU course 02476, which includes exercises and additional materials for the machine learning operations course. The course spans three weeks and covers topics such as development practices, reproducibility, automation, cloud services, deployment, and advanced topics like monitoring and scaling for machine learning applications.

8. MLOps Course by Goku Mohandas

GitHub Link: GokuMohandas/mlops-course

The course focuses on teaching students how to design, develop, deploy, and iterate on production-grade ML applications using best practices, scaling ML workloads, integrating MLOps components, and creating CI/CD workflows for continuous improvement and seamless deployment.

9. MLOps ZoomCamp

GitHub Link: DataTalksClub/mlops-zoomcamp

One of my favorite courses for learning a new concept by building a project. The MLOps course from DataTalks.Club teaches the practical aspects of putting machine learning services into production, from training and experimentation to model deployment and monitoring. It is designed for data scientists, ML engineers, software engineers, and data engineers who are interested in learning how to operationalize machine learning workflows.

10. Serverless ML Course

GitHub Link: featurestoreorg/serverless-ml-course

This course focuses on developing complete Machine Learning systems with serverless capabilities. It allows developers to create predictive services without requiring expertise in Kubernetes or cloud computing. They can do so by writing Python programs and using serverless features, inference pipelines, feature stores, and model registries.

Conclusion

Mastering MLOps is essential for ensuring the reliability, scalability, and efficiency of machine learning projects in production. The repositories listed above offer a wealth of knowledge, practical examples, and essential tools to help you understand and apply MLOps principles effectively. Whether you're a beginner looking to get started or an experienced practitioner seeking to deepen your knowledge, these resources provide valuable insights and guidance on your journey to mastering MLOps.

Please check out the AI learning platform called Travis, which can help you master MLOps and its concepts faster. Travis generates explanations about the topic, and you can ask follow-up questions. Moreover, you can conduct your own research as it provides links to blogs and tutorials published by top publications on Medium, Substacks, independent blogs, official documentation, and books.

Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in technology management and a bachelor's degree in telecommunication engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.

More On This Topic

  • Learn MLOps From These GitHub Repositories
  • 10 GitHub Repositories to Master Machine Learning
  • 25 Github Repositories Every Python Developer Should Know
  • Learn Data Science From These GitHub Repositories
  • Learn Data Engineering From These GitHub Repositories
  • Learn Machine Learning From These GitHub Repositories

5 Learning Platforms That Every Techie Should Sign Up To

5 Learning Platforms That Every Techie Should Sign Up To
Photo by Peter Olexa

As we already know, the tech industry is moving faster than we can blink. If you are already a tech professional, you will know that the learning never stops. You have to move with the industry, and there’s always something new popping out.

If you are looking to enter the tech industry, the main piece of advice that many current tech professionals will give you is that you have to be a keen learner, because that is what you will be doing for most of your career.

With that being said, some may not know where they can get knowledge from or continuously learn. Therefore, I have put this article together to provide you with different platforms that will cater to different needs.

Coursera

Link: Coursera

Coursera is a well-known learning platform and is not specific to technology courses and certifications only. They cater to individuals, businesses, universities and governments. Over the years, Coursera has become more and more popular, especially in its partnership with many popular universities such as the University of Michigan and Princeton University.

They offer a wide range of courses, from data science, computer science, personal development and more. You may be interested in learning a new programming language whilst building your public speaking or leadership skills. Coursera has it all!

If you are looking to become a Data Scientist, check out A Free Data Science Learning Roadmap: For All Levels with IBM or if you’re looking into SQL, check out Boost Your Data Science Skills: The Essential SQL Certifications You Need.

edX

Link: edX

Another platform that provides you with a variety of online courses, in different backgrounds is edX. You can take a course, earn a certificate or even earn a degree!

The areas of interest on the edX platforms are up to date, for example, you can specifically learn about artificial intelligence and ChatGPT if you’re looking to become a prompt engineer. Or you could dive into the Finance industry and learn about blockchain. The courses provided on edX once upon a time were for specific sectors, however with the way the tech industry is moving — the skills are highly transferable and can benefit anybody!

For example, if you are looking to be an all-rounder data professional, check out 7 Free Harvard University Courses to Advance Your Skills.

DataCamp

Link: DataCamp

DataCamp is a well-known learning platform that is specific to individuals who are looking to build their data and AI skills. The platform offers a range of free courses which you can take on the free plan, or you could access their full content library with the premium version which is under £11 a month! You can also learn as a team and also implement the learning platform in your company!

They have a variety of courses from beginners to experts, ensuring data professionals have all the skills and knowledge they need to keep up with the tech world. You can also benefit from their webinars, code-alongs, blogs, podcasts, and real-world projects!

If you are looking to jump into the tech industry, check out 4 Certifications to Become Job-Ready in 30 Days.

O’Reilly

Link: O’Reilly

O’Reilly is well known for being a publisher, but that’s not all they offer. Over the years they have produced a lot of tech conferences, live training, interactive learning, certification experiences, videos, and more.

Their online learning and training platform caters to people who like to learn through books and also those who prefer learning through classes. You can learn as an individual or you can learn as a team, through your business, government or higher education institution.

Outside of the courses, certifications and interactive learning, O’Reilly also provides insight reports where they analyze data to discuss what is currently happening in the tech market.

If you would like to learn through O’Reilly, check out Books, Courses, and Live Events to Learn Generative AI with O’Reilly or if you’d like to get your hands on a free report, check out 2024 Tech Trends: AI Breakthroughs & Development Insights from O’Reilly’s Free Report.

Pluralsight

Link: Pluralsight

You probably haven’t heard of this platform and I’m here to tell you that you’re missing out! Pluralsight stands for helping individuals and teams learn to cut cycle times. They want organizations to speed up their onboarding and ensure that your team has the right skills to do what they need to do! They aim to cut out the challenges and priorities making work more efficient and effective.

The learning platform believes that technology teams are only successful if the skills are relevant. The learning platform has a wide range of courses, assessments, and labs to ensure individuals are tech-fluent in the business world.

Wrapping it up

As a tech professional, learning should be part of your career goals. If not — you will unfortunately fall short and feel like you’re on a constant run of catching up with the industry. Learning platforms are here to eradicate those challenges so that you are up-to-date and feel secure in your career.

Self-development should not look like a cost, but how you can elevate your career.

Nisha Arya is a data scientist, freelance technical writer, and an editor and community manager for KDnuggets. She is particularly interested in providing data science career advice or tutorials and theory-based knowledge around data science. Nisha covers a wide range of topics and wishes to explore the different ways artificial intelligence can benefit the longevity of human life. A keen learner, Nisha seeks to broaden her tech knowledge and writing skills, while helping guide others.

More On This Topic

  • 5 Machine Learning Skills Every Machine Learning Engineer Should…
  • The 6 Python Machine Learning Tools Every Data Scientist Should Know About
  • KDnuggets News, May 25: The 6 Python Machine Learning Tools Every…
  • Every Engineer Should and Can Learn Machine Learning
  • 5 Podcasts Every Machine Learning Enthusiast Should Follow
  • 25 Github Repositories Every Python Developer Should Know

Elon Musk’s xAI Unveils Grok-1.5 with Improved Reasoning Capabilities, 128K Context Window

Elon Musk’s xAI Open Sources Grok

Elon Musk’s xAI has announced Grok-1.5, which comes with improved reasoning capabilities and a context length of 128,000 tokens. It will be available to our early testers and existing Grok users on the 𝕏 platform in the coming days.

Grok-1.5 boasts notable improvements, particularly in coding and math-related tasks. It beats Mistral Large on various benchmarks including MMLU, GSM8K and HumanEval.

During tests, Grok-1.5 demonstrated exceptional performance, achieving a remarkable 50.6% score on the MATH benchmark and an impressive 90% score on the GSM8K benchmark. These benchmarks cover a wide range of math problems, showcasing Grok-1.5’s versatility and problem-solving capabilities.

In addition to its prowess in math-related tasks, Grok-1.5 excelled in the HumanEval benchmark, scoring 74.1%. This benchmark evaluates code generation and problem-solving abilities, further highlighting Grok-1.5’s comprehensive skill set.

A standout feature of Grok-1.5 is its long context understanding capability, enabling it to process contexts of up to 128K tokens within its window. This significant enhancement represents a sixteen-fold increase in memory capacity compared to previous models, allowing Grok-1.5 to utilize information from substantially longer documents.

The Grok-1.5 infrastructure is built on a custom distributed training framework based on JAX, Rust, and Kubernetes. This robust training stack ensures reliability and uptime of training jobs, minimising downtime and maximizing efficiency during large-scale model training.

xAI recently released the model weights and network architecture of Grok-1. As the model is gradually rolled out to a wider audience, xAI plans to introduce several new features to Grok 1.5 in the coming days.

The post Elon Musk’s xAI Unveils Grok-1.5 with Improved Reasoning Capabilities, 128K Context Window appeared first on Analytics India Magazine.

Google will soon roll out on-device AI-powered features on Pixel 8

Google will soon roll out on-device AI-powered features on Pixel 8 Ivan Mehta 12 hours

Google announced today that it will soon roll out on-device AI-powered features such as recording summaries and smart replies on the Pixel 8. These features will be based on Gemini Nano, a small-sized model Google released last year, primed to run on devices.

The company said that Summarizer in Recorder and Smart Reply in Gboard will be released as a developer preview in the next Pixel feature drop. The company announced these features last October during the Pixel 8 launch. Until now, Gemini Nano-powered features were only available on the Pixel 8 Pro and and the Galaxy S24.

“Running large language models on phones with different memory specs can deliver different user experiences, so we have been testing and validating this on Pixel 8. We’re excited to provide the opportunity for more enthusiasts and developers to try out Gemini Nano, where we hope to get more feedback and see more innovation,” the company said.

Until now, Google has primarily worked on AI-powered features that rely on the cloud, even if they are targeted toward mobile devices.

In January, Google introduced “Circle to Search” for select Pixel and Samsung Devices. The company announced this week that more Pixel and Samsung phones along with select tablets will get Circle to Search feature. The feature itself will get an update with support for instant translation of the content that’s on the screen.