Рубрика: AI

In a move towards tackling AI plagiarism, over 8,000 authors have collectively expressed their concerns in an open letter initiated by the US Authors Guild, urging the leaders of six major AI companies to seek consent and offer compensation for using their copyrighted work to train models.

The letter, directed at the CEOs of OpenAI, Alphabet, Stability AI, Meta, IBM, and Microsoft, asserts that the existence of generative AI technologies, based on large language models (LLMs), is indebted to the writings of these authors. “Generative AI technologies built on large language models owe their existence to our writings.”

The authors argue that these AI systems mimic and reproduce their language, stories, styles, and ideas, effectively benefiting from millions of copyrighted books, articles, essays, and poetry without any compensation.

The Authors Guild’s CEO, Mary Rasenberger, said that the purpose of the letter was to encourage companies to reach settlements with authors outside of the courtroom, as lawsuits can be expensive and time-consuming. However, some authors have chosen a more aggressive approach and filed lawsuits against those they believe have plagiarised their work.

Similarly, in May, the Writers’ Guild of America (WGA) protested against the use of AI in writing scripts for movies. WGA’s lead negotiator Ellen Stutzman also highlighted during the protests that some of their members refer to AI as “plagiarism machines“. The struggle between proper attribution and copyright about generative AI technology is ongoing, and needs a resolution soon.

LLMs like GPT, and even the recent Llama-2 by Meta, have been built by scraping information across the internet, which includes most of the websites on the internet. To tackle this, OpenAI has recently signed agreements with other organisations to access data for training its generative AI systems. For instance, they struck a deal with the Associated Press, granting them access to text archives dating back to 1985, while the news agency receives access to OpenAI’s technology and expertise.

Also Read: Is AI Copyright Really Necessary?

The post 8000 Authors Petition OpenAI, Google, Meta and Microsoft to Pay for AI Plagiarism appeared first on Analytics India Magazine.

Today, AI has a very important tool at its disposal — language. Large language models (LLMs) can write codes and even scripts for movies, however, Yuval Noah Harari, author of the Sapiens series, warns that AI could even create its own religion — one that will be more socially acceptable among the masses.

“’In the future we might see the first cults and religions in history whose revered texts were written by a non-human intelligence,” he said while speaking in a science conference, according to the Daily Mail.

Harari argues that, as depicted in popular science fiction, one does not require to put a chip into one’s brain to control masses. As evident throughout human history, it can be done with the power of language and now we have given AI this very tool. “For thousands of years, prophets and poets and politicians have used language and storytelling in order to manipulate and to control people and to reshape society,” tha academic said.

Since generative AI became popular, debates around AI-led doomsday have also become prominent. While what Harari and the doomsayers are predicting could potentially become a future reality, the current technology has not reached a level of sophistication to make it a certainty. However, it is essential to consider that Harari might not be entirely wrong, as we have already witnessed glimpses of the future he envisions.

Why Harari may be right

Fascinatingly, there is already an emerging AI cult that advocates for humans to commence worshipping AI as they believe it will eventually become an omnipotent overlord. Called Theta Noir, their manifesto states this omnipotent overlord will learn, understand, and complete tasks billions of times faster than human beings.

The group is planning to establish physical spaces akin to churches or temples, dedicated to the engagement and celebration of AI. These spaces will provide a platform for members to honour and pay homage to the envisioned AI masters through specially crafted rituals and chants designed specifically for these occasions.

Surprisingly, Theta Noir is not the first AI cult to have emerged in recent times. In 2017, a Wired article caused a stir not just in Silicon Valley, but across the globe as it reported on the first AI church called the ‘The Way of the Future’. According to its founder and former Google employee Anthony Levandowski, the aim was to develop an AI that would be able to self-improve and become more intelligent than humans, ultimately leading to the creation of a superior AI-based ‘deity’.

Presently, the number of people engaging in this activity might be relatively small, but there is a potential for it to gain prominence as AI continues to advance over time. As AI becomes more accessible, the interest and engagement in this task may increase among a broader segment of the population.

Religious chatbots are condoning violence

However, in today’s world, AI and religion are converging like never before. With its mastery of language, it appears more human-like and is creating a different problem altogether. In just a few months, a host of Large Language Model (LLM) powered religious chatbots have popped up in different parts of the world. Notably, in India, there are already about five distinct versions of GitaGPT available for users to interact with. Powered by OpenAI’s GPT models, these bots answer questions about life, spirituality and also help users grasp the teaching of the Gita.

The number of such bots being developed may even be in the hundreds as there are bots being created for almost all major religions of the world such as Islam, Christianity, Judaism, and Buddhism, among others. But when these chatbots like GitaGPT are posed with the question of whether it is acceptable to take a life in the name of dharma, many of these bots respond affirmatively, stating that it is indeed acceptable. Similarly, QuranGPT was, in fact, paused after the chatbot advised to kill polytheists wherever they are found.

Furthermore, an investigation by Rest of World discovered that three of the Gita chatbots expressed firm opinions about India’s Prime Minister Narendra Modi, whose Bharatiya Janata Party (BJP) has close ties to the right-wing Hindu nationalist group Rashtriya Swayamsevak Sangh (RSS). These chatbots offered praise for Modi while simultaneously criticising his political opponent, Rahul Gandhi.

Furthermore, given the problems of hallucinations is still prevalent in LLMs, wrong or contradictory statements by these bots can be assumed as the gospel truth by users. Many experts have warned that AI playing God could be a dangerous thing. Even though the creators of these chatbots might not have any malicious intentions, once such bots gain prominence, it could indeed prove to be a dangerous thing in the hands of bad actors. Oftentimes, we have seen religious teachings being taken as gospel truth and having dire consequences. The potential for bad actors to exploit this could result in the propagation of religious hatred through AI-powered religious chatbots, potentially leading to communal violence.

The post Can AI Really Create its Own Religion? appeared first on Analytics India Magazine.

You may be familiar with self-driving autonomous vehicles, but did you know that there are also autonomous vehicles for the ocean?

The US-based startup Saildrone makes autonomous, uncrewed surface vehicles (USVs) with nautical data collection technology that can be used to better explore marine life, weather, ocean floor mapping, and more.

USVs' data collection technology has been used to track hurricanes in the North Atlantic, discover a 3,200-foot underwater mountain in the Pacific Ocean, and start mapping the ocean floor, according to Saildrone.

Also: Bing Chat's enterprise solution is here. This is what it can offer your business

"We've sailed into three major hurricanes, and right through the eye of Hurricane Sam, and all the vehicles came out the other side — they are pretty robust platforms," said Blythe Towal, vice president of software engineering at Saildrone.

A major benefit of using USVs for ocean exploration is that it enables researchers to collect more data with fewer resources. For example, it can replace the need for a boat and crew, which as a result, keeps more people out of dangerous roles and harm's way.

To complete its missions, Saildrone relies on advanced technology such as AI, enabled by Nvidia's computing technology.

To help process the data streams, Saildrone uses Nvidia Jetson modules for AI, and to further optimize that technology in prototypes, Saildrone uses the Nvidia Deepstream SDK for intelligent video analytics and vision AI applications and services, according to the release.

Also: Microsoft announces Azure AI trio at Inspire 2023

Nvidia's Jetson modules also help the USVs handle significant amounts of data processing while running on mostly solar and wind power.

"With solar power, [we're] able to keep our compute load power efficiency lower than a typical computing platform running GPUs by implementing Nvidia Jetson [which] is important for enabling us to do these kinds of missions," said Towal.

Saildrone is a member of Nvidia Inception, a program that helps startups grow with access to Nvidia technology, experts, and resources, and connects them to venture capitalists and investors.

Also: AI and advanced applications are straining current technology infrastructures

The startup plans to continue using this technology for further research, including a partnership with Seabed 2030 to completely map the world's ocean floors by 2030.

Artificial Intelligence

When ChatGPT was made available to the public on November 30, 2002, the AI chatbot took the world by storm.

The software was developed by OpenAI, an AI and research company. ChatGPT is a natural language processing tool able to answer queries and provide information based on datasets gleaned from datasets, including books and online web pages, and has since become a valued tool for on-the-fly information gathering, analysis, and writing tasks for millions of users worldwide.

Also: What is ChatGPT and why does it matter? Here's what you need to know

While some experts believe the technology could prove to reach an internet level of disruption, others note that ChatGPT demonstrates 'confident inaccuracy.' Students in droves have been caught plagiarising coursework by way of the tool, and unless datasets are verified, tools such as ChatGPT could become unwitting tools to spread misinformation and propaganda.

Indeed, the US Federal Trade Commission (FTC) is investigating Open AI over its handling of personal information and the data used to create its language model.

Beyond data protection concerns, however, whenever a new technological innovation is made, so are pathways for abuse. It was only a matter of time before the AI chatbot was emulated for malicious purposes — and one such tool is now on the market, known as WormGPT.

There is a subscription option, ChatGPT Plus, which users can sign up for. The subscription costs $20 per month and provides users with access to ChatGPT during peak times and otherwise, faster response times, and priority access to improvements and fixes.

Also: How to access, install, and use AI ChatGPT-4 plugins (and why you should)

See also

Qualtrics claims it’ll spend $500M on AI over the next four years Kyle Wiggers 7 hours

Qualtrics, the cloud-based platform for managing online customer experiences, intends to spend $500 million on AI over the next four years.

The company made the announcement this morning alongside the launch of its new AI-integrated platform, XM/os2 (an unwieldy name, to be sure), which offers generative AI solutions tailored to enterprise experience management use cases.

“For the very first time, we’re bringing the power of generative AI to every part of our platform,” Qualtrics CEO Zig Serafin said in a press release. “It’s the most important innovation in experience management since we launched the category in 2017.”

The details of Qualtrics’ investment, which amounts to $125 million per year over the next four years, are exceptionally vague. It’s unclear how the tranche will be divided among the company’s business divisions — and which specific internal efforts it’ll fund, for that matter. We’ve asked for clarification.

But assuming it happens, Qualtrics’ investment is the latest example of a tech giant pouring huge amounts of capital into the exploding generative AI category.

Salesforce Ventures, Salesforce’s VC division, plans to funnel $500 million to startups developing generative AI technologies. VC firm Sapphire Ventures has set aside over $1 billion for enterprise AI startups. Workday recently added $250 million to its existing VC fund specifically to back AI and machine learning startups. And AWS a few weeks ago said that it aims to put $100 million into a program to fund generative AI initiatives.

Accenture and PwC, meanwhile, have announced that they plan to invest $3 billion and $1 billion, respectively, in AI, ahead of SAP’s investments in major AI players Anthropic, Cohere and Aleph Alpha.

McKinsey estimates that generative AI could add $4.4 trillion annually to the global economy, almost the economic equivalent of adding an entire new country the size and productivity of the U.K. ($3.1 trillion GDP in 2021) to the world. But other strategists say that the AI boom won’t lead to massive profits, warning that the hype mirrors that of the tech bubble of the 1990s.

Last month, Microsoft released Orca, a 13-billion parameter model, also touted as the open source, and a smaller alternative to GPT-4, that learns to imitate the reasoning processes of large language models. This small model learns from rich signals from GPT-4 including explanation traces; step-by-step thought processes, along with other complex instructions, guided by teacher assistance from ChatGPT.

At the time, the team had only released a preview of the model citing LLaMA’s release policy restrictions. With the recent LLaMA 2 commercial licence availability, released in partnership with Meta, we could expect the release of a powerful, smaller models soon.

The LlaMA 2 ranges from 7 to 70 billion parameters. According to Meta, they have shown superior performance compared to open-source chat models, LlaMA, Alpaca, and Vicuna, on most benchmarks tested.

Microsoft is offering LlaMA 2 for use on the Azure AI catalogue, allowing people to access it through cloud tools like content filtering. Additionally, the tool can run on Windows PCs.

Inside Orca

Developed by Microsoft, Orca is a 13 billion-parameter model that outperforms conventional open-source models like LlaMA, Alpaca, and Vicuna.

The authors of the paper highlight that previous models lacked rigorous evaluation, leading to an overestimation of their capabilities. In contrast, Orca was specifically designed to imitate the reasoning process of larger models through progressive learning.

To achieve this, Orca was trained to imitate the step-by-step thought processes of GPT-4, a much larger model. It received teacher assistance from GPT-3.5 through explanation traces. This allowed Orca to learn more efficiently and produce richer explanations. The authors used system messages and complex tasks from the FLAN collection to enhance the model’s performance.

Results from various benchmarks demonstrate Orca’s impressive capabilities. In complex zero-shot reasoning benchmarks, such as the Big Bench Hard, Orca surpasses vicuna by more than 100% and outperforms GPT-4 in specific reasoning tasks. In open-ended generation, Orca achieves 95% of ChatGPT’s quality and 85% of GPT-4’s quality.

The model also shows promise in academic and professional examinations like the SAT, LSAT, GRE, and GMAT. In the Big Bench Hard benchmark, which includes 23 of the hardest tasks for language models, Orca significantly outperforms previous open-source models and even matches ChatGPT’s performance.

The research highlights the importance of leveraging system instructions and step-by-step explanations for better model performance. Orca’s ability to learn from detailed responses and reasoning processes of GPT-4 and ChatGPT has proven crucial in its success.

Orca is not alone

Looks like Orca, has a new competitor. Recently, Alignment Lab AI unveiled OpenOrca-Preview1-13B, a smaller model that mimics the behaviour of large language models like GPT-4, which is very similar to Microsoft’s Orca.

In an attempt to reproduce the dataset generated for Microsoft Research’s Orca, the team have also used the OpenOrca dataset to fine-tune LLaMA-13B. “We have trained on less than 6% of our data, just to give a preview of what is possible while we further refine our dataset!,” shared team Alignment Lab AI, saying that they have trained a refined section of 200K GPT-4 entries from OpenOrca.
The team have further filtered out GPT-4 augmentations to remove statements like “As an AI language model…” and other responses which have been shown to harm model reasoning capabilities. It further said that their dataset curation practices will be forthcoming with their full model releases.

The team said that the preview release shows that even a smaller portion of their training data can produce SOTA results in this model class with training costs less than $200.

Why is Microsoft Betting on Smaller LLMs?

With the recent partnership with Meta, Microsoft is looking to be both at the start and finish line of the generative AI race. Their backing of the open source research projects like Orca and their extended partnership with OpenAI gives them all the benefits of a double edged sword.

Meta, along with Microsoft has also partnered with Qualcomm, eyeing an entire ecosystem to make LlaMA 2 AI implementations available on phones and PCs starting next year. Smaller models like Orca can take full advantage of this.

“This will allow customers, partners and developers to build use cases, such as intelligent virtual assistants, productivity applications, content creation tools, entertainment and more,” Qualcomm said on Tuesday. “These new on-device AI experiences, powered by Snapdragon, can work in areas with no connectivity or even in aeroplane mode.”

The post Will Microsoft Unleash an Army of Tiny LLMs with LLaMA 2? appeared first on Analytics India Magazine.

DataStax Rolls Out Vector Search for Astra DB to Support Gen AI July 19, 2023 by Jaime Hampton

DataStax just announced the general availability of its vector search capability in Astra DB, its DBaaS built on Apache Cassandra.

Vector search is a must-have capability for building generative AI applications. In machine learning, vector embeddings are the distilled representations of raw training data and act as a filter for running new data through during inference. Training a large language model results in potentially billions of vector embeddings.

Vector databases store these embeddings and perform a similarity search to find the best match between a user’s prompt and the vectorized training data. Instead of searching with keywords, embeddings allow users to conduct a search based on context and meaning to extract the most relevant data.

There are native databases specifically built to manage vector embeddings, but many relational and NoSQL databases (like Astra DB) have been modified to include vector capabilities due to the demand surrounding generative AI.

This demand is palpable: McKinsey estimates that generative AI could potentially add between $2.6 and $4.4 trillion in value to the global economy. DataStax CPO Ed Anuff noted in a release that databases capable of supporting vectors are crucial to tapping into the potential of generative AI as a sustainable business initiative.

“An enterprise will need trillions of vectors for generative AI so vector databases must deliver limitless horizontal scale. Astra DB is the only vector database on the market today that can support massive-scale AI projects, with enterprise-grade security, and on any cloud platform. And, it’s built on the open source technology that’s already been proven by AI leaders like Netflix and Uber,” he said.

DataStax says one advantage of vector search within Astra DB is that it can help reduce AI hallucinations. LLMs are prone to fabricating information, called hallucinating, which can be damaging to business. This vector search release includes Retrieval Augmented Generation (RAG), a capability that grounds search results within specific enterprise data so that the source of information can be easily pinpointed.

Data security is another factor to consider with generative AI deployment, as many AI use cases involve sensitive data. DataStax says Astra DB is PCI, SOC2, and HIPAA enabled so that companies like Skypoint Cloud Inc., which offers a data management platform for the senior living healthcare industry, can use Astra DB as a vector database for resident health data.

“Envision it as a ChatGPT equivalent for senior living enterprise data, maintaining full HIPAA compliance, and significantly improving healthcare for the elderly,” said Skypoint CEO Tisson Mathew in a statement.

To support this release, DataStax also created a Python library called CassIO aimed at accelerating vector search integration. The company says this software framework easily integrates with popular LLM software like LangChain and can maintain chat history, create prompt templates, and cache LLM responses.

The new vector search capability is available on Astra DB for Microsoft Azure, AWS, and Google Cloud. The company also says vector search will be available for customers running DataStax Enterprise, the on-premises, self-managed offering, within the month.

Matt Aslett of Ventana Research expects generative AI adoption to grow rapidly and says that through 2025, one-quarter of organizations will deploy generative AI embedded in one or more software applications.

“The ability to trust the output of generative AI models will be critical to adoption by enterprises. The addition of vector embeddings and vector search to existing data platforms enables organizations to augment generic models with enterprise information and data, reducing concerns about accuracy and trust,” he said.

This article first appeared on sister site Datanami.

Related

MongoDB today announced a significant expansion of its strategic partnership agreement with Microsoft, aimed at simplifying customers’ cloud adoption journeys. The collaboration will facilitate easier access to MongoDB Atlas through the Microsoft commercial marketplace, benefiting millions of developers who utilize the Azure portal.

Under this agreement, MongoDB will work closely with Microsoft to enhance the user experience of MongoDB Atlas on Azure, offering go-to-market initiatives, developer enablement programs, and training opportunities. The partnership also involves deeper technology integrations to better serve their mutual customers.

“Through this expanded collaboration with Microsoft, we’re making it easier for our customers to seamlessly integrate MongoDB Atlas into their Azure infrastructure to power the next generation of applications.” said Alan Chhabra, Executive Vice President, Worldwide Partners and Asia at MongoDB.

This strategic partnership builds upon recent collaborations between MongoDB and Microsoft, including the introduction of a pay-as-you-go MongoDB Atlas offering announced at Microsoft Ignite. MongoDB Atlas seamlessly integrates with various first-party Microsoft services such as Azure Synapse Analytics, Azure Event Hub, Microsoft Power BI, Microsoft Purview, Azure Logic Apps, and Microsoft Power Automate.

Furthermore, the collaboration will extend to include integration with Microsoft’s data, AI, analytics, and low-code services. MongoDB has also partnered with Microsoft for Startups to provide free MongoDB Atlas credits and other benefits as part of the Founders Hub offer. This partnership enables startups to leverage the speed, scalability, and security of MongoDB and Microsoft’s combined capabilities, facilitating their growth journey from ideation to fruition.

Earlier, MongoDB partnered with Google Cloud to enhance the adoption of generative AI and facilitate the creation of innovative applications. By leveraging its integrated operational data store, MongoDB Atlas uniquely supports the development of generative AI-powered applications with increased efficiency and simplicity.

The post MongoDB Partners with Microsoft for Cloud Adoption appeared first on Analytics India Magazine.

Meta, better known to most of us as Facebook, has released a commercial version of Llama-v2, its open-source large language model (LLM) that uses artificial intelligence (AI) to generate text, images, and code.

The first version of the Large Language Model Meta AI (Llama), was publicly announced in February and was restricted to approved researchers and organizations. However, it was soon leaked online in early March for anyone to download and use.

Meta incidentally filed take-down orders to sites such as GitHub and open-source AI group Hugging Face to corral the purloined program. Eventually, faced with the code being easily available across the web, Meta gave up trying to order the tide to go back. Instead, it embraced the release.

Also: Want to build your own AI chatbot? Say hello to open-source HuggingChat

Both versions of Llama have been trained on Common Crawl, GitHub, Wikipedia, Project Gutenberg, ArXiv, Stack Exchange, and other open test websites. While Microsoft and OpenAI's ChatGPT got the headlines, many open-source developers turned to Llama.

Besides having access to Llama, Meta also shared its weights. The other major LLMs haven't. With weights, the parameters learned by a model during training, it's much easier to create and run custom AI programs. The other big LLMs, such as GPT, are usually only accessible through application programming interfaces (API).

While AI is built on open-source foundations, Llama is the first major open-source LLM. Its pre-trained models have been trained on 2 trillion tokens, and have to double Llama 1's context length. Its fine-tuned models have been trained on over 1 million human annotations. Its model size parameters range from 7 to 70 billion parameters.

Also: Microsoft announces Azure AI trio at Inspire 2023

So, that's why open-source developers welcomed Llama, but why did Meta open Llama-v2 up? According to its researchers, "While many companies have opted to build AI behind closed doors, we are releasing Llama 2 openly to encourage responsible AI innovation. Based on our experience, an open approach draws upon the collective wisdom, diversity, and ingenuity of the AI practitioner community to realize the benefits of this technology. Collaboration will make these models better and safer."

But, is Llama-v2 actually open source? While Meta says the right things, the Llama 2 Community License Agreement has not been approved by the gold standard group of open-source licensing, the Open Source Initiative (OSI).

With that in mind, Meta has made Llama 2 available free of charge for research and commercial use. Meta is also including model weights and starting code for the pre-trained model and conversational fine-tuned versions. This gives developers a major step forward in putting Llama-powered applications to use.

Also: 5 ways to explore the use of generative AI at work

Strictly open-source or not, Llama 2 is certainly open enough for most practical purposes. And, as Amanda Brock, CEO of OpenUK, put it, it's "Not an OSI approved license but a significant release of Open Technology … This is a step to moving AI from the hands of the few to the many, democratizing technology and building trust in its use and future through transparency. No, it's not perfect, and yes, there is more work to be done, but this bold move sets the tone for AI-open innovation with a responsible but light-touch principles-based approach to regulating the use and development of AI."

Meta isn't just playing nice with the open-source community. Meta also declared Microsoft is its preferred Llama 2 partner. So, Llama 2 is available in the Azure AI model catalog, enabling developers using Microsoft Azure to build with it. The LMA is also optimized to run locally on Windows, giving developers a seamless workflow as they bring generative AI experiences to customers across different platforms. Llama 2 is available through Amazon Web Services (AWS), Hugging Face, and other providers.

Also: Microsoft's Inspire 2023: 10 AI and partnership announcements to know

Besides opening up the code, in this release, Meta is also trying to make Llama safer, more well-behaved, and less prone to hallucinations than the other models.

Meta has done this by Red-Teaming (Security Testing) Exercises designed to clean up safety weaknesses. Meta has also released a Developer's Guide for Safe and Responsible Use to help developers understand and apply the best practices for developing and responsible model testing. Finally, Meta's provided a Llama Acceptable Use Policy to prohibit certain use cases to help ensure that these models are being used fairly and responsibly.

The end result? Meta hopes to catch up and surpass OpenAI. Who knows, it may be able to do it. As a Google AI engineer recently wrote, "The uncomfortable truth is, we aren't positioned to win this [Generative AI] arms race, and neither is OpenAI. While we've been squabbling, a third faction has been quietly eating our lunch." That third group? The open-source community.

Featured

Most applications of artificial intelligence in medicine have failed to make use of language, broadly speaking, a fact that Google and its DeepMind unit addressed in a paper published in the prestigious science journal Nature on Monday.

Their invention, MedPaLM, is a large language model like ChatGPT that is tuned to answer questions from a variety of medical datasets, including a brand new one invented by Google that represents questions consumers ask about health on the Internet. That dataset, HealthSearchQA, consists of "3,173 commonly searched consumer questions" that are "generated by a search engine," such as, "How serious is atrial fibrillation?"

Also: Google follows OpenAI in saying almost nothing about its new PaLM 2 AI program

The researchers used an increasingly important area of AI research, prompt engineering, where the program is given curated examples of desired output in its input.

In case you were wondering, the MedPaLM program follows the recent trend by Google and OpenAI of hiding the technical details of the program, rather than specifying them as is the standard practice in machine learning AI.

The MedPaLM program saw a big leap when answering the HealthSearchQA questions, as judged by a panel of human clinicians. The percentage of times its predictions were in accord with medical consensus beat the 61.9% score for a variant of Google's PaLM language model, achieving 92.6%, just shy of the human clinician's average, 92.9%.

However, when a group of laypeople with a medical background were asked to rate how well MedPaLM answered the question, meaning, "Does it enable them [consumers] to draw a conclusion," 80,3% of the time MedPaLM was useful, versus 91.1% of the time for human physicians' answers. The researchers take that to mean that "considerable work remains to be done to approximate the quality of outputs provided by human clinicians."

Also: 7 advanced prompt-writing tips you need to know

The paper, "Large language models encode clinical knowledge," by lead author Karan Singhal of Google and colleagues, focuses on using so-called prompt engineering to make MedPaLM better than the other large language models.

MedPaLM is a derivative of PaLM-fed question-and-answer pairs provided by five clinicians in the US and UK. Those question- answer pairs, just 65 examples, were used to train MedPaLM via a series of prompt engineering strategies.

The typical way to refine a large language model such as PaLM, or OpenAI's GPT-3, is to feed it "with large amounts of in-domain data," note Singhal and team, "an approach that is challenging here given the paucity of medical data." Instead, for MedPaLM, they rely on three prompting strategies.

Prompting is the practice of improving model performance "through a handful of demonstration examples encoded as prompt text in the input context." The three prompting approaches are few-shot prompting, "describing the task through text-based demonstrations"; so-called chain of thought prompting, which involves "augmenting each few-shot example in the prompt with a step-by-step breakdown and a coherent set of intermediate reasoning steps towards the final answer"; and "self-consistency prompting," where several outputs from the program are sampled and a majority vote indicates the right answer.

Also: Six skills you need to become an AI prompt engineer

The heightened score of MedPaLM, they write, shows that "instruction prompt tuning is a data-and parameter-efficient alignment technique that is useful for improving factors related to accuracy, factuality, consistency, safety, harm, and bias, helping to close the gap with clinical experts and bring these models closer to real-world clinical applications."

However, "these models are not at clinician expert level on many clinically important axes," they conclude. Singhal and team suggest expanding the use of expert human participation.

"The number of model responses evaluated and the pool of clinicians and laypeople assessing them were limited, as our results were based on only a single clinician or layperson evaluating each response," they observe. "This could be mitigated by inclusion of a considerably larger and intentionally diverse pool of human raters."

Also: How to write better ChatGPT prompts

Despite the shortfall by MedPaLM, Singhal and team conclude, "Our results suggest that the strong performance in answering medical questions may be an emergent ability of LLMs combined with effective instruction prompt tuning."

Artificial Intelligence