Рубрика: AI

In a groundbreaking study, Cambridge scientists have taken a novel approach to artificial intelligence, demonstrating how physical constraints can profoundly influence the development of an AI system.

This research, reminiscent of the developmental and operational constraints of the human brain, offers new insights into the evolution of complex neural systems. By integrating these constraints, the AI not only mirrors aspects of human intelligence but also unravels the intricate balance between resource expenditure and information processing efficiency.

The Concept of Physical Constraints in AI

The human brain, an epitome of natural neural networks, evolves and operates within a myriad of physical and biological constraints. These limitations are not hindrances but are instrumental in shaping its structure and function. I

n the words of Jascha Achterberg, a Gates Scholar from the Medical Research Council Cognition and Brain Sciences Unit (MRC CBSU) at the University of Cambridge, “Not only is the brain great at solving complex problems, it does so while using very little energy. In our new work, we show that considering the brain's problem-solving abilities alongside its goal of spending as few resources as possible can help us understand why brains look like they do.”

The Experiment and Its Significance

The Cambridge team embarked on an ambitious project to create an artificial system that models a highly simplified version of the brain. This system was distinct in its application of ‘physical' constraints, much like those in the human brain.

Each computational node within the system was assigned a specific location in a virtual space, emulating the spatial organization of neurons. The greater the distance between two nodes, the more challenging their communication, mirroring the neuronal organization in human brains.

This virtual brain was then tasked with navigating a maze, a simplified version of the maze navigation tasks often given to animals in brain studies. The importance of this task lies in its requirement for the system to integrate multiple pieces of information—such as the start and end locations, and the intermediate steps—to find the shortest route. This task not only tests the system's problem-solving abilities but also allows for the observation of how different nodes and clusters become critical at various stages of the task.

Learning and Adaptation in the AI System

The journey of the artificial system from novice to expert in maze navigation is a testament to the adaptability of AI. Initially, the system, akin to a human learning a new skill, struggled with the task, making numerous errors. However, through a process of trial and error and subsequent feedback, the system gradually refined its approach.

Crucially, this learning occurred through alterations in the strength of connections between its computational nodes, mirroring the synaptic plasticity observed in human brains. What's particularly fascinating is how the physical constraints influenced this learning process. The difficulty in establishing connections between distant nodes meant the system had to find more efficient, localized solutions, thus imitating the energy and resource efficiency seen in biological brains.

Emerging Characteristics in the Artificial System

As the system evolved, it began to exhibit characteristics startlingly similar to those of the human brain. One such development was the formation of hubs – highly connected nodes acting as information conduits across the network, akin to neural hubs in the human brain.

More intriguing, however, was the shift in how individual nodes processed information. Instead of a rigid coding where each node was responsible for a specific aspect of the maze, the nodes adopted a flexible coding scheme. This meant that a single node could represent multiple aspects of the maze at different times, a feature reminiscent of the adaptive nature of neurons in complex organisms.

Professor Duncan Astle from Cambridge’s Department of Psychiatry highlighted this aspect, stating, “This simple constraint – it's harder to wire nodes that are far apart – forces artificial systems to produce some quite complicated characteristics. Interestingly, they are characteristics shared by biological systems like the human brain.”

Broader Implications

The implications of this research extend far beyond the realms of artificial intelligence and into the understanding of human cognition itself. By replicating the constraints of the human brain in an AI system, researchers can gain invaluable insights into how these constraints shape brain organization and contribute to individual cognitive differences.

This approach provides a unique window into the complexities of the brain, particularly in understanding conditions that affect cognitive and mental health. Professor John Duncan from the MRC CBSU adds, “These artificial brains give us a way to understand the rich and bewildering data we see when the activity of real neurons is recorded in real brains.”

Future of AI Design

This groundbreaking research has significant implications for the future design of AI systems. The study vividly illustrates how incorporating biological principles, particularly those related to physical constraints, can lead to more efficient and adaptive artificial neural networks.

Dr. Danyal Akarca from the MRC CBSU underscores this, stating, “AI researchers are constantly trying to work out how to make complex, neural systems that can encode and perform in a flexible way that is efficient. To achieve this, we think that neurobiology will give us a lot of inspiration.”

Jascha Achterberg further elaborates on the potential of these findings for building AI systems that closely mimic human problem-solving abilities. He suggests that AI systems tackling challenges similar to those faced by humans will likely evolve structures resembling the human brain, particularly when operating within physical constraints like energy limitations. “Brains of robots that are deployed in the real physical world,” Achterberg explains, “are probably going to look more like our brains because they might face the same challenges as us.”

The research conducted by the Cambridge team marks a significant step in understanding the parallels between human neural systems and artificial intelligence. By imposing physical constraints on an AI system, they have not only replicated key characteristics of the human brain but also opened new avenues for designing more efficient and adaptable AI.

We are living in a time of unprecedented change and innovation. Generative AI (GenAI) has created new horizons for us to explore the possibilities of AI in creating novel and diverse content. But the real revolution lies in the next step of the journey – autonomous analytics, an emerging category of analytics that can learn, adapt, and act with minimal human intervention as they interact with their environment. Autonomous analytics can bring transformative benefits to every aspect of society, including enhancing the availability and quality of healthcare, tackling environmental challenges, addressing transportation safety and bottlenecks, accelerating manufacturing excellence, expanding entertainment innovation, fostering social and economic equity, and much more (Figure 1).

Figure 1: Analytics Transformation: From Optimizing To Autonomous Analytics

What are Autonomous Analytics?

First, some definitions:

• Traditional analytics is the process of collecting, processing, analyzing, and visualizing data to generate insights and recommendations for decision-making. It requires human intervention to define the analysis’s goals and methods and supervise outcomes. Traditional analytics relies on rules and models that must be manually updated and adjusted as market, economic, business, and social conditions change. Examples of traditional analytics applications include Business Intelligence, regression analysis, association rules, clustering, segmentation, data mining, and machine learning.
• Generative AI (GenAI) is a type of AI that can create new data or content, such as text, images, music, code, etc. GenAI uses generative models that learn the probability distribution of the data they are trained on and can sample new data from that distribution. GenAI can be used for various purposes, such as data augmentation, content creation, and data analysis.
• Autonomous Analytics is a type of AI that can learn and adapt to its environment and make optimal decisions with minimal human intervention. Autonomous Analytics is often based on reinforcement learning (RL) that learns from experience and feedback. Autonomous Analytics can be used for various purposes, such as self-driving cars, robotics, complex games, and dynamic optimization problems.
• Artificial General Intelligence (AGI) is a hypothetical form of AI that can achieve or surpass human intelligence across all domains and tasks without being limited by specific goals or contexts. AGI can learn from any data and experience and can transfer its knowledge and skills to new situations it has not encountered before.

GenAI provided a massive leap forward in analytic capabilities unimagined one year ago. Autonomous analytics has the potential for an even more significant leap forward in the quest for Artificial General Intelligence (AGI) or “Super Intelligence.” Autonomous analytics provide the following benefits:

• Autonomous analytics can automatically discover the optimal methods, techniques, and pathways to achieve desired outcomes based on the data and feedback.
• Autonomous analytics dynamically update and adjust its rules and model weights based on the learning and adaptation process.
• Autonomous analytics can support dynamic, constantly changing, and complex operational situations.
• Autonomous analytics can adjust more quickly to changing operational conditions to deliver more accurate and relevant results and actions.
• Autonomous analytics leverages a real-time feedback process to learn and adapt from the results of every decision it makes.

The key to autonomous analytics is the analytics feedback loop. The analytics feedback loop assesses the results of the analytic outcomes (i.e., comparing predicted versus actual outcomes), identifies and codifies learnings from the outcome’s assessment, and feeds those learnings back to the analytics models to update and adjust the model’s weights automatically. This feedback loop enables the autonomous analytic models to learn from each decision and interaction, update the model parameters and weights based upon those learnings, and adapt to new environmental and operational situations with minimal human intervention (Figure 2).

Figure 2: Autonomous Analytics Feedback Loop

Autonomous Analytics can handle complex and dynamic operational situations, provide faster, more accurate, and more relevant outcomes, reduce human effort and error, and create new customer, product, service, and operational value creation opportunities.

What are some Autonomous Analytic Use Cases?

Maybe the best way to envision the potential of autonomous analytics is through a few industry use cases:

• Manufacturing: optimize the manufacturing process by monitoring the production data, detecting anomalies, predicting failures, and adjusting the parameters in real time.
• Healthcare: improve healthcare quality by analyzing medical data, diagnosing conditions, recommending treatments, and providing personalized medical advice and treatments to patients.
• Retail: enhance the customer experience by generating personalized content, such as emails, ads, and social media posts, based on customer profiles and preferences. It can also optimize the marketing strategy and improve customer retention by analyzing customer feedback and behavior.
• Transportation: enable autonomous vehicles by processing the sensor data, recognizing the objects, planning the routes, and controlling the actions. It can also reduce traffic congestion and accidents by coordinating the vehicles and optimizing the traffic flow.
• Agriculture: optimize crop yield and quality by monitoring the soil, weather, and plant data, detecting pests and diseases, predicting harvest time, and adjusting irrigation and fertilization.
• Energy: improve energy efficiency and reliability by analyzing the power grid data, forecasting demand and supply, detecting faults and outages, and controlling generation and distribution.
• Tourism: enhance the travel experience by generating personalized recommendations, such as destinations, activities, and hotels, based on the traveler’s data and preferences. It can also optimize the travel itinerary and budget by analyzing the travel data and feedback.
• Security: prevent and detect cyberattacks by processing network data, identifying anomalies and threats, predicting vulnerabilities, and taking countermeasures.
• Smart Cities: optimize urban services and infrastructure by monitoring the data from sensors, cameras, and IoT devices, detecting anomalies and emergencies, predicting traffic and demand, and adjusting the parameters and policies in real-time.
• Smart Hospitals: improve the quality and efficiency of healthcare by analyzing the data from medical records, devices, and wearables, diagnosing the conditions, recommending treatments, and providing personalized feedback and care to the patients.
• Smart Manufacturing: enhance the productivity and quality of manufacturing by analyzing the data from machines, processes, and products, detecting faults and failures, predicting maintenance and performance, and controlling operations and actions.

Yeah, it’s pretty cool what could be coming our way!

Summary: Autonomous Analytics

Generative AI has opened our eyes to the fantastic possibilities of AI in creating novel and diverse content. And the next big breakthrough lies in autonomous analytics, an emerging category of analytics that can learn, adapt, and act with minimal human intervention as it interacts with its environment.

Autonomous analytics is the ultimate goal of digital transformation, where organizations create a continuously learning and adapting culture, both AI-driven and human-empowered, that seeks to optimize AI-human interactions and leverage customer, product, and operational insights to create new value and opportunities. Traditional analytics, which focuses on reporting and optimizing specific use cases, are not enough to cope with the dynamic and complex challenges of the 21st century. Instead, we are witnessing the emergence of a new family of analytics that is more focused on learning and adapting than just optimizing.

Autonomous analytics can impact all aspects of our lives – healthcare, education, housing, employment, entertainment, safety, equity, and the environment – if we are rigorous in understanding the problem we are trying to solve, the desired outcomes, and the KPIs and metrics against which we will measure outcomes’ progress and success.

Like any new shiny tool, GenAI and Autonomous Analytics are a means to an end, not an end in themselves. As always, begin with that end in mind.

While the OpenAI fiasco came to an end (so, we believe) before Thanksgiving, people were quick to compare it to the Game of Thrones series with Sam Altman being Jon Snow. However, the real Altman was not a Snow who “knew nothing,” but was more like the courageous, skilled, intelligent archer Arjun from the epic Hindu mythology, Mahabharat. Interestingly, there are a number of people from the OpenAI drama that align with the multifaceted and complex characters from the Sanskrit epic.

In Mahabharat, when Pandavas fought the Kauravas (both being set of cousins- 5 against 100) in a huge war, there were a number of alliances and powerful characters that took sides. Similarly, OpenAI saw a similar plot (minus the battle and deaths) with allies, advisors and foes coming together. So, who plays who in the Mahabharat of OpenAI?

Arjun

Arjun is one of the central characters who led the Kurukshetra war from the Pandavas’ end, and can be depicted by none other than Sam Altman. Like Arjun, Altman’s intelligence and high capabilities makes him a natural leader, whose importance is incomparable. Similar to the epic, he has even undergone moments of moral dilemma, probably with AI safety, rallying for regulation of powerful models.

Krishna

The character that is considered the driver of the war and a literal driver to Arjuna’s chariot during the war, Lord Krishna is believed to be the vital force for all that ensued. While we may not be able to conclude that Microsoft Chief Satya Nadella had any part to play that led to the OpenAI fiasco, however, like Krishna, he stood beside Altman as his biggest supporter and best friend. Nadella, who was diplomatic when he mentioned that he would support Altman regardless of whether he stayed at Microsoft or OpenAI, is one of the striking personalities reminiscent of Krishna. He believes that outcomes are driven by people’s actions, and he sees himself as an enabler.

Yudhishthir

Righteousness, wisdom and loyalty being some of the most important traits of Arjuna’s older brother Yudhishthir, is reflective of co-founder Greg Brockman’s actions. After Altman’s ousting, and Brockman being removed from the board, like a true brother, he decided to quit the company and stand by Altman’s side. Like a just ruler who wanted to give clarity to the people, he was the only one who tweeted, explaining about the firing situation that unfolded at OpenAI’s board. After he was back, he even shared a picture with the whole OpenAI staff, showcasing his solidarity and support to his people.

Karna

The most debated character in Mahabharat was Karna. A virtuous man who had his reasons to take sides with Kauravas, Karna is a complex character who had a sad fate: was even framed as the bad guy. Ilya Sutskever was part of the 4-member board that united to oust Altman, but he had his reasons, which till date is not clarified. Misunderstood but powerful, Sutskever seems to tread Karna’s path.

Kunti

Displaying resilience, strength, calmness at the time of any adversity, Kunti has been a central character to bring unity. She has always taken the side of her Pandava children and has stood by their side adapting to any situation. However, being Karna’s birth mother too, she was torn to see him side with Kauravas, and always wished that all her children were united. When the ousting happened, CTO Mira Murati was made the interim CEO, however her support for Altman and Brockman did not fade.

Bheem, Nakul, Sahadev

Like the three Pandava brothers who stood by their brothers’ side, key researchers at OpenAI, Aleksander Madry, Jakub Pachocki, and Szymon Sidor quit the company shortly after Altman and Brockman’s exit. They decided to follow their brothers to pursue their next goal.

Shakuni and Duryodhan

Oldest brother and Kaurava leader Duryodhan was influenced by his maternal uncle Shakuni to take the path of wrongdoings and foster hatred towards his cousins. Though not comparable to the volumes of the tactics Duryodhan and Shakuni pulled that led to the war, the context of the OpenAI drama, Tasha McCauley and Helen Toner could fit these roles. They were crucial members that led to the ousting and were ultimately thrown out of board when Altman returned – similar to the vanquishing of Shakuni and Duryodhan at the end of the war.

Yuyutsu

Though on Kauravas’s side, Yuyutsu was the only brother of Kaurava who sided with the Pandavas. Adam D’Angelo who was part of the old board that ousted Altman, is also the only member who is part of the newly appointed OpenAI board. Thus, indicating his trusted allegiance to Altman and team.

Bhishma

As the wise, able, loyal and strong central character of Mahabharat, the Bhishma of OpenAI fiasco could probably be the CEO of Airbnb, Brian Chesky. Akin to Bhishma with his profound quotes, after the OpenAI fiasco ended, Chesky said that ‘you learn a lot about people in a crisis,’ possibly referring to the behaviour of different stakeholders during the drama. Though not directly involved in the debacle, his unwavering support to his long-time friend Altman was evident. He even sided with Altman during the discussions that led to reinstating Altman as CEO.

Dhritarashtra

The blind king and father of Kauravas, Dhritarashtra was unable to rule the kingdom. Like the titular position, Emmet Shear’s presence in OpenAI can be equated to Dhritarashtra’s. The CEO who lasted for only 72 hours had to resign his position when Altman was reinstated. In Mahabharat, after the war ended, Dhritarashtra had to give up his position too.

Eklavya

Ekalavya, an aspiring archer who used to self-train by watching Dronacharya teach his students (Kauravas and Pandavas), was asked to cut off his thumb as an offering, so as to prevent him from surpassing the royal sons’ capabilities. Though Elon Musk, a brilliant entrepreneur and visionary, did not have to cut any part of his body, his bitterness after leaving OpenAI in 2018 has been evident over the last year, especially with the meteoric rise the company witnessed with ChatGPT.

Though he voluntarily left OpenAI in 2018, citing a conflict of interest with his company Tesla, it was recently reported that Musk left, owing to a failed attempt of taking over OpenAI. He did not miss a chance to take a dig at OpenAI during the Altman fiasco.

Drupad

Drupad, the king of Panchala and father of Draupadi, who belonged to a royal lineage, fought alongside the Pandavas during the war. He was believed to be a calculative king who built strategic alliances that served both parties’ interests. Similarly, VC and businessman, Vinod Khosla, one of the first investors in OpenAI’s for-profit subsidiary, had a lot at stake during Altman’s ousting. He ensured that he supported Altman and praised his capabilities. Khosla also dissed Emmett on X.

Gandhari

The capable queen who chose to be blind in solidarity to her husband Dhritrarashtra, ended up siding with the wrong, owing to her ‘blind love’ for her Kaurava children. While Ashneer Grover had no part to play in Altman’s ousting, his blindness to the whole OpenAI situation and incorrectly comparing it to his ousting from his company, and even giving two cents to Altman on how the company will go behind his shares (which Altman doesn’t have), would make him fit the bill of the blind queen. He later deleted his tweets, but the damage was done.

In this article, the comparison between OpenAI and Mahabharat characters aim for thematic exploration rather than exact replication.

The post The Mahabharat of OpenAI appeared first on Analytics India Magazine.

Altman Returns Amid Swirl of Questions About Project Q-Star November 24, 2023 by Alex Woodie

Sam Altman’s wild weekend had a happy ending, as he reclaimed his CEO position at OpenAI earlier this week. But questions over the whole ordeal remain as rumors of a powerful new AI capability developed at OpenAI called Project Q-Star are swirling.

Altman returned to OpenAI after a tumultuous four days in exile. During that time, Altman nearly reclaimed his job at OpenAI last Saturday, was rebuffed again, and the next day took a job at Microsoft, where he was to head an AI lab. Meanwhile, the majority of OpenAI’s 770 or so employees threatened to quit en masse if Altman was not reinstated.

The employee’s open revolt ultimately appeared to convince OpenAI Chief Scientist Ilya Sutskever, the board member who led Altman’s ouster–reportedly over concerns that Altman was rushing the development of a potentially unsafe technology–to back down. Altman returned to his job at OpenAI, which reportedly is worth somewhere between $80 billion and $90 billion, on Tuesday.

Just when it seemed as if the story couldn’t get any stranger, rumors started to circulate that the whole ordeal was due to OpenAI being on the cusp of releasing a potentially groundbreaking new AI technology. Dubbed Project Q-Star (or Q*), the technology purportedly represents a major advance toward artificial general intelligence, or AGI.

Sam Altman, CEO of OpenAI (left), and Microsoft CEO Satya Nadella

Reuters said it learned of a letter wrote by several OpenAI staffers to the board warning them of the potential downsides of Project Q-Star. The letter was sent to the board of directors before they fired Altman on November 17, and is considered to be one of several factors leading to his firing, Reuters wrote.

The letter warned the board “of a powerful artificial intelligence discovery that they said could threaten humanity,” Reuters reporters Anna Tong, Jeffrey Dastin and Krystal Hu wrote on November 22.

The reporters continued:

“Given vast computing resources, the new model was able to solve certain mathematical problems, the person said on condition of anonymity because the individual was not authorized to speak on behalf of the company. Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said.”

OpenAI hasn’t publicly announced Project Q-Star, and little is known about it, other than that it exists. That, of course, hasn’t stopped rampant speculation about its supposed capabilities on the Internet, particularly around a branch of AI called Q-learning.

(SuPatMaN/Shutterstock)

The board intrigue and AGI tease come on eve of the one-year anniversary of the launch of ChatGPT, which catapulted AI into the public spotlight and caused a gold rush to develop bigger and better large language models (LLMs). While the emergent capabilities of LLMs like GPT-3 and Google LaMDA were well-known in the AI community before ChatGPT, the launch of OpenAI’s Web-based chatbot supercharged interest and investment in this particular form of AI, and the buzz has been resonating around the world ever since.

Despite the advances represented by LLMs, many AI researchers have stated that they do not believe humans are, in fact, close to achieving AGI, with many experts saying it was still years if not decades away.

AGI is considered to be the Holy Grail in the AI community, and marks an important point at which the output of AI models is indiscernible from a human. In other words, AGI is when AI becomes smarter than humans. While LLMs like ChatGPT display some characteristics of intelligence, they are prone to output content that is not real, or hallucinate, which many experts say presents a major barrier to AGI.

Editor's note: This article originally appeared on Datanami.

Related

About the author: Alex Woodie

Alex Woodie has written about IT as a technology journalist for more than a decade. He brings extensive experience from the IBM midrange marketplace, including topics such as servers, ERP applications, programming, databases, security, high availability, storage, business intelligence, cloud, and mobile enablement. He resides in the San Diego area.

Two-Thirds of Organizations Already Using GenAI, O’Reilly Says November 24, 2023 by Ali Azhar

(JLStock/Shutterstock)

Large language and image AI models, also referred to as generative artificial intelligence (GenAI), have been the big story of 2023. It has opened a new set of opportunities for professionals and businesses. While most businesses acknowledge the potential for GenAI, however, there are also some concerns about its use. In enterprises, we’ve seen a wide range of opinions about GenAI, ranging from wholesale adoption to severely restricted or even forbidden use.

O’Reilly, the premier source for insight-driven learning in technology and business, recently conducted a study of more than 2,800 technology professionals from various industries around the globe to uncover the realities of GenAI use for enterprises. The findings of the report reveal the rapid rise of GenAI, bottlenecks to AI adoption, and the skills needed to move forward with these technologies.

According to Mary Treseler, chief content officer at O’Reilly, GenAI offers a lot of opportunities for enterprises however “Without the proper talent in place to manage it, this rapidly evolving technology can quickly outpace enterprise resources. As this groundbreaking report unveils, we are far from reaching the peak of what generative AI can achieve, and organizations still have time to invest in the critical skills development required to be at the forefront of the AI revolution.”

One of the key findings of the report is that GenAI has seen rapid adoption, more than any other technology in recent times. Two-thirds (67 percent) of companies are currently using GenAI and over a third (38 percent) have been working with AI for less than a year. This is contrary to Gartner, who reported that AI is close to reaching the peak of its inflated expectations. The result of the O’Reilly report indicates there is plenty more headroom.

The O’Reilly report shows that 54 percent of AI users believe that AI tools will lead to better productivity, but only 4 percent believe that it would result in lower head counts. The most commonly used applications for AI include programming (77 percent), data analysis (70 percent), and customer-facing applications (70 percent).

The increased GenAI adoption has enabled enterprises to train models more easily and deploy more complex applications on those models. Even with the rapid adoption, many enterprises are still in the early stages with 18 percent of respondents reporting having applications in production.

While 23 percent of respondents are using one of the GPT models, enterprises are also building on top of open-source models. This indicates an active and vital world beyond GPT.

Enterprises are facing multiple bottlenecks that are restricting faster adoption. Chief constraints are the challenges in identifying appropriate use cases (53 percent), followed by legal issues, risk, and compliance (38 percent).

Only 4% of respondents to O'Reilly's survey say they believe AI will reduce head count (FGC/Shutterstock)

While the accelerated adoption of GenAI has created a demand for technology workers, a significant skill gap remains. The most needed skills include AI programming (66 percent), data analysis (59 percent), and AI/ML operations (54 percent). Unexpected outcomes, security, safety, fairness and bias, and privacy are the biggest risks for which adopters are testing.

“The adoption of generative AI is certainly explosive, but if we ignore the risks and hazards of hasty adoption, it is certainly possible we can slide into another AI winter,” said Mike Loukides, vice president of content strategy at O’Reilly and author of the report. “By taking a pragmatic approach versus rushing into production, investing in training and resources, and thinking creatively about how to put AI to work, enterprises have an enormous opportunity in front of them. As the report concludes, ‘AI won’t replace humans, but companies that take advantage of AI will replace companies that don’t.”

The O’Reilly report is further proof that enterprises are optimistic about GenAI’s future. However, there are some concerns related to security, bias, correctness, and fairness. Some early adopters who ignore these risks are likely to suffer consequences. To close the AI skills gap, companies will have to invest heavily in training for both software developers and AI users. White AI won’t be replacing humans anytime soon, those who can integrate AI into their work will benefit the most.

Editor's note: This article originally appeared on Datanami.

Related

About the author: Alex Woodie

Alex Woodie has written about IT as a technology journalist for more than a decade. He brings extensive experience from the IBM midrange marketplace, including topics such as servers, ERP applications, programming, databases, security, high availability, storage, business intelligence, cloud, and mobile enablement. He resides in the San Diego area.

Inside Microsoft’s AI Announcements at Ignite November 24, 2023 by Alex Woodie

From new copilots and AI development tools, to vector search and AI chips, artificial intelligence featured prominently in Microsoft’s annual Ignite developers conference held last week. It also unveiled some data news around OneLake and Microsoft Fabric.

It would be an understatement to say that Microsoft is bullish on copilots. “Microsoft is the Copilot company,” the company claims, “and we believe in the future there will be a Copilot for everyone and for everything you do.”

To that end, the company made a slew of copilot-related announcements and updates at Ignite 2023. For starters, it announced the general availability of Copilot for Microsoft 365, which it originally unveiled in March.

Since early adopters first started working with Copilot for Microsoft 365, Microsoft has made several additions, including a new dashboard that shows what the copilot is doing, new personalization capabilities, and new whiteboarding and note-taking capabilities in Copilot for Outlook. Additional updates have been added for Copilots for PowerPoint, Excel, and Microsoft Viva.

Outlook is getting a Copilot too

There’s also a new Copilot for Service, which is targeted at customer service professionals. Security Copilot, which it launched earlier this year, will play a prominent role in the system resulting from the combination of Sentinel security analytics and Microsoft Defender XDR platforms.

Copilot for Azure, meanwhile, serves as an AI companion for cloud administrators. “More than just a tool,” Microsoft declares, “it is a unified chat experience that understands the user’s role and goals, and enhances the ability to design, operate and troubleshoot apps and infrastructure.”

The company also rolled out Copilot Studio, a low-code tool designed to allow Microsoft 365 users to build their own custom copilots and connect them to business data. Its Bing Chat and Bing Chat Enterprise offerings have been replaced with (you’ll never guess) Copilot. “When you give Copilot a seat at the table,” the company says, “it goes beyond being your personal assistant to helping the entire team.”

Organizations that use Microsoft Teams to collaborate will soon be able to spin up 3D virtual meeting places using GenAI. Microsoft says its Teams customers will be able to request the creation of 3D meetings and objects using its AI Copilot system. The virtual reality (VR) version of Teams is due in January.

OpenAI and Nvidia Partnerships

Microsoft has a close partnership with OpenAI and is invested in the company. All of the recent new capabilities that OpenAI announced two weeks ago at its DevDay event–such as GPT-4 Turbo and GPSs apps–will be offered by Microsoft via Azure OpenAI Service too.

“As OpenAI innovates, we will deliver all of that innovation as part of Azure OpenAI,” Microsoft CEO Satya Nadella said.

As far as the timeline goes, GPT-3.5 Turbo model with a 16K token prompt length will be generally available soon, and GPT-4 Turbo will be available by the end of the month. GPT-4 Turbo with Vision will soon be available as a preview.

Another partner critical for Microsoft’s ambitions is Nvidia. The GPU chipmaker and the software giant unveiled that its new AI foundry service, which will include Nvidia tools like AI Foundation Models, NeMo framework, and DGX Cloud AI supercomputing, will be available on Azure.

Nvidia CEO Jensen Huang joined Microsoft CEO Nadella on stage. “You invited Nvidia’s ecosystem, all of our software stacks, to be hosted on Azure,” Huang said. “There’s just a profound transformation in the way that Microsoft works with the ecosystem.”

AI Development

The company made several announcements around AI development, including rolling out Azure AI Studio, which the company describes as a “hub” for exploring, building, testing, and deploying GenAI apps, or even your own custom copilots.

The company also unveiled a new offering called Windows AI Studio that allows developer to build and run AI models directly on the Windows operating system. Windows AI Studio will allow developers to access and play with a variety of language models, such as its own Microsoft Phi, Meta’s Llama2, and open source models sourced from Azure AI Studio or Hugging Face.

It also rolled out Model-as-a-Service, which will give developers access to the latest AI models from its model catalog. AI developers will be able to Llama 2, upcoming premium models from Mistral, and Jais from G42, as an API endpoint, the company says.

Vector Search, which is a feature of Azure AI Search, is now generally available, the company says. It also added a new “prompt flow” capability to Azure Machine Learning. This will “streamline the entire development lifecycle” of GenAI and LLM apps, the company says.

New Chips

Microsoft CEO Satya Nadella holding up a new Arm chip (Image source: Microsoft)

Microsoft unveiled a new Arm-based CPU this week. Dubbed the Azure Cobalt, the new chip is 40% faster than the commercial Arm chips it currently uses, the company says. The Azure Cobalt will be offered exclusively in the Azure cloud and is designed for cloud workloads.

It also announced Azure Maia, which it calls an “AI accelerator chip” that’s designed to run cloud-based training and inferencing for AI workloads such as OpenAI models, Bing, GitHub Copilot and ChatGPT.

Some Data Stuff Too

It wasn’t all models all of the time at Ignite. Data, after all, lies at the heart of AI, and Microsoft made some data-related announcements at Ignite.

For instance, it announced that Microsoft Fabric OneLake, which it announced earlier this year, is available as a data store in Azure Machine Learning. The company says this will make it easier for data engineers to share “machine learning-ready data assets developed in Fabric.”

Microsoft announced the GA of Azure Data Lake Storage Gen2 (ADLS Gen2) “shortcuts,” which will allow data engineers “to connect to data from external data lakes in ADLS Gen2 into OneLake through a live connection with target data.”

The company also supports “Amazon S3 shortcuts” in OneLake, which it says will allow customers to “create a single virtualized data lake” that spans Amazon S3 buckets and OneLake, thereby eliminating the latency involved with copying data.

You can access Microsoft’s full slate of AI news from Ignite 2023 here. The full “book ‘o news,” including all 100 product announcements made at the show, is available here.

Editor's note: This article originally appeared on Datanami.

Related

About the author: Alex Woodie

Alex Woodie has written about IT as a technology journalist for more than a decade. He brings extensive experience from the IBM midrange marketplace, including topics such as servers, ERP applications, programming, databases, security, high availability, storage, business intelligence, cloud, and mobile enablement. He resides in the San Diego area.

What's the Black Friday deal?

Amazon dropped the price for the AI-powered Eufy X9 Pro robot vacuum and mop to only $650 as part of its Black Friday deals.

Why this deal is ZDNET-recommended

The Eufy Clean X9 Pro CleanerBot, a new 2-in-1 robot vacuum, boasts a deep cleaning, hands-free mopping experience, coupled with 5,500pa of suction power. It also uses some AI navigation features to maneuver throughout your house.

Also: Best robot vacuum deals: Get a Roomba or Shark on sale now

Initially, I was less than enthusiastic about trying out yet another robot mop vacuum (I'd tested a similar one recently), but once I watched the Eufy X9 Pro work its way across my home floors, my mind was changed.

ZDNET RECOMMENDS

Eufy Clean X9 Pro CleanerBot

This is the perfect robot vacuum and mop for homes with hard floors, even if there are carpets and rugs in between.

View at Amazon

The CleanerBot truly lives up to the name, outperforming my old Roborock and the Yeedi MopStation Pro in vacuum and mop functions. The suction power, 5,500pa at maximum capacity, is outstanding. And the main brush is bristle-less, made of silicone wedges instead that are just as effective at cleaning floors.

In my limited experience (as I've only tested this model for about a week), the primary silicone brush makes it less likely for the X9 Pro to get tangled, as it's easier to scoop debris up than sweep it.

The mopping function on the Eufy X9 Pro CleanerBot is one of the two features that impressed me the most. The X9 Pro has two rotating mop pads — which I love in a robot vac/mop combo — which put 2.2 lbs of downward pressure to break down tough stains, a particularly useful feat for my home of children and pets.

Review: Roborock S8 Pro Ultra: This 2-in-1 vacuum can do just about everything

The other outstanding feature, and probably my favorite, is the use of AI for navigation, obstacle avoidance, and mapping. The CleanerBot has time-of-flight sensors and an AI camera system, called AI See, that helps detect and avoid objects so the vacuum doesn't suck up your kids' socks or stuffed animals.

It also uses iPath Laser Navigation to create maps of your home, which separates the rooms by color in the Eufy Clean app and even shows you the obstacles that the robot has found in each room. When you review the map after cleaning, you'll find things like power cords, shoes, and trash cans marked on the map.

Eufy isn't the first to use this technology for obstacle avoidance and mapping, but it is a great feature. I hate having to pick up every last bit of paper my kids dropped before I can start cleaning — only to have the robot vacuum get stuck anyway on a power cord somewhere.

Also: This robot vacuum has a brilliant self-cleaning feature I didn't know I needed

The Eufy Clean app lets you customize settings for charging, cleaning intensity, voice, and more. And it also enables you to choose from the rooms that the robot automatically created on the map so you can send it to clean just that area, like a muddy entryway. You can choose to clean zones as small as 1.6 ft by 1.6 ft on the map in case of spills.

Beyond the AI See camera set, the CleanerBot has a sensor to detect floor types in case you're running the X9 Pro in vacuum and mopping mode and it reaches a carpet or a rug. Once the robot detects a rug or carpet, it raises the mop pads to keep them off the mat and only vacuums on the soft surface.

Also: Best robot vacuums you can buy right now

Here's another thing I was glad to see: The X9 returns dutifully to its station to wash the mop pads rather than wait until they're overdue for a cleaning. I don't want to see my robot mop dragging dry, dirty mopping pads minutes after it should've returned for a refresh, but I haven't found this to be a problem with the X9.

ZDNET's buying advice

The Eufy Clean X9 Pro CleanerBot is available for sale at $650 and is the perfect option for someone looking for a robot vacuum and mop combination for a home with a lot of hard floors, whether that's tile or hardwood, with some carpet or rugs mixed in.

It doesn't have a self-emptying dustbin, and the dustbin itself has to be emptied after each cleaning as it's pretty tiny. Still, the mopping feature and the suction power are impressive, especially as the mop can pick up stains and dirt that my Yeedi MopStation Pro left behind.

Featured reviews

Image by Author Introduction

Data visualization is a crucial skill for anyone working with data. But creating beautiful, informative data viz can be time-consuming and require specialized tools. That's where ChatGPT comes in. With its latest updates, ChatGPT makes data visualization faster and easier than ever before.

The latest update has improved the ChatGPT experience significantly. Now, instead of having to switch between different options like the original GPT-4, GPT4 with advanced analysis, or DALLE-3, you simply need to type in a prompt, and ChatGPT will automatically interpret your request and generate the desired results.

Image from ChatGPT

In this blog post, we'll explore how to instantly generate various data visualizations using plain English prompts. Thanks to ChatGPT's advanced data analysis, you don't need to process the data or run the Python code. We'll walk through simple pie and bar charts, then tackle more complex visualizations using real-world datasets.

Simple Visualization

In this part, we will write a simple prompt to generate plots. The prompt includes data in the form of a Python dictionary.

Pie Chart

Before we create a prompt, please ensure that you are using the GPT-4 model, as it is the only one that supports generating visualizations.

We will write a prompt to generate a pie chart visualization based on various nutrient data. Additionally, we have requested that ChatGPT use a lighter color combination, as the default colors are pretty bright.

Prompt:  Generate a pie chart of values {"Vitamin A":5, "Vitamin B": 1, "Vitamin C": 4, "Water": 90} to keep the color combination light.

As you can see, we got great results.

If you want to see the Python code behind the visualization, you have to click on the terminal logo at the end of the result.

After that, a window will appear containing the source code that you can modify and execute on your own. However, this step is not mandatory, as ChatGPT will simply run the code and display the visualizations as images. You can save these images for your presentation or report.

Bar Chat

In the next part, we provide CO2 emission data for the car and let ChatGPT do the magic.

Prompt:  Generate a bar plot co2 emissions of values {"Car A":30, "Car B": 25, "Car C": 20}.

It has added the title, x and y labels, and ensured descending order. Perfect!!!

Exploratory Data Analysis

Instead of excessively controlling ChatGPT's output, you can ask it to create results independently, similar to various Python AutoViz libraries. By simply providing the dataset and requesting a complete exploratory data analysis to generate the necessary plots for you to review.

In our case, we are providing it with a Customer Shopping Trends dataset that offers valuable insights into consumer behavior and purchasing patterns.

Prompt:  Perform exploratory data analysis on customer shopping trends dataset and display only plots.

The ChatGPT delivered quick results, processing and analyzing consumer trends in under a minute, a task that typically takes at least 30 minutes for me to code and run.

You can improve the results by providing follow-up prompts regarding the type of visualization you are interested in.

Prompt:  Improve the analysis by plotting  a correlation chart, bar chart, pie chart, boxplot, and relplot.

If you want to see multilevel complex visualization, you have to ask ChatGPT for it specifically.

Prompt:  Use the dataset to plot various complex visualizations.

Model Evaluation

Data visualization plays a crucial role in evaluating models. In this section, we will be using the Diabetes Dataset from Kaggle and ask ChatGPT to train and evaluate multiple models. To make the most out of ChatGPT's capabilities, we will request it to display a confusion matrix, precision-recall, and a chart comparing different models.

Prompt:  Multiple machine learning models should be trained using the target column "Outcome", and the resulting model evaluation visualization should include a confusion matrix, precision-recall, and model comparison chart.

It is evident that ChatGPT has performed exceptionally well. Although the models didn't perform well on the dataset, we are impressed with its fast and accurate data visualization capabilities. It can be used to quickly analyze datasets or answer questions during interviews or take-home assignments.

Conclusion

ChatGPT has revolutionized how we can create data visualizations with ease. With its advanced data analysis capabilities, you can generate stunning and informative data viz in seconds using simple English prompts.

In this post, we have learned how ChatGPT can instantly produce various plots like pie charts, bar graphs, correlation matrices, and even complex visualizations like relplots on request.

ChatGPT also exceeded expectations when asked to train ML models on the diabetes dataset and generate evaluation metrics and comparison plots. The entire model building and visualization process took barely a minute.

Whether you need a simple bar chart, advanced model analysis, or just a quick way to understand datasets, ChatGPT delivers exceptional results with minimal effort. With capabilities improving every day, it's an exciting time to level up your data viz skills using this AI assistant.

Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in Technology Management and a bachelor's degree in Telecommunication Engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.

More On This Topic

• How to Create Stunning Web Apps for your Data Science Projects
• How to Process a DataFrame with Millions of Rows in Seconds
• How to Use Kafka Connect to Create an Open Source Data Pipeline for…
• Amazon Web Services Webinar: Leverage data sets to create a…
• How to Create a Sampling Plan for Your Data Project
• Data Science Project Infrastructure: How To Create It

Back in March, a glitch within an open-source library knocked OpenAI’s ChatGPT offline and inadvertently gave a few users access to others’ chat titles and ChatGPT Plus subscribers’ payment information. Redis stepped in to troubleshoot for OpenAI, quickly restoring service and communication and keeping the conversation going.

Traditionally known for its simple key-value pair data handling, the company has expanded its capabilities to include vector search functionality in the latest Redis 7.2 release.

“This addition addresses the inefficiency of using key-value pairs for vector embeddings, as users often need to search across these embeddings to analyse the relative distance between a query and the stored data,” said Tim Hall, chief product officer, in an exclusive interaction with AIM. Thus, Redis now supports not just the storage but also the effective searching of vector data.

Relevance of Vector Db in the Generative AI Era

“We’ve also been advancing our ecosystem integration, including collaborating with LangChain. This led to Harrison’s OpenGPT on Rediscloud, leveraging our vector embedding and search capabilities,” said Hall, highlighting that Rediscloud exemplifies how developers can build generative AI applications and recommendation systems, especially when real-time and interactive responsiveness is needed.

OpenGPTs, an open-source initiative by LangChain, offers a flexible approach to generative AI, allowing users to select models, control data retrieval, and manage data storage.

Redis has a suite of premium customers like X (formerly Twitter), Stack Overflow, Snapchat and Craigslist, among others, for its versatility, high performance, ease of use, and customisation options, particularly suited for AI applications. Its enterprise-grade solutions offer robust security certifications and reliable handling of new data structures, like vector embeddings.

Discussing the same, Hall emphasised the practicality and flexibility of Redis for AI applications, particularly in handling vector embeddings for retrieval tasks with an example. The Retrieval Augmented Generation (RAG) framework showcased in the image embodies this by using Redis alongside OpenAI’s embedding layer.

Regardless of type, data is converted into vector embeddings and stored in Redis’s vector database, which is then queried to find relevant information on asking a question. This efficient, real-time process, which eliminates the need for fine-tuning with sensitive data, underscores Redis’s capabilities for rapid and secure AI development, perfectly aligning with use cases like document analysis and chatbot interaction.

In the space of generative AI, which is swiftly becoming a staple in numerous applications, the necessity for databases capable of managing intricate and real-time data is paramount. Redis is ideal for such real-time requirements, particularly storing and searching vector embeddings in real-time applications.

So, will vector databases be irrelevant soon because of peaking generative AI models? Short answer: no.

“I don’t think vector databases will become irrelevant quickly. It’s typical for vendors to expand their capabilities, but they will still need to have these under your control. You should be wary of ceding too much control to someone else, especially in this space,” he commented.

Solving Multiple Problems at One Go

In dialogue with customers, Redis addresses the nuanced demands of adopting LLMs. Businesses are increasingly interested in utilising LLMs to automate customer support and enhance interactive retail experiences. Such AI implementations can efficiently handle complex queries, streamlining customer interactions and potentially boosting sales while reducing costs associated with human support staff.

However, the deployment of LLMs has its challenges. “A primary concern among customers is hallucinations, where the AI provides irrelevant or incorrect responses for which we have created a hybrid search mechanism that merges vector embeddings with metadata, which refines search results and keeps responses relevant,” Hall explained.

A further concern is the control and privacy of proprietary data during the embedding process. Customers fear their unique data might be utilised without consent for other model training. Redis’s solution is a vector database that allows for constructing bespoke, private AI systems akin to having a personal ChatGPT. This empowers users with the autonomy to control their data, ensuring privacy and preventing unauthorised usage.

Growth of Database Solutions Market

As for the evolution of the database solutions market, Hall foresees a shift towards more unified and flexible systems. The past decade has transitioned from conventional relational databases to specialised solutions for specific data types or tasks, such as JSON documents and time series data. The inclusion of vector databases for vector data management is a recent trend, becoming increasingly significant with the rise of AI and machine learning.

However, Hall notes that the sector has reached a critical juncture where the operational complexity of juggling multiple specialised databases for a singular application is no longer tenable.

There are slower databases that might be suitable if real-time interactivity is optional. “The key is determining whether one database can satisfy most of your use cases, especially if you deal with one, two, or three different kinds of data. Everyone is trying to find the sweet spot for their specific use case and the problem they’re trying to solve, considering the vast array of data platforms available—around 450 globally. Our specific focus at Redis is on real-time solutions,” Hall explained.

Hall’s prognosis for the future of the database market is one of consolidation and integration, a departure from the previous era of rapid expansion and specialisation. This trend reflects a broader industry movement towards enhancing operational efficiency and diminishing complexity, especially as generative AI matures and integrates more deeply into the technological fabric.

India as a Market

Since 2020, Redis has doubled its headcount within India, expanded its presence across the country (Delhi, Mumbai, and Pune), and opened its first dedicated regional office in Bangalore in 2019. Groww, Apna, AngelOne, Purplle, Motilal Oswal and Zee are among the company’s customers in India.

Sharing a personal anecdote on economic growth, all humorously compared India’s surging GDP to the early internet days in the U.S., where the digital boom was just beginning to light up screens. He fondly recounts how his English friends were once mystified by the concept of ’24/7’

Fast forward, and it’s India’s turn, with its tech leapfrog driven by a mobile device explosion, ushering in a 24/7 economy where services never sleep. Redis, seizing the moment, is powering a myriad of India’s mobile applications, banking on the country’s digital transformation to fortify its own growth in this dynamic market.

“India presents a tremendous opportunity for interactive applications, particularly those that Redis can power. We’re working with various organisations in India, especially in the banking and entertainment sectors, to power their mobile applications. India is an exciting and vital market for us right now,” concluded Hall.

The post How Redis is Fueling the Generative AI Wave appeared first on Analytics India Magazine.