Рубрика: AI

Image by Editor

I’ve been reading, writing, and speaking since late last year on the intersection of open source software and machine learning, trying to understand what the future might bring.

When I started, I expected that I would be talking mostly about how open source software is used by the machine learning community. But the more I’ve explored, the more I’ve realized that there are a lot of similarities between the two areas of practice. In this article I’ll discuss some of those parallels — and what machine learning can and can’t learn from open source software.

Building, from the Beginning

The easy and obvious parallel is that both modern machine learning and modern software are built almost entirely with open source software. For software, that is compilers and code editors; for machine learning, it is training and inference frameworks like PyTorch and TensorFlow. These spaces are dominated by open source software, and nothing appears ready to change that.

There is one notable, apparent exception to this: all of these frameworks depend on the very proprietary Nvidia hardware and software stack. This actually is more parallel than it might look at first. For a long time, open source software ran mostly on proprietary Unix operating systems, sold by proprietary hardware vendors. It was only after Linux came along that we began to take for granted that an open “bottom” of the stack was even possible, and much open development is done these days on MacOS and Windows. It is unclear how this will play out in machine learning. Amazon (for AWS), Google (for both cloud and Android), and Apple are all investing in competing chips and stacks, and it’s possible that one or more of those could follow the path laid by Linus (and Intel) of freeing the entire stack.

Training Data: The New Accidental Supply Chain?

A more critical parallel between how open source software is built, and how machine learning is built, is the complexity and public availability of the data that each are built on.

As detailed in this preprint paper "The Data Provenance Project,” which I co-authored, modern machine learning is built on literally thousands of data sources, just as modern open source software is built on hundreds of thousands of libraries. And just like each open library brings with it legal, security, and maintenance challenges, each public data set brings with it the exact same set of difficulties.

At my organization, we’ve talked about open source software’s version of this challenge as being an “accidental supply chain.” The software industry started building things because the incredible building blocks of open source libraries meant that we could. This meant the industry started treating open source software as a supply chain—which came as a surprise to many of those “suppliers.”

To mitigate these challenges, open source software has developed lots of sophisticated (though imperfect) techniques, like scanners for identifying what is being used, and metadata for tracking things after deployment. We’re also starting to invest in humans, to try to address the mismatch between industrial needs and volunteer motivations.

Unfortunately, the machine learning community seems ready to plunge into the exact same “accidental” supply chain mistake—doing lots of things because it can, without stopping to think much about the long-term implications once the entire economy is based on these data sets.

Open Expands to fill many Niches

A last important parallel is that I strongly suspect that machine learning will expand to fill many, many niches, just as open source software has. At the moment, the (deserved) hype is about large, generative models, but there are also many small models out there, as well as tweaks on larger models. Indeed, hosting site HuggingFace, machine learning’s primary hosting platform, reports the number of models on their site is growing exponentially.

These models will likely be plentiful and available for improvement, much like small pieces of open source software. That will make them incredibly flexible and powerful. I’m using a small machine learning-based tool to do cheap, privacy-sensitive traffic measurement on my street, for example, a use case that wouldn’t have been possible except on expensive devices a few years ago.

But this proliferation means that they’ll need to be tracked—models may become less like mainframes and more like open source software or SaaS, which pop up all over the place because of low cost and ease of deployment.

Metadata is not a Cure-all, but it is a first step

So if there are these important parallels (particularly of complex supply chains and proliferating distribution) what can machine learning learn from open source software?

The first parallel lesson we can draw is simply that to understand its many challenges, machine learning will need metadata and tooling. Open source software stumbled into metadata work through copyright and licensing compliance, but as the accidental supply chain for software has matured, metadata has proven immensely useful on a variety of fronts.

In machine learning, metadata tracking is a work in progress. A few examples:

• A key 2019 paper, widely cited in the industry, urged developers of models to document their work with “model cards.” Unfortunately, recent research suggests their implementation in the wild is still weak.
• Both the SPDX and CycloneDX software bills of materials (SBOM) specifications are working on AI bills of materials (AI BOMs) to help track machine learning data and models, in a more structured manner than model cards (befitting the complexity one would expect if this truly does parallel open source software).
• HuggingFace has created a variety of specs and tools to allow model and dataset authors to document their sources.
• The MIT Data Provenance paper cited above tries to understand the “ground truth” of data licensing, to help flesh out the specifications with real-world data.
• Anecdotally, many companies doing machine learning training work appear to have somewhat casual relationships with data tracking, using “more is better” as an excuse to shovel data into the hopper without necessarily tracking it well.

If we’ve learned anything from open, it’s that getting the metadata right (first, the specs, then the actual data) is going to be a project of years and may require government intervention. machine learning should take that metadata plunge sooner rather than later.

Security is going to be a Real Problem

Security has been another major driver of open source software’s metadata demand—if you don’t know what you’re running, you can’t know if you’re susceptible to the seemingly endless stream of attacks.

Machine learning isn’t subject to most types of traditional software attacks, but that doesn’t mean they’re invulnerable. (My favorite example is that it was possible to poison image training sets because they often drew from dead domains.) Research in this area is hot enough that we’ve already gone past “proof of concept” and into “there are enough attacks to list and taxonomize.”

Unfortunately, open source software can’t offer machine learning any magic bullets for security—if we had them, we’d be using them. But the history of how open source software spread to so many niches suggests that machine learning must take this challenge seriously, starting with tracking usage and deployment metadata, exactly because it is likely to be applied in so many ways beyond those in which it is currently deployed.

Regulation and Liability will Expand

The motivations that drove open source metadata (licensing, then security) point to the next important parallel: as the importance of a sector grows, the scope of things that must be measured and tracked will expand, because regulation and liability will expand.

In open source software, the primary government “regulation” for many years was copyright law, and so metadata developed to support that. But open source software now faces a variety of security and product liability rules—and we must mature our supply chains to meet those new requirements.

AI will similarly be regulated in an ever-growing multitude of ways as it becomes ever-more important. The sources of regulation will be extremely diverse, including on content (both inputs and outputs), discrimination, and product liability. This will require what is sometimes called “traceability”—understanding how the models are built, and how those choices (including data sources) impact the outcomes of the models.

This core requirement—what do we have? how did it get here?—is now intimately familiar for enterprise open source software developers. However, it may be a radical change for machine learning developers and needs to be embraced.

“Long term” is Longer than People think

Another parallel lesson machine learning can draw from open source software (and indeed from many waves of software before it, dating back at least to the mainframe) is that its useful life will be very, very long. Once a technology is “good enough,” it will be deployed and therefore must be maintained for a very, very long time. This implies that we must think about maintenance of this software as early as possible, and think about what it will mean that this software might survive for decades. “Decades” is not an exaggeration; many customers I encounter are using software that is old enough to vote. Many open source software companies, and some projects, now have so-called “Long Term Support” versions that are intended for these sorts of use cases.

In contrast, OpenAI kept their Codex tool available for less than two years—leading to a lot of anger, especially in the academic community. Given the rapid pace of change in machine learning, and that most adopters are probably interested in using the very cutting edge, this probably wasn’t unreasonable—but the day will come, sooner than the industry thinks, where it needs to plan for this sort of “long term”—including how it interacts with liability and security.

Financial Incentives are not Necessarily Aligned

Finally, it’s clear that—like open source software—there is going to be a lot of money flowing into machine learning, but most of that money will pool around what one author has called the “processor rich” companies. If the parallels to open source software play out, those companies will have very different concerns and spending priorities than the median creator (or user) of models.

Our company, Tidelift, has been thinking about this problem of incentives in open source software for some time, and entities like the world’s largest purchaser of software—the US government—are looking into the problem as well.

Machine learning companies, especially those seeking to create communities of creators, should think hard about this challenge. If they’re dependent on thousands of data sets, how will they ensure those are funded for maintenance, legal compliance, and security, for decades? If large companies end up with dozens or hundreds of models deployed around the company, how will they ensure those with the best specialist knowledge—those who created the models—are still around to work on new problems as they are discovered?

Like security, there are no easy answers for this challenge. But the sooner machine learning takes the problem seriously—not as an act of charity, but as a key component of long-term growth—the better off the entire industry, and the entire world, will be.

Conclusion

Machine learning’s deep roots in academia’s culture of experimentalism, and Silicon Valley’s culture of fast iteration, has served it well, leading to an amazing explosion of innovation that would have seemed magical less than a decade ago. Open source software’s course in the past decade has perhaps been less glamorous, but during that time it has become the underpinning of all enterprise software—and learned a lot of lessons along the way. Hopefully machine learning will not reinvent those wheels.
Luis Villa is co-founder and general counsel at Tidelift. Previously he was a top open source lawyer advising clients, from Fortune 50 companies to leading startups, on product development and open source licensing.

More On This Topic

• Closed Source VS Open Source Image Annotation
• Baby AGI: The Birth of a Fully Autonomous AI
• How to Build Data Frameworks with Open Source Tools to Enhance Agility and…
• 8 Open-Source Alternative to ChatGPT and Bard
• Software Developer vs Software Engineer
• Easy, Open-Source AutoML in Python with EvalML

One of the persistent challenges in satellite imaging is the dearth of data, especially while using multispectral or optical camera satellite imaging, where the presence of clouds hampers the consistent capturing of data.

Clouds act as the natural barrier to optical cameras and multispectral imaging systems, creating data gaps that impede the monitoring and analysis of Earth’s surface. This challenge is particularly significant in time-sensitive scenarios like environmental monitoring, disaster response, and agricultural assessments, where the obstruction caused by clouds not only disrupts the continuity of observations but also compromises the accuracy and reliability of the gathered data.

GalaxEye to the Rescue

Recognising the severity of this challenge, Bengaluru-based space tech startup GalaxEye is in the process of developing its own satellite — the world’s first multi-sensor micro-satellite called ‘Drishti Mission’ with data fusion capabilities.

Founded in 2021 by Suyash Singh, Denil Chawda, Kishan Thakkar, Pranit Mehta, Rakshit Bhatt, and Prof S R Chakravarthy, GalaxEye operates on a Data-as-a-Service Model, enabling businesses and governments to make informed decisions by leveraging insights from satellite data.

Originating from members of Avishkar Hyperloop, a global competition linked to SpaceX, GalaxEye emerged as part of the FalconX immersion program 2022 in Silicon Valley, California. Currently based in Bengaluru, it has evolved from the Hyperloop team at IIT-Madras.

They plan to launch Drishti Mission in collaboration with ISRO in the fourth quarter of 2024. They have synchronised two complementary sensors, bridging the gap in visual imaging using generative AI through data fusion and integrated synthetic aperture radar (SAR), known for its ability to penetrate the clouds and provide X-ray-like imagery.

“Since the very beginning, our aim has been to render Earth observation feasible across various industries,” said Suyash Singh, CEO and co-founder of GalaxEye, in an exclusive interview with AIM.

He added that he combined multispectral and SAR technologies into a single sensor stack, creating an all-weather, intuitive imagery system.

Addressing the alignment of GalaxEye’s initial mission with its current endeavours, Singh explained, “Our strategy has always involved minimising costs and risks by gradually demonstrating and maturing the technology. This approach, starting with aerial demonstrations before moving to space, allows for iterative development and learning from failures in a stepwise manner.”

The space tech startup recently secured $3.5 million in seed funding, with Special Invest leading the investment round in December 2022. Other contributors include Artha India Ventures, Veda VC, Anicut Capital, Upsparks, and a consortium of notable entrepreneurs such as Nithin Kamath of Zerodha, Prashant Pitti of EaseMyTrip, Abhishek Goyal of Traxn and more.

Tech Stack

“While design and development activities are entirely internal, the manufacturing and assembly were outsourced to the ecosystem,” said Singh. On the software side, the algorithms for geospatial analysis, spanning applications from insurance to agriculture, are also made by the 35+ strong tech team of GalaxEye.

The team primarily works with machine learning and deep learning frameworks. “We are also experimenting with generative AI algorithms to make Synthetic Aperture Radar (SAR) imagery more interpretable,” he added. However, in Singh’s experience, there exist talent gaps in specialised fields like radar algorithms and signal processing.

Discussing the internally developed data fusion technology, he emphasised its potential to provide unique insights and data from space as it aims to enable satellite constellations to perform all-weather imaging consistently, overcoming atmospheric challenges that affect existing single-sensor satellites.

Once fully operational, the constellation is anticipated to achieve global coverage within a 12-hour time frame. The ability to conduct continuous all-weather, all-time imaging, coupled with precise object geometry analysis, offers significant benefits across various industries. These include applications in insurance, precision agriculture, accurate property tax assessments, and the monitoring of utilities such as transmission lines, among others.

Position of Indian Space Ecosystem

Singh, discussing India’s evolving space tech ecosystem, highlighted the complexity of determining the minimum investment for a space mission, attributing it to the dynamic nature of the market.

Despite this challenge, he acknowledged, “We are witnessing a positive trend of growing interest and investments in India’s space tech sector, making it a good time to be in this field.”

Additionally, collaboration between government and private players is one of the strong pillars of Indian space tech, given the geopolitical significance that it holds. Singh stressed that such collaboration plays a pivotal role, signifying a transition from a phase of learning to a more active investment phase.

Last week, GalaxEye and drone tech firm ideaForge partnered to develop an innovative UAV FOPEN Radar, enhancing aerial surveillance for security forces. The radar, overcoming obstacles like clouds and foliage, will provide all-weather surveillance with real-time ground photography, identifying concealed objects behind camouflage or dense vegetation.

GalaxEye has solidified collaborations and business agreements with prominent entities within the space technology domain, such as Antaris Inc., a space software provider based in the US. Notably, the company has forged partnerships with QL Space, XDLINX Labs, Ananth Technologies, and Dassault System.

GalaxEye is committed to further broadening its network of alliances and clientele in the coming months. Demonstrating its ambition, the company has submitted a proposal to IN-SPACe, seeking endorsement and support from ISRO for its initiatives.

Read more: India’s Space Techs Don’t Get Enough Investments

The post This Indian Space Tech Startup is Building World’s First Multisensor Satellite appeared first on Analytics India Magazine.

Earlier this week Google DeepMind released resources for general purpose robots and they took it a step further. This is touted to be the ImageNet moment for robotics where it won’t be necessary to individually train robotic models to each specific task but it explores knowledge transfer between robots.

In line with this, let’s take a look at other models and robots for general purpose tasks and operations.

RT-X

RT-X, or the Robotics Transformer X, is a general-purpose robotics model developed by Google DeepMind to enhance robotics research and performance across a variety of robots and tasks.

The key goal was to create a model that can generalize and transfer its skills to various robots and tasks. It can be used for actions, vision, and language understanding, making it a comprehensive tool for general robotics research.

RT-X is built upon the foundation of two previous robotics transformer models: RT-1 and RT-2. RT-1 is a model developed for real-world robotic control at scale, and RT-2 is a vision-language-action (VLA) model that learns from both web and robotics data. By incorporating the architectures and knowledge from these models, RT-X achieves higher performance, especially due to the extensive and diverse cross-embodiment data it is trained on.

The development of RT-X involves training the model on a vast and diverse dataset, known as the Open X-Embodiment dataset. This dataset is a collection of data from 22 different robot embodiments, encompassing a wide range of skills and tasks, a total of 150,000 tasks in more than 500 distinct skills according to the paper.

RoboCat

RoboCat developed by Google DeepMind is an AI agent in robotics that can learn various tasks across different robotic systems. What sets RoboCat apart is its ability to not only master multiple tasks but also generate its own training data to enhance its skills.

With access to a broad and varied dataset, it can quickly grasp a new task in as little as 100 demonstrations. This rapid learning potential is a promising advancement in robotics research, diminishing the need for extensive human-guided training.

RoboCat is a product of merging Google’s Gato multimodal model, proficient in processing language, images, and actions in both simulated and real-world contexts. Gato’s architecture was integrated with an extensive training dataset comprising image-action sequences from diverse robot arms engaged in a myriad of tasks.

Dactyl

Dactyl, built by OpenAI, was trained in a computer simulation within, with its learnings then applied in the real world, despite some discrepancies in simulation accuracy. The specialized Shadow Dexterous Hand enables Dactyl to manipulate objects like blocks or prisms, employing a learning approach akin to that used for OpenAI Five.

Dactyl specifically addresses the task of altering an object’s position while held by the robotic hand, such as rotating a block. For precise grips, like the Tip Pinch grasp, Dactyl cleverly uses its thumb and little finger, which is quite like how humans use their thumb along with either the index or middle finger.

However, Dactyl’s little finger on the robot hand is more flexible due to an extra way it can move. That’s why Dactyl often opts for this finger. Essentially, Dactyl can independently discover grips like humans do but adjusts them to suit its own hand’s capabilities and limitations.

Jaco Arm

JACO arm built by Kinova Robotics is a lightweight assistive robot designed to compensate for lost arm movements. It consists of six linked segments, including a three-fingered hand. Users control hand movements in 3D space and object grasping/releasing with an adaptable controller, utilizing two or three fingers.

The arm can be mounted on a wheelchair, workstation, or table, fitting under the wheelchair’s armrest without widening it. JACO enhances daily activities for individuals with physical disabilities without hindering wheelchair mobility.

Key components include actuators driven by DC brushless motors with Harmonic Drive tech and various sensors for precise control. Grippers are designed to grasp diverse daily objects, with underactuated grippers simplifying control and adapting to object shapes.

The interface options include a Software Development Kit (SDK), a ROS package, and a Joystick, providing users with flexible control over the robot’s movements.

Panda Arm

Panda Arm was developed and built by Franka Emika, a German robotics platform company. The robot has a range of features and is designed with sensors to stop moving when there’s unexpected interference to prevent accidents. It has a special hand that can hold things tightly, like exerting a strong grip. This hand can hold up to 3 kg of weight. This allows the robot to pick up various objects almost like a human arm.

The arm is used in production lines and its uses are non-exhaustive. It is being used in nursing homes, laboratories and universities, logistics platforms, etc.

The post 5 Must-Know General Purpose Robots appeared first on Analytics India Magazine.

Image created by Author with Midjourney
Key Takeaways

• Chain of Density (CoD) is a novel prompt engineering technique designed for optimizing summarization tasks in Large Language Models like GPT-4
• The technique deals with controlling the information density in the generated summary, providing a balanced output that is neither too sparse nor too dense
• CoD has practical implications for data science, especially in tasks that require high-quality, contextually appropriate summarizations

Selecting the "right" amount of information to include in a summary is a difficult task.

Introduction

Prompt engineering is the fuel that powers advancements in the efficacy of generative AI. While existing prompting stalwarts such as Chain-of-Thought and Skeleton-of-Thought focus on structured and efficient output, a recent technique called Chain of Density (CoD) aims to optimize the quality of text summarizations. This technique addresses the challenge of selecting the "right" amount of information for a summary, ensuring it is neither too sparse nor too dense.

Understanding Chain of Density

Chain of Density is engineered to improve the summarization capabilities of Large Language Models like GPT-4. It focuses on controlling the density of information in the generated summary. A well-balanced summary is often the key to understanding complex content, and CoD aims to strike that balance. It uses special prompts that guide the AI model to include essential points while avoiding unnecessary details.

Figure 1: The Chain of Density process using an example (From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting) (Click to enlarge) Implementing Chain of Density

Implementing CoD involves the use of a series of chained prompts that guide the model in generating a summary. These prompts are designed to control the model's focus, directing it toward essential information and away from irrelevant details. For example, you might start with a general prompt for summarization and then follow up with specific prompts to adjust the density of the generated text.

Steps of the Chain of Density Prompting Process

1. Identify the Text for Summarization: Choose the document, article, or any piece of text that you wish to summarize.
2. Craft the Initial Prompt: Create an initial summarization prompt tailored to the selected text. The aim here is to guide the Large Language Model (LLM) like GPT-4 towards generating a basic summary.
3. Analyze the Initial Summary: Review the summary generated from the initial prompt. Identify if the summary is too sparse (missing key details) or too dense (containing unnecessary details).
4. Design Chained Prompts: Based on the initial summary's density, construct additional prompts to adjust the level of detail in the summary. These are the "chained prompts" and are central to the Chain of Density technique.
5. Execute Chained Prompts: Feed these chained prompts back to the LLM. These prompts are designed to either increase the density by adding essential details or decrease it by removing non-essential information.
6. Review the Adjusted Summary: Examine the new summary generated by executing the chained prompts. Ensure that it captures all essential points while avoiding unnecessary details.
7. Iterate if Necessary: If the summary still doesn't meet the desired criteria for information density, return to step 4 and adjust the chained prompts accordingly.
8. Finalize the Summary: Once the summary meets the desired level of information density, it is considered finalized and ready for use.

Chain of Density Prompt

The following CoD prompt is taken directly from the paper.

Article: {{ ARTICLE }}

You will generate increasingly concise, entity-dense summaries of the above Article.

Repeat the following 2 steps 5 times.

Step 1. Identify 1-3 informative Entities ("; " delimited) from the Article which are missing from the previously generated summary.
Step 2. Write a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities.

A Missing Entity is:
— Relevant: to the main story.
— Specific: descriptive yet concise (5 words or fewer).
— Novel: not in the previous summary.
— Faithful: present in the Article.
— Anywhere: located anywhere in the Article.

Guidelines:
— The first summary should be long (4-5 sentences, ~80 words) yet highly non-specific, containing little information beyond the entities marked as missing. Use overly verbose language and fillers (e.g., "this article discusses") to reach ~80 words.
— Make every word count: rewrite the previous summary to improve flow and make space for additional entities.
— Make space with fusion, compression, and removal of uninformative phrases like "the article discusses".
— The summaries should become highly dense and concise yet self-contained, e.g., easily understood without the Article.
— Missing entities can appear anywhere in the new summary.
— Never drop entities from the previous summary. If space cannot be made, add fewer new entities.

Remember, use the exact same number of words for each summary.

Answer in JSON. The JSON should be a list (length 5) of dictionaries whose keys are "Missing_Entities" and "Denser_Summary".

Chain of Density is not a one-size-fits-all solution. It requires careful crafting of chained prompts to suit the specific needs of a task. However, when implemented correctly, it can significantly improve the quality and relevance of AI-generated summaries.

Conclusion

Chain of Density offers a new avenue in prompt engineering, specifically geared towards improving summarization tasks. Its focus on controlling information density makes it an invaluable tool for generating high-quality summaries. By incorporating CoD into your projects, you can tap into the advanced summarization capabilities of next-generation language models.

Matthew Mayo (@mattmayo13) holds a Master's degree in computer science and a graduate diploma in data mining. As Editor-in-Chief of KDnuggets, Matthew aims to make complex data science concepts accessible. His professional interests include natural language processing, machine learning algorithms, and exploring emerging AI. He is driven by a mission to democratize knowledge in the data science community. Matthew has been coding since he was 6 years old.

More On This Topic

• Unraveling the Power of Chain-of-Thought Prompting in Large Language Models
• Summarization with GPT-3
• Text Summarization Development: A Python Tutorial with GPT-3.5
• GPT-2 vs GPT-3: The OpenAI Showdown
• Meet Gorilla: UC Berkeley and Microsoft’s API-Augmented LLM Outperforms…
• Exploring Tree of Thought Prompting: How AI Can Learn to Reason Through…

Level 5 autonomous vehicles might arrive earlier than expected. Wayve recently published a technical report detailing their generative world model, GAIA-1 (Generative AI for Autonomy), previewed in June 2023.

If you think this is a normal driving video, you'd be wrong.
This entire video is generated by @wayve_ai's generative AI model, GAIA-1.
The model was built to generate realistic driving scenes into the future to improve safety of self-driving cars in the real world. pic.twitter.com/Dg9fwIZugB

— Rowan Cheung (@rowancheung) October 5, 2023

GAIA-1 is a generative world model that leverages video, text, and action inputs to generate realistic driving scenarios while offering fine-grained control over ego-vehicle behavior and scene features. Interestingly, Meta’s AI chief is one of the investors of Wavye.

Nicely consistent and variable generations from a world model.
Video generation is done in representation space.
Pixel generation is the final step, only useful for visualization and data generation. https://t.co/7HQuSwdEdt

— Yann LeCun (@ylecun) October 5, 2023

It is the first generative world model designed specifically for self-driving systems. It was introduced earlier this year and builds on Wayve’s research in future prediction, dreaming about driving, predicting in a bird’s eye view and learning a world model.

How it Works

Wayve collects huge amounts of driving videos and uses generative AI to create new datasets of videos to train the cars. Interestingly, the videos created by Wayve are so realistic that it is pretty much impossible to figure out if they are real or created using generative AI.

GAIA-1 uses special encoders for different inputs like video and text, aligning them in time. Its main part, a world model with 6.5 billion parameters, predicts image tokens based on past data. These predictions are turned back into images using a video model with 2.6 billion parameters.

Trained on 4,700 hours of driving data, GAIA-1 has 9 billion parameters, making it powerful for generating accurate visuals.

Alex Kendall, chief of Wayve shared on X that unlike other models out there, it is not just a simulation but a full world model that understands how the world works. He elaborated on different scenarios demonstrating how autonomous vehicles can be trained using videos, eliminating the necessity of conducting actual road tests.

For instance, below is an example where Wayve prompted a vehicle to swerve right.

Moreover, Wayve’s model also takes into account the different weather conditions. To fully train a self-driving car, giving information about road conditions, lanes, and traffic is not enough. Wayne’s world model can be prompted to generate various driving scenes with the text prompt ‘It is’ followed by either ‘sunny’, ‘rainy’, ‘foggy’, or ‘snowy’.

Kendall said that one of the biggest challenges for self-driving is long-tail robustness: dealing with the enormous magnitude of edge cases we see on the road. An advantage of generative AI is its incredible ability to recombine experiences in new ways.

For example, one can experience driving in fog, and experience jay-walking pedestrians, and GAIA can learn from these experiences to understand how to generate a fog+jaywalking scenario.

A user from X raised concerns about ensuring that synthetic data’s diversity truly represents real-world scenarios without bias. In response, Kendall stated that with the right approach, it’s possible to build a world model that balances realism and diversity.

The Wayve method of training self-driving cars is different from Tesla. Tesla leverages its cars on the roads to collect the driving data. The company utilises an AI system that collects real-time visual data from eight car cameras.

This data is processed into a 3D output, identifying obstacles, their movement, lanes, roads, and traffic lights. Tesla gathers additional visual data from its car network, enhancing its training model. The newfound knowledge is integrated into cars through FSD software updates.

Tesla uses simulated data to train its autonomous vehicles. The simulation is created using a combination of real-world data and computer-generated imagery. Also, Tesla was one of the first companies to use neural networks for self-driving applications.

Ashok Elluswamy, director of autopilot software at Tesla, earlier this year at Investor day said that Tesla will not just stop at computer vision but reach out to techniques in other areas such as language modelling and reinforcement learning to train its fleet of FSD vehicles.

Wayve Brings Autonomous Cars to UK

It’s common to see self-driving taxis operated by Waymo and Tesla’s FSD cars in San Francisco. However, since both Tesla and Waymo are based in the US, most of their training data is collected from the US driving style.

Wayve interestingly is a UK company where until now self driving vehicles were not allowed on the road. However, earlier this year, Ford’s electric flagship Mustang Mach-E became the first autonomous car model to cruise in the UK, following regulatory approval in April. Mustang Mach-E features Ford’s driving assistance technology “BlueCruise”.

Currently, Wayve is actively testing its self-driving technology on a daily basis on roads in the UK. Additionally, the company is conducting the largest last-mile autonomous grocery delivery trial in Europe with Asda, the UK’s second-largest online supermarket.

Moreover, Wayve recently introduced a unique AI model called LINGO-1, which utilizes natural language to provide commentary on driving scenes and clarify its decision-making processes.

The post Generative AI Turns Level 5 Autonomy Real appeared first on Analytics India Magazine.

Meta announced its Ray-Ban smart glasses a few days back and everyone has been excited about owning one for themselves. The experience is like recording everything that you see from your point of view. Undoubtedly, with this news, Meta has ushered in its metaverse dream back into reality. But at the same time, things can be a little concerning when it comes to privacy.

Simply put, the smart glasses would be integrated with microphones, speakers, and a camera. The small camera lenses on the glasses record 1080p videos and take 12-megapixel photos, along with sharing them on social media platforms. Added to that is the fact that you can also livestream to Facebook and Instagram.

Meta AI in RayBan glasses that can live stream, take photo, etc.
Google Glasses all over again.
I do not want anyone around me who wears this. Taking photos, videos, streaming without my consent!!
This privacy was the main issue with the Google Glasses. I dont hear them… pic.twitter.com/DHPHfrAExR

— Oykun (@oykun) September 29, 2023

Why would Meta be so careless

Imagine you are in a public place and someone is wearing these smart glasses and starts live recording. To mitigate this privacy risk, Meta has introduced a white light that indicates the people around that it is recording. But is that really enough?

Meta did introduce these glasses with Ray-Ban two years ago and no one really knew what to do with them. This resulted in around 90 percent of the users dropping them. The introduction of cameras and speakers, along with live streaming, may attract a lot of users this time, but it just introduces a lot of concerns for people who are not wearing them.

As discussions on HackerNews point out, the advent of more and more smart devices always brings up the question of privacy. “smart” glasses come with a unique set of privacy challenges. Unlike traditional cameras and cell phones that require conscious effort to capture content, glasses can discreetly stream and record without the knowledge or consent of those around. This inherent invasion of privacy sparks legitimate concerns about how these devices will be used and regulated in various social settings.

Meta's smart glasses look great but…at $299?
That's almost how much my non-tech RayBan glasses cost me a few years ago. I guess the data collected is much more valuable and this way they can get everyone to buy them. Got it! #dataisthenewcurrency #smartglasses

— Tiago Barbosa (@t1agoB) September 28, 2023

In a recent podcast with Lex Fridman, Mark Zuckerberg was confident about the future of Metaverse with such glasses. “I think we’re moving towards a world where we’re gonna have something that looks like normal glasses where you can just, you see the physical world, but you’ll see holograms,” said Zuckerberg.

One of the most significant concerns with Meta’s smart glasses is the potential for invasive surveillance. While the white light indicator is intended to signal when recording is taking place, it’s not foolproof, and there may still be situations where people are recorded without their knowledge or consent. This raises legitimate worries about the erosion of personal privacy in public spaces.

Meta goes in for data

Earlier, Meta launched its Threads platform, competing with Elon Musk’s X, to know what we think. Musk has said that it will use the data from the social media platform to train AI models. Possibly, Zuckerberg wanted to do the same thing. Now, with the race towards multimodality, Meta wants real-life data from people to build its model.

The smart glasses are always going to be on our noses and watching everything. It seems as though, instead of onboarding people onto the metaverse, Meta now wants to see what people are doing in their lives, capturing people’s BTS, as Meta put it at the Meta Connect 2023. Anyway, Meta is no stranger to messing with people’s privacy all these years as well.

Mark Zuckerberg reveals collaboration with Rayban!!!
Great idea but the person who blocks camera on his devices wants us to wear sunglasses with camera and microphone.
Think about it !!#Meta #privacy pic.twitter.com/Sr7aJmv02J

— Aakarsh Bajpai (@aakybajpai) September 30, 2023

When Apple announced Vision Pro, it felt like it was trying to make everyone into a zombie. Apple at least knows a little bit about privacy. Now with Meta smart glasses, the trend continues, and it raises important questions about the intersection of technology and privacy in our everyday lives.

The question of data ownership and usage looms large. Meta’s appetite for data is well-documented, and it’s reasonable to assume that the company will gather a significant amount of personal information through these smart glasses. What will Meta do with this data, and how will it be protected? These are crucial questions that demand answers as we move further into the age of wearable technology.

The post Ban Meta Ray-Ban? appeared first on Analytics India Magazine.

Edtech Unicorn, PhysicsWallah recently said that it is introducing Alakh AI, a personalised assistant for students, which would act like a study companion and provide access to learning.

This new platform is expected to include various generative AI capabilities and features that would help students to understand and learn new concepts easily and in a simplified manner, trained on PhysicsWallah’s teaching materials and style.

In an exclusive interview with AIM, PyshicsWallah chief technology officer Devesh Mishra said that the first beta version of Alakh AI is still in the development stage, and looks to release it by end of this year.

Founded in 2016, by Alakh Pandey, PhysicsWallah offers a variety of courses for students of all grades, from Class 6 to Class 12, as well as for students preparing for competitive exams such as the Joint Entrance Examination (JEE) and the National Eligibility cum Entrance Test (NEET).

PhysicsWallah became a unicorn in June last year, after raising $100 million in a Series A funding round from WestBridge Capital and GSV Ventures. The company was valued at $1.1 billion after the funding round. Coincidentally, the name Alakh AI has also been named after the founder.

Alakh AI will serve as the powerhouse behind PhysicsWallah’s problem-solving engine, enabling personalisation and adaptive learning paths. The AI will empower the Study Companion and various educational nuggets, ensuring a comprehensive and tailored learning experience for students.

He emphasised that Alakh AI isn’t just another tutoring service or an AI tutor (of any kind); it’s designed to be a unique “Study Companion.” Unlike traditional tutoring platforms, Alakh AI aims to be a student’s dedicated companion for both academic and non-academic support.

The companion will assist students in group study sessions, resolving academic and non-academic queries, providing motivation, and even creating personalised study planners. Mishra emphasised that it isn’t a personal tutor but a companion to study alongside.

One of the most ambitious features of Alakh AI is its language agnosticism. PhysicsWallah is striving to make this innovative learning tool accessible to a global audience, supporting multiple languages to cater to diverse learners.

Moreover, PhysicsWallah is harnessing the power of generative AI to translate its extensive YouTube content into various languages. This move will not only enhance accessibility but also significantly reduces operational costs for the edtech company.

PhysicsWallah is going big on AI. Recently Alakh said: “No one has really used AI at scale in edtech, it will be great if we can do that because we have the scale. We won’t have to train AI on open AI – we will train it on PhysicsWallah’s own datasets.”

The post [EXCLUSIVE] PhysicsWallah to Introduce Generative AI Platform, Alakh AI appeared first on Analytics India Magazine.

With the ongoing Israel-Palestine crisis, reporters from media publications have been scrambling to cover the condition. However, citizens in these areas turned to X to share the situation at ground zero. Alarming visuals from every corner has found its way to the internet, and today, X is serving as the quintessential news outlet that everyone needs.

Donning a black cowboy hat, aviator shades and a casual tee, Elon Musk, the Chief of Tesla and the head of X, recently visited the Eagle Pass Border (Texas-Mexico), to meet with local politicians and law enforcement, with the intention of capturing the ground reality of the illegal immigrant crisis. Streaming the coverage live on X, Musk used the opportunity to also demonstrate how one can use his social media platform to cover news as and when it happens. Thus, encouraging ‘citizen journalism’, and laying the foundation for making X the next big news platform.

Citizen journalism is the path to better future!
I strongly encourage people around the world to post news about events as they’re happening, in both text & video. https://t.co/c8PxqinKAm

— Elon Musk (@elonmusk) October 4, 2023

Leading By Example

Elon Musk has always been clear about his intentions on making his social media platform, a free space to express anything. After acquiring Twitter (now called X), Musk hailed ‘free speech’ and called it the ‘bedrock of a functioning democracy.’ True to his word, as a first step, Musk reinstated accounts of banned users including a number of journalists and ring-wing supporters. Former President of the US, Donald Trump’s account was also brought back to the platform – setting a fair play ground for all. Interestingly, Musk recently went a step further by even removing the option of blocking any user on X (only DM messages can be blocked).

Move Over, Legacy Media

Musk’s conflict with legacy media houses has been an ongoing tussle. He has been vocal about promoting the platform as a place for any political leaning supporter. He even recently took a dig at publications that are always left-leaning and assured X to be a place that will have both ‘left and right’ perspectives. Interestingly, a number of publications have been extensively covering Musk.

Really weird how Elon Musk started breaking all the laws in every area of life from securities to civil rights ever since Democratic politicians started to hate him. pic.twitter.com/Yk2ElYwm14

— Richard Hanania (@RichardHanania) October 5, 2023

Musk believes that Community Notes, a feature on X that is controlled by the users as a whole, allows flagging misleading posts, which he believes, makes it ‘harder to lie on the platform.’

When Tucker Carlson, a political commentator and TV talk show host, was ousted by Fox News, Elon Musk offered him to exclusively premier his talk show ‘Tucker on X’ with no form of gate-keeping. The episode where Carlson interviewed Donald Trump garnered over 250M views in 24 hours.

Furthermore, the video feature on X is tested for streaming games– a new feature that Musk has been experimenting with off late.

Incentivising User Content

If inviting people to post anything without any restriction was not enough, X also started paying content creators. CEO of X, Linda Yaccarino confirmed that the company has paid close to $20 million to their creator community. Content creators are paid for ad revenue based on impressions of their original content- a feature available for paid users on the platform.

Innovation strategist and tech blogger Robert Scoble, famous Youtuber Marques Brownlee, among others, have started posting full-length interviews and product review videos on X.

Word of Caution

The simplicity and ease with which one can easily cover and report anything from anywhere with a mere phone in hand, has popularised citizen journalism for a while. Social media platforms such as Facebook, Tik Tok and even Twitter have been the choice for users to host images and videos on critical issues. May it be the Russia-Ukraine crisis or the recent Israel-Palestine conflict, live video footage of the situation has been uploaded on the platform.

While citizen journalism can empower people to share first-hand accounts of events, accuracy and verification of information becomes crucial. With the rise of deepfake and AI-generated content, the likelihood of spreading fake news still looms. A few months ago, a fake image of an explosion near the Pentagon went viral, with leading news houses sharing the image. The incident rocked the financial markets and highlighted the danger of fast-spreading misinformation on social platforms.

While the act of individual reporting can never become fool-proof, community notes is one of X’s strongest feats to counter it. Irrespective, the platform’s ambition to drive traffic and boost user engagement is already working. A new era of journalism might have already begun.

The post X for Free Speech and Citizen Journalism appeared first on Analytics India Magazine.

Internet of Things (IoT) establishes a network for connecting physical objects, such as devices, machines, sensors, or any equipment with processing abilities that can connect to the Internet. It refers to a digitally connected universe built on smart devices like fitness trackers, home voice assistants, smart thermostats, etc.

IoT market is growing rapidly. It is reaching every home across the globe. According to McKinsey, the global IoT market will amount to $12.6 trillion by 2030. But, today, it is not strictly limited to machine-to-machine (M2M) communication.

Internet of Everything (IoE) broadens the scope of IoT to encompass people, data, and processes. It simply means that “everything” is or will be connected digitally. This increased connectedness creates an intelligence ecosystem beyond anything we have encountered.

But to enable IoE globally, we need technologies that can connect “everything” and understand massive amounts of information to deliver intelligent outcomes. Artificial Intelligence (AI) and 6G are prominent technologies facilitating connected intelligence in IoE systems.

In this article, we’ll look at the concept of the Internet of Everything in detail and shed some light on the relationship between AI and 6G technologies to enable global connectivity.

What is the Internet of Everything (IoE)?

First coined by Cisco in 2012, the Internet of Everything builds on IoT by extending connections beyond machine-to-machine communication. It unifies people, organizations, data, and devices into a well-connected ecosystem. It facilitates a higher level of interconnectivity by seamlessly combining numerous technologies, like cloud computing, edge computing, AI, IoT, 6G, and data analytics, along with various gadgets, sensors, and machines to gather, transmit, and analyze data in real-time.

Overview of the Internet of Everything Framework

The IoE industry is expanding rapidly. According to CMSWire, the IoE market is expected to reach $4,205.50 billion by 2030, compared to $928.11 billion in 2020. When this technology is fully developed, it will affect numerous fields, like healthcare, manufacturing, agriculture, and mining.

Let's consider the example of an IoE-based smart grid system. The smart grid contains interconnected components, including distribution networks, transmission lines, power generation units, and meters. By deploying an IoE system, data collected from these components can be analyzed in real-time using AI-based systems and advanced analytics. Based on real-time intelligent insights, the smart grid can deliver electricity to consumers with improved efficiency. It can optimize energy management, detect faults, and enhance the safety of the grid systems.

Enabling the Internet of Everything With 6G Connectivity

Sixth-generation or 6G technology is the next wave of wireless communication systems, succeeding 5G cellular networks. With ultra-high-speed data transfer, low latency, and hyper-connectivity, 6G is a critical component in supporting and releasing the full potential of the Internet of Everything-based systems.

Evolution of Network Generations

Let’s check out the following ways in which 6G can facilitate IoE:

• 6G hyper-connectivity would enable super-secure authorization procedures and security frameworks to safeguard private information in networked IoE environments.
• 6G can effectively improve the network's latency and bandwidth distribution to meet the different needs of IoE applications.
• 6G would enable intelligent edge computing by putting computational capacity closer to the IoE-enabled devices.
• 6G can deliver hyperpersonalization and customization of consumers’ network experiences.

Applications of AI & 6G-Based Internet of Everything Systems

1. Healthcare

AI and the 6G have the capacity to transform the healthcare industry. AI-powered IoE systems can provide real-time medical data examination, personalized therapies, and remote monitoring of patients. Combining it with 6G enables dependable and low-latency connectivity for primary telemedicine services, enabling healthcare professionals to spot irregularities and react quickly.

2. Manufacturing

IoE systems based on AI and 6G foster productivity and cost-effectiveness in manufacturing operations. Such applications can extract large amounts of data from various sources, including sensors, machines, and networks. AI models can analyze this data to optimize production processes, forecast service requirements, and offer predictive quality assurance. 6G connectivity would power continuous interaction between machines, networks, and staff, allowing for real-time data exchange and decision-making.

3. Agriculture

6G-based IoE applications can improve precision farming by combining real-time data from weather gauges, land sensors, and farming equipment. The acquired data can be analyzed using AI-based agriculture systems trained to generate meaningful insights, helping farmers and land owners make timely decisions about their yields. It allows for improved distribution of resources, optimized irrigation plans, increased crop yields, and the adoption of sustainable agricultural practices.

4. Environmental Sustainability

AI and 6G-based IoE systems can aid in environmental surveillance and sustainability activities. Connected sensors and satellite monitoring can capture air quality, water resources, and energy consumption data, allowing for real-time analysis and effective resource management. Furthermore, AI systems can recognize trends, predict environmental hazards, and aid decision-making to promote environmentally friendly practices.

5. Transportation

AI and 6G-based systems have the potential to revolutionize how we commute. IoE connectivity provides transportation systems with real-time data flows. It allows them to track and control traffic movement, traffic jams, and road conditions. For example, AI-based transportation systems can analyze data from IoE devices to optimize traffic signals, redirect vehicles, and execute adaptive congestion control measures to provide a safer environment for self-driving cars.

Internet of Everything – Where Are We Headed?

“The 21st century will be equivalent to 20,000 years of progress at today’s rate — about 1,000 times greater than the 20th century” – The Law of Accelerating Returns by Ray Kurzweil, Technologist and Futurist.

Integrating AI and 6G in the context of the Internet of Everything opens up significant opportunities for futuristic systems. Automation across sectors powered by AI and 6G will increase operational productivity, lower expenses, and increase production. Consumers will have more personalized and frictionless interactions with linked devices.

However, as AI and 6G grow, tackling ethical issues and privacy concerns becomes increasingly essential. Effective AI frameworks for governance are required to promote transparency, equity, and responsibility in deploying and using AI within the IoE landscape. Building the long-term viability of AI and 6G-enabled IoE systems will largely depend on the protection of user data and observance of privacy rights.

To learn more about futuristic technologies, read the following blogs:

• Quantum Algorithms Could Investigate Larger Molecules
• Quantum Computing Algorithm Could Lead to Design of New Materials
• MetaGPT: Complete Guide to the Best AI Agent Available Right Now
• Mind Over Machines: Revolutionizing Human-Device Interaction
• How Human Genome Sequencing & Deep Learning (AI) Can Revolutionize Personalized Healthcare