AI — Страница 893

Just How Easy is it to Publish Malicious Extension on VSCode?

Recently, a group of Israeli researchers were able to create and publish a malicious VSCode extension in 30 minutes. Surprisingly, the extension was trending, and had 100+ downloads within the first 24 hours, shockingly exposing the vulnerability of the platform.

Built with Flaws

The experiment showcased that over 1,280 extensions had malicious dependencies packaged in them with a combined total of 229 million installations. Also, there were 87 extensions that attempted to read the /etc/passwd file on the host system.

Amit Assaraf, the co-founder of real estate investing app Landa, and one of the Israeli researchers who experimented to expose the flaw, said, “Unlike Google Chrome extensions, VSCode extensions are practically apps/executables running on your machine with zero limitations on what they can do on the host.

“This means extensions pose a threat to spawn child processes, system calls or even import any NodeJS package they’d like.”

In the experiment conducted by Assaraf and his team members Itay Kruk, and Idan Dardikman, it was found that 2,304 extensions used another publisher’s GitHub repo as their official listed repository. This means you have no way to tell if the code of your extension and the linked GitHub repository are the same.

A major flaw in the popular open-source code editor VSCode is that extensions are not sandboxed. Though there’s a provision for sandboxing code, it is not applicable for extensions.

Since extensions are not sandboxed, they can access anything inside the IDE and execute anything on the host machine without the developer receiving any feedback.

Apart from sandboxing, VSCode also lacks permission management. You will find multiple feature requests to add permission manager in VSCode, similar to what we have in our smartphones to know that is being accessed by extensions.

This way, a theme extension that is built to change colours of IDE, may execute code and read or write files without any visibility or explicit authorisation from the user.

Auto Update of Extension

VSCode extensions automatically update to the latest version behind the scenes. This means any developer can initially create an extension without any malicious intent and later update the extension where he can introduce the malicious code.

The same happened with xz utility, which was safe for years and later found to have a backdoor.

$5 to Verify VSCode Extension

Extension verification allows extension authors to verify the ownership of a domain to establish their authenticity and credibility, with the verified domain displayed alongside their name.

Isidor Nikolic, a senior product manager at Microsoft, VS Code, mentions that extension authors can become verified by checking the ownership of an eligible domain associated with your brand or identity.

Interestingly, to verify your extension, all you have to do is get a domain for your extension (which usually costs around $5) and soon you’ll receive a badge from VSCode suggesting that your extension is verified.

How easy it is to verify malicious extension on VSCode Market place

A GitHub user critiqued the process, calling the process for verification badge as totally useless and misleading. “The verified blue check mark merely means that whoever the publisher is has proven the ownership of a domain. And that means any domain. In reality, a publisher could buy any domain and register it to get that verified check mark,” he said.

Sure, there are manual steps involved in the verification process but these are not rock solid. You can use a different name while applying for verification and as soon as you get verified, it can be changed to look exactly like the original name of the extension.

“Security is not a high priority for Microsoft as they want as many extensions on their marketplace as possible,” the Israeli researcher also mentioned further.

Solutions to Navigate

Vignesh Rajan, a lead engineer at GenAI startup MachineHack, suggested, “You may use VSCode but with as few extensions as possible to minimise the risk. If necessary, a developer should do a thorough research on which extension is official and can be trusted before installation.”

By nature, VSCode heavily relies on extensions to enhance its functionality. It is a bare-bone IDE where developers can install extensions of their choice to get the job done.

You may also switch to closed-source IDE such as IntelliJ. A user while praising IntelliJ for Java development mentioned that “VS Code is not only unsuitable (still) for larger enterprise-level projects, it is also less reliable, responsive and stable than IntelliJ.”

On November 18, 2015, the source code of VS Code was released under the MIT License and made available on GitHub. The idea was to create a lightweight platform powered by extension and it was an instant success.

Transformers Can Now Work Pixel by Pixel, Says Meta AI’s New Study

A latest research by Meta AI and the University of Amsterdam have shown that transformers, a popular neural network architecture, can operate directly on individual pixels of an image without relying on the locality inductive bias present in most modern computer vision models.

The study, exploring “Transformers on Individual Pixels,” challenges the long-held belief that locality – the notion that neighboring pixels are more related than distant ones – is a fundamental requirement for vision tasks.

Traditionally, computer vision architectures like Convolutional Neural Networks (ConvNets) and Vision Transformers (ViTs) have incorporated locality bias through techniques such as convolutional kernels, pooling operations, and patchification, assuming neighboring pixels are more related.

However, researchers introduced Pixel Transformers (PiTs), which treat each pixel as an individual token, removing any assumptions about the 2D grid structure of images. Surprisingly, PiTs achieved highly performant results across various tasks.

Following the architecture of Diffusion Transformers (DiTs), PiTs operating on latent token spaces from VQGAN achieved better quality metrics like Fréchet Inception Distance (FID) and Inception Score (IS) than their locality-biased counterparts.

Perceiver IO Transformers (PiTs) are computationally expensive due to longer sequences, but they challenge the need for locality bias in vision models. Advances in handling large sequence lengths may make PiTs more practical.

The study highlights reducing inductive biases in neural architectures, potentially leading to more versatile and capable systems for diverse vision tasks and data modalities.

Image generation using transformers

There are different models for image generation, such as Midjourney, Stable Diffusion, and Invoke, whose images can be reimagined with these technologies. Recently Midjourney has released the new feature “Character Reference” claiming to generate consistent characters across multiple reference images.

Stability AI announced Stable Diffusion 3, the most capable text-to-image model, featuring significantly enhanced performance in multi-subject prompts, image quality, and spelling abilities.

Unlocking Data Insights: Key Pandas Functions for Effective Analysis

Image by Author | Midjourney & Canva

Pandas offers various functions that enable users to clean and analyze data. In this article, we will get into some of the key Pandas functions necessary for extracting valuable insights from your data. These functions will equip you with the skills needed to transform raw data into meaningful information.

Data Loading

Loading data is the first step of data analysis. It allows us to read data from various file formats into a Pandas DataFrame. This step is crucial for accessing and manipulating data within Python. Let's explore how to load data using Pandas.

import pandas as pd  # Loading pandas from CSV file  data = pd.read_csv('data.csv')

This code snippet imports the Pandas library and uses the read_csv() function to load data from a CSV file. By default, read_csv() assumes that the first row contains column names and uses commas as the delimiter.

Data Inspection

We can conduct data inspection by examining key attributes such as the number of rows and columns and summary statistics. This helps us gain a comprehensive understanding of the dataset and its characteristics before proceeding with further analysis.

df.head(): It returns the first five rows of the DataFrame by default. It's useful for inspecting the top part of the data to ensure it's loaded correctly.

     A    B     C  0  1.0  5.0  10.0  1  2.0  NaN  11.0  2  NaN  NaN  12.0  3  4.0  8.0  12.0  4  5.0  8.0  12.0

df.tail(): It returns the last five rows of the DataFrame by default. It's useful for inspecting the bottom part of the data.

     A    B     C  1  2.0  NaN  11.0  2  NaN  NaN  12.0  3  4.0  8.0  12.0  4  5.0  8.0  12.0  5  5.0  8.0   NaN

df.info(): This method provides a concise summary of the DataFrame. It includes the number of entries, column names, non-null counts, and data types.

<class 'pandas.core.frame.DataFrame'>  RangeIndex: 6 entries, 0 to 5  Data columns (total 3 columns):   #   Column  Non-Null Count  Dtype    ---  ------  --------------  -----     0   A       5 non-null      float64   1   B       4 non-null      float64   2   C       5 non-null      float64  dtypes: float64(3)  memory usage: 272.0 bytes

df.describe(): This generates descriptive statistics for numerical columns in the DataFrame. It includes count, mean, standard deviation, min, max, and the quartile values (25%, 50%, 75%).

              A         B          C  count  5.000000  4.000000   5.000000  mean   3.400000  7.250000  11.400000  std    1.673320  1.258306   0.547723  min    1.000000  5.000000  10.000000  25%    2.000000  7.000000  11.000000  50%    4.000000  8.000000  12.000000  75%    5.000000  8.000000  12.000000  max    5.000000  8.000000  12.000000

Data Cleaning

Data cleaning is a crucial step in the data analysis process as it ensures the quality of the dataset. Pandas offers a variety of functions to address common data quality issues such as missing values, duplicates, and inconsistencies.

df.dropna(): This is used to remove any rows that contain missing values.

Example: clean_df = df.dropna()

df.fillna():This is used to replace missing values with the mean of their respective columns.

Example: filled_df = df.fillna(df.mean())

df.isnull(): This identifies the missing values in your dataframe.

Example: missing_values = df.isnull()

Data Selection and Filtering

Data selection and filtering are essential techniques for manipulating and analyzing data in Pandas. These operations allow us to extract specific rows, columns, or subsets of data based on certain conditions. This makes it easier to focus on relevant information and perform analysis. Here’s a look at various methods for data selection and filtering in Pandas:

df['column_name']: It selects a single column.

Example: df[“Name”]

0      Alice  1        Bob  2    Charlie  3      David  4        Eva  Name: Name, dtype: object

df[['col1', 'col2']]: It selects multiple columns.

Example: df["Name, City"]

0      Alice  1        Bob  2    Charlie  3      David  4        Eva  Name: Name, dtype: object

df.iloc[]: It accesses groups of rows and columns by integer position.

Example: df.iloc[0:2]

    Name  Age  0  Alice   24  1   Bob   27

Data Aggregation and Grouping

It is crucial to aggregate and group data in Pandas for data summarization and analysis. These operations allow us to transform large datasets into meaningful insights by applying various summary functions such as mean, sum, count, etc.

df.groupby(): Groups data based on specified columns.

Example: df.groupby(['Year']).agg({'Population': 'sum', 'Area_sq_miles': 'mean'})

         Population  Area_sq_miles  Year                                2020       15025198     332.866667  2021       15080249     332.866667

df.agg(): Provides a way to apply multiple aggregation functions at once.

Example: df.groupby(['Year']).agg({'Population': ['sum', 'mean', 'max']})

      Population                                      sum          mean       max  Year                                    2020  15025198  5011732.666667  6000000  2021  15080249  5026749.666667  6500000

Data Merging and Joining

Pandas provides several powerful functions to merge, concatenate, and join DataFrames, enabling us to integrate data efficiently and effectively.

pd.merge(): Combines two DataFrames based on a common key or index.

Example: merged_df = pd.merge(df1, df2, on='A')

pd.concat(): Concatenates DataFrames along a particular axis (rows or columns).

Example: concatenated_df = pd.concat([df1, df2])

Time Series Analysis

Time series analysis with Pandas involves using the Pandas library to visualize and analyze time series data. Pandas provides data structures and functions specially designed for working with time series data.

to_datetime(): Converts a column of strings to datetime objects.

Example: df['date'] = pd.to_datetime(df['date'])

     date       value  0 2022-01-01     10  1 2022-01-02     20  2 2022-01-03     30

set_index(): Sets a datetime column as the index of the DataFrame.

Example: df.set_index('date', inplace=True)

    date     value    2022-01-01     10  2022-01-02     20  2022-01-03     30

shift(): Shifts the index of the time series data forwards or backward by a specified number of periods.

Example: df_shifted = df.shift(periods=1)

  date       value  2022-01-01    NaN  2022-01-02   10.0  2022-01-03   20.0

Conclusion

In this article, we have covered some of the Pandas functions that are essential for data analysis. You can seamlessly handle missing values, remove duplicates, replace specific values, and perform several other data manipulation tasks by mastering these tools. Moreover, we explored advanced techniques such as data aggregation, merging, and time series analysis.

Jayita Gulati is a machine learning enthusiast and technical writer driven by her passion for building machine learning models. She holds a Master's degree in Computer Science from the University of Liverpool.

This MagSafe accessory gives you iOS 18’s call recording feature right now

Magmo Pro Magnetic Snap-On Call Recorder for iPhone

ZDNET's key takeaways

The Magmo Pro Magnetic Snap-On Call Recorder is available on Amazon for $129 in Starlight (white), Navy, and Space Black.
If you want to record phone calls without relying on a third-party app or waiting until iOS 18 finally launches, this gadget will get the job done.
The device excels at recording, not transcribing, so depending on your needs, it may not be worth the investment.

Recording a phone call can be handy in many instances, such as conducting an interview or collecting evidence. However, on iPhones, there is currently no native or easy way to record phone calls. This gadget can help.

Users have highly requested the ability to record calls on an iPhone, so much so that at Apple's Worldwide Developer's Conference (WWDC) this year, the company unveiled a feature that uses Apple Intelligence to record, transcribe, and summarize iPhone calls.

Also: Apple's iOS 18 will let you record phone calls without a third-party app

However, iOS 18 will not be released until the fall, and not all iPhone users will be eligible for the full iOS 18 experience. Many advanced Apple Intelligence features require the A17 Pro chip, which is currently only found on the iPhone 15 Pro and iPhone Pro Max.

Until then, users interested in recording calls will either need to record from a separate device or rely on third-party phone call apps that usually conference into the call and let the parties know it is present, ruining the discreetness factor. That is where the Magmo Pro iPhone Call Recorder can help.

View at Amazon

I first encountered the gadget at CES, which was recognized as a CES 2024 Innovation Awards Honoree in the Mobile Devices, Accessories & Apps category. The gadget uses MagSafe to snap on the back of your iPhone and record calls for you. Sound too good to be true? Well, I put it to the test.

In the box, you get the accessory and a USB-C to USB-A charger. The charger is impressively thin at 6.88mm, much thinner than any of my MagSafe power banks, and very light, weighing only 50g. It takes no time to fully charge.

Once you snap the device onto the back of your phone, follow the directions on the screen and pair it to Bluetooth. Then, you can set the device in Auto mode, which records every call you get automatically, or manual mode, in which you move the toggle to record a call. I started with auto mode.

The first call I recorded was promising but could have been better. The device started recording automatically when I got the call, which I loved. However, the other person on my call sounded very distant, and you could hear some muffling from me accidentally placing my hand on the mic.

Also: How to record a call on your iPhone (and check if it's legal in your state)

In the next call, I made the volume as high as possible and held my phone so that my palm didn't block the microphone. You'll want to be mindful of this for the best results. The quality of the call was significantly better, and when I played it back, the call sounded as if it had been recorded natively, even though it wasn't.

Despite accomplishing its recording function well, there are downsides to the device, including the fact that the battery life dies quickly when it is in Auto mode, which it even warns you about, as seen below. I don't find this to be a deal breaker because I can't think of many situations where you'd need every call to be recorded for long periods.

There's also a manual mode that greatly reduces battery consumption, and it worked as advertised. You simply switch the toggle on the back of the device when you want to start recording. Even if the accessory isn't paired to your phone at the time of recording, it still syncs the on-device recording to your app once you do.

One downside is the app's user interface, which looks like something I would have found on my iPod Touch in 2009. It isn't intuitive, looks clunky, and is unnecessarily difficult to complete simple tasks.

Also: Don't wait for iOS 18's AI. ChatGPT offers these same 4 features now

For example, to listen to your recordings, you first need to refresh your app, click on the recording you need to download, click through several popups, and then download it to give it a listen. Again, this isn't a dealbreaker for me because all I want is the ability to listen to crisp phone call recordings and export them.

Lastly, the app's transcription feature, which is still in beta, shows promise but is unreliable. To power the AI transcriptions, the application uses "Apple's very own API to transcribe audio to written text," as stated below. As a result, the transcriptions are just as unformatted and jumbled as the ones found on your iPhone's Voicemails. Here's to hoping they'll improve with time.

The biggest problem with the Magmo Pro is that, at the moment, the device doesn't transcribe the entire call but rather just a paragraph. Magmo says the recorder will be upgraded shortly, but it may not be suitable for you if you want a transcription first, recording second device.

ZDNET's buying advice

If you need to record phone calls daily, whether for professional or personal reasons, the Magmo Pro Call Recorder can help you do so discreetly. Of course, you would still have to check your state's laws to see if it is legal to record another party without disclosing the fact.

As a reporter, I am excited because, typically, I would have to record phone interviews on another device, such as my iPad or Mac, and now all I have to do is slap the recorder on the back of my phone. However, if you buy it for accurate transcriptions, as advertised, you may want to reconsider because the technology is not ready yet.

Featured reviews

Bengaluru’s HSR is All About AI Startups

Bengaluru, often dubbed the ‘Silicon Valley of India’, has always been synonymous with startups and innovation. In recent years, the city has embraced the AI trend, making it a significant player in the global AI landscape.

Srikrishna Swaminathan, the CEO of Factor.AI, wrote on LinkedIn, “One building, and all AI companies. Factors.ai can claim to attracted all AI here, as we were the first occupants, Amal Mishra, Urban Vault.”

Urban Vault, a building in HSR Layout, is currently housing five notable AI companies, namely Loop AI, Factors AI, Raga AI, Frinks AI, and Actyv AI, thereby, alluding to HSR’s pivotal role in nurturing AI startups.

HSR: The AI Hub

In an earlier conversation with AIM, Ganesh Gopalan, the CEO and co-founder of GNANI.AI, highlighted that they wished to establish the startup in HSR, calling it a hub for tech talent.

Some of the vibrant AI startups buzzing in HSR Layout include MachineHack Gen AI, a GenAI startup for the developer community; Zolnoi Innovations Pvt. Ltd., which leverages AI to enhance production efficiency; and HyperVerge, a deep-tech AI company.

Then there’s SigTuple, specialising in intelligent solutions for medical diagnosis using state-of-the-art AI techniques; Invento Robotics, creating intelligent robots for a range of applications; and Talview, offering an AI-powered recruitment platform for global enterprises.

This unique ecosystem thrives due to the diversity of startups focusing on various sectors, including logistics, sales, developer tools, manufacturing, and fintech. The proximity of these companies fosters collaboration and innovation, making mergers and acquisitions more seamless.

AI Startups in Bengaluru

Speaking to AIM at IGIC 2024, Sanjeev Kumar Gupta, the CEO of Karnataka Digital Economy Mission, said, “There are over 1,000 AI startups in Bengaluru. With initiatives like Beyond Bengaluru, we are also promoting AI startups, in Mysuru, Hubli, and Mangaluru.”

The emergence of companies like Sarvam AI, which focuses on developing advanced AI models, reflects Bengaluru’s commitment to technological innovation.

Krutrim AI, led by Ola’s founder Bhavish Aggarwal, has raised $50 million in funding at a $1 billion valuation and is now India’s first AI startup to reach the unicorn status.

Innovation in AI isn’t limited to corporate giants. Startups like KOGO OS are disrupting markets with their AI operating systems, offering modular AI assistants tailored for diverse industries.

Meanwhile, Karya AI is pioneering in the rural employment sector, leveraging AI to create job opportunities through tasks in local languages, with notable partnerships with tech giants like Microsoft and Google.

These home-grown initiatives underscore Bengaluru’s status as a hub for AI innovation, promising transformative impacts across multiple sectors.

Karnataka Government’s Support to Bengaluru Startups

According to a report released by NASSCOM, Bengaluru is home to over 7,000 startups, solidifying its status as India’s leading startup hub and accounting for 20% of the country’s overall startup activity.

Priyank Kharge, Karnataka minister for IT/BT, recently unveiled a new scale-up program, Hypergrowth Global Karnataka, at London Tech Week. This initiative aims to accelerate the global commercialisation and international market expansion of the best tech companies in Bengaluru and Karnataka.

The program offers local companies access to global mentors, expert scaling advice from leading executives, go-to-market support, and connections to potential new customers and investors to enhance their international growth capabilities.

NVIDIA Rolls Out HelpSteer2 Dataset to Align LLMs

NVIDIA has released HelpSteer2, an open-source dataset designed to train state-of-the-art reward models for aligning LLMs with human preferences. The permissively licensed dataset under CC-BY-4.0 contains 10,681 prompt-response pairs annotated across five attributes on a Likert scale by over 1,000 US-based annotators.

Read the full paper here.

The HelpSteer2 dataset achieves a state-of-the-art 92.0% accuracy on RewardBench’s primary dataset when used to train a reward model with NVIDIA’s 340B Nemotron-4 base model, outperforming all other open and proprietary models as of June 12, 2024.

It is highly data-efficient, requiring only 10,000 response pairs compared to the millions used in other preference datasets, thus significantly reducing computational costs.

It enables the training of reward models that can effectively align large language models like Llama 3 70B to match or exceed the performance of models such as Llama 3 70B Instruct and GPT-4 on major alignment metrics. Additionally, it introduces SteerLM 2.0, a novel model alignment approach that leverages multi-attribute reward predictions to train LLMs on complex, multi-requirement instructions.

“High-quality preference data is crucial for aligning AI systems with human values, but existing datasets are often proprietary or of inconsistent quality,” said Zhilin Wang, senior research scientist at NVIDIA. “

HelpSteer2 provides an open, permissively licensed alternative for both commercial and academic use.

The HelpSteer2 dataset is available on the Hugging Face hub, and the code is open-sourced on NVIDIA’s NeMo-Aligner GitHub repository.

Sentiment Analysis Datasets

HelpSteer2 trains and guides models to behave in ways that people prefer. Additionally, there are many other sentiment analysis models with applications in various fields, helping enterprises accurately understand and learn from their clients or customers.

Some examples include Amazon product data, the multi-domain sentiment dataset, and Sentiment140.

TCS, HCLTech and Others Join Govt’s INR 6,000 Crore Quantum Tech Project

The Indian government is set to partner with leading Information Technology (IT) firms like Tata Consultancy Services (TCS), HCLTech and Tech Mahindra in a INR 6,000 crore initiative aimed at developing software for quantum technologies according to a recent report.

This scheme, designed to support Indian startups and scientists in the emerging quantum tech field, signifies a significant step towards harnessing quantum computing capabilities.

Ajai Chowdhry, co-founder of HCL and chairman of the mission governing board of the National Quantum Mission, revealed that the government plans to engage major IT companies in researching quantum technologies due to the extensive algorithmic requirements in this domain.

“We are going to involve our software companies like TCS, HCL, Tech Mahindra, etc., in this effort. We want them to conduct research on quantum technologies because there is a huge amount of algorithms required for quantum,” he said, according to the report.

The mission is laying groundwork to establish four separate hubs focusing on quantum hardware, quantum encryption, sensors, and materials by August. These hubs, to be located in esteemed research institutions such as Tata Institute of Fundamental Research (TIFR) and Indian Institute of Science (IISc), will operate as non-profit Section 8 companies, empowering their leaders to make strategic decisions.

Chowdhry emphasised the mission’s objective to involve startups in quantum tech, with approximately 50 startups already active in the sector. These startups will receive mentoring and initial seed funding through the thematic hubs, fostering innovation and growth in quantum computing.

“A bank or an electrical grid in India can be attacked by an adequate quantum computer sitting in China. We must start working on making our country quantum secure. We are going to work with different agencies in the government to make them aware that something like this has to be done, or the Reserve Bank of India (RBI) should start to work on creating a policy for making all banks,” said Chowdhry.

“We will create our own quantum computers. For the period we don’t have a quantum computer, we’ll buy a few quantum computers to do research on. But we are not going to use quantum computers only on the cloud, because it’s very expensive,” he added.

Quantum computing, leveraging quantum physics principles, offers unparalleled computational capabilities compared to traditional computing. A quantum computer’s ability to represent multiple values simultaneously enables rapid solution of complex problems that would be impractical for conventional computers.

Global tech giants like Google, IBM, and Intel have heavily invested in quantum computing, with China leading significant funding efforts.

The mission’s long-term goals include developing intermediate-scale quantum computers, secure quantum communications, quantum materials, and quantum devices to advance various sectors, including banking, communications, and navigation.

Chowdhry highlighted the urgency of making India quantum secure to mitigate potential cyber threats, advocating for strategic collaborations and policy development in quantum technology adoption. The mission’s comprehensive agenda encompasses a range of quantum technologies, aiming to position India at the forefront of quantum computing and applications.

6 Best CRMs for Small Business in 2024

Error loading HTML

AWS Teams Up with Accel to Support GenAI Startups in APJ

AWS has announced the launch of the new AWS Generative AI Spotlight program in Asia Pacific and Japan (APJ). This is a four-week accelerator program to support early-stage startups in the region that are building generative AI applications.

In India, the company is partnering with the venture capital firm Accel for this program. Last year, the company and Accel launched ML Elevate 2023, a six-week accelerator that supported 35 generative AI startups in India.

The program will select up to 120 early-stage startups across the region, including 40 from India.

For example, with help from AWS, fintech start-up Fibe has improved customer support efficiency by 30%.

The participants will also have access to the company’s Activate program for startups. They can receive up to $100,000 in AWS credits.

The company’s Generative AI Spotlight program in APJ is collaborating with venture capital firms and organisations in key cities across the region.

Generative AI Accelerator

Additionally, the company has announced a $230 million commitment for generative AI startups to accelerate the creation of generative AI applications worldwide. This funding will provide early-stage companies with AWS credits, mentorship, and education to further their use of AI and ML technologies.

A major portion of the commitment will fund the second cohort of the AWS Generative AI Accelerator program. This 10-week program will provide hands-on expertise and up to one million dollars in AWS credits to each of the top 80 early-stage startups using generative AI to solve complex challenges.

The program will identify top startups in areas such as financial services, healthcare, media, entertainment, business, and climate change. Participants will receive sessions on ML performance enhancement, stack optimisation, and go-to-market strategies, along with business and technical mentorship based on their industry vertical.

AWS’s Support for Startups

Matt Wood, VP of AI products at the company, stated, “With this new effort, we will help startups launch and scale world-class businesses, providing the building blocks they need to unleash new AI applications that will impact all facets of how the world learns, connects, and does business.”

AWS has a long history of supporting startups, with 96% of all AI/ML unicorns running on its platform. The new commitment aims to further accelerate the growth of generative AI startups by providing them with the necessary resources and mentorship.

This is not the first time AWS has committed to helping startups, last month AWS collaborated with Shellkode, a cloud company, to train one lakh women developers in generative AI.

The Future of India’s AI is 1-bit LLMs

Generative AI has frequently sparked discussions about electricity and the need for diverse energy sources. Recently, Ola CEO Bhavish Aggarwal made an interesting comparison between training AI models and running an Ola S1 scooter.

Aggarwal said, “1 H100 NVIDIA GPU consumes 30x electricity in a year as an Ola S1 scooter.” He said that an H100 GPU requires around 8.7 MWh of energy per year, whereas an S1 requires 0.25 MWh/year. “Need a lot of electricity in the future!” he added.

Krishan S Iyer, the CEO of NDR InvIT, called it an incorrect comparison.

Aggarwal pointed out that a necessary step is needed to make AI models efficient within the country. “Not sure why the comparison between GPU and S1 scooter, but the problem is real. Grid capacity is becoming challenging for EV adoption, a lot more so in India,” replied Ganesh Raju, the co-founder of RapidEVchargE.

On the other hand, Pranav Mistry, the founder and CEO of TWO, which recently launched its Sutra line of AI models, disagreed with Aggarwal completely. “No, you need optimised AI models like SUTRA Light and innovations like 1-bit LLM.” Though this might simply be a promotion of TWO’s new AI model, the part about the 1-bit LLM does make sense.

India Needs 1-bit LLMs

The conversation around 1-bit LLM started around February, when Microsoft released its paper titled The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits.

With the conversation shifting towards 1-bit LLMs, this might also be a shifting paradigm for designing hardware specifically optimised for these LLMs. “One-bit LLMs open new doors for designing custom hardware and systems specifically optimised for 1-bit LLMs,” said Furu Wei, one of the researchers of the 1-bit LLM paper.

Wei explained that these quantised models have multiple advantages as they fit on smaller chips, require less memory, and have faster processing.

As for Krutrim, Aggarwal claims that the company is shifting to its own hardware and cloud for AI. “Only if Krutrim has plans for customised GPUs for its native market…,” replied Jainul Thakar.

Mistry further said that Aggarwal probably overlooked the fact that most of the energy consumption by H100 is during training the model, which is a one-time process. “ Inference is the main energy-hungry task,” he said. For inference, H100s are not the most-efficient GPUs.

Since the claim is that we need more electricity to run AI models in the future, the solutions that Krutrim builds should be on the efficient and energy-saving side, which would also be ideal for inference on AI models.

During a discussion with AIM, Adithya S Kolavi, the founder of CongitiveLab, and Adarsh Shirawalmath, the founder of Tensoic, also said that there needs to be better quantisation and optimisation techniques for LLMs to run efficiently for the India market.

Though the performance of these 1-bit LLMs in Indic language is yet to be measured and evaluated, discussions on Hacker News point to the fact that the existing models can also be converted into 1-bit LLMs in the future using the changed hardware.

The Era of 1-bit LLMs

The crux of this innovation lies in the representation of each parameter in the model, commonly known as weights, using only 1.58 bits. Unlike traditional LLMs, which often employ 16-bit floating-point values (FP16) or FP4 by NVIDIA for weights, BitNet b1.58 restricts each weight to one of three values: -1, 0, or 1.

This substantial reduction in bit usage is the cornerstone of the proposed model. It performs as well as the traditional ones with the same size and training data in terms of end-task performance.

Most importantly, it is more cost-effective in terms of factors like latency, memory usage, throughput, and energy consumption. This is why it is necessary for Indian AI research in the future.

This 1.58-bit LLM introduces a new way of scaling and training language models, offering a balance between high performance and cost-effectiveness. Additionally, it opens up possibilities for a new way of computing and suggests the potential for designing specialised hardware optimised for these 1-bit LLMs.

But for now, you still need to train the model from scratch for this optimisation, and the current paradigm is mostly on the NVIDIA H100, which might change soon as well.

India is mostly experimenting with AI’s inference part. One-bit LLMs are the way forward for India’s AI models, not just increasing the electricity requirement since that may not be the ideal way forward.

Рубрика: AI