Рубрика: AI

Image from Pexels

Last winter, I gave a presentation on ‘More predictable time-series model with BQML’ at GDG DevFest Tashkent 2022 in Tashkent, the capital of Uzbekistan.

I was going to share some of the material and code after DevFest I used in the presentation, but time has passed, and new features have been released in BQML that overlap some of the content.

Therefore, I’ll instead mention the new features and some of the things that are still valid briefly.

Time-Series Model and BQML

Time series data is used by many organizations for a variety of purposes, and it’s important to note that “predictive analytics” is all about the “future” in time. Time series predictive analytics has been used in the short, medium, and long term, and while it has many inaccuracies and risks, it has also been steadily improving.

Since “prediction” seems to be so useful, you might be tempted to apply a time series prediction model if you have time series data. But time series prediction models are usually computationally intensive, and if you have a lot of data, it will be more computationally intensive. So it’s cumbersome and hard to process it, load it to the analytics environment and analyze it

If you are using Google BigQuery for data management, you can use BQML (BigQuery ML) to apply machine learning algorithms to your data in a simple, easy, and fast way. A lot of people use BigQuery to process a lot of data, and a lot of that data is often time series data. And BQML also supports time series models.

The basis of the time series model currently supported by BQML is the AutoRegressive Integrated Moving Average (ARIMA) model. The ARIMA model predicts using only existing time series data and is known to have good short-term prediction performance, and since it combines AR and MA, it is a popular model that can cover a wide range of time series models.

However, this model is computationally intensive overall, and since it only utilizes time series data with normality, it is difficult to use it in cases with trends or seasonality. Therefore, ARIMA_PLUS in BQML includes several additional features as options. You can add time series decomposition, seasonality factors, spikes and dips, coefficient changes, and more to your model, or you can go through them separately and manually adjust the model. I also personally like the fact that you can adjust for periodicity by automatically incorporating holiday options, which is one of the benefits of using a platform that doesn’t require you to manually add informations related to dates.

Structure of ARIMA_PLUS (from BQML Manual)

You can refer to this page for more information.

However, when it comes to real-world applications, time series prediction is not as simple as this. Of course, we’ve been able to identify multiple cycles and add interventions to multiple time series with ARIMA_PLUS, but there are many external factors related to time series data, and only very few events happen in isolation. Stationarity can be hard to find in time-series data.

In the original presentation, I looked at how to deal with these real-world time series data for making prediction model- to decompose these time series, clean up the decomposed data, import it into Python, and then weave it with other variables to create a multivariate time series function, estimate causality and incorporate it into a prediction model, and estimate the degree to which the effect varies with changes in events.

New Feature: ARIMA_PLUS_XREG

And in the only last few months, a new feature for creating multivariate time series functions with external variables(ARIMA_PLUS_XREG, XREG below) has become an outright feature in BQML.

You can read all about it here(it’s in preview as of July 2023, but I’m guessing it’ll be available later this year).

I apply the official tutorial to see how it compares to a traditional univariate time series model and we can see how it works.

The steps are the same as in the tutorial, so I won’t duplicate them, but here are the two models I created. First, I created a traditional ARIMA_PLUS model and then an XREG model using the same data but adding the temperature and wind speed at the time.

# ARIMA_PLUS

# ARIMA_PLUS  CREATE OR REPLACE MODEL test_dt_us.seattle_pm25_plus_model  OPTIONS (   MODEL_TYPE = 'ARIMA_PLUS',   time_series_timestamp_col = 'date',   time_series_data_col = 'pm25') AS  SELECT date, pm25  FROM test_dt_us.seattle_air_quality_daily  WHERE date BETWEEN DATE('2012-01-01') AND DATE('2020-12-31')  #ARIMA_PLUS_XREG  CREATE OR REPLACE  MODEL test_dt_us.seattle_pm25_xreg_model   OPTIONS (     MODEL_TYPE = 'ARIMA_PLUS_XREG',     time_series_timestamp_col = 'date',     time_series_data_col = 'pm25') AS  SELECT  date, pm25, temperature, wind_speed  FROM test_dt_us.seattle_air_quality_daily  WHERE  date BETWEEN DATE('2012-01-01') AND DATE('2020-12-31')

A model that uses these multiple data would look something like this

Structure ARIMA_PLUS_XREG (from BQML Manual)

Two models are compared with ML.Evaluate.

SELECT  *   FROM  ML.EVALUATE           (  MODEL test_dt_us.seattle_pm25_plus_model,            (  SELECT  date,  pm25             FROM  test_dt_us.seattle_air_quality_daily              WHERE  date > DATE('2020-12-31')  ))  SELECT  *   FROM  ML.EVALUATE            (  MODEL test_dt_us.seattle_pm25_xreg_model,             (  SELECT  date,  pm25,  temperature,  wind_speed                FROM  test_dt_us.seattle_air_quality_daily                WHERE  date > DATE('2020-12-31')  ),            STRUCT(  TRUE AS perform_aggregation,  30 AS horizon))

Results are below.

ARIMA_PLUS

ARIMA_PLUS_XREG

You can see that the XREG model is ahead on basic performance metrics such as MAE, MSE, and MAPE. (Obviously, this is not a perfect solution, data-dependent, and we can just say that we got another useful tool.)

Multivariate time series analysis is a much-needed option in many cases, but it’s often difficult to apply due to various reasons. Now, we can use it if the reasons are in data and analysis steps. It looks like we have a good option for that, so it’s good to know about it and hopefully it will be useful in many cases.
JeongMin Kwon is a freelance senior Data Scientist in 10+ years of hands-on experience leveraging machine learning models and data mining.

More On This Topic

• Multivariate Time Series Analysis with an LSTM based RNN
• Building a Tractable, Feature Engineering Pipeline for Multivariate Time…
• Top 5 Time Series Methods
• Using PyCaret’s New Time Series Module
• Multiple Time Series Forecasting with PyCaret
• How to label time series efficiently – and boost your AI

Unlocking Generative AI for Content Understanding in Enterprises July 31, 2023 by Clemens Mewald, Head of Product, Instabase

AI is shaping up to be a true paradigm shift across all industries. OpenAI's groundbreaking GPT technology is the most widely known example of how AI is reshaping our daily experiences in ways we couldn’t predict just a few years ago.

Although AI has been around for decades, its practical application to real-world problems has only recently become feasible due to advancements in computing power and the availability of data at a large scale. We are now entering the era of "applied AI," where the breakthroughs achieved by academic research groups are finally becoming accessible for businesses to unlock real value. With the latest developments in generative models, adopting AI has become easier than ever, transforming the landscape of how we utilize this technology.

This article will explore the tangible impact AI is already making in businesses and discuss key factors needed to unleash its full potential.

Applying AI in Business

Enterprises that have been able to leverage AI to solve complex business problems frequently emerge as “category leaders” and command a premium because of their ability to innovate. However, even the most advanced companies have mostly focused on structured data – anything that you would find in a spreadsheet or a database. As it turns out, the vast majority of data (and its potential value) is unstructured and lives in documents, images, audio, or video.

How we deal with this unstructured data has changed over time. Traditionally we relied on template-based methods, where handwritten rules were crafted for each document type. These rules proved reliable as long as the documents were, which only worked for well-structured documents like tax forms that change at the pace of tax codes.

More recently, deep learning techniques have gained popularity to address variability and complexity in content. Some of the models used for this task have been pre-trained on millions of pages, which makes them good at many things but not great at anything in particular. In other words, they are able to comprehend any data, as long as they are fine-tuned on relevant training examples. These examples have to be created by humans who laboriously teach models complex tasks, like processing invoices. If the task changes, those same humans have to go back to the classroom to teach a new generation of models.

Fast forward to today, the latest innovation in Large Language Models (LLMs) promises yet another step-function in performance. GPT-4, trained by OpenAI, is several orders of magnitude larger than previous models, and can be said to be great at many things out-of-the-box (it passes most standardized tests and even the bar exam). These models exhibit a remarkable aptitude for comprehending the nuances of intent and the intricate relationships embedded within text. The implications are twofold: (1) Natural language is now a viable form factor for most products. You can express your intent in natural language and GPT writes code, SQL queries, poems, emails, even entire books. (2) For most use cases, GPT is able to answer content-related questions without any fine-tuning, eliminating the need to laboriously create example data. The value of applying GPT-like models to content hasn’t gone unnoticed by OpenAI’s competitors, who are launching models that are better suited to skim through thousands of pages.

As a result, we expect a tidal wave in business value coming from unstructured data. Enterprises used to have to pick just a handful of use cases where the laborious task of fine-tuning models was justified by a costly-to-create business case. With GPT-like models, new products will emerge that are both easier to use (because of natural language interfaces) and more powerful (because they can comprehend all of your content out of the box). This will transform legal processes, streamline medical practice management, accelerate financial analysis, expedite mortgage approvals, and more generally supersede most legacy workflows. For the first time in history, this technology will impact the day-to-day operations not only of big tech but also of traditional enterprises, small businesses, and professionals, truly ushering in the era of applied AI.

Challenges and Looking Forward

As businesses look forward to applying these technologies, they should keep a few things in mind to truly derive value:

• Align AI with Business Knowledge: To implement AI in practice, it’s important that impactful applications come from the overlap of what’s possible with AI and deep knowledge about the business. Otherwise, there is a risk of numerous small AI projects that fail to deliver significant value. AI initiatives should not be viewed merely as technology center driven endeavors; they must be closely aligned with the business lines they serve and be driven out of business units.
• Assess and Optimize Current Processes: To begin, companies should start by obtaining a good understanding of their current processes. Take for example a process like customer onboarding which requires complexity of systems, sensitive data, and multiple processes. Map out such processes to identify the parts which can be automated. Next, evaluate multiple processes to identify which one would gain the most from the application of AI.
• Manage the Roll Out: Once ready to roll out the solution do so in small experiments. It's critical for the success of AI initiatives that there is no financial or reputation damage. Invest in technologies that enable quick integration of new LLM models, validations, and human checks by your teams to ensure there is a low risk of major hallucinations. Additionally, almost all serious applications of AI should have policy layers that sit on top of the AI system to enforce guardrails. For example, when you generate text, you should probably ensure that it doesn’t include profanity. Or whenever AI is used to predict important economic values like house prices, it is important to have humans review whenever the new prediction deviates significantly from a previous estimate.

As we embrace the era of applied AI, businesses that navigate these challenges with foresight will unlock the full potential of AI, driving value creation across their operations and have agility to adopt the next wave of AI innovation that enters the market. By embracing this new technology, enterprises and small businesses alike can thrive in the transformative AI landscape.

About the Author

Clemens Mewald is the Head of Product at Instabase. With over 15 years of experience in the industry, Clemens Mewald has built a successful track record as a product and technology leader in the AI and machine learning space. Previously Clemens held leadership positions at Databricks, where he spent more than three years leading the product team for Machine Learning and Data Science. Before Databricks, Clemens served on the Google Brain Team building AI infrastructure for Alphabet, where his product portfolio included TensorFlow and TensorFlow Extended (TFX). Clemens holds an MSc in computer science from UAS Wiener Neustadt, Austria, and an MBA from MIT Sloan.

Related

For India to be a leader in AI, it must invest heavily on its research. However, the country’s research and development spending ranks among the world’s lowest in terms of Gross Domestic Product (GDP) percentage. In 2022, India spent just 0.65% of its GDP on R&D, significantly lower than what the US or China spends, which is 2.9% and 2.2% respectively.

Moreover, India also ranks far below than the US and China when it comes to AI for peer-reviewed journals, accounting for just 5.56% of AI journals published in 2021, according to Stanford University’s AI Index 2023 report. But the report does acknowledge India’s growing significance in the AI ecosystem, with India’s share increasing from just 1.3% in 2010. In comparison, China remains the clear leader with 39.78% share with the US contributing 10.03% share,

“Although the US and China are ahead of India at present, the acceleration in our country is higher,” Prof. Arnab Bhattacharya, Dept. of Computer Science & Engineering, IIT Kanpur, told AIM. And now, the National Research Foundation (NRF) could help further accelerate the state of AI research in the country. The Indian government’s decision to clear a bill enabling the setting up of the NRF with a corpus of INR 50,000 crore (USD 6 billion), has been welcomed by most researchers and academics. “By fostering a supportive and nurturing environment for AI research, we can capitalise on this momentum and potentially surpass the advanced nations within the next decade,” Prof. Bhattacharya said.

Can NRF boost AI research?

Currently, one of the biggest challenges for AI researchers in the country is allocation of funds. Expensive hardware like GPU/TPU and robot parts, often imported, limits affordability. Timely and adequate funds are crucial to address this challenge effectively. Hence, the government’s INR 50,000 crore allocation over five years could boost researchers’ finances; however, the distribution of funds will encompass all dimensions of research, leaving uncertainty about the specific allocation for AI researchers.

Additionally, brain drain is a significant concern within the Indian research ecosystem. Today, many Indian researchers are employed by the top AI labs in the world such as Google Deepmind, OpenAI, Microsoft etc. According to a Moneycontrol report, there are only around 500 active AI researchers in the country. Most researchers move out of the country to states offering better opportunities both in terms of funding, resources and also attractive career prospects. The NRF could indirectly address this issue by streamlining funding channels for researchers, removing bureaucratic hurdles, and improving access to resources.

The NRF is indeed an affirmative step for fostering research in India, according to Prof. Bhattacharya. “One core advantage it promises is its assimilation of the various scientific research funding bodies into one, including the social sciences, et al. This can be instrumental in streamlining funding processes and increasing accountability.”

Moreover, he believes there is a need for larger and more consolidated AI centres that bring together researchers and experts from different institutes and the NRF could be a step in the right direction. In most cases, individual researchers face several limitations to contribute significantly to path-breaking research. “The NRF can definitely help in tackling these challenges. For starters, it can take a lead in promoting and supporting large collaborative research centres, which would ensure the confluence of researchers with diverse expertise. This would help enrich collaborative research in the booming AI sector,” he said.

AI research needs a moonshot approach

Nonetheless, what AI research in India needs is a moonshot approach, according to Prof Amrutur Bharadwaj, research head and director, ARTPARK. In a previous interaction with AIM, discussing why an Indian equivalent of OpenAI is unlikely, Prof. Bharadwaj said that to boost AI research, certain amounts of funds need to be allocated to the field and for a certain period of time. “I think it has to be driven by the government and they need to pump in money into the ecosystem for a certain period of time.”

But given the wide scope of the NRF, it is unlikely that NRF could be a moonshot approach. However, the NRF aims to forge collaborations among academia, industry, and government departments and research institutions. In India, there is enough research happening in educational institutions and universities; however, they often find it difficult to actually translate that research into useful products or companies that can scale.

Perhaps, NRF can change this. By bringing together different stakeholders in AI, such as researchers, scientists, industry professionals, and academics, the NRF can foster a collaborative ecosystem for knowledge exchange, joint research projects, and technology transfer. But much of it will depend on NRF’s successful implementation, allocation of resources, and active participation from all stakeholders.

Problems with NRF

Even though the draft of the National Research Foundation Bill has not been made public yet, it has been made known that the NRF emulates the model of the National Science Foundation of the US. The NRF bill 2023 will replace the existing Science and Engineering Board (SERB) Act, 2008. But researchers have expressed concerns that replacing SERB could disrupt ongoing schemes and funding, potentially impacting the already precarious state of science granting.

Prof. Bhattacharya raises another significant concern about the centralisation of the entire process by NRF. “Centralisation necessarily brings with it its own issues. Policies may become too dependent on the incumbent board/body that controls the agency, or NRF may turn out to be less favourable for a particular kind of field/project, for example AI, leaving little scope of improvement. If we can address such concerns comprehensively with due deliberations, then the road may be smoother.”

Moreover, out of the to-be-allocated INR 50,000 crore, only INR 14,000 crore will come from the government, whereas, the remaining INR 36,000 is expected to be raised from the private sector. Many feel this is a very ambitious target, which won’t be fulfilled, atleast, in the first few years. India’s R&D investment from industry stands at 36.8%, contrasting with 80-90% in tech-heavy economies like Japan, South Korea or even Israel. Attracting corporate investors for the NRF may be challenging and it is not known how the government plans to source such a substantial sum from the industry players and other philanthropic sources.

The post Can India’s Billion-Dollar Funding Agency Propel AI Research? appeared first on Analytics India Magazine.

Recently, Uncle Sam went ahead with Worldcoin scanning iris using orb-sized devices in exchange for crypto tokens, which he assumes is going to be the human identifier needed to survive. He thinks that in the near future, AI will not only take our jobs but also might steal our identity and it would become extremely difficult to distinguish between a human and AI entity.

After scanning your iris, it will generate a World ID that can be used on the World Wallet. This lets you prove you are a real and unique person online while remaining completely private.

The three key problems Altman is specifically hoping to address are – identification, decentralization and providing universal basic income (UBI) in an AI dominated world.

While these things seem revolutionary, people are raising several questions:

Why Iris Scanning?

The most common question that comes to our mind is why Iris scans for biometrics to prove you are a human. According to Worldcoin co-founder Alex Blania, the company explored various methods to verify uniqueness and humanity, but ultimately, iris biometrics emerged as the only viable solution.“For a number of reasons, we didn’t want to go down that path,” he says. “But it really was the only solution,” he said.

Altman, during his India visit, said that iris scanning is one of the approaches. However, they are working on other solutions to verify humans. “I think it’s good as a world for us to try many approaches to this problem” he expressed, saying that eventually these systems will help in governance, alongside enhancing the ability of equitably sharing the benefits of access.

Why do you need Worldcoin, when you have Aadhar?

In 2009, Government of India launched Aadhar Card to provide a unique identity to the citizens of the country. It was meant to help the government fund a number of social welfare schemes and programs that are focused on the poor and most vulnerable sections of society and to streamline delivery mechanisms under welfare schemes, thereby ensuring transparency and efficiency. WorldCoin’s goal is pretty much similar to that but on a global level. It aims to give back the wealth generated by AI back to society.

Worldcoin and Aadhar Card
Before worldcoin, indian government run a huge drive to collect biometric data of indian citizens, and issue a national identity to them.
Aadhar Number is an Indian government issued identity, and UPI is the payment infrastructure built on top of…

— hitesh.eth (@hmalviya9) July 27, 2023

However, Worldcoin has yet to present a convincing argument to instill more trust in their platform and identification system than in the existing government and established methods of identification.

Five days after its launch it came under scrutiny in France .“The legality of [Worldcoin’s data] collection seems questionable, as do the conditions for storing biometric data,” a CNIL spokesperson confirmed according to the reports.

Worldcoin says its data collection including The World Coin Eye Scan and World Coin Orb is based on blockchain technology. Every blockchain is made of nodes. Nodes are generally computer systems that contain a copy of a blockchain’s primary protocol and its entire transaction history.

One of the key advantages of blockchain technology is its ability to create immutable records. Once a transaction is recorded on the blockchain, it becomes unchangeable and permanent, ensuring a transparent and secure record of all transactions.

Altman seems to be unfazed by the criticism he is receiving. In a tweet post he said Worldcoin may or may not work out similar to any really ambitious project but he won’t stop trying because this is how progress happens.

“in either case, we especially love our haters, it gives us energy, please keep it coming!” he exaggerated.

like any really ambitious project, maybe it works out and maybe it doesn’t, but trying stuff like this is how progress happens.
in either case, we especially love our haters, it gives us energy, please keep it coming!

— Sam Altman (@sama) July 24, 2023

This also brings us to question, if we should simply trust Altman’s vision of the world he is trying to create? While many associate AI with a dystopian future, Altman doesn’t completely share that belief. Altman regularly talks about AI being the next revolution, and how it is going to take away jobs.

Altman is looking at a utopian society, where no one will be impacted because of AI, and would reap the benefits of technological progress. Two years back in a blog post, Altman wrote “we need technology to create more wealth, and policy to fairly distribute it.” According to Altman’s piece titled “Moore’s Law for Everything,” AI has the potential to create enough wealth to provide every adult in the U.S. with $13,500 per year within a decade.

The post Should You Trust Worldcoin? appeared first on Analytics India Magazine.

Image Created by Author with Playground AI

Before we dive into the details of this exciting development, let's explore a scenario to understand it better. Picture yourself as a Senior Data Scientist leading a sophisticated image classification project. Your TensorFlow-based model is performing remarkably well. However, as you add more functionalities, you notice that some team members prefer JAX for scalability, while others favor PyTorch for its user-friendly nature. As a team lead, how can you ensure seamless collaboration while maintaining the model's efficiency across various deep-learning frameworks?

Recognizing this challenge, the Keras team introduces Keras Core—an innovative multi-backend implementation of the Keras API with support for TensorFlow, JAX, and PyTorch. This library will evolve into Keras 3.0 by Fall 2023. But before we jump straight to Keras 3.0, let's first take a brief look at Keras' history.

A Brief History of Keras and the Road to 3.0

In 2015, François Chollet introduced Keras, an open-source deep-learning library written in Python. This simple yet powerful API quickly gained popularity among researchers, students, and professionals by streamlining complex neural network building. Over time, Keras received significant enhancements, making it even more appealing to the deep-learning community. Eventually, Keras became an integral part of TensorFlow, Google's cutting-edge deep learning framework. Around the same time, Facebook's AI Research lab developed PyTorch, known for its intuitive and flexible model building. At the same time, JAX emerged as another powerful framework for high-performance machine learning research. As these frameworks gained momentum, developers started to experience the dilemma of choosing between the frameworks. This led to further fragmentation in the deep learning community.

Acknowledging the challenges faced posed by the fragmented frameworks, the developers of Keras decided to revolutionize the library once again, giving rise to Keras 3.0.

Salient Features of Keras 3.0

Keras 3.0 empowers you to collaborate with your team effectively. You can develop sophisticated models with a unified front end by combining the strengths of TensorFlow, JAX, and PyTorch. Here are some of the features due to which Keras 3.0 is an absolute game changer:

1. Multi-Backend Support

Keras 3.0 acts as a super-connector, enabling seamless use of TensorFlow, JAX, and PyTorch together. Developers have the freedom to mix and match the best tool for their specific tasks without changing the code.

2. Performance Optimization

Performance Optimization is the key feature of Keras 3.0. By default, Keras 3.0 leverages the XLA (Accelerated Linear Algebra) compilation. XLA Compilation optimizes your mathematical computations making them run much faster on hardware like GPUs and TPUs. It also allows you to dynamically select the best backend for your AI model ensuring optimal efficiency. Such performance optimization features are a big deal allowing you to train more models, experiment more, and get results much quicker.

3. Expanded Ecosystem Surface

Your Keras models can be used as the PyTorch Modules, TensorFlow SavedModels, or part of JAX's large-scale TPU training infrastructure. This means that you can take advantage of the strengths of each framework. So, with Keras 3.0’s expanded ecosystem you are not locked into a single ecosystem. It is just like a universal adapter that lets you connect your favorite device to any machine.

4. Cross-Framework Low-Level Language

The introduction of the keras_core.ops namespace is a groundbreaking feature that allows you to write custom operations once and use them across different deep-learning frameworks effortlessly. The keras_core.ops provide a set of tools and functions that resemble the popular NumPy API, which is a widely used library for numerical computations in Python. This level of cross-framework compatibility fosters code reusability and encourages collaboration.

5. Progressive Disclosure of Complexity

This design approach of Keras 3.0 sets it apart from other deep learning frameworks. Imagine that you are a beginner and you want to build a simple neural network using Keras 3.0. It presents you with the most straightforward workflows at the start. Once you get comfortable with the basics, you can access all the advanced features and the low-level functionalities. It doesn’t restrict you to predefined workflows only. The beauty of this approach lies in its adaptability and is welcoming for both beginners and seasoned deep learning practitioners.

6. Stateless API for Layers, Models, Metrics, and Optimizers

In the context of deep learning, state refers to the internal variables and parameters that change during the training process. However, JAX operates on the principle of statelessness meaning that the functions do not have mutable variables or internal states. Keras 3.0 embraces the statelessness of JAX via a Stateless API. It allows the essential component of deep learning i.e. layers, models, metrics and optimizers to be designed in a stateless manner. This unique compatibility makes Keras 3.0 an indispensable tool in modern AI development.

Getting Started with Keras 3.0

Keras Core is compatible with Linux and MacOS systems. Setting up Keras 3.0 is a straightforward process. Here is the step-by-step guide for you to follow:

1. Clone and Navigate to the Repository

Clone the repository into your local system using the following command

git clone https://github.com/keras-team/keras-core.git

Change your root directory to the clone keras-core using:

cd keras-core

2. Install Dependencies

Open your interval and run the following command to install the required dependencies.

pip install -r requirements.txt

4. Run the Installation Command

Run the following script to handle the installation process:

python pip_build.py --install

5. Configure the Backend

By default, Keras Core strictly requires TensorFlow as the backend framework but you can configure it using the following two ways:

Option 01: You can set the KERAS_BACKEND environment variable to your preferred backend option.

export KERAS_BACKEND="jax"

Option 02: You can edit the local Keras configuration file located at ~/.keras/keras.json. Open the file in a text editor and change the "backend" option to your preferred backend.

{      "backend": "jax",      "floatx": "float32",      "epsilon": 1e-7,      "image_data_format": "channels_last"  }

6. Verify the Installation

To ensure that Keras Core is correctly installed with your chosen backend, you can test it by importing the library. Open a Python interpreter or a Jupyter Notebook and run the following:

import keras_core as keras

Ending Notes

While Keras 3.0 has some limitations, such as its current TensorFlow dependency and limited tf.data support with other backends, the future potential of this framework is promising. Keras has released the beta version as of now and they encourage the developers to provide their valuable feedback. If you are interested to explore more about it, you can find the documentation of Keras Core (Keras 3.0) here. Don’t be afraid to try new ideas. Keras 3.0 is a powerful tool and it's an exciting time to be part of the evolution.
Kanwal Mehreen is an aspiring software developer with a keen interest in data science and applications of AI in medicine. Kanwal was selected as the Google Generation Scholar 2022 for the APAC region. Kanwal loves to share technical knowledge by writing articles on trending topics, and is passionate about improving the representation of women in tech industry.

More On This Topic

• Working With The Lambda Layer in Keras
• Building and Training Your First Neural Network with TensorFlow and Keras

Every maths lover on the internet has once in their lifetime come across the name Professor Gilbert Strang. The YouTube star, even before it was a thing, devoted his entire life to the MIT maths department.

In the early 2000s, MIT decided to create OpenCourseWare and instead of trying to sell courses, made them open to everyone. Then they recorded Strang’s ‘linear algebra course maths 18.06’. “In fact, I think India has more people who know that, of course, than any other country in the world,” Strang shared in an exclusive interview with AIM.

Read more: Inside The Mind of Gilbert Strang

Strang’s teaching methods remain irreplaceable even today as he “structures the class so that ideas seem to flow from the students into proofs”, said a former student, Jesse Michel. “Every class includes a cool maths trick or joke that keeps the class laughing,” he added.

The icon continues to stay relevant even after his retirement through his online courses and teaching methods. We have picked out his 6 most viewed and loved topics at MIT.

Linear Algebra 18.06

This lecture by Strang has achieved an impressive milestone, surpassing 10 million views on OpenCourseWare (OCW). This remarkable feat tells us why he is considered one of the most renowned mathematicians globally. At 88, Professor Strang delivered this lecture as his last, and received a standing ovation for his lifelong commitment to the field of mathematics.

Here’s the link to the course.

Mathematical Methods For Engineers (2)

Regarded as a graduate-level course, this lecture stands out as one of the finest discussions on numerical analysis available. Delivered during the Fall of 2000, the lecture delves profoundly into the difference methods for ordinary differential equations.

Here’s the link to the course.

Matrix Methods in Data Analysis, Signal Processing, and Machine Learning

A comprehensive understanding of linear algebra is key to mastering and developing machine learning algorithms, particularly in deep learning and neural networks. This course offers a thorough review of linear algebra, its practical applications in probability, statistics, and optimisation. Moreover, it presents an in-depth and clear explanation of the field of deep learning.
https://www.youtube.com/watch?v=t36jZG07MYc&t=1s

Here’s the link to the course.

Computational Science and Engineering (1)

Crafted in the Fall of 2008, this course is acclaimed for being a practical and valuable mathematics course. Students often express that it stands as one of the most useful maths courses they have ever encountered. The primary objectives of the course are to see the underlying patterns in numerous significant applications and to equip learners with methods to compute solutions.

Here’s the link to the course.

Highlights of Calculus

In this course, Strang presented a concise series, providing a foundational introduction to the fundamental concepts of calculus, shedding light on its mechanics and significance. With an inclusive target audience, this series is resourceful for beginners as well as for educators pursuing self-learning.

Here’s the link to the course.

Wavelength, Filter Banks and Applications

The course written in Spring 2003 was co-instructed by Prof Kevin Amaratunga. The professors aim for the right balance of theory and “applications”. The course has no specific prerequisites, although a basic knowledge of Fourier transforms is recommended. The introduction begins with time-invariant filters and basic wavelets and further covers the analysis of filter banks and wavelets, design methods and so on.

Here’s the link to the course.

The post 6 Courses by Professor Gil Strang that the Internet Loves appeared first on Analytics India Magazine.

Amazon Web Services (AWS) and venture capital firm Accel has announced ML Elevate 2023, a six-week accelerator programme aimed at empowering Indian startups that are building generative AI applications.

ML Elevate aims to support generative AI startups by providing them access to impactful AI models and tools, business and technical mentorship, curated resources, the AWS Activate programme, and up to USD 200,000 in AWS Credits.

Other benefits include peer support from a community of leading AI and ML startup founders and the opportunity to scale production-ready generative AI applications on Amazon SageMaker JumpStart.

Generative AI startups that have already developed a Minimum Viable Product (MVP) and intend to seek funding in the next 12-18 months are eligible to apply.

Selected startups will undergo live virtual masterclasses, including fireside chats and panel discussions by investors, industry leaders, and AWS experts.

The panel of speakers includes Tom Mason, Chief Technology Officer, Stability AI; Vishal Dhupar, Managing Director, Asia-South, NVIDIA Graphics Pvt Ltd; Poonacha Kongetira, SVP Engineering at SambaNova (US); Anupam Datta, co-founder, President and Chief Scientist, TruEra (US); Apurva Kalia, Senior Researcher, Tufts University (USA), and others.

The cohort will also have a fundraising opportunity through a dedicated Demo Week to pitch leading VC funds, angel investors, and industry leaders.

“Generative AI offers an unprecedented opportunity to revolutionize and transform businesses globally. Through ML Elevate, our aim is to support generative AI startups to develop industry-focused solutions and innovate to drive the digital economy, ” Vaishali Kasture, Director of AWS India and South Asia at Amazon Web Services India Pvt. Ltd, said.

The post AWS and Accel announce ML Elevate 2023 to Support Indian GenAI Startups appeared first on Analytics India Magazine.

With rising moderator controversies, decline in user numbers, Stack Overflow announced the integration of generative AI on their platform with Overflow AI – something the company has been hinting about since April. Through this, Stack Overflow aims to improve quality and relevance of resources to user queries. The feature will not only be available on the public platform but also extend to Stack Overflow for Teams. Will this be a promising step for Stack Overflow towards getting back to being relevant again?

OverflowAI helps you create content within your Stack Overflow for Teams community, by taking existing sources within your instance and creating a tagging framework and series of questions and answers. pic.twitter.com/TDIcIu7DNU

— Stack Overflow (@StackOverflow) July 27, 2023

Arms Against ChatGPT

Ever since the ChatGPT rage began last December, Q and A community platform for programmers Stack Overflow, has been in news, either for their new policies and regulations or simply for their user switch. Developers who have been using Stack Overflow for copying code started switching to ChatGPT. With the launch of Code Interpreter plugin on ChatGPT, it has only made it even more coveted as a coder platform.

When ChatGPT was finding its presence everywhere, in December last year, the company took the bold step of banning content generated by ChatGPT on their platform. In addition to attributing high inaccuracy as a reason for not allowing content from ChatGPT and other LLMs, the volume of answers that can be easily produced via GPT platforms which require a detailed review by an SME has burdened their volunteer-based quality curation infrastructure.

The company faced a 12% slump in the number of user visits – 247 million visits in December from 279 million in November. However, the company confirmed to AIM that it did not face any material impact to traffic and that the dip was a normal one brought about by a seasonal shift owing to holidays in December.

You cannot have the cake and eat it too

ChatGPT is believed to be trained on Stack Overflow data. GPT-3 paper that talks about training datasets includes Common Crawl which implies everything on the internet, including Stack Overflow.

For a company losing customers to another platform, owing to better and smoother experience for query resolution, it was only obvious that Stack overflow will next prevent ChatGPT from feeding on their datasets to better themselves or get paid – that’s pretty much what happened.

Stack Overflow, closely following Reddit, announced that it will charge AI developers for accessing its programming driven community questions. The company believes that community platforms that support LLM models should in some way be compensated for their contribution, which will help these companies reinvest back to grow the community.

The Internal Tussle

With initially blocking AI generated content on the platform leading to a number of users being suspended and posts being removed, Stack Overflow then went back to allowing AI content on the platform. With lack of uniformity, the company’s stance on AI-generated content came under scrutiny with curators, contributors and moderators participating in moderation strikes from June.

The moderators then came to an interim solution where AI-generated content will be checked against ‘strong’ and ‘weak’ heuristics, to serve as guidelines to evaluate a content’s quality and appropriateness. This would determine if a post needs to be removed or allowed on the platform. Furthermore, the moderators were able to convince Stack Overflow to continue providing data dumps and API access – something the company had been contemplating removing. Amidst the tussle, Stack Overflow finally announced generative AI.

What’s Ahead?

With Overflow AI, the company will be utilising their vast database consisting of over 58 million questions and answers from their community – a rich database exclusive to the company. Stack Overflow will integrate this database into IDEs (Integrated Development Environments) which are software applications that help developers write and manage codes. By this integration, the developers are facilitated with simpler coding resolution- akin to the multiple functions that ChatGPT’s Code Interpreter brings in.

While Stack Overflow is definitely late to the AI race, with players such as GitHub Copilot X having implemented GPT-4 in March itself, the vast database it brings might prove helpful for the company. Whether this would help bring back lost users, is something that needs to be seen.

The post Stack Overflow’s Bumpy Road to Gen AI Adoption appeared first on Analytics India Magazine.

If you track the investments in the generative AI field by VCs like Andreessen Horowitz, Sequoia Capital, or Y Combinator, you will notice that most of the exits are because of them getting acquired by Meta, Google, or other big tech firms. Case in point: Startups such as MosaicML and Neeva.AI, which got acquired within two years of their formation. And yet, it becomes difficult for investors to assess which one would actually give them the exit they need.

The field of generative AI is moving so fast with new models, new approaches, and new startups that it becomes difficult to track what each offers that is different from the thousand others in the field. When it comes to investing in generative AI startups, the most important aspect for any investor is identifying the startups’ unique approach and how exactly they can offer the ROI.

“We have to ensure that everything is new and I can come back and generate a new picture or a kind of application that is unique and cannot be done by any other 500 startups in the field,” said Som Pal Choudhury from Bharat Innovation Fund (BIF) in an exclusive interview with AIM.

Choudhury said that BIF has been focused on investing in deep tech startups since the beginning, and generative AI is no different from investments in AI, blockchain, or 5G networks. The firm hasn’t invested in any generative AI startups at the moment as Choudhury explained that it is very essential to deeply assess what is the moat for generative AI startups and if it is even worth investing in them since the big tech is already leading the way.

“There are mostly four types of startups that are there in the generative AI space and recognising this makes it easier to mitigate the risks related to these investments,” Choudhury explained. According to him, speaking with startups in the field, there are mostly five type of “generative AI startups”-

1. Core Elements and LLMs: Companies developing core elements and Large Language Models (LLMs).
2. Enterprise Application Focus: Startups creating specific enterprise applications using generative AI technologies.
3. Open Source Modifications: Companies using open-source models and modifying them with their proprietary data for enterprise applications.
4. Orchestrating Multiple Models: Startups combining various generative AI models to create more specific enterprise applications.
5. Basic Wrappers: Companies that provide basic wrappers around existing models, without significant differentiation.

Eximius Ventures, a firm that has already invested in two startups in the field also is employing a similar strategy. Pearl Agarwal, the founder of the firm told AIM that there are two types of startups in the field – Horizontal platforms that are focusing on core innovation such as OpenAI and vertical platforms that are utilising technologies from these companies to create applications in diverse use cases. “In both these cases, the exit strategy is similar to other tech investments mostly about getting acquired by a tech-giant or going into IPO,” Agarwal said.

Horizontal platform driven companies that are driving infrastructural innovation in the field, there are very few startups, not just in India, but globally. It is very hard for smaller VCs to invest in these companies as they require a huge amount of capital and there it is also difficult to assess if these would even work in the long run.

“We are essentially peeling the layers of these generative AI startups to understand if a startup is actually doing something innovative or is just building a wrapper around ChatGPT or such models and selling itself as a startup. We don’t want to invest in any of these companies,” emphasised Choudhury.

Unlike big VCs, BIF is still figuring out if any startups are worth investing in since they are just building wrappers around already existing technology. On the other hand, Eximius Ventures decided to focus on the vertical platform companies, who are leveraging these platforms for specific use cases instead of building their own.

India has never been ahead in the AI infrastructure race anyway and the same should be expected from startups coming out of the country right now in the generative AI field. Choudhury from BIF said it is hard to assess if a startup would generate revenue or not, no matter in which of the five layers it is working on. “We’ll see a lot more applications based and enterprise focused applications come up, and that’s where I think Indian startups will contribute the most,” said Choudhury.

Open source has made it easier for new startups to form, but difficult for investors to assess if they are worth investing

By the time a product is built, newer and potentially superior models may already have flooded the market. Venkat Vallabhaneni, co-founder and managing partner at Inflexor Ventures told AIM that this dynamic environment actually serves as an essential litmus test for startup founders. “We value and seek out founders and companies that demonstrate a deep understanding of this highly evolutionary field. These are entrepreneurs who factor the rapid pace of development into their product design and strategies,” he said.

Similar to Choudhury, Vallabhaneni also said that some successful startups decouple their products from the foundational models, allowing their offerings to evolve and adapt in tandem with the market. But even then it becomes very hard to find the distinguishing factors of these startups.

Additionally, part of an investment firm’s examination includes a deep dive into the technical aspects of the startup. “We need to understand the types of model they are using and assess whether they have a solid grasp of these models and their implications. While it may be challenging to build intellectual property (IP) on the model itself, startups can create their unique IP based on the implementation of the model and the specific use cases they are addressing,” said Vallabhaneni.

While IP might not always be the paramount concern, possessing enforceable IP is a definite advantage. It could prove to be an important asset that enhances the attractiveness of the company in potential acquisition scenarios.

One of the ways that the investors believe that startups could stand out in the generative AI field is through their intellectual property data. Even if the startups are leveraging open source and wrapping their technology around other technologies, leveraging proprietary data and then solving for use cases would be essential to stand apart.

“Our due diligence process for generative AI startups mirrors that of other deep tech startups, with an added focus on regulatory aspects,” said Vallabhaneni. It’s crucial that these startups are cognizant of the shifting regulatory landscape for generative AI and are flexible enough to adapt to any new developments.

Amid investors like Sequoia Capital, Andreessen Horowitz, and Y Combinator, who are investing in big-tech backed companies such as Inflection or Anthropic, other VCs find it easier to invest in more application driven generative AI startups that are building out of open source and hope to get acquired by a big tech soon. Looking at the trend, it looks like it might take mostly 2-3 years.

The post Generative AI Startups May Get Profitable in 2 Years appeared first on Analytics India Magazine.

A key element of a robotics future will be how humans can instruct machines on a real-time basis. But just what kind of instruction is an open question in robotics.

New research by Google's DeepMind unit proposes that a large language model, akin to OpenAI's ChatGPT, when given an association between words and images, and a dash of data recorded from a robot, creates a way to type instructions to a machine as simply as one converses with ChatGPT.

The paper by DeepMind, "RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control," authored by Anthony Brohan and colleagues, and posted within a blog post, introduces RT-2, what it calls a "vision-language-action" model. (There is a companion GitHub repository as well.) The acronym RT stands for "robotics transformer."

The challenge is how to get a program that consumes images and text to produce as output a series of actions that are meaningful to a robot. "To enable vision-language models to control a robot, they must be trained to output actions," as they put it.

The key insight of the work is, "we represent robot actions as another language," write Brohan and team. That means that actions recorded from a robot can become the source of new actions the way being trained on text from the Internet makes Chat GPT generate new text.

The actions of the robot are encoded in robotics transformer as coordinates in space, known as degrees of freedom.

"The action space consists of 6-DoF [degree of freedom] positional and rotational displacement of the robot end-effector, as well as the level of extension of the robot gripper and a special discrete command for terminating the episode, which should be triggered by the policy to signal successful completion."

The tokens are fed into the program during training in the same phrase as the language tokens of words and the image tokens of pictures. Robot coordinates become just another part of a phrase.

The use of coordinates is a significant milestone. Usually, the physics of robots are specified via low-level programming that is different from language and image neural nets. Here, it's all mixed together.

The RT program builds upon two prior Google efforts, called PaLI-X and PaLM-E, both of which are what are called vision-language models. As the name implies, vision-language models are programs that mix data from text with data from images, so that the program develops a capacity to relate the two, such as assigning captions to images, or to answer a question about what's in an image.

While PaLI-X focuses only on image and text tasks, PaLM-E, introduced recently by Google, takes it a step farther by using the language and image to drive a robot by generating commands as its output. RT goes beyond PaLM-E in generating not just the plan of action but also the coordinates of movement in space.

In the case of RT-2, it is a successor to the version from last year, RT-1. The difference between RT-1 and RT-2 is that the first RT was based on a small language and vision program, EfficientNet-B3. But RT-2 is based on the PaLI-X and PaLM-E, so-called large language models. That means they have many more neural weights, or, parameters, which tends to make programs more proficient. PaLI-X has 5 billion parameters in one version and 55 billion in another. PaLM-E has 12 billion.

Once the RT-2 has been trained, the authors run a series of tests, which require the robot to pick things up, move them, drop them, etc., all by typing natural-language commands, and a picture, at the prompt, just like asking ChatGPT to compose something.

For example, when presented with a prompt,

Given  Instruction: Pick the object that is different from all other objects

where the image shows a table with a bunch of cans and a candy bar, the robot will generate an action accompanied by coordinates to pick up the candy bar

Prediction: Plan: pick rxbar chocolate. Action: 1 128 129 125 131 125 128 127

where the three-digit numbers are keys to a code book of coordinate movements.

A key aspect is that many elements of the tasks might be brand-new, never-before-seen objects. "RT-2 is able to generalize to a variety of real-world situations that require reasoning, symbol understanding, and human recognition," they relate.

"We observe a number of emergent capabilities," as a result. "The model is able to re-purpose pick and place skills learned from robot data to place objects near semantically indicated locations, such as specific numbers or icons, despite those cues not being present in the robot data.

"The model can also interpret relations between objects to determine which object to pick and where to place it, despite no such relations being provided in the robot demonstrations."

In tests against RT-1 and other programs, the RT-2 using either PaLI-X or PaLM-E is much more proficient at completing tasks, on average achieving about 60 percent of tasks with previously unseen objects, versus less than 50 percent for the previous programs.

There are also differences between PaLI-X, which is not developed specifically for robots, and PaLM-E, which is. "We also note that while the larger PaLI-X-based model results in better symbol understanding, reasoning and person recognition performance on average, the smaller PaLM-E-based model has an edge on tasks that involve math reasoning." The authors attribute that advantage to "the different pre-training mixture used in PaLM-E, which results in a model that is more capable at math calculation than the mostly visually pre-trained PaLI-X."

The authors conclude that using vision-language-action programs can "put the field of robot learning in a strategic position to further improve with advancements in other fields," so that the approach can benefit as language and image handling get better.

There is one caveat, however, and it goes back to the idea of control of the robot in real time. The large language models are very compute-intensive, which becomes a problem for getting responses.

"The computation cost of these models is high, and as these methods are applied to settings that demand high-frequency control, real-time inference may become a major bottleneck," they write. "An exciting direction for future research is to explore quantization and distillation techniques that might enable such models to run at higher rates or on lower-cost hardware."