Рубрика: AI

Watsonx Brings AI Visibility to Banking Systems September 27, 2023 by Doug Eadline

A new set of AI-based code conversion tools is available with IBM watsonx. Before introducing the new "watsonx," let's talk about the previous generation Watson, perhaps better known as "Jeopardy!-Watson."

The original Watson system was initially developed as an experiment to answer questions on the quiz show Jeopardy! and in 2011, the Watson computer system competed against champions Brad Rutter and Ken Jennings, winning the first place prize of $1 million (IBM donated 100% of Watson's winnings to charity, with 50% of those winnings going to World Vision and 50% going to World Community Grid.) Except for one embarrassing blunder ("Its largest airport is named for a World War II hero; its second largest for a World War II battle," Watson "What is Toronto?"), Watson crushed the human competition.

In terms of hardware, Jeopardy!-Watson was no slouch. The Watson cluster was composed of ninety IBM Power 750 servers, each with a 3.5 GHz POWER7 server; the system provided 2,880 POWER7 processor threads and 16 TBytes of RAM.

Despite the Jeopardy! success and multi-billion dollar investment, Watson's further development and the creation of Watson Health never gained wide acceptance. In January 2022, IBM sold Watson Health for a reported $1 billion to private equity firm Francisco Partners.

Building on the Foundation

Building AI systems like ChatGPT, Bard, or Llama takes time and resources. Grown from information found across the internet, these massive large language modes (LLM) can answer complex questions on many different topics but lack specific knowledge. One way to envision these foundation LLMs is to consider them a well-built house. The basics include windows, walls, stairs, doors, rooms, closets, and pathways, but no furnishings exist. One might ask a house-foundation model, "How do I get from the living room to the upstairs bathroom?" to which the LLM will answer. It may provide different (answers) pathways through the house, but it will get you to your destination.

If one were to ask, "How big is the flat screen TV in the basement?" the house-foundational model would be a loss. This information is specific to a particular home and thus would require augmenting the foundational model with information about your home.

Providing foundational models with more domain-specific data is the gateway to widespread use. Of course, asking ChatGPT to write an essay on War and Peace or to provide Python code to multiply two matrices is impressive, but asking ChatGPT to write a 300-word report on last year's sales numbers won't work. You need to add furniture to the house.

Adding furniture means tuning the model specifically for your organization or business. This process introduces many "foundational questions" that may have a considerable importance in the success of your new model. For instance,

1. What Foundational model should I use?
2. What if the foundational provides biased answers?
3. What if my training data is biased?"

As suggested by these questions, putting AI to work with domain knowledge requires careful consideration of data providence and quality.

Meet watsonx

In May of 2023, IBM introduced watsonx. It might be convenient to describe watsonx as next-generation or evolutionary, but in reality, watsonx has little connection to Jeopardy!-Watson. watsonx is designed to assist with the GenAI technologies sweeping through virtually every market sector. That is, watsonx will help answer the types of questions mentioned above.

Watsonx has three main components listed below. These components offer advanced machine learning, data management, and generative AI capabilities to train, validate, tune, and deploy AI systems across the business with speed, trusted data, and governance. It helps facilitate the entire data and AI lifecycle, from data preparation to model development, deployment, and monitoring.

1. watsonx.ai allows users to train, validate, tune, and deploy generative AI, foundation models, and machine learning capabilities easily and build AI applications in a fraction of the time with a fraction of the data.
2. watsonx.data is an open lakehouse storage solution for AI projects, providing seamless data transfer and management. It provides a single entry point to data regardless of location and allows real-time or batch, all in the context of a robust governance framework.
3. watsonx.governance is an end-to-end governance solution. It covers the full AI lifecycle data provenance, ensuring traceability, transparency, explainability, and bias detection.

Getting Smart About The Future

Jen McComas, Executive Architect, Principal Global Industry Engineer for IBM Global Industries

In a recent interview with Jen McComas, Executive Architect and Principal Global Industry Engineer for IBM Global Industries, Financial Services, about watsonx in the banking industry, the topic of modernization was discussed. While most enterprises need to address modernization at some point, the Banking industry is under pressure to find increased revenue, reduce costs, and find growth opportunities.

One of the challenges facing the banking industry is the continued use of COBOL applications. COBOL is the second oldest high-level programming language and has been the mainstay of many banking and financial institutions. COBOL is still in use because it works, and the applications are trusted. There are issues, however. The first is the shortage of COBOL programmers. The second is the need for modern features and capabilities in a programming language to address the challenges mentioned above. New banking approaches that include hyper-personalized service need modern tools that support sophisticated analysis. Banks are pressured to modernize or build new systems to address their needs.

Generative AI provides a path forward for many of these challenges. Reports of using ChatGPT to translate code from one language to another are impressive. However, these success stories are usually based on small examples and may need further vetting to be trusted in production environments.

The challenges of code translation are where watsonx comes into play. Based on the watsonx.ai foundation models, IBM has developed the watsonx Code Assistant for Z that uses a fine-tuned coding model that can make it easier for developers to incrementally modernize COBOL business services and selectively convert to high-quality Java code. In addition, using watsonx, developers can gain insights and recommendations for existing COBOL code that may not have been possible without the original author's knowledge (the person who retired right after doing the Y2K fixes).

Code Assistant for Z allows users to tune and customize the underlying foundation model with their standards and best practices. At the same time, data-source attribution will provide transparency into the potential origins of the generated code.

When translating COBOL to Java, the resulting Java code is "human readable, " allowing the developer to modify or enhance the generated code. To provide confidence and verification for the new code, automatic generation of test cases for the existing COBOL and the new Java codes are produced.

While discussing these new tools with IBM's McComas, she mentioned attributes not generally associated with GenAI tools. Attributes such as traceability, transparency, explainability, and bias detection are part of watsonx.governance component that allows organizations to create open and observable conversion processes. With emerging AI (and existing) regulation, end-to-end data provenance will be essential for risk management and compliance in regulated industries.

Not Just for COBOL

Providing rigorous code conversion is not just limited to the banking industry. Almost every sector has code that needs trusted modernization or conversion.

Another application for IBM watsonx Code Assistant is a generative AI service that helps developers create Ansible content more efficiently. The Red Hat Ansible Lightspeed assistant reads plain English entered by a user and then interacts with IBM watsonx foundation models to generate code recommendations for automation tasks that are then used to create Ansible Playbooks. Results are written in YAML code and adhere to accepted best practices for Ansible experienced users. The Ansible Lightspeed assistant reduces the need for highly skilled Ansible developers.

It should be noted Watsonx Code Assistant for Z is a new addition to the watsonx Code Assistant product family, along with IBM watsonx Code Assistant for Red Hat Ansible Lightspeed. These solutions are powered by IBM's watsonx.ai code model, which is expected to have knowledge of 115 coding languages, having learned from 1.5 trillion tokens. At 20 billion parameters, it is on target to become one of the most significant generative AI foundation models for code automation. The watsonx Code Assistant product portfolio will extend over time to address other programming languages, improve time to value for modernization and address growing skills challenges for developers.

Converting that legacy Fortran application to Java, C, or even Python may be possible very soon because watsonx knows what you are talking about.

Related

OpenAI is reportedly getting into the hardware business. WHOOP recently announced that it is partnering with OpenAI to launch WHOOP Coach today, the most advanced generative AI feature to ever be released by a wearable.

WHOOP said that most guidance, from wearables or elsewhere, is based on information that might not apply to your body, and your goals. WHOOP Coach acts like a search engine for your body, using GPT-4, OpenAI’s most advanced generative AI system, to generate highly personalized, highly specific recommendations and guidance.

WHOOP Coach uses special WHOOP algorithms, a custom-made computer program that learns from data, the latest in sports science, and your own body information to find patterns and links in your WHOOP data. Leveraging OpenAI’s latest technology, WHOOP Coach instantly generates personalized, conversational answers to your inquiries about health, fitness, and well-being.

On the privacy aspect WHOOP said that it doesn’t share users data with other companies. When one uses WHOOP Coach, their information is made anonymous and analyzed by a third-party language learning model partner.

Also, renowned iPhone designer Jony Ive and OpenAI CEO Sam Altman have been in discussions regarding the development of a new AI hardware device, sources familiar with the matter told The Information.

Moreover, OpenAI recently announced that ChatGPT can now “see, hear and speak”. The company plans to introduce these updates to its paying users within the next two weeks. Voice functionality will be restricted to the iOS and Android apps, but image processing capabilities will be accessible across all platforms.

The post OpenAI to Challenge Apple Watch appeared first on Analytics India Magazine.

Sponsored Content

By Jim Dowling, Co-Founder & CEO, Hopsworks

Introduction

This article introduces a unified architectural pattern for building both Batch and Real-Time machine learning (ML) Systems. We call it the FTI (Feature, Training, Inference) pipeline architecture. FTI pipelines break up the monolithic ML pipeline into 3 independent pipelines, each with clearly defined inputs and outputs, where each pipeline can be developed, tested, and operated independently. For a historical perspective on the evolution of the FTI Pipeline architecture, you can read the full in-depth mental map for MLOps article.

In recent years, Machine Learning Operations (MLOps) has gained mindshare as a development process, inspired by DevOps principles, that introduces automated testing, versioning of ML assets, and operational monitoring to enable ML systems to be incrementally developed and deployed. However, existing MLOps approaches often present a complex and overwhelming landscape, leaving many teams struggling to navigate the path from model development to production. In this article, we introduce a fresh perspective on building ML systems through the concept of FTI pipelines. The FTI architecture has empowered countless developers to create robust ML systems with ease, reducing cognitive load, and fostering better collaboration across teams. We delve into the core principles of FTI pipelines and explore their applications in both batch and real-time ML systems.

Unified Architecture for ML Systems as
Feature/Training/Inference Pipelines

The FTI approach for this architectural pattern has been used to build hundreds of ML systems. The pattern is as follows — a ML system consists of three independently developed and operated ML pipelines:

• a feature pipeline that takes as input raw data that it transforms into features (and labels)
• a training pipeline that takes as input features (and labels) and outputs a trained model, and
• an inference pipeline that takes new feature data and a trained model and makes predictions.

In this FTI, there is no single ML pipeline. The confusion about what the ML pipeline does (does it feature engineer and train models or also do inference or just one of those?) disappears. The FTI architecture applies to both batch ML systems and real-time ML systems.

Figure 1: The Feature/Training/Inference (FTI) pipelines for building ML Systems

The feature pipeline can be a batch program or a streaming program. The training pipeline can output anything from a simple XGBoost model to a parameter-efficient fine-tuned (PEFT) large-language model (LLM), trained on many GPUs. Finally, the inference pipeline can be a batch program that produces a batch of predictions to an online service that takes requests from clients and returns predictions in real-time.

One major advantage of FTI pipelines is that it is an open architecture. You can use Python, Java or SQL. If you need to do feature engineering on large volumes of data, you can use Spark or DBT or Beam. Training will typically be in Python using some ML framework, and batch inference could be in Python or Spark, depending on your data volumes. Online inference pipelines are, however, nearly always in Python as models are typically training with Python.

Figure 2: Choose the best orchestrator for your ML pipeline/service.

The FTI pipelines are also modular and there is a clear interface between the different stages. Each FTI pipeline can be operated independently. Compared to the monolithic ML pipeline, different teams can now be responsible for developing and operating each pipeline. The impact of this is that for orchestration, for example, one team could use one orchestrator for a feature pipeline and a different team could use a different orchestrator for the batch inference pipeline. Alternatively, you could use the same orchestrator for the three different FTI pipelines for a batch ML system. Some examples of orchestrators that can be used in ML systems include general-purpose, feature-rich orchestrators, such as Airflow, or lightweight orchestrators, such as Modal, or managed orchestrators offered by feature platforms.

Some of the FTI pipelines, however, will not need orchestration. Training pipelines can be run on-demand, when a new model is needed. Streaming feature pipelines and online inference pipelines run continuously as services, and do not require orchestration. Flink, Spark Streaming, and Beam are run as services on platforms such as Kubernetes, Databricks, or Hopsworks. Online inference pipelines are deployed with their model on model serving platforms, such as KServe (Hopsworks), Seldon, Sagemaker, and Ray. The main takeaway here is that the ML pipelines are modular with clear interfaces, enabling you to choose the best technology for running your FTI pipelines.

Figure 3: Connect your ML pipelines with a Feature Store and Model Registry

Finally, we show how we connect our FTI pipelines together with a stateful layer to store the ML artifacts — features, training/test data, and models. Feature pipelines store their output, features, as DataFrames in the feature store. Incremental tables store each new update/append/delete as separate commits using a table format (we use Apache Hudi in Hopsworks). Training pipelines read point-in-time consistent snapshots of training data from Hopsworks to train models with and output the trained model to a model registry. You can include your favorite model registry here, but we are biased towards Hopsworks’ model registry. Batch inference pipelines also read point-in-time consistent snapshots of inference data from the feature store, and produce predictions by applying the model to the inference data. Online inference pipelines compute on-demand features and read precomputed features from the feature store to build a feature vector that is used to make predictions in response to requests by online applications/services.

Feature Pipelines

Feature pipelines read data from data sources, compute features and ingest them to the feature store. Some of the questions that need to be answered for any given feature pipeline include:

• Is the feature pipeline batch or streaming?
• Are feature ingestions incremental or full-load operations?
• What framework/language is used to implement the feature pipeline?
• Is there data validation performed on the feature data before ingestion?
• What orchestrator is used to schedule the feature pipeline?
• If some features have already been computed by an upstream system (e.g., a data warehouse), how do you prevent duplicating that data, and only read those features when creating training or batch inference data?

Training Pipelines

In training pipelines some of the details that can be discovered on double-clicking are:

• What framework/language is used to implement the training pipeline?
• What experiment tracking platform is used?
• Is the training pipeline run on a schedule (if so, what orchestrator is used), or is it run on-demand (e.g., in response to performance degradation of a model)?
• Are GPUs needed for training? If yes, how are they allocated to training pipelines?
• What feature encoding/scaling is done on which features? (We typically store feature data unencoded in the feature store, so that it can be used for EDA (exploratory data analysis). Encoding/scaling is performed in a consistent manner training and inference pipelines). Examples of feature encoding techniques include scikit-learn pipelines or declarative transformations in feature views (Hopsworks).
• What model evaluation and validation process is used?
• What model registry is used to store the trained models?

Inference Pipelines

Inference pipelines are as diverse as the applications they AI-enable. In inference pipelines, some of the details that can be discovered on double-clicking are:

• What is the prediction consumer — is it a dashboard, online application — and how does it consume predictions?
• Is it a batch or online inference pipeline?
• What type of feature encoding/scaling is done on which features?
• For a batch inference pipeline, what framework/language is used? What orchestrator is used to run it on a schedule? What sink is used to consume the predictions produced?
• For an online inference pipeline, what model serving server is used to host the deployed model? How is the online inference pipeline implemented — as a predictor class or with a separate transformer step? Are GPUs needed for inference? Is there a SLA (service-level agreements) for how long it takes to respond to prediction requests?

MLOps Principles

The existing mantra is that MLOps is about automating continuous integration (CI), continuous delivery (CD), and continuous training (CT) for ML systems. But that is too abstract for many developers. MLOps is really about continual development of ML-enabled products that evolve over time. The available input data (features) changes over time, the target you are trying to predict changes over time. You need to make changes to the source code, and you want to ensure that any changes you make do not break your ML system or degrade its performance. And you want to accelerate the time required to make those changes and test before those changes are automatically deployed to production.

So, from our perspective, a more pithy definition of MLOps that enables ML Systems to be safely evolved over time is that it requires, at a minimum, automated testing, versioning, and monitoring of ML artifacts. MLOps is about automated testing, versioning, and monitoring of ML artifacts.

Figure 4: The testing pyramid for ML Artifacts

In figure 4, we can see that more levels of testing are needed in ML systems than in traditional software systems. Small bugs in data or code can easily cause a ML model to make incorrect predictions. From a testing perspective, if web applications are propeller-driven airplanes, ML systems are jet-engines. It takes significant engineering effort to test and validate ML Systems to make them safe!
At a high level, we need to test both the source-code and data for ML Systems. The features created by feature pipelines can have their logic tested with unit tests and their input data checked with data validation tests (e.g., Great Expectations). The models need to be tested for performance, but also for a lack of bias against known groups of vulnerable users. Finally, at the top of the pyramid, ML-Systems need to test their performance with A/B tests before they can switch to use a new model.

Finally, we need to version ML artifacts so that the operators of ML systems can safely update and rollback versions of deployed models. System support for the push-button upgrade/downgrade of models is one of the holy grails of MLOps. But models need features to make predictions, so model versions are connected to feature versions and models and features need to be upgraded/downgraded synchronously. Luckily, you don’t need a year in rotation as a Google SRE to easily upgrade/downgrade models — platform support for versioned ML artifacts should make this a straightforward ML system maintenance operation.

Example ML Systems

Here is a sample of some of the open-source ML systems available built on the FTI architecture. They have been built mostly by practitioners and students.

Batch ML Systems

• Electricity Demand Prediction (452 github stars)
• NBA Game Prediction (152 github stars)
• Premier league football score predictions (101 github stars)
• Churn prediction (113 github stars)

Real-Time ML System

• Online Credit Card Fraud (113 github stars)
• Crypto Price Prediction (65 github stars)
• Loan application approval (113 github stars)

Summary

This article introduces the FTI pipeline architecture for MLOps, which has empowered numerous developers to efficiently create and maintain ML systems. Based on our experience, this architecture significantly reduces the cognitive load associated with designing and explaining ML systems, especially when compared to traditional MLOps approaches. In corporate environments, it fosters enhanced inter-team communication by establishing clear interfaces, thereby promoting collaboration and expediting the development of high-quality ML systems. While it simplifies the overarching complexity, it also allows for in-depth exploration of the individual pipelines. Our goal for the FTI pipeline architecture is to facilitate improved teamwork and quicker model deployment, ultimately expediting the societal transformation driven by AI.

Read more about the fundamental principles and elements that constitute the FTI Pipelines architecture in our full in-depth mental map for MLOps.

More On This Topic

• Feature Store Summit 2022: A free conference on Feature Engineering
• How causal inference lifts augmented analytics beyond flatland
• Building Machine Learning Pipelines using Snowflake and Dask
• Answering Questions with HuggingFace Pipelines and Streamlit
• Kedro-Airflow: Orchestrating Kedro Pipelines with Airflow
• Natural Language Processing Pipelines, Explained

On September 27, 2023, the technology realm experienced a momentous event with the emergence of Luda, a groundbreaking company striving to popularize Artificial Intelligence (AI). Luda empowers the general populace to construct, nurture, and deploy their unique AI entities, by integrating play into the learning process via an engaging sandbox experience.

Composed of seasoned experts from renowned organizations like Google Research, Disney, and Zynga, Luda has unveiled an unprecedented system for Real-Time Reinforcement Learning (RT-RL). This innovative system unlocks new realms of artistic expression and social interaction. Alongside this, the company has successfully secured $7 million in funding, orchestrated by BITKRAFT and Compound, and featuring notable contributors such as Jeff Dean and Illia Polosukhin, a co-author of the Transformers paper.

A New Creative Horizon: Mels

Luda has developed Mels, a pioneering browser-based simulator, providing a real-time platform for the generative AI to breathe life into abstract concepts. With this innovation, users can seamlessly construct AI entities, referred to as agents, using essential components within a virtual sandbox. These self-assembled characters are then invigorated by Luda’s RT-RL, following the laws of real-world physics, providing an array of interactive and unpredictable situations without the intricacies of coding or manual animation.

This avant-garde approach by Luda is reshaping the landscape of AI agent development through accelerated learning, significantly reducing the training duration from days to mere minutes. It also offers Low-Latency Inference, allowing these AI agents to operate on everyday consumer devices rather than being reliant on expensive server data centers, making interaction with agents instantaneous and user-friendly.

Empowering Human Creativity in AI

Vijay Sundaram, Luda’s Founder and CEO, emphasized, “At Luda, we believe people belong at the center of AI – not the other way around. That’s why we’re building UGC gameplay for the AI generation. The exciting thing about technology paradigm shifts is the chance to create fundamentally new experiences that weren’t possible before. We started researching and developing novel generative AI techniques like RT-RL five years ago in Google Research, to create a new consumer category: immersive, interactive AI agents anyone can create through an experience that’s as accessible and joyful as play – a new kind of playground bringing together human creativity, social connection, and user-generated agents.” He further shared the vision for Luda to be a convergence of human ingenuity, and user-generated agents, providing an experience that’s exhilarating and accessible.

Luda aims to provide a platform that goes beyond being just a game or intellectual property. It is shaping a new creative medium, allowing every user to construct their AI agents using Lego-like building blocks. This approach aligns with the revolutionary creativity and user-generated content witnessed in platforms like Minecraft and Roblox, positioning Luda as a harbinger of a new era of social and immersive experiences.

Changing the Consumer Interaction Landscape

Scott Rupp, a Partner at BITKRAFT Ventures, articulated his thoughts on Luda’s potential impact, “Luda’s generative AI breakthrough has the potential to transform how consumers interact with AI, and gameplay is the ideal use case. Luda isn’t just building an IP, game or platform, but developing an entirely new creative medium, one that empowers every user to truly craft AI agents through Lego-like building blocks. Just as Minecraft and Roblox ushered in a new era of active creativity and UGC, Luda is poised to introduce a fundamentally new category of social and immersive experiences, ”

Additionally, Michael Dempsey, Managing Partner at Compound, expressed his excitement about Luda's advancements in creating intelligent agents that are scalable and customizable with negligible inference cost, enabling the construction of a unique AI company. “Luda’s breakthroughs in crafting custom, scalable, and physically intelligent agents with almost zero inference cost have enabled them to build a full-stack AI company unlike any other. We've known the team since they started their R&D at Google Research five years ago and they've since assembled one of the few AI labs with world-class researchers and creatives working shoulder to shoulder. We couldn’t be more excited for category-defining experiences like Mels that were never before possible,” said Dempsey.

Conclusion

In summary, Luda's emergence marks the inception of an innovative approach to AI development, democratizing AI through playful learning and creative exploration. With its pioneering technologies, Luda is poised to redefine the realms of interactive and social experiences, opening doors to uncharted territories in AI and creative expression. The technological landscape keenly watches as Luda steps into a new epoch, merging human creativity with advanced AI, and crafting experiences that were previously deemed impossible.