Рубрика: AI

Enabling spatial understanding in vision-language learning models remains a core research challenge. This understanding underpins two crucial capabilities: grounding and referring. Referring enables the model to accurately interpret the semantics of specific regions, while grounding involves using semantic descriptions to localize these regions.

Developers have introduced Ferret, a Multimodal Large Language Model (MLLM), capable of understanding spatial referring across any granularity or shape in an image and accurately grounding open-vocabulary descriptions. Ferret uses a novel hybrid representation combining continuous features and discrete coordinates to represent image regions. Its spatial-aware visual sampler handles varying sparsity in shapes, allowing it to process diverse region inputs like free-form shapes, bounding boxes, and points.

Ferret's approach enables it to excel in classical grounding and referring tasks and surpass other MLLMs in localization-demanding and region-based multimodal communication. This article delves into Ferret's architecture and methodology, highlighting its impressive performance in various multimodal language tasks. Let's explore this further.

Ferret : Superior Performance in Referring and Grounding Tasks

Referring in a model is a capability that allows the model to comprehend the semantics of given specific regions accurately whereas grounding makes it essential for the model to use the given semantic descriptions to localize the regions. Although they might differ in their respective tasks, both referring and grounding have the same fundamental concept: alignment of spatial semantics and information. However, despite sharing the same concept, existing models learn grounding and referring individually. Although the method works, it poses a hurdle in achieving human-like capabilities since humans can learn from one task, and apply the learnings to other tasks seamlessly, and are able to effortlessly integrate grounding/referring capabilities with reasoning and daily dialogue. The Ferret framework takes inspiration from the above mentioned gap in existing MLLM frameworks and studies three main questions:

1. How to unify grounding and referring capabilities in the framework, and how will their unison benefit one another?
2. Humans use versatile types of regions like box, point, scribble, free-form shapes for referring? How to represent these versatile regions?
3. How to make grounding and referring instruction-following, robust, and open-vocabulary, that are critical for their practical and real-time applications?

The Ferret framework is a novel refer and ground Multimodal Large Language Model that attempts to target these questions. The Ferret framework chooses a Multimodal Large Language Model as its foundation owing to their remarkable global vision and language understanding capabilities. Furthermore, to unify the grounding and referring capabilities, the Ferret framework represents the coordinates of regions in natural language numerical form. However, in practice, it is inefficient to use box coordinates or even single points to represent versatile region shapes like scribbles, strokes, or complex polygons as these shapes are critical for enhanced precision and more universal human-model interaction. To tackle this issue, the Ferret framework employs a spatial-aware visual sampler that acquires the visual regions for regions irrespective of the shape, thus negotiating with varying sparsity in these shapes. The framework then combines the continuous visual features with discrete coordinates to represent the visual regions in the input, resulting in the creation of a hybrid region representation in Ferret.

The Ferret framework deploys the above methods to resolve input that mixes free-form text with referred regions, and is able to seamlessly generate the coordinates for each groundable object with generating text to ground the mentioned objects in the output. By doing so, Ferret is the first framework to process free-formed input regions in Multimodal Large Language Models. Furthermore, the Ferret framework absorbs remarkable open-vocabulary capabilities of spatial localization and understanding, allowing the framework to achieve superior performance when evaluated on conventional grounding and referring tasks.

Moving along, the Ferret framework seeks inspiration from three existing AI frameworks including Multimodal Large Language Models, MLLMs for Referring and Grounding, and Unifying Grounding and VL Understanding.

The introduction of Large Language Models including GPT, DALL-E, PaLM, LLaMA, and BLOOM, has changed the landscape in NLP research, resulting in significant advancements of multimodal language models. The earlier multimodal language models focussed primarily on large scale image-text generation with some notable examples being PaLI, SimVLM, GIT, BLIP-2, FLAMINGO, CM3, and PaLI-X. However, since the Flamingo framework achieved efficient integration of LLMs with a pre-trained CLIP image encoder through cross-gated attention blocks resulting in remarkable multimodal few-shot learning capabilities. The current research is looking for ways to utilize pre-trained large language models for visual instruction tuning with notable examples being MiniGPT-4, Otter, InstructBLIP and more. What’s more is that recent models like Emu and GILL have shown remarkable success in using MLLMs for image generation and image retrieval. The Ferret framework also refers to prior research that focuses on unifying text and bounding box output for Vision Language models.

Ferret : Methodology and Architecture

Hybrid-Region Representations

Point, box, and free-form shapes are the three dominant formats that a language model uses when referring to specific regions. On one hand, the point and the box format can be accurately represented by coordinates, mapping free form shapes is a bit challenging since free-form shapes are versatile. Being versatile, free-form shapes can encompass a wide array of regions including masks, polygons, and scribbles. Using coordinates to depict free-form shapes is a complex task that hinders the model’s capability to learn to establish a correlation between the regions and the corresponding coordinates. Furthermore, the use of coordinates for free-form shapes is computationally expensive and obscure.

To tackle this problem and to generalize across all three formats, the Ferret framework proposes a hybrid region representation that synergizes continuous visual features with discrete coordinates to refer to a particular region.

For continuous visual features, for a given region, the Ferret framework first constructs a 2D binary mask of the same size as the image, and marks a value 1 within the targeted region while assigning a value 0 outside the region. The model then extracts the binary mask together with the extracted image feature map, and then sends it to the spatial-aware visual sampler.

Architecture

The architecture of the Ferret model comprises three main components

1. An image encoder to extract image embeddings.
2. A Spatial Aware Visual Samples to extract regional continuous features.
3. A Large Language Model to model text, image, and region features jointly.

The image is first feeded into the pre-trained visual encoder to extract the image embeddings. For text inputs, the framework first uses a pre-trained LLM tokenizer to tokenize the text sequence, and then projects these tokens into text embeddings. For referred regions, Ferret appends a special token and the coordinates as a placeholder for continuous features after the region name. If the region’s name is unknown or is complex to describe as a result of inclusion of several objects, the framework just uses area or region name.

One of the major challenges dealing with referred regions is that their shape can be quite varying, meaning they can have different shapes, and are not just limited to rectangle boxes or points. Referred regions with irregular shapes cannot be processed with traditional methods like Grid-based processing including patch attention or convolution techniques. To tackle this issue, the Ferret framework proposes a Spatial-Aware Visual Sampler. For a given extracted feature map with a binary region mask, the Ferret model first randomly samples N number of points within the binary region mask.

For every individual point, the model obtains its feature by performing bilinear interpolation. The N points are then fed into a waterfall of blocks with each of them passing through three different stages: sampling, gathering, and pooling. In the Sampling phase, a fixed number of points are sampled from N number of points available using FPS or Farthest Point Sampling algorithm that guarantees adequate coverage. In the second step, for each sample point, the framework searches for its k nearest neighbors from the pool of available N points. For each group, the model then fuses the features of a sample point with its neighbor points. In the final step, the Ferret framework conducts a max pooling to fuse k neighbor features into one feature to act as the representation for the point sampled. By performing these three steps, the Ferret framework is left with fewer points but features space with higher density because it not only incorporates the features of local neighbors but also their relative positions.

GPT-Assisted Visual Data Generation

Dialogue Instruction Tuning Data is of critical importance to Multimodal Large Language Models are they not only help in converting existing dataset by templates, but they also help the model understand human intention and generate appropriate response. A majority of MLLMs use a few-shot prompting method to obtain visual instruction tuning data, where the model provides textual description of scenes in the image along with human annotated dialogues as few-shot demonstrations. However, existing instruction tuning methods focus primarily on describing the entire image without specifying spatial-related information explicitly. The Ferret framework emphasizes on region-based knowledge to collect refer and ground instruction tuning data in three steps.

1. In addition to using global captions and objects, the framework provides symbolic scene description that describes the physical relationship between the region captions and objects while also providing their coordinates.
2. For human-annotated dialogues, the framework adds coordinates after groundable objects or regions either in input or output or both with the dialogues focussing primarily on specific regions that helps in prompting the language model implicitly to follow the similar patterns for new dialogue generation.
3. It might be possible that the dialogue generated by the framework might not follow the rules and patterns as instructed by few-shot examples and the system prompts. To tackle this issue, the framework again uses a language model to refine the dialogues generated by the model initially.

Spatial Negative Mining

Prior research has demonstrated that multimodal large language models have a high probability of hallucinating when responding to Yes or No questions. To ensure the Ferret model does not hallucinate in similar conditions, the framework employs Spatial Negative Mining approach with Image-Conditioned Category Localization and Semantics-conditioned Category Localization. Both these methods ask the model to localize specific object categories that enable the model to recognize the absence of certain objects in the image.

Ferret : Results and Experimentation

To analyze its performance, the Ferret framework is evaluated on conventional grounding and referring benchmarks after which the framework is evaluated in a more complex multimodal chatting task and testing its refer-and-ground capabilities.

The model’s capability to understand referring is evaluated by how accurately a model can understand the semantics of the referred region given a referred region in the image or the question. To measure the model’s accuracy, objects, the most basic semantics are considered first as it is not only fundamental but also easy to define. To mimic human-level versatility, the framework replaces the location of the object within the image with a free form shape, a box, and a point. For a free-form shape, the model randomly generates strokes within the Ground Truth object for simulation. For box, the Ferret framework uses the ground truth bounding box provided by the LVIS component. Finally, for point, the model randomly samples a point within the ground truth object that is also near the boundary of the ground truth object. The results on the three types of referring are demonstrated in the following image.

The Ferret framework demonstrates remarkable performance in referential dialogue tasks, making room for integration with different visual learning tasks, especially the ones with grounding outputs. To assess its grounding capability, the Ferret framework first subjects itself to benchmark visual grounding tasks with a generative paradigm. The framework then evaluates its ability on grounded captioning tasks to measure the alignment between the regions and the words.

In visual grounding tasks, the framework aims to ground language queries into aligned regions of the image, and as it can be seen in the following image, the Ferret framework demonstrates remarkable performance across all benchmarks, and the performance is comparable to the one achieved by specialized fine-tuning methods.

For grounded captioning tasks, the model needs to generate a caption, and then ground the generated noun phrases to image regions. The final prediction made by the model consists of three components: visual regions as boxes, text captions, and grounding alignments between boxes and words. The results are demonstrated in the following image, and as it can be observed, the framework delivers performance comparable to state of the art methods.

Finally, multimodal chatting is one of the most desired capabilities within a MLLM, and existing MLLMs primarily evaluate detailed descriptions, conversation, and complex reasoning with the language model as a judge. However, as no dataset evaluates multimodal chatting with mandatory referring or grounding actions, it leaves a gap. To bridge this gap, the Ferret framework covers three region-based questions to evaluate its referring and grounding capabilities in multimodal chatting tasks. The results are demonstrated in the following image.

Finally, the Ferret framework is compared directly against the state of the art GPT framework, and the results are demonstrated below.

Final Thoughts

In this article, we have talked about Ferret, a multimodal large language model demonstrating remarkable grounding and referring capabilities. The Ferret framework can refer to image regions irrespective of its shape, and can establish grounding for text predicted by the model automatically. Ferret employs a spatial-aware visual sampler capable of handling varying sparsity displayed by different shapes to extract the continuous features of versatile regions. As a result, the Ferret framework can input diverse region inputs including free-form shapers, bounding boxes, and points.

Hello,

I hope this message finds you in the midst of exciting projects and breakthroughs in the ever-evolving field of Machine Learning.

Today, we're proud to announce a significant addition to our catalog at Machine Learning Mastery. Known for our detailed, code-centric guides, we're taking a leap further into the realms of Computer Vision with our latest offering.

Introducing "Machine Learning in OpenCV." This ebook is designed to open the door for you to use the machine learning module in OpenCV. It is to lift you up to a new level on your OpenCV projects and to unleash the tremendous potential of computer vision beyond mending pixels. It seamlessly bridges foundational concepts on image processing with advanced applications in machine learning, offering an enriching experience for both beginners and seasoned professionals in the field.

Inside, you'll discover:

🔍 In-depth tutorials on image processing and transformation techniques.
🖥️ Step-by-step guides on implementing image classification and object detection with OpenCV.
💡 Advanced methodologies in image segmentation and video processing in real-time.
📚 Practical use-cases and projects to apply your skills in real-world scenarios.
🔧 Integrating advanced ML models with OpenCV for cutting-edge image analysis.

And much more!

Celebrate this launch with us by redeeming an exclusive 20% early bird discount using the code "20offearlybird" at checkout. But hurry, this special offer is available for a limited time only!

👉 Machine Learning in OpenCV

This ebook is more than just a guide; it will lead you into the depths of Machine Learning and Computer Vision, empowering you to leverage the full potential of OpenCV. Whether you are a seasoned professional or just starting, "Machine Learning in OpenCV" is designed to enhance your skills and broaden your horizons in this dynamic field.

If you have any questions or need further information, feel free to reply to this email directly. Let's embark on this journey of discovery and mastery in the world of computer vision together.

Jason, Machine Learning Mastery Founder

GET YOUR COPY NOW

More On This Topic

• How our Obsession with Algorithms Broke Computer Vision: And how…
• TensorFlow for Computer Vision — Transfer Learning Made Easy
• Open Source Datasets for Computer Vision
• Computer Vision in Agriculture
• 10 AI Project Ideas in Computer Vision
• 5 Applications of Computer Vision

Traditional relational databases struggle with unstructured data – the text, images, videos, and social media feeds that flood our modern world. But graph databases, with their unique structure, offer a powerful tool for taming this chaos and extracting valuable insights. Here’s how they bring a game-changing perspective to unstructured data analytics:

Modeling relationships, not just tables

Forget rows and columns. In the world of graph databases, data takes on a whole new dimension, with relationships taking center stage. Think of it as shifting from a static photo to a dynamic movie, where the connections between things matter as much as the things themselves. Here’s how this paradigm shift unlocks hidden value in unstructured data:

1. Representing the Interconnected World: Relational databases treat data as isolated entities in neatly organized tables. This approach falls short when dealing with unstructured data, where the context and connections are often as crucial as the information itself. Imagine analyzing customer reviews – in a graph database, you can connect reviews of different products by the same user, uncovering patterns and preferences that wouldn’t be visible in isolated tables.

2. Capturing the Nuances of Relationships: Relationships in graph databases are not just binary connections. They can have direction, weight, and specific types. This lets you capture the subtle nuances of real-world interactions. For example, you can differentiate between “follows” and “retweets” in a social network and assign weights to connections based on frequency or sentiment. This richer representation unlocks possibilities for advanced analyses like influence detection or community identification.

3. Unveiling Hidden Stories: By traversing the web of relationships, you can discover surprising connections and patterns invisible to traditional methods. Imagine analyzing news articles related to a political election. A graph database might reveal unexpected links between seemingly unrelated articles, exposing potential narratives or hidden agendas. This ability to connect the dots leads to a more holistic understanding of complex data.

4. Breaking Down Silos and Integrating Data: Unstructured data often resides in scattered sources, creating silos of information. Graph databases excel at bridging these gaps. You can build a comprehensive picture of your data landscape by linking entities across different data sources. For example, connecting customer purchase history with social media sentiment data can reveal valuable insights into consumer behavior and preferences.

5. Adapting to the Evolving Data Landscape: Graph databases are malleable and flexible, unlike rigid relational schemas. New connections and nodes can be seamlessly added as your data grows or evolves. This makes them ideal for dealing with the dynamic nature of unstructured data, where new information and relationships emerge constantly.

AI/ML and graph databases

When it comes to unlocking the secrets of unstructured data, two powerful forces join hands: Artificial Intelligence and Machine Learning. Their synergy brings unique insights and unlocks hidden patterns in unstructured data.

1. Graph Embeddings: Imagine turning complex relationships into numerical representations – that’s the magic of graph embeddings. Using AI algorithms, graph databases can transform the intricate connections between nodes into vectors, creating a language that AI and ML models can understand. This opens doors to powerful applications like anomaly detection in sensor data or predicting user behavior in social networks.

2. Unsupervised Learning on Unstructured Terrain: Graph databases provide fertile ground for unsupervised learning, where ML models autonomously discover patterns and clusters within the data. With the guidance of the relationships captured in the graph, the models can identify hidden communities in social networks, group similar documents based on content, or suggest new research directions in scientific papers.

3. Personalized Recommendations and Predictive Power: By understanding the connections between users, their preferences, and interactions, AI algorithms can leverage graph databases to deliver hyper-personalized recommendations. This can benefit e-commerce platforms, streaming services, or news aggregators, significantly improving user engagement and satisfaction. Furthermore, graph-based analytics can predict future trends and patterns, enabling businesses to make informed decisions based on data-driven insights.

4. Explainable AI: Traditional AI models often operate like black boxes, leaving users wondering, “Why?” Graph databases bring transparency to the game. By visualizing the relationships and insights derived from the data, they explain how AI and ML models arrived at their conclusions. This builds trust and creates a better understanding, allowing teams to make better decisions that are based on explicit reasoning.

5. Continuous Learning and Evolution: The dynamic nature of unstructured data demands adaptability. Graph databases empower AI and ML models to learn and refine as new information or connections emerge continuously. This adaptability ensures that insights remain relevant and accurate even as the data landscape evolves.

Challenges and considerations

While the possibilities offered by graph databases in unstructured data management are vast, it’s important to acknowledge the challenges and considerations of this exciting realm. Here’s a closer look at the roadblocks you might encounter:

1. Scalability and Performance

As your data volume grows, ensuring efficient queries and analysis with a graph database can become a hurdle. Choosing the right technology optimized for your specific needs and implementing proper data partitioning is crucial for maintaining performance with large datasets.

1. Query Complexity and Expertise:

While some graph databases offer user-friendly query languages like Cypher, navigating complex relationships and traversing paths efficiently requires specialized knowledge and understanding of graph algorithms. Building a team with the necessary expertise is essential for maximizing the effectiveness of your graph database.

1. Schema Flexibility vs. Data Governance:

The flexible nature of graph databases can be a double-edged sword. While it allows for adaptation to evolving data, it also challenges data governance and consistency. Implementing transparent data modeling practices and access controls becomes crucial to maintaining data integrity and quality.

1. Integration with Existing Systems:

Integrating a graph database with relational databases or other data sources can be complex. Choosing the right integration tools and ensuring data consistency across different systems requires careful planning and execution.

1. Lack of Standardization:

The graph database landscape is evolving, with vendors offering varying query languages and functionalities. This lack of standardization can make switching between platforms or leveraging existing tools and resources challenging.

1. Cost and Availability:

Implementing and maintaining a graph database can be resource-intensive. Careful cost considerations and choosing the suitable deployment model (cloud, on-premises, etc.) are necessary to ensure a cost-effective solution.

Conclusion

In conclusion, despite these challenges, the potential of graph databases in unlocking the true potential of unstructured data remains highly promising. By considering the scalability, query complexity, governance, integration, and cost considerations, you can navigate the roadblocks and leverage this powerful technology to gain valuable insights and unlock innovation within your organization.

Image by svstudioart on Freepik

As enterprises rapidly adopt cloud-native technologies, the demand for tools that can seamlessly manage their containerized applications has soared over recent years.

To help you find the right solution for your organization, this article aims to help guide you through the leading solutions that are out there. We’ll provide some practical insights to help you choose the most suitable container management solution for your organization’s specific needs.

Whether you're a small business owner, a developer, or an IT professional, understanding the nuances of these top-tier solutions is essential to helping you make an informed decision when it comes to managing your cloud computing resources.

1. Google Cloud Run

Google Cloud Run is a fully managed platform that enables developers to deploy containerized applications quickly and securely.

This platform uses Google's robust cloud infrastructure to provide an environment where containers can be run in a serverless state, meaning users don't have to worry about the underlying infrastructure management.

Google Cloud Run is famous for its high degree of usability, which is why companies use it for so many different purposes, from data migration, CI/CD pipelines, and API development and hosting to implementing SAP staff augmentation measures. It stands out for its ability to scale up or down automatically based on traffic, ensuring cost-effectiveness and efficient resource utilization for organizations of all sizes.

Key Features:

• Serverless: Cloud Run automatically scales your applications based on the demand, efficiently managing traffic fluctuations without manual intervention.
• Integration with Google Cloud Services: It offers seamless integration with a wide range of Google services such as Cloud Storage, Cloud SQL, and more, enhancing the overall functionality and convenience.
• Custom domains and SSL: It supports the use of custom domains, coupled with automatic provisioning of SSL certificates, enhancing security and brand identity.
• Container-to-container networking: It provides enhanced security measures and facilitates smooth container communication.
• Continuous deployment: It easily integrates with Google Cloud Build, allowing for continuous deployment directly from source code repositories, thereby streamlining the development process.

2. Podman

Podman, also known as Pod Manager, is an open-source container management tool, part of the Red Hat family, designed as a drop-in replacement for Docker.

What sets Podman apart is its daemon-less architecture, which enhances security and reduces complexity. Likewise, Podman can also be incredibly useful for less complex but still speed-oriented actions, such as those in the finance world. Everything from peer-to-peer transactions to asset protection and even invoice factoring can benefit from proper container management.

It uses standard container images from Docker and Open Container Initiative registrars. On top of this, it also supports almost all Docker CLI commands, making it easy for users to transition from Docker to Podman.

Key Features:

• Daemonless architecture: Podman increases security and reduces system complexity by operating without a central daemon.
• Rootless containers: It enables running containers without root privileges, significantly enhancing security and reducing risks.
• OCI compatible: It’s fully compatible with OCI-compliant container images, ensuring broad compatibility and ease of use.
• Pod concept: Podman mimics Kubernetes' pod structure by grouping multiple containers into a single pod for better resource management.
• Systemd integration: It offers improved control and management of the container lifecycle through integration with systemd.

3. Digital Ocean

Digital Ocean's container service, DigitalOcean Kubernetes or DOKS, is tailored for simplicity and ease of use. It's an ideal solution for small to medium-sized businesses or individual developers requiring a straightforward container deployment and management approach.

Digital Ocean automates much of the process involved, including updates and maintenance of the Kubernetes cluster.

Key Features:

• Managed Kubernetes: Digital Ocean simplifies the setup and management of Kubernetes clusters, making it more accessible, especially for small to medium-sized businesses.
• Easy-to-use interface: It features an intuitive user interface that simplifies the management of Kubernetes clusters.
• Marketplace for quick deployment: It provides a marketplace with a variety of pre-configured applications and stacks for rapid deployment.
• Block storage and load balancers: DO integrates seamlessly with Digital Ocean's block storage and load balancing services for enhanced performance.
• Monitoring and alerts: It includes built-in monitoring tools for effective performance tracking and configurable alerts for system events.

4. Vultr

Vultr Kubernetes Engine, or VKE for short, provides a highly scalable and user-friendly platform for deploying, managing, and scaling containerized applications.

Vultr distinguishes itself from the competition with its global footprint, offering data centers around the world, which proves to be particularly beneficial for businesses requiring high availability and low-latency access across different geographical locations.

Key Features:

• Global reach: Vultr offers a global network of data centers to provide high availability and low-latency access across various geographical locations.
• Fully managed Kubernetes: VKE actively alleviates the complexity associated with Kubernetes cluster management to provide organizations with a far more streamlined experience.
• Block storage and load balancers: It easily integrates with Vultr’s native block storage and load balancer services for enhanced storage and traffic management.
• Private networking: The platform provides secure, private networking options for safe inter-communication between containers.
• API and CLI access: The platform features robust API and command-line tools for enhanced automation and easy management of container environments.

5. Dockerize.io

Dockerize.io is a relatively new entrant in the container management space that focuses primarily on Docker-based container management. It provides a streamlined platform for managing Docker containers that puts a key focus on continuous integration and continuous deployment of CI/CD workflows.

Dockerize.io is particularly useful for development teams looking to automate their deployment pipeline.

Key Features:

• CI/CD integration: It focuses on streamlining the continuous integration and deployment process, making it ideal for development teams looking to automate their deployment pipeline.
• Docker-centric management: The platform is specifically designed for managing Docker containers, offering tailored functionality and support.
• Webhook triggers: It enables automated deployments triggered by code commits or other specified events.
• Real-time monitoring: Dockerize provides real-time insights into container performance, aiding in effective management and troubleshooting.
• User-friendly interface: It offers a simplified user interface for easy and efficient management of Dockerized applications.

6. Red Hat OpenShift

Red Hat OpenShift is a leading enterprise Kubernetes platform, offering a comprehensive solution for container-based applications. It provides a full-stack automated operations model with a strong focus on enterprise security.

OpenShift is suitable for enterprises seeking a scalable and secure platform to manage complex containerized applications.

Key Features:

• Enterprise Kubernetes: The platform delivers an enterprise-grade Kubernetes environment that’s suitable for managing complex, large-scale applications.
• Developer and operations-centric: It balances the needs of both developers and IT operations, fostering collaboration and efficiency.
• Automated operations: Openshift actively automates installation, upgrades, and lifecycle management to help significantly reduce manual efforts in maintaining your operation.
• Built-in CI/CD: It integrates continuous integration and deployment toolchains, streamlining the development process.
• Advanced security features: It incorporates strong security controls and compliance features, ensuring a secure environment for enterprise applications.

7. Portainer

Portainer is a lightweight management UI that allows users to easily manage different Docker environments. It is known for its simplicity and is ideal for those new to Docker or who require a straightforward tool to help manage their containers, images, networks, and volumes.

Key Features:

• User-friendly interface: Portainer has an easy-to-use and intuitive interface, making it accessible for both beginners and experienced users.
• Docker compatibility: It’s fully compatible with Docker and Docker Swarm, facilitating seamless management of container environments.
• Multi-environment support: It manages local Docker hosts, Docker Swarm clusters, and even allows you to enhance Kubernetes clusters from a single pane of glass.
• Role-based access control or RBAC: The platform provides robust access control mechanisms, allowing for precise user role definition and permissions management.
• Templates for quick deployment: Portainer offers a range of application templates to simplify the deployment of common services.

8. SUSE Rancher

The Rancher platform from SUSE is an open-source container management platform that enables organizations to deploy, manage, and secure Kubernetes at scale.

It's well-known and respected for its extensive Kubernetes distribution support, straightforward interface, and robust security features.

Key Features:

• Multi-cluster management: Rancher actively simplifies the operation of Kubernetes clusters across different computing environments, including on-premises, cloud, and edge.
• Extensive Kubernetes support: It readily works with any CNCF-certified Kubernetes distribution.
• Integrated security: The platform boasts comprehensive security features for cluster management, including role-based access control, a.k.a RBAC, and pod security policies.
• User-friendly interface: Rancher offers an intuitive UI and API to easily manage your Kubernetes clusters.
• DevOps tooling integration: It easily integrates with a range of CI/CD tools and supports GitOps workflows.

Effectively Manage Your Containerized Applications

When it comes to cloud container management, it's clear that the choice of a management solution hinges on a variety of factors that must be carefully considered.

These factors include the business size, specific use cases, budget constraints, and the desired level of control and security. From the fully managed, serverless offerings of Google Cloud Run to the open-source flexibility and security focus of Rancher, each container management platform brings its unique strengths to the table.

The diversity of these solutions underscores the importance of assessing your organizational needs and considering future scalability. As container technology continues to evolve, spanning applications ranging from edge computing applications to advanced AI integrations, staying informed and adaptable will be key to leveraging the full potential of these tools.

Whether you're a startup looking to innovate quickly or a large enterprise seeking robustness and security, the array of options available ensures that there’s an effective container management solution out there that can meet your company’s specific needs and requirements.

Nahla Davies is a software developer and tech writer. Before devoting her work full time to technical writing, she managed—among other intriguing things—to serve as a lead programmer at an Inc. 5,000 experiential branding organization whose clients include Samsung, Time Warner, Netflix, and Sony.

More On This Topic

• 2024 Data Management Crystal Ball: Top 4 Emerging Trends
• 5 Data Management Challenges with Solutions
• Top 10 Kaggle Machine Learning Projects to Become Data Scientist in 2024
• Overcoming Barriers in Multi-lingual Voice Technology: Top 5…
• The 5 Best Vector Databases You Must Try in 2024
• 11 Best Practices of Cloud and Data Migration to AWS Cloud

Customer success stories illuminate how hardware accelerators speed up the necessary infrastructure to support all aspects of an accelerated AI and HPC computing data center.

Customer success stories illuminate how hardware accelerators speed up necessary infrastructure to support all aspects of an accelerated AI and HPC computing data center.

The real-world impact of accelerated AI technology extends far beyond any single data center or workload. While undeniably important, AI technology is but one of several software components that now comprise what experts term the 5th epoch of distributed computing. [1] [2]

To meet both productivity and efficiency goals, commercial, academic, and HPC data centers rely on heterogeneous computing environments that utilize computational accelerators to speed the tsunami of fast-becoming-ubiquitous AI-based workloads. Cloud computing represents an off-premises alternative that is now the go-to solution for many commercial AI workloads. Cloud computing is also being considered as an additional (and even replacement) platform for some HPC workloads. All this accelerated computational performance can be rendered useless if the data center, cloud provider, or hardware procurement does not address the data management aspects of user workloads.

The key concepts to realize the benefits of heterogeneity and accelerators were discussed in the previous articles in this series (The 5th Epoch, Multiarchitecture Software Frameworks, Investment Guidelines). Succinctly, Intel and the Exascale Computing Project (ECP) have both taken leadership roles in the creation of general-purpose and mutually compatible software frameworks that others can use to achieve performance portability across many data centers (e.g., oneAPI and E4S).

Experts Explain the Data Conundrum

The extreme scale of exascale supercomputers highlights the challenges that must be addressed by accelerated computational devices. All currently announced DOE exascale supercomputers utilize GPUs to achieve their high floating-point performance on AI and HPC workloads. These hybrid systems are inherently unbalanced as the computational accelerators can deliver orders of magnitude greater parallelism and data throughput than their corresponding storage subsystems.[3] This requires new approaches to working with data as exemplified by the addition of multiple on-package CPU accelerators, some of which are focused on data security and management (Figure 1).

Although High bandwidth memory (HBM) is not in itself considered an accelerator, memory bandwidth needs to be considered. As Mark Kachmarek (HPC marketing manager, Intel) noted in his HBM overview presentation, “Memory BW [Bandwidth] sensitive workloads get accelerated performance benefit from High Memory Bandwidth.” [4] Similarly, the addition of Intel advanced matrix extensions (AMX) can greatly accelerate the performance of reduced-precision machine learning workloads. Overall, HBM and Intel AMX plus AVX-512 can increase the efficiency of these with computational accelerators to provide new levels of mobility, security, and performance.[5] It can also make CPUs — as opposed to GPUs — the platform of choice for many HPC and AI workloads.

Customers Need to Interact and Collaborate

CS Chang (Managing Principal Research Physicist at Princeton Plasma Physics Lab) is preparing to run his code on the Aurora supercomputer equipped with the Intel Xeon CPU Max Series and Intel GPU Max Series. He observed that a very important aspect of extreme-scale computing assisted by AI is the creation of surrogate models that can return results in minutes rather than days spent waiting for big, large-scale simulations.

Chang observed, “from the large-scale simulation data, we are beginning to build surrogate models.” These surrogate models utilize machine learning to produce a very simple formula from big simulation data.” Chang continued, “there are many, many of those problems in fusion where surrogate models are just as good as the physics they’re putting into it. PPPL [Princeton Plasma Physics Laboratory] scientists have created an AI model which reduces the [runtime from] days into just minutes.”

Customers also Need to Visualize, Interact, and Train

Also preparing to run on the Argonne National Laboratory managed Aurora exascale supercomputer, Amanda Randles, (Alfred Winborne Mordecai and Victoria Stover Mordecai Associate Professor of Biomedical Engineering at Duke University) noted, “We’re working with the Argonne teams to try to create techniques and capabilities to visualize the data, interact with the data, use it to train machine learning models while it’s still in memory on the system, and really make use of it without having to store it after the fact.”

We’re working with the Argonne teams to try to create techniques and capabilities to visualize the data, interact with the data, use it to train machine learning models while it’s still in memory on the system, and really make use of it without having to store it after the fact. – Amanda Randles, Alfred Winborne Mordecai, and Victoria Stover Mordecai Associate Professor of Biomedical Engineering at Duke University

Randles continued, “We’re just getting into the machine learning and artificial intelligence side of things. The goal is run a minimal number of simulations per patient, train a machine learning model, and then be able to predict and infer what other instances, like what other physiological conditions there may be, and to kind of optimize our use of the resources on that end.”

HPC, Cloud, and Industry Stories

Numerous academic, government, and enterprise customer stories highlight the need for AI data and security-related accelerators to enhance their HPC efforts.

The Texas Advanced Computing Center

TACC is a leading academic supercomputer research facility.

When evaluating hardware for their next procurement, TACC observed that Intel Xeon Max processors with High Bandwidth Memory (HBM) can provide significant performance benefits. These are the processors that were chosen to support future academic research across the U.S. as part of the recently announced a $10M U.S. National Science Foundation (NSF) grant for the new Stampede3 supercomputer. “We believe the high bandwidth memory of the Intel Xeon CPU Max Series nodes will help deliver better performance than any other CPU that our users have seen before,” TACC director Dan Stanzione said. “They offer more than double the memory bandwidth performance per core over the current 2nd and 3rd Gen Intel Xeon processor nodes in Stampede2.”

The 14 applications that were assessed include widely used codes from the earthquake and astrophysics communities, software for large international experiments and custom codes that explore innovative approaches to machine learning and black hole modeling. Performance results are shown in Figure 2.

A special configuration of the Community Earth System Model (CESM) developed by the NSF-sponsored EarthWorks project also demonstrated notable performance improvements when running on an HBM-enabled processor with internal accelerators.

CESM is one of the principal climate codes used by the earth science community. It is developed and maintained by the National Center for Atmospheric Research (NCAR) in collaboration with the research community. The TACC benchmarks found that the EarthWorks configuration of CESM ran 2.5× faster on the Intel Xeon CPU Max Series with DDR5 memory than on Frontera. This same code achieved a further 30 percent improvement (to 3.2×) in HBM-only mode.

TACC also observed the 3D earthquake code developed by Yifeng Cui of the San Diego Supercomputer Center, the Anelastic Wave Propagation (AWP) code, ran significantly faster on an HBM-enable Intel Xeon CPU with accelerators. TACC determined this code ran 3.7× faster on Intel Xeon CPU Max Series than on Frontera and showed a 100 percent boost with HBM.

Addressing Regulatory and Privacy Requirements

To create successful algorithms for use in healthcare, financial, government and other institutions, developers need to find ways to make their models generalizable to diverse populations even when confronted with stringent regulatory and privacy requirements.

Equideum Health

Equideum Health provides one customer story that utilizes Intel Software Guard Extensions (Intel SGX shown in Figure 1) to support their Data Integrity and Learning Networks (DILNs). According to Equideum, “The lack of a unified, comprehensive health record for each individual renders effective data analysis nearly impossible. To perform high-quality, trusted analytics, stakeholders must engage in multi-party data sharing, which introduces privacy and security risks.” Through the use of blockchain and AI technology, researchers can work with data compartmentalized among stakeholders such as health insurance companies, pharmacies, provider offices, health systems, and laboratories.[7]

Ropers Majseki

The law firm of Ropers Majeski partnered with Intel, Activeloop and ZERO Systems to create a secure AI-driven solution. Embedded accelerators such as Intel Advanced Vector Extensions 512 (Intel AVX-512) in the 4th generation of Intel Xeon Scalable Processors provided the performance and efficiency gains to run a custom Large Language Model (LLM) so the customer did not have to invest in a proprietary solution. This coupled with a secure, enterprise-grade database for AI made it possible to store both embeddings and multi-modal data for millions of sensitive documents and provided advanced role-based access controls. The system was able to boost worker productivity while meeting strict regulatory and security requirements. “This kind of strategic partnership is just the beginning of integrating generative AI into the legal profession and other highly regulated industries.” Said Gevorg Karapetyan, Co-founder and CTO of Zero Systems.

Accelerated Instruction Set Architectures (ISAs)

On-package and on-chip accelerators are also making CPUs competitive with GPUs in terms of computational performance for many AI workloads. The following customer stories reflect the benefits of this trend.

The Laboratory for Laser Energetics at the University of Rochester

Scientists at the Laboratory for Laser Energetics (LLE) at the University of Rochester are studying and experimenting with inertial confinement fusion (ICF) to harvest energy from the same process that powers stars. The new LLE processor-based Conesus system helps scientists explore machine learning and artificial intelligence (AI) technologies to garner insight into how to harvest power from nuclear energy. William Scullin, HPC Lead at LLE, expects scientists will take advantage of Intel Advanced Vector Extensions 512 (Intel AVX-512), which provides significant acceleration for floating point calculations.

Numenta

Numenta has developed technology to accelerate inference performance for Natural Language Processing applications that customers utilize to analyze extensive collections of documents. Numenta leverages Intel AMX to achieve high inference performance on production-ready Transformer models – from BERTs to multi-billion parameter GPTs — without using a GPU. Numenta observed a 35× throughput improvement versus NVIDIA A100 GPUs for BERT-Large inference on short text sequences and batch size.[8] [9] [10] Batch size 1 is ideal for low-latency inference applications used in production NLP workloads as this batch size provides the most flexibility in real-time scenarios where input data constantly changes.

Siemens

Siemens is using 4th Gen Intel Xeon Scalable processor with Intel Advanced Matrix Extensions, and the Intel Distribution of the OpenVINO Toolkit to support radiation therapy professionals with AI-based auto contouring technology. The company claims they can increase workload efficiency, improve consistency, and help free up staff to focus on value-adding work.

Taboola

Taboola is using Intel AMX accelerated processors to deliver targeted content and recommendations that are most relevant to each Internet visitors’ unique preferences. “We are the cloud for our customers, and we need a CPU that is flexible, versatile, and efficient, so it can perform multiple jobs,” explained Ariel Pisetzky (Taboola’s VP of Information Technology & Cyber), “It [the CPU] needs to be able to scale up and down through the day in terms of power use and have enough memory to do in-memory calculations that we need done really fast.”

Summary

The 5th epoch of computing provides many AI benefits, but only when the data security and accessibility infrastructure are also accelerated to meet user needs. Gordon Moore, anticipated this 5th epoch day of reckoning when we will need to build larger systems out of smaller functions, combining heterogeneous and customized solutions. [11] Happily, many of these solutions fit nicely into small, power-efficient packages. More than that, specialized hardware accelerators help facilitate AI and HPC applications across data silos that have severe restrictions on data sharing.

Rob Farber is a global technology consultant and author with an extensive background in HPC and machine learning technology.

[1] https://www.datasciencecentral.com/ushering-in-the-5th-epoch-of-distributed-computing-with-accelerated-ai-technologies/

[2] https://www.youtube.com/watch?v=27zuReojDVw

[3] https://www.exascaleproject.org/supporting-scientific-discovery-and-data-analysis-in-the-exascale-era/

[4] See https://www.intel.com/content/www/us/en/content-details/762147/video-high-bandwidth-memory-hbm-overview.html?wapkw=hbm at time 2.52.

[5] https://www.nextplatform.com/2021/10/21/how-high-bandwidth-memory-will-break-performance-bottlenecks/

[6] Applications compared by TACC were WRF CONUS 2.5km, Parsec liquid_water_64H2O_0.3A, Parsec Si1947H604_0.9A, Amber STMV_production_BP4_4fs, AWP-ODC Fortran, Seissol tpv5_1node, CESM PFS_Ld5_P56.mpasa120_mpasa, EWP MgB2_16, MILC grid 32x32x32x32, Enzo-E 128^3 root mech 512 root blocks, ISSM test435, MuST muffin_56x32, PSDNS 768x768x768, Plascom, and athena++. These were run on systems installed at TACC that use Intel® Xeon® 8280, Intel® Xeon® Max 9480 (Flat Mode, DDR only), and Intel® Xeon® Max 9480 (HBM only). TACC observed a 2.6x average performance increase from Frontera (Intel® Xeon® 8280) to Intel® Xeon® Max 9480 (HBM only) on these workloads.

[7] https://www.intel.com/content/www/us/en/customer-spotlight/stories/equideum-health-customer-story.html

[8] For more, see: https://www.intel.com/content/www/us/en/products/details/processors/xeon/max-series.html Numenta BERT-Large: AMD Milan: Tested by Numenta as of 11/28/2022. 1-node, 2x AMD EPYC 7R13 on AWS m6a.48xlarge, 768 GB DDR4-3200, Ubuntu 20.04 Kernel 5.15, OpenVINO™ Toolkit 2022.3, BERT-Large, Sequence Length 512, Batch Size 1. Intel® Xeon® 8480+ processor: Tested by Numenta as of 11/28/2022. 1-node, 2x Intel® Xeon® 8480+ processor, 512 GB DDR5-4800, Ubuntu 22.04 Kernel 5.17, OpenVINO™ Toolkit 2022.3, Numenta-Optimized BERT-Large, Sequence Length 512, Batch Size 1. Intel® Xeon® Max 9468 processor: Tested by Numenta

[9] Tested by Numenta as of 03/22/2023. 1-node, pre-production platform with 2x Intel® Xeon® 8480+ processor, 512 GB DDR5-4800, Ubuntu 22.04 Kernel 5.17, OpenVINO™ Toolkit 2022.3, Numenta-Optimized BERTLarge, Sequence Length 128, Batch Size 1.

[10] For more, see: https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT#inference-performance-nvidia-dgx-a100-1x-a100-40gb

[11] https://www.datasciencecentral.com/ushering-in-the-5th-epoch-of-distributed-computing-with-accelerated-ai-technologies/#_edn2

This article was produced as part of Intel’s editorial program, with the goal of highlighting cutting-edge science, research and innovation driven by the HPC and AI communities through advanced technology. The publisher of the content has final editing rights and determines what articles are published.

Image Generated with DALL-E

In a time where data analytic processing is the critical difference between a successful business and not, we need a tool stack that could support the needs. The advancement of technology has helped advance all these data tools that we need, namely DuckDB and MotherDuck.

DuckDB is an open-source, in-process SQL Online Analytical Processing (OLAP) database management system. The database system is designed for swiftly handling data analytical queries, regardless of the data size. The system implements in-memory processing and OLAP systems that effectively improve our data analytical process.

DuckDB is perfect for storing and processing tabular data involving data analysis (table join, data aggregation, etc.) and when our workflow usually involves significant changes in the table. On the other hand, DuckDB isn’t suitable for high-volume data activity and multiple concurrent processes in one database.

MotherDuck is a managed DuckDB-in-the-cloud service. It’s free to use and open-source while maintained by the DuckDB Community. It’s a service built by partnering with DuckDB Lab to create a cloud service platform that the public can use.

With a combination of DuckDB and Motherduck, we can create an analytics engine that is readily useable in every scenario. How do we do that? Let’s get into it.

MotherDuck UI

We would use the native MotherDuck UI to give you an example of how the service works and why DuckDB is a powerful tool for data analytics. Please register on the website and acquire the MotherDuck account if you haven’t already.

Once you successfully register for the MotherDuck account, we will be taken to the MotherDuck UI. Try to familiarize yourself with the UI, and you will realize that the UI is similar to the Jupyter Notebook if you ever use one.

We will experiment with the DBduck power in the MotherDuck UI with the DS Salary data from Kaggle. Upload the data using the Add Files button, and a new cell will be shown with the query to execute. The query should look like this.

CREATE OR REPLACE TABLE ds_salaries AS SELECT * FROM read_csv_auto(['ds_salaries.csv']);

Once you create the table, try to query the data with the following code.

select * from my_db.ds_salaries limit 10;

As you can see, MotherDuck is pretty much like doing data analysis in Notebook, but with SQL queries. Let’s try out the query to do data analysis in the MotherDuck.

select job_title,          avg(salary_in_usd) as average_salary_in_usd   from my_db.ds_salaries  GROUP BY job_title  ORDER BY job_title

You can execute the query in the cell; the table result is shown similarly to the image below.

You can filter out the data, pivot the table, or download the result with the selection button available in the UI.

MotherDuck Python

MotherDuck also allows the user to access the database via Python on your Notebook. We need to install the DuckDB package using the following code.

pip install duckdb==v0.9.2

The current version that MotherDuck supports is DuckDB 0.9.2; that’s why we installed that version.

When the installation is successful, we need to connect the DuckDB with the Motherduck. There are a few ways to authenticate the connection, but we would use the service token. This token is acquired in your MotherDuck settings.

import duckdb    token = "insert token here"  # initiate the MotherDuck connection  con = duckdb.connect(f'md:?motherduck_token={token}')

If we didn’t set any database name, MotherDuck would access using the default database, which is my_db. Next, let’s use the same query we did previously in the Notebook.

q = """  select job_title,         avg(salary_in_usd) as average_salary_in_usd  from my_db.ds_salaries  GROUP BY job_title  ORDER BY job_title  """    con.sql(q).show()

You will see the output similar to the table below.

┌─────────────────────────────────────┬───────────────────────┐  │              job_title              │ average_salary_in_usd │  │               varchar               │        double         │  ├─────────────────────────────────────┼───────────────────────┤  │ 3D Computer Vision Researcher       │              21352.25 │  │ AI Developer                        │     136666.0909090909 │  │ AI Programmer                       │               55000.0 │  │ AI Scientist                        │            110120.875 │  │ Analytics Engineer                  │    152368.63106796116 │  │ Applied Data Scientist              │              113726.3 │  │ Applied Machine Learning Engineer   │               99875.5 │  │ Applied Machine Learning Scientist  │    109452.83333333333 │  │ Applied Scientist                   │     190264.4827586207 │  │ Autonomous Vehicle Technician       │               26277.5 │  │            ·                        │                  ·    │  │            ·                        │                  ·    │  │            ·                        │                  ·    │  │ Principal Data Engineer             │              192500.0 │  │ Principal Data Scientist            │            198171.125 │  │ Principal Machine Learning Engineer │              190000.0 │  │ Product Data Analyst                │               56497.2 │  │ Product Data Scientist              │                8000.0 │  │ Research Engineer                   │    163108.37837837837 │  │ Research Scientist                  │    161214.19512195123 │  │ Software Data Engineer              │               62510.0 │  │ Staff Data Analyst                  │               15000.0 │  │ Staff Data Scientist                │              105000.0 │  ├─────────────────────────────────────┴───────────────────────┤  │ 93 rows (20 shown)                                2 columns │  └─────────────────────────────────────────────────────────────┘

With the query above, you can use the following code to process them into the Pandas DataFrame.

import pandas as pd    df = con.sql(q).fetchdf()

Lastly, you can load another dataset to the database using the following query.

con.sql("CREATE TABLE mytable AS SELECT * FROM '~/filepath.csv'")

The above query assumes your data is a CSV file. Other options include S3 or the local DuckDB to the MotherDuck database.

Conclusion

DuckDB is an open-source database system that was developed specifically for data analysis. The system is designed to handle data processing swiftly and efficiently. MotherDuck is an open-source managed cloud-based service for DuckDB.

By combining DuckDB and MotherDuck, we can turn our laptops into a personal analytics engine by having our data in the cloud and quickly processing them with DuckDB.

Cornellius Yudha Wijaya is a data science assistant manager and data writer. While working full-time at Allianz Indonesia, he loves to share Python and Data tips via social media and writing media.

More On This Topic

• Building a Visual Search Engine — Part 2: The Search Engine
• Why is DuckDB Getting Popular?
• Explore LLMs Easily on Your Laptop with openplayground
• KDnuggets: Personal History and Nuggets of Experience
• Data storytelling: brains are built for visuals, but hearts turn on stories
• Inflection-1: The Next Frontier of Personal AI