Рубрика: AI

Generative AI models have been a hot topic of discussion within the AI industry for a while. The recent success of 2D generative models has paved the way for the methods we use to create visual content today. Although the AI community has achieved remarkable success with 2D generative models, generating 3D content remains a major challenge for deep generative AI frameworks. This is especially true as the demand for 3D generated content reaches an all-time high, driven by a wide array of visual games, applications, virtual reality, and even cinema. It is worth noting that while there are 3D generative AI frameworks that deliver acceptable results for certain categories and tasks, they are unable to efficiently generate 3D objects. This shortfall can be attributed to the lack of extensive 3D data for training the frameworks. Recently, developers have proposed leveraging the guidance offered by pre-trained text-to-image AI generative models, an approach that has shown promising results.

In this article, we will discuss the DreamCraft3D framework, a hierarchical model for generating 3D content that produces coherent and high-fidelity 3D objects of high quality. The DreamCraft3D framework uses a 2D reference image to guide the geometry sculpting stage, enhancing the texture with a focus on addressing consistency issues encountered by current frameworks or methods. Additionally, the DreamCraft3D framework employs a view-dependent diffusion model for score distillation sampling, aiding in sculpting geometry that contributes to coherent rendering.

We will take a closer dive into the DreamCraft3D framework for 3D content generation. Furthermore, we will explore the concept of leveraging pretrained Text-to-Image (T2I) models for 3D content generation and examine how the DreamCraft3D framework aims to utilize this approach to generate realistic 3D content.

DreamCraft3D : An Introduction

DreafCraft3D is a hierarchical pipeline for generating 3D content. The DreamCraft3D framework attempts to leverage a state of the art T2I or Text to Image generative framework to create high-quality 2D images using a text prompt. The approach allows the DreamCraft3D framework to maximize the capabilities of state of the art 2D diffusion models to represent the visual semantics as described in the text prompt while retaining the creative freedom offered by these 2D AI generative frameworks. The image generated is then lifted to 3D with the help of cascaded geometric texture boosting, and geometric sculpting phases, and the specialized techniques are applied at each stage with the help of decomposing the problem.

For geometry, the DreamCraft3D framework focuses heavily on the global 3D structure, and multi-view consistency, thus making room for compromises on the detailed textures in the images. Once the framework gets rid of geometry-related issues, it shifts its focus on optimizing coherent & realistic textures by implementing a 3D-aware diffusion that bootstraps the 3D optimization approach. There are two key design considerations for the two optimization phases namely the Geometric Sculpting, and Texture Boosting.

With all being said, it would be safe to describe the DreamCraft3D as an AI generative framework that leverages a hierarchical 3D content generation pipeline to essentially transform 2D images into their 3D counterparts while maintaining the holistic 3D consistency.

Leveraging Pretrained T2I or Text-to-Image Models

The idea to leverage pretrained T2I or Text-to-Image models for generating 3D content was first introduced by the DreamFusion framework in 2022. The DreamFusion framework attempted to enforce a SDS or Score Distillation Sample loss to optimize the 3D framework in a way that the renderings at random viewpoints would align with the text-conditioned image distributions as interpreted by an efficient text-to-image diffusion framework. Although the DreamFusion approach delivered decent results, there were two major issues, blurriness, and over saturation. To tackle these issues, recent works implement various stage-wise optimization strategies in an attempt to improve the 2D distillation loss, which ultimately leads to better quality, and realistic 3D generated images.

However, despite the recent success of these frameworks, they are unable to match the ability of 2D generative frameworks to synthesize complex content. Furthermore, these frameworks are often riddled with the “Janus Issue”, a condition where 3D renderings that appear to be plausible individually, show stylistic & semantic inconsistencies when examined as a whole.

To tackle the issues faced by prior works, the DreamCraft3D framework explores the possibility of using a holistic hierarchical 3D content generation pipeline, and seeks inspiration from the manual artistic process in which a concept is first penned down into a 2D draft, after which the artist sculpts the rough geometry, refines the geometric details, and paints high-fidelity textures. Following the same approach, the DreamCraft3D framework breaks down the exhaustive 3D content or image generation tasks into various manageable steps. It starts off by generating a high-quality 2D image using a text prompt, and proceeds to use texture boosting & geometry sculpting to lift the image into the 3D stages. Splitting the process into subsequent stages helps the DreamCraft2D framework to maximize the potential of hierarchical generation that ultimately results in superior-quality 3D image generation.

In the first stage, the DreamCraft3D framework deploys geometrical sculpting to produce consistent & plausible 3D-geometric shapes using the 2D image as a reference. Furthermore, the stage not only makes use of the SDS loss for photometric losses and novel views at the reference view, but the framework also introduces a wide array of strategies to promote geometric consistency. The framework aims to leverage the Zero-1-to-3, a viewpoint-conditioned off the shelf image translation model to use the reference image to model the distribution of the novel views. Additionally, the framework also transitions from implicit surface representation to mesh representation for coarse to fine geometrical refinement.

The second stage of the DreamCraft3D framework uses a bootstrapped score distillation approach to boost the textures of the image as the current view-conditioned diffusion models are trained on a limited amount of 3D data which is why they often struggle to match the performance or fidelity of 2D diffusion models. Thanks to this limitation, the DreamCraft3D framework finetunes the diffusion model in accordance with multi-view images of the 3D instance that is being optimized, and this approach helps the framework in augmenting the 3D textures while maintaining multi-view consistency. When the diffusion model trains on these multi-view renderings, it provides better guidance for the 3D texture optimization, and this approach helps the DreamCraft3D framework achieve an insane amount of texture detailing while maintaining view consistency.

As can be observed in the above images, the DreamCraft3D framework is capable of producing creative 3D images & content with realistic textures, and intricate geometric structures. In the first image, is the body of Son Goku, an anime character mixed with the head of a running wild boar, whereas the second picture depicts a Beagle dressed in the outfit of a detective. Following are some additional examples.

DreamCraft3D : Working and Architecture

The DreamCraft3D framework attempts to leverage a state of the art T2I or Text to Image generative framework to create high-quality 2D images using a text prompt. The approach allows the DreamCraft3D framework to maximize the capabilities of state of the art 2D diffusion models to represent the visual semantics as described in the text prompt while retaining the creative freedom offered by these 2D AI generative frameworks. The image generated is then lifted to 3D with the help of cascaded geometric texture boosting, and geometric sculpting phases, and the specialized techniques are applied at each stage with the help of decomposing the problem. The following image briefly sums up the working of the DreamCraft3D framework.

Let’s have a detailed look at the key design considerations for the texture boosting, and geometric sculpting phases.

Geometry Sculpting

Geometry Sculpting is the first stage where the DreamCraft3D framework attempts to create a 3D model in a way it aligns with the appearance of the reference image at the same reference view while ensuring maximum plausibility even under different viewing angles. To ensure maximum plausibility, the framework makes use of SDS loss to encourage plausible image rendering for every individual sampled view that a pre-trained diffusion model can recognize. Furthermore, to utilize guidance from the reference image effectively, the framework penalizes photometric differences between the reference and the rendered images at the reference view, and the loss is computed only within the foreground region of the view. Additionally, to encourage scene sparsity, the framework also implements a mask loss that renders the silhouette. Despite this, maintaining appearance and semantics across back-views consistently still remains to be a challenge which is why the framework employs additional approaches to produce detailed, and coherent geometry.

3D Aware Diffusion Prior

The 3D optimization methods making use of per-view supervision alone is under-constrained which is the primary reason why the DreamCraft3D framework makes use of Zero-1-to-3, a view-conditioned diffusion model, as the Zero-1-to-3 framework offers an enhanced viewpoint awareness since it has been trained on a larger scale of 3D data assets. Furthermore, the Zero-1-to-3 framework is a fine-tuned diffusion model, that hallucinates the image in relation with the camera pose given the reference image.

Progressive View Training

Deriving free views directly in 360 degree might lead to geometrical artifacts or discrepancies like an extra leg on the chair, an event that might be credited to the ambiguity inherence of a single reference image. To tackle this hurdle, the DreamCraft3D framework enlarges the training views progressively following which the well-established geometry is gradually propagated to obtain results in 360 degrees.

Diffusion Time Step Annealing

The DreamCraft3D framework employs a diffusion time step annealing strategy in an attempt to align with the 3D optimization’s coarse-to-fine progression. At the start of the optimization process, the framework gives priority to sample a larger diffusion timestep, in an attempt to provide the global structure. As the framework proceeds with the training process, it linearly anneals the sampling range over the course of hundreds of iterations. Thanks to the annealing strategy, the framework manages to establish a plausible global geometry during early optimization steps prior to refining the structural details.

Detailed Structural Enhancement

The DreamCraft3D framework optimizes an implicit surface representation initially to establish a coarse structure. The framework then uses this result, and couples it with a deformable tetrahedral grid or DMTet to initialize a textured 3D mesh representation, that disentangles the learning of texture & geometry. When the framework is done with the structural enhancement, the model is able to preserve high-frequency details obtained from the reference image by refining the textures solely.

Texture Boosting using Bootstrapped Score Sampling

Although the geometry sculpting stage emphasizes on learning detailed and coherent geometry, it does blur the texture to a certain extent that might be a result of the framework’s reliance on a 2D prior model operating at a coarse resolution along with restricted sharpness on offer by the 3D diffusion model. Furthermore, common texture issues including over-saturation, and over-smoothing arises as a result of a large classifier-free guidance.

The framework makes use of a VSD or Variational Score Distillation loss to augment the realism of the textures. The framework opts for a Stable Diffusion model during this particular phase to get high-resolution gradients. Furthermore, the framework keeps the tetrahedral grid fixed to promote realistic rendering to optimize the overall structure of the mesh. During the learning stage, the DreamCraft3D framework does not make use of the Zero-1-to-3 framework since it has an adverse effect on the quality of the textures, and these inconsistent textures might be recurring, thus leading to bizarre 3D outputs.

Experiments and Results

To evaluate the performance of the DreamCraft3D framework, it is compared against current state of the art frameworks, and the qualitative & quantitative results are analyzed.

Comparison with Baseline Models

To evaluate the performance, the DreamCraft3D framework is compared against 5 state of the art frameworks including DreamFusion, Magic3D, ProlificDreamer, Magic123, and Make-it-3D. The test benchmark comprises 300 input images that are a mix of real-world images, and those generated by the Stable Diffusion framework. Each image in the test benchmark has a text prompt, a predicted depth map, and an alpha mask for the foreground. The framework sources the text prompts for the real images from an image caption framework.

Qualitative Analysis

The following image compares the DreamCraft3D framework with the current baseline models, and as it can be seen, the frameworks that rely on text-to-3D approach, often face multi-view consistency issues.

On one hand, you have the ProlificDreamer framework that offers realistic textures, but it falls short when it comes to generating a plausible 3D object. Frameworks like the Make-it-3D framework that rely on Image-to-3D methods manage to create high-quality frontal views, but they cannot maintain the ideal geometry for the images. The images generated by the Magic123 framework offer better geometrical regularization, but they generate overly saturated and smoothed geometric textures and details. When compared to these frameworks, the DreamCraft3D framework that makes use of a bootstrapped score distillation method, not only maintains semantic consistency, but it also improves the overall imagination diversity.

Quantitative Analysis

In an attempt to generate compelling 3D images that not only resembles the input reference image, but also conveys semantics from various perspectives consistently, the techniques used by the DreamCraft3D framework is compared against baseline models, and the evaluation process employs four metrics: PSNR and LPIPS for measuring fidelity at the reference viewpoint, Contextual Distance for assessing pixel-level congruence, and CLIP to estimate the semantic coherence. The results are demonstrated in the following image.

Conclusion

In this article, we have discussed DreamCraft3D, a hierarchical pipeline for generating 3D content. The DreamCraft3D framework aims to leverage a state-of-the-art Text-to-Image (T2I) generative framework to create high-quality 2D images using a text prompt. This approach allows the DreamCraft3D framework to maximize the capabilities of cutting-edge 2D diffusion models in representing the visual semantics described in the text prompt, while retaining the creative freedom offered by these 2D AI generative frameworks. The generated image is then transformed into 3D through cascaded geometric texture boosting and geometric sculpting phases. Specialized techniques are applied at each stage, aided by the decomposition of the problem. As a result of this approach, the DreamCraft3D framework can produce high-fidelity and consistent 3D assets with compelling textures, viewable from multiple angles.

A staggering 75% of Indian companies express the belief that they have a mere year, at most, to have an AI strategy before their business faces repercussions. The urgency to deploy AI technologies has spiked in the past six months, with IT infrastructure and cybersecurity emerging as the top priority areas for AI deployments.

A hint of anxiety is also visible in the startup ecosystem with companies like Zomato, OYO, ixigo, Freshworks and others having launched generative AI-enabled products and services in their respective domains. Even a company like Zerodha, which initially had a no-AI policy, has decided to explore the spectrum.

These sentiments were reflected in the AI Readiness Index conducted by Cisco. In the survey, it was observed that only 26% of organisations in India are fully equipped to deploy and leverage AI-powered technologies. The index, a product of a comprehensive survey involving 8,161 business and IT leaders from the private sector across 30 markets, talks about the AI readiness among companies with 500 or more employees.

On a positive note, Indian companies are actively taking strides to prepare for an AI-centric future. About 95% already have a sturdy AI strategy in place or are in the process of developing one. For instance, Project Indus by Tech Mahindra, which is an Indic-based foundational model, is expected to be launched in the next month or two.

“As companies rush to deploy AI solutions, they must assess where investments are needed to ensure their infrastructure can best support the demands of AI workloads,” said Liz Centoni, executive vice president and general manager, applications, and chief strategy officer, Cisco. “Organisations also need to be able to observe with context how AI is being used to ensure ROI, security, and especially responsibility.”

Globally, 95% of businesses acknowledge that AI will ramp up infrastructure workloads. However, in India, only 39% believe their infrastructure is highly scalable. This same group contends with limited or no scalability when confronting new AI challenges within their existing IT frameworks.

To meet the power and computing demands of AI, over two-thirds (68%) of Indian companies anticipate the need for additional data centre graphics processing units (GPUs) to support current and future AI workloads.

GitHub Report Tells the Tale

GitHub’s State of the Octoverse 2023 report, which was released just a week ago, shows Indian developers’ trajectory in AI development. With a community of 13.2 million developers now active on GitHub within its borders, India has firmly established itself as the globe’s second-largest contributor to AI projects, after the United States.

Notably, 3.5 million new developers joined GitHub’s ranks in 2023 alone.

The significance of this surge is not lost on Sharryn Napier, VP of APAC at GitHub as she said, “Just imagine what India will be able to achieve if its 13.2M developers are empowered with AI. Not only will this transform enterprise innovation and productivity, but it will elevate developer happiness and make a substantial impact on India’s economy and society as a whole.”

Examining the data shows a consistent year-on-year growth rate of 148% within the Indian developer community, as underscored by GitHub’s projections forecasting India’s overtaking of the United States in total developer population by the year 2027.

This momentum signifies a seismic shift in the tech landscape due to AI, more specifically, generative AI. The stark difference between researchers and enterprise-level businesses in India highlights how tricky the AI scene in the country is.

India has always adopted new technologies launched in the West, but now with generative AI, things are changing. According to GitHub, India is now one of the “top contributors” to the global AI open-source ecosystem. This shift is a big deal showing India’s involvement in the AI landscape.

The post Why is There an AI Gap Between Companies and Researchers in India? appeared first on Analytics India Magazine.

Image by Author

In the ever-evolving landscape of technology, the surge of large language models (LLMs) has been nothing short of a revolution. Tools like ChatGPT and Google BARD are at the forefront, showcasing the art of the possible in digital interaction and application development.

The success of models such as ChatGPT has spurred a surge in interest from companies eager to harness the capabilities of these advanced language models.

Yet, the true power of LLMs doesn't just lie in their standalone abilities.

Their potential is amplified when they are integrated with additional computational resources and knowledge bases, creating applications that are not only smart and linguistically skilled but also richly informed by data and processing power.

And this integration is exactly what LangChain tries to assess.

Langchain is an innovative framework crafted to unleash the full capabilities of LLMs, enabling a smooth symbiosis with other systems and resources. It's a tool that gives data professionals the keys to construct applications that are as intelligent as they are contextually aware, leveraging the vast sea of information and computational variety available today.

It's not just a tool, it's a transformational force that is reshaping the tech landscape.

This prompts the following question:

How will LangChain redefine the boundaries of what LLMs can achieve?

Stay with me and let’s try to discover it all together.

What is LangChain?

LangChain is an open-source framework built around LLMs. It provides developers with an arsenal of tools, components, and interfaces that streamline the architecture of LLM-driven applications.

However, it is not just another tool.

Working with LLMs can sometimes feel like trying to fit a square peg into a round hole.

There are some common problems that I bet most of you have already experienced yourself:

• How to standardize prompt structures.
• How to make sure LLM’s output can be used by other modules or libraries.
• How to easily switch from one LLM model to another.
• How to keep some record of memory when needed.
• How to deal with data.

All these problems bring us to the following question:

How to develop a whole complex application being sure that the LLM model will behave as expected.

The prompts are riddled with repetitive structures and text, the responses are as unstructured as a toddler's playroom, and the memory of these models? Let's just say it's not exactly elephantine.

So… how can we work with them?

Trying to develop complex applications with AI and LLMs can be a complete headache.

And this is where LangChain steps in as the problem-solver.

At its core, LangChain is made up of several ingenious components that allow you to easily integrate LLM in any development.

LangChain is generating enthusiasm for its ability to amplify the capabilities of potent large language models by endowing them with memory and context. This addition enables the simulation of "reasoning" processes, allowing for the tackling of more intricate tasks with greater precision.

For developers, the appeal of LangChain lies in its innovative approach to creating user interfaces. Rather than relying on traditional methods like drag-and-drop or coding, users can articulate their needs directly, and the interface is constructed to accommodate those requests.

It is a framework designed to supercharge software developers and data engineers with the ability to seamlessly integrate LLMs into their applications and data workflows.

So this brings us to the following question…

How is LangChain trying to solve all these problems?

Knowing current LLMs present 6 main problems, now we can see how LangChain is trying to assess them.

Image by Author

1. Prompts are way too complex now

Let’s try to recall how the concept of prompt has rapidly evolved during these last months.

It all started with a simple string describing an easy task to perform:

Hey ChatGPT, can you please explain to me how to plot a scatter chart in Python?

However, over time people realized this was way too simple. We were not providing LLMs enough context to understand their main task.

Today we need to tell any LLM much more than simply describing the main task to fulfill. We have to describe the AI’s high-level behavior, the writing style and include instructions to make sure the answer is accurate. And any other detail to give a more contextualized instruction to our model.

So today, rather than using the very first prompt, we would submit something more similar to:

Hey ChatGPT, imagine you are a data scientist. You are good at analyzing data and visualizing it using Python.   Can you please explain to me how to generate a scatter chart using the Seaborn library in Python

Right?

However, as most of you have already realized, I can ask for a different task but still keep the same high-level behavior of the LLM. This means that most parts of the prompt can remain the same.

This is why we should be able to write this part just one time and then add it to any prompt you need.

LangChain fixes this repeat text issue by offering templates for prompts.

These templates mix the specific details you need for your task (asking exactly for the scatter chart) with the usual text (like describing the high-level behavior of the model).

So our final prompt template would be:

Hey ChatGPT, imagine you are a data scientist. You are good at analyzing data and visualizing it using Python.   Can you please explain to me how to generate a  using the  library in Python?

With two main input variables:

• type of chart
• python library

2. Responses Are Unstructured by Nature

We humans interpret text easily, This is why when chatting with any AI-powered chatbot like ChatGPT, we can easily deal with plain text.

However, when using these very same AI algorithms for apps or programs, these answers should be provided in a set format, like CSV or JSON files.

Again, we can try to craft sophisticated prompts that ask for specific structured outputs. But we cannot be 100% sure that this output will be generated in a structure that is useful for us.

This is where LangChain’s Output parsers kick in.

This class allows us to parse any LLM response and generate a structured variable that can be easily used. Forget about asking ChatGPT to answer you in a JSON, LangChain now allows you to parse your output and generate your own JSON.

3. LLMs Have No Memory — but some applications might need them to.

Now just imagine you are talking with a company’s Q&A chatbot. You send a detailed description of what you need, the chatbot answers correctly and after a second iteration… it is all gone!

This is pretty much what happens when calling any LLM via API. When using GPT or any other user-interface chatbot, the AI model forgets any part of the conversation the very moment we pass to our next turn.

They do not have any, or much, memory.

And this can lead to confusing or wrong answers.

As most of you have already guessed, LangChain again is ready to come to help us.

LangChain offers a class called memory. It allows us to keep the model context-aware, be it keeping the whole chat history or just a summary so it does not get any wrong replies.

4. Why choose a single LLM when you can have them all?

We all know OpenAI’s GPT models are still in the realm of LLMs. However… There are plenty of other options out there like Meta’s Llama, Claude, or Hugging Face Hub open-source models.

If you only design your program for one company's language model, you're stuck with their tools and rules.

Using directly the native API of a single model makes you depend totally on them.

Imagine if you built your app's AI features with GPT, but later found out you need to incorporate a feature that is better assessed using Meta’s Llama.

You will be forced to start all over from scratch… which is not good at all.

LangChain offers something called an LLM class. Think of it as a special tool that makes it easy to change from one language model to another, or even use several models at once in your app.

This is why developing directly with LangChain allows you to consider multiple models at once.

5. Passing Data to the LLM is Tricky

Language models like GPT-4 are trained with huge volumes of text. This is why they work with text by nature. However, they usually struggle when it comes to working with data.

Why? You might ask.

Two main issues can be differentiated:

• When working with data, we first need to know how to store this data, and how to effectively select the data we want to show to the model. LangChain helps with this issue by using something called indexes. These let you bring in data from different places like databases or spreadsheets and set it up so it's ready to be sent to the AI piece by piece.

• On the other hand, we need to decide how to put that data into the prompt you give the model. The easiest way is to just put all the data directly into the prompt, but there are smarter ways to do it, too.

In this second case, LangChain has some special tools that use different methods to give data to the AI. Be it using direct Prompt stuffing, which allows you to put the whole data set right into the prompt, or using more advanced options like Map-reduce, Refine, or Map-rerank, LangChain eases the way we send data to any LLM.

6. Standardizing Development Interfaces

It's always tricky to fit LLMs into bigger systems or workflows. For instance, you might need to get some info from a database, give it to the AI, and then use the AI's answer in another part of your system.

LangChain has special features for these kinds of setups.

• Chains are like strings that tie different steps together in a simple, straight line.
• Agents are smarter and can make choices about what to do next, based on what the AI says.

LangChain also simplifies this by providing standardized interfaces that streamline the development process, making it easier to integrate and chain calls to LLMs and other utilities, enhancing the overall development experience.

Conclusion

In essence, LangChain offers a suite of tools and features that make it easier to develop applications with LLMs by addressing the intricacies of prompt crafting, response structuring, and model integration.

LangChain is more than just a framework, it's a game-changer in the world of data engineering and LLMs.

It's the bridge between the complex, often chaotic world of AI and the structured, systematic approach needed in data applications.

As we wrap up this exploration, one thing is clear:

LangChain is not just shaping the future of LLMs, it's shaping the future of technology itself.

Josep Ferrer is an analytics engineer from Barcelona. He graduated in physics engineering and is currently working in the Data Science field applied to human mobility. He is a part-time content creator focused on data science and technology. You can contact him on LinkedIn, Twitter or Medium.

More On This Topic

• LangChain + Streamlit + Llama: Bringing Conversational AI to Your…
• LangChain 101: Build Your Own GPT-Powered Applications
• Transforming AI with LangChain: A Text Data Game Changer
• LangChain Cheat Sheet
• More Performance Evaluation Metrics for Classification Problems You…
• How to troubleshoot memory problems in Python

LangChain and Microsoft announced a collaboration today which aims to provide deeper product integrations for their joint customers, enhancing the Azure ecosystem’s enterprise assurances. LangChain is also now part of the Microsoft for Startups Pegasus Program.

LangChain 🤝🏾 Microsoft
Today, we’re thrilled to announce our collaboration with @microsoft @azure.
Our joint customers will enjoy deeper product integrations that live up to our commitment to AI with the enterprise assurances the Azure ecosystem provides.
more on the blog and…

— LangChain (@LangChainAI) November 15, 2023

The AI industry is evolving rapidly, with large enterprises quickly adopting new technologies. LangChain offers speed of iteration, flexibility, and advanced cognitive architectures.

The combination of LangChain, LangSmith, and Azure services like Azure Machine Learning and Azure AI Search is considered essential for building GenAI applications.

Eric Boyd, CVP of Azure AI Platform, stated, “We’re excited to work closely with LangChain to streamline product adoption and to explore continued integration opportunities across the Azure and LangChain ecosystem to better serve our customers.”

Harrison Chase, CEO of LangChain, commented, “We have many customers of LangSmith – Rakuten Group, Moody’s to name a couple — that build with Azure services, and we’re excited to meet them where they are.”

The collaboration will result in deeper product integrations. Users will benefit from LangChain’s orchestration and LangSmith’s monitoring of Azure services. Microsoft and LangChain will work on improvements to make LangServe easier to deploy on Azure.

Post the Microsoft for Startups Pegasus Program, LangSmith will be available in the Azure Marketplace. Customers can use their Microsoft Azure Consumption Commitment for purchases. LangSmith deployments within the customer’s Azure VPC will ensure data security. LangChain and Microsoft are encouraging companies using their services to contact them for better support.

The post LangChain and Microsoft Announce Partnership to Enhance AI Collaboration appeared first on Analytics India Magazine.

At Ignite 2023, Microsoft announced the newest iteration of the Phi Small Language Model (SLM) series termed Phi-2. This comes at a time when many industry members are voicing their opinions that smaller models are going to be more useful for enterprises in comparison to Large Language Models (LLMs).

“Microsoft loves SLMs,” said Satya Nadella, Chairman and CEO at Microsoft in the keynote, and added that Phi-2, which has been developed by Microsoft’s Research Wing on highly specialised datasets, can rival models 150 times bigger.

Phi-2 has 2.7 billion parameters and demonstrates state-of-the-art performances against benchmark testing parameters such as common sense, language understanding and logical reasoning. “Phi-2 is open-source and soon will be coming to Microsoft’s catalogue of models-as-a-service,” Nadella said.

Moreover, in a blog post, Microsoft said that with the right fine-tuning and customisation, these SLMs are incredibly powerful tools for applications both on the cloud and on the edge.

( Microsoft Chairman and CEO Satya Nadella at Microsoft Ignite 2023)

Smaller Language Models are on the rise

In the last year or so, LLMs have captivated our attention, from GPT3.5, GPT-4, PaLM-2 to open-source models like Falcon and LLaMA. However, SLMs today are finding a growing emphasis.

When Meta released LLaMA, which has four variations -7 bn, 13 bn, 33 bn and 65 bn, it heralded the way for SLMs, at least, in some sense. It prompted the realisation that smaller models with fewer parameters can perform admirably.

Given the training of LLMs costs a fortune is one of the primary barriers to adoption. Smaller models present notable cost savings in contrast to GPT-3.5 and GPT-4. The expense for generating a paragraph summary with LLaMA 2, which has three variations—7 bn, 13 bn, and 70 bn— is approximately 30 times lower than that of GPT-4, all while preserving an equivalent level of accuracy.

Smaller models not only offer cost-efficiency but also excel in accuracy. Unlike their larger counterparts trained on vast and diverse datasets, smaller models focus on carefully vetted data tailored to specific business use cases, ensuring precision and relevance.

“Most companies will realise that smaller, cheaper, more specialised models make more sense for 99% of AI use-cases,” Clem Delangue, CEO at HuggingFace predicts.

Sam Altman, OpenAI’s CEO, echoes the sentiment. In a discussion at MIT, Altman envisioned a future where the number of parameters decreases, and a group of smaller models outperforms larger ones.

Microsoft’s efforts in developing smaller models underscore their belief in the significant benefits that SLMs will bring to enterprises in the future.

Microsoft loves SLMs

Earlier this year, besides releasing Phi and Phi 1.5, Microsoft also released Ocra, an open-source model with 13 billion parameters based on Vicuna which can imitate and learn from GPT-4 size LLMs.

During Ignite 2023, Nadella also unveiled ‘models-as-a-service’ offerings, providing enterprises access to various open-source models on platforms like Hugging Face, including models from Mistral and LLaMA 2.

Moreover, Phi-2, which is also available to enterprises in the Azure AI catalogue, could also be seen as a contender for the LLaMA series of models. Earlier this year, Microsoft already claimed that Phi-1.5, which has 1.3 billion parameters, outperforms LlaMA 2’s 7-billion parameters model on several benchmarks.

When LLaMA was released to the public, it had neither Reinforcement Learning with Human Feedback (RLHF), nor instruction or conversation tuning.

However, its open-source nature sparked excitement within the community, leading to a cascade of variants featuring instruction tuning, human evaluations, multimodality, RLHF, and more. It made LLaMA one of the most popular models. Now, Microsoft could look to replicate or surpass the success of LLaMA with Phi-2.

According to Sebastien Bubeck, who leads the ML Foundations team at Microsoft Research, Phi-2 is the perfect model to be fine-tuned. Small enterprises or startups looking to leverage generative AI models could find it to be beneficial.

“I’m sure that there are tons of small AI products that have used non-commercial LLMs like Llama. Phi-2 is going to supplant all of those,” Mark Tenenholtz, VP of Data Science at Predelo said.

Open-source for research purposes only

At the keynote, Nadella revealed that Phi-2 is open source. However, a quick glance at the licence revealed that the model is stated for research purposes only for now. The same was pointed out by many X ( previously Twitter) users.

Ok so looking at the model in azure it has a research license again… pic.twitter.com/u2BGcjWLFH

— anton (@abacaj) November 15, 2023

‘Open-source-for-research-purpose-only’ has a familiar ring, reminiscent of the earlier LLaMA release. In February of this year, Meta shared LLaMA’s model weights with the research community under a noncommercial license. However, it later surfaced on 4Chan with accessible weights, inadvertently making it available for commercial use.

If Microsoft is looking to replicate the success of LLaMA with Phi-2, it needs to make the model available for commercial use. Moreover, over time, the idea of ‘open source’ has faced scrutiny. Although models like LLaMA are touted as open source, some argue they don’t truly fit the definition as Meta has not disclosed the datasets used in their training.

The post Microsoft’s Strategic Shift: Embracing Smaller Language Models with Phi-2 appeared first on Analytics India Magazine.

Introduction

GitHub Copilot is an artificial intelligence-powered code completion assistant developed by GitHub and in collaboration with OpenAI, leveraging the ChatGPT model. It's designed to assist developers in accelerating their coding process while minimizing errors. The underlying model is trained on a blend of licensed code from GitHub's own repositories as well as publicly available code, equipping it with a broad understanding of programming paradigms.

On the other hand, Databricks, an open analytics and cloud-based platform founded by the original creators of Apache Spark, empowers organizations to construct data analytics and machine learning pipelines seamlessly, thereby accelerating innovation. Additionally, it fosters collaborative work among users.

Integrating GitHub Copilot with Databricks empowers data analytics and machine learning engineers to deploy solutions efficiently and in a time-effective manner. This integration facilitates smoother code development, enhances code quality and standardization, boosts cross-language efficiency, speeds up prototype development, and aids in documentation, consequently elevating the productivity and efficiency of engineers.

Prerequisites for GitHub Copilot and Databricks Integration:

Databricks account setup.

Setting up GitHub Copilot.

Download and install Visual Studio Code.

Steps for Integration

Install Databricks Plugin in Visual Studio Code Marketplace.

Configure the Databricks Plugin in Visual Studio Code. If you have used Databricks CLI before, then it’s already configured for you locally in databrickscfg file. If not, create the following contents in ~/.databrickscfg file.

[DEFAULT]  host = https://xxx  token = <token>  jobs-api-version = 2.0

Click the “Configure Databricks” option, then choose the first option from the dropdown, which displays the hostname configured in the above step, and continue with the “DEFAULT” profile.

After completing the configuration, a Databricks connection is established with Visual Studio Code. You can see the workspace and cluster configuration details when you click on the Databricks plugin.

Once a user completes the GitHub Copilot account setup, make sure you have access to GitHub Copilot. Install GitHub Copilot and GitHub Copilot Chat Plugins in VSCode via Marketplace.

Once a user installs GitHub Copilot & Copilot Chat plugins, it will be prompted to sign in to GitHub Copilot through Visual Studio IDE. If it’s not prompted to authorize, then click the bell icon in the bottom panel of Visual Studio code IDE.

Now, it’s time development with GitHub Copilot

Developing a Data Engineering Pipeline

Data Engineers can utilize GitHub Copilot to write data engineering pipelines at fingertips at a faster pace, including documentation, within no time. Below are the steps to create a simple data engineering pipeline with prompting techniques.

Read files from the S3 bucket using Python and Spark framework.

Write data frame to S3 bucket using Python and Spark framework

Execute the functions through the main method: Represented same in prompt and resulted from the code with execution steps

Benefits of Using GitHub Copilot for Data Engineering and Machine Learning in Databricks

• Good AI pair programming tool for quick sensible suggestions and provides boilerplate code.
• Top-notch suggestions to optimize the code & run time.
• Better documentation and ASCII representation for logical steps.
• Faster data pipeline implementation with minimal errors.
• Explain existing simple/complex functionality in detail and suggest intelligent code refactoring techniques.

Cheat Sheet

• Opens a Co-pilot text/search bar where you can enter your prompts.

  
  Windows: [Cltr] + [I]

  Mac: Command + [I]

• Opens a separate window on the right with the top 10 code suggestions.

  Windows: [Cltr] + [Enter]

  Mac: [control] + [return]

• Open a separate copilot chat window on the left side.

  Windows: [Cltr] + [Alt] + [I]

  Mac: [Control] + [Command] + [I]

• Dismiss an inline suggestion.

  Windows/Mac: Esc

• Accept a suggestion.

  Windows/Mac: Tab

• Refer to previous suggestions.

  Windows: [Alt] + [

  Mac: [option] + [

• Check for next suggestion

  Windows: [Alt] + ]

  Mac: [option] + ]

Conclusion

Integration of AI pair programming tools with integrated development environments helps developers speed up the development with real-time code suggestions, reducing time spent on referring to documentation for boilerplate code and syntaxes, and enabling developers to focus on innovations and business problem-solving use cases.

Further Resources

• https://app.pluralsight.com/library/courses/getting-started-prompt-engineering-generative-ai/table-of-contents
• https://docs.github.com/en/copilot/quickstart

Naresh Vurukonda is a Principal Architect with 10 plus years of experience in building Data Engineering and Machine learning projects in Healthcare and Life Sciences and Media Network organizations.

More On This Topic

• GitHub Copilot: Your AI pair programmer — what is all the fuss about?
• GitHub Copilot Open Source Alternatives
• GitHub Copilot and the Rise of AI Language Models in Programming Automation
• Integrating ChatGPT Into Data Science Workflows: Tips and Best Practices
• Cutting Down Implementation Time by Integrating Jupyter and KNIME
• Optimizing Data Storage: Exploring Data Types and Normalization in SQL

Oracle is bolstering healthcare organizations’ inventory management capabilities with new mobile features integrated into Oracle Fusion Cloud Inventory Management, part of Oracle Fusion Cloud Supply Chain & Manufacturing (SCM). The aim is to enhance stock availability optimization, reduce errors, and ultimately improve patient care.

“With Oracle Cloud SCM’s new inventory management capabilities, healthcare providers can improve the efficiency and accuracy of inventory management processes to expand visibility, reduce costs, and ensure the right stock is available in the right place, at the right time.” said Chris Leone, executive vice president, applications development, Oracle.

These features include full mobile device and barcode scanning support for various inventory processes such as receiving, putaway, picking, cycle counting, material transfers, and issuing of materials across different stocking locations within a hospital. The PAR Management Workbench allows organisations to monitor inventory levels across all locations, predict demand, and manage replenishment efficiently. Bulk-maintenance capabilities streamline bulk stock ordering processes, enhancing user productivity.

The Mobile PAR Counting App facilitates efficient PAR management from any location, especially beneficial for hospitals with poor connectivity. The app’s offline mode enables continuous tracking of inventory at each location, triggering a restock once connected to the internet.

Another valuable addition is the Last-Mile Logistics App, which focuses on improving logistics management, materials delivery, and restocking efficiency. The app records proof of delivery to enhance transparency during stock hand-offs, and its offline mode optimizes internal logistics for hospitals with connectivity challenges.

These new inventory management capabilities align with Oracle’s broader strategy to help healthcare organizations optimize planning, automate processes, and enhance visibility across the supply chain. In the past year, Oracle has introduced various healthcare-specific supply chain features, covering logistics, product lifecycle management, planning, procurement, and channel revenue management.

Part of the Oracle Fusion Applications Suite, Oracle Cloud SCM allows seamless connectivity of supply chain processes, aiding organizations in responding swiftly to changing demand, supply, and market conditions. Oracle’s self-updating platform ensures continuous innovation, with new features introduced every 90 days.

The post Oracle Empowers Healthcare Inventory Management with Mobile Capabilities appeared first on Analytics India Magazine.

Image by Author

As a data professional, you’re probably familiar with the cost of poor data quality. For all data projects—big or small—you should perform essential data quality checks.

There are dedicated libraries and frameworks for data quality assessment. But if you are a beginner, you can run simple yet important data quality checks with pandas. And this tutorial will teach you how.

We’ll use the California Housing Dataset from scikit-learn for this tutorial.

An Overview of the California Housing Dataset

We’ll use the California housing dataset from Scikit-learn’s datasets module. The data set contains over 20,000 records of eight numeric features and a target median house value.

Let’s read the dataset into a pandas dataframe df:

from sklearn.datasets import fetch_california_housing  import pandas as pd    # Fetch the California housing dataset  data = fetch_california_housing()    # Convert the dataset to a Pandas DataFrame  df = pd.DataFrame(data.data, columns=data.feature_names)    # Add target column  df['MedHouseVal'] = data.target

For a detailed description of the dataset, run data.DESCR as shown:

print(data.DESCR)

Output of data.DESCR

Let's get some basic information on the dataset:

df.info()

Here’s the output:

Output >>>      RangeIndex: 20640 entries, 0 to 20639  Data columns (total 9 columns):   #   Column   	Non-Null Count  Dtype    ---  ------   	--------------  -----     0   MedInc   	20640 non-null  float64   1   HouseAge 	20640 non-null  float64   2   AveRooms 	20640 non-null  float64   3   AveBedrms	20640 non-null  float64   4   Population   20640 non-null  float64   5   AveOccup 	20640 non-null  float64   6   Latitude 	20640 non-null  float64   7   Longitude	20640 non-null  float64   8   MedHouseVal  20640 non-null  float64  dtypes: float64(9)  memory usage: 1.4 MB

Because we have numeric features, let us also get the summary starts using the describe() method:

df.describe()

Output of df.describe() 1. Check for Missing Values

Real-world datasets often have missing values. To analyze the data and build models, you need to handle these missing values.

To ensure data quality, you should check if the fraction of missing values is within a specific tolerance limit. You can then impute the missing values using suitable imputation strategies.

The first step, therefore, is to check for missing values across all features in the dataset.

This code checks for missing values in each column of the dataframe df:

# Check for missing values in the DataFrame  missing_values = df.isnull().sum()  print("Missing Values:")  print(missing_values)

The result is a pandas series that shows the count of missing values for each column:

Output >>>    Missing Values:  MedInc     	0  HouseAge   	0  AveRooms   	0  AveBedrms  	0  Population 	0  AveOccup   	0  Latitude   	0  Longitude  	0  MedHouseVal	0  dtype: int64

As seen, there are no missing values in this dataset.

2. Identify Duplicate Records

Duplicate records in the dataset can skew analysis. So you should check for and drop the duplicate records as needed.

Here’s the code to identify and return duplicate rows in df. If there are any duplicate rows, they will be included in the result:

# Check for duplicate rows in the DataFrame  duplicate_rows = df[df.duplicated()]  print("Duplicate Rows:")  print(duplicate_rows)

The result is an empty dataframe. Meaning there are no duplicate records in the dataset:

Output >>>    Duplicate Rows:  Empty DataFrame  Columns: [MedInc, HouseAge, AveRooms, AveBedrms, Population, AveOccup, Latitude, Longitude, MedHouseVal]  Index: []

3. Check Data Types

When analyzing a dataset, you’ll often have to transform or scale one or more features. To avoid unexpected errors when performing such operations, it is important to check if the columns are all of the expected data type.

This code checks the data types of each column in the dataframe df:

# Check data types of each column in the DataFrame  data_types = df.dtypes  print("Data Types:")  print(data_types)

Here, all numeric features are of float data type as expected:

Output >>>    Data Types:  MedInc     	float64  HouseAge   	float64  AveRooms   	float64  AveBedrms  	float64  Population 	float64  AveOccup   	float64  Latitude   	float64  Longitude  	float64  MedHouseVal	float64  dtype: object

4. Check for Outliers

Outliers are data points that are significantly different from other points in the dataset. If you remember, we ran the describe() method on the dataframe.

Based on the quartile values and the maximum value, you could’ve identified that a subset of features contain outliers. Specifically, these features:

• MedInc
• AveRooms
• AveBedrms
• Population

One approach to handling outliers is to use the interquartile range, the difference between the 75th and 25th quartiles. If Q1 is the 25th quartile and Q3 is the 75th quartile, then the interquartile range is given by: Q3 — Q1.

We then use the quartiles and the IQR to define the interval [Q1 - 1.5 * IQR, Q3 + 1.5 * IQR]. And all points outside this range are outliers.

columns_to_check = ['MedInc', 'AveRooms', 'AveBedrms', 'Population']    # Function to find records with outliers  def find_outliers_pandas(data, column):  	Q1 = data[column].quantile(0.25)  	Q3 = data[column].quantile(0.75)  	IQR = Q3 - Q1  	lower_bound = Q1 - 1.5 * IQR  	upper_bound = Q3 + 1.5 * IQR  	outliers = data[(data[column] < lower_bound) | (data[column] > upper_bound)]  	return outliers    # Find records with outliers for each specified column  outliers_dict = {}    for column in columns_to-check:  	outliers_dict[column] = find_outliers_pandas(df, column)    # Print the records with outliers for each column  for column, outliers in outliers_dict.items():  	print(f"Outliers in '{column}':")  	print(outliers)  	print("n")

Outliers in 'AveRooms' Column | Truncated Output for Outliers Check 5. Validate Numeric Ranges

An important check for numeric features is to validate the range. This ensures that all observations of a feature take on values in an expected range.

This code validates that the 'MedInc' value falls within an expected range and identifies data points that do not meet this criteria:

# Check numerical value range for the 'MedInc' column  valid_range = (0, 16)    value_range_check = df[~df['MedInc'].between(*valid_range)]  print("Value Range Check (MedInc):")  print(value_range_check)

You can try for other numeric features of your choice. But we see that all values in the 'MedInc' column lie in the expected range:

Output >>>    Value Range Check (MedInc):  Empty DataFrame  Columns: [MedInc, HouseAge, AveRooms, AveBedrms, Population, AveOccup, Latitude, Longitude, MedHouseVal]  Index: []

6. Check Cross-Column Dependency

Most data sets contain related features. So it's important to include checks based on logically relevant relationships between columns (or features).

While features—individually—may take on values in the expected range, the relationship between them may be inconsistent.

Here is an example for our dataset. In a valid record, the ‘AveRooms’ should typically be greater than or equal to the ‘AveBedRms’.

# AveRooms should not be smaller than AveBedrooms  invalid_data = df[df['AveRooms'] < df['AveBedrms']]  print("Invalid Records (AveRooms < AveBedrms):")  print(invalid_data)

In the California housing dataset we’re working with, we see that there are no such invalid records:

Output >>>    Invalid Records (AveRooms < AveBedrms):  Empty DataFrame  Columns: [MedInc, HouseAge, AveRooms, AveBedrms, Population, AveOccup, Latitude, Longitude, MedHouseVal]  Index: []

7. Check for Inconsistent Data Entry

Inconsistent data entry is a common data quality issue in most datasets. Examples include:

• Inconsistent formatting in datetime columns
• Inconsistent logging of categorical variable values
• Recording of reading in different units

In our dataset, we’ve verified the data types of columns and have identified outliers. But you can also run checks for inconsistent data entry.

Let’s whip up a simple example to check if all the date entries have a consistent formatting.

Here we use regular expressions in conjunction with pandas apply() function to check if all date entries are in the YYYY-MM-DD format:

import pandas as pd  import re    data = {'Date': ['2023-10-29', '2023-11-15', '23-10-2023', '2023/10/29', '2023-10-30']}  df = pd.DataFrame(data)    # Define the expected date format  date_format_pattern = r'^d{4}-d{2}-d{2}$'  # YYYY-MM-DD format    # Function to check if a date value matches the expected format  def check_date_format(date_str, date_format_pattern):  	return re.match(date_format_pattern, date_str) is not None    # Apply the format check to the 'Date' column  date_format_check = df['Date'].apply(lambda x: check_date_format(x, date_format_pattern))    # Identify and retrieve entries that do not follow the expected format  non_adherent_dates = df[~date_format_check]    if not non_adherent_dates.empty:  	print("Entries that do not follow the expected format:")  	print(non_adherent_dates)  else:  	print("All dates are in the expected format.")

This returns the entries that do not follow the expected format:

Output >>>    Entries that do not follow the expected format:       	Date  2  23-10-2023  3  2023/10/29

Wrapping up

In this tutorial, we went over common data quality checks with pandas.

When you are working on smaller data analysis projects, these data quality checks with pandas are a good starting point. Depending on the problem and the dataset, you can include additional checks.

If you’re interested in learning data analysis, check out the guide 7 Steps to Mastering Data Wrangling with Pandas and Python.

Bala Priya C is a developer and technical writer from India. She likes working at the intersection of math, programming, data science, and content creation. Her areas of interest and expertise include DevOps, data science, and natural language processing. She enjoys reading, writing, coding, and coffee! Currently, she's working on learning and sharing her knowledge with the developer community by authoring tutorials, how-to guides, opinion pieces, and more.

More On This Topic

• Data Quality Dimensions: Assuring Your Data Quality with Great Expectations
• When to Retrain an Machine Learning Model? Run these 5 checks to…
• Deep Learning For Compliance Checks: What's New?
• 10 Essential Pandas Functions Every Data Scientist Should Know
• Data Observability, Part II: How to Build Your Own Data Quality…
• Inside the Architecture Powering Data Quality Management at Uber