Not many are aware that IT consulting giant Infosys, together with Elon Musk, AWS, YC Research, and a few others, had donated a sizable $1 billion to OpenAI back in 2015, when the latter began as a non-profit organisation.
But what transpired thereafter and why is Infosys not the Microsoft of IT consulting?
The donation to OpenAI, previously undisclosed, occurred during Vishal Sikka’s tenure as the leader of Infosys. Sikka, a strong advocate for AI, played a significant role in facilitating the donation to the freshly-minted AI company. Additionally, he acted as an advisor to OpenAI, along with Yoshua Bengio, Sergey Levine, Pieter Abbeel, and Alan Kay.
“Our wish is that together the OpenAI team will do unfettered research in the most important, most relevant dimensions of AI, no matter how long it takes to get there, not limited to just identifying dancing cats in videos, but to creating ideas and inventions that amplify our humanity,” Sikka said, in an Infosys blog post in 2015, identifying with OpenAI’s mission.
A donation that didn’t become an investment
Sikka’s decision to invest in OpenAI was influenced by its open-source philosophy. “If complex systems are not open, not open to be used, extended, and learned about, they end up becoming yet another mysterious thing for us, ones that we end up praying to and mystifying. The more open we make AI, the better,” he said, pointing out that openness was the most important factor behind investing in OpenAI.
Sikka had previously said that Infosys would benefit heavily from OpenAI because of the products and intelligent software systems that Altman was hoping to build and maintain for all domains and industries. “In addition, as a large services company, many parts of our work can transform fundamentally with AI,” he said.
In August 2017, Sikka stepped down from his role as managing director and chief executive at Infosys. With his departure, the initiative to integrate OpenAI’s technology into Infosys came to an abrupt halt.
In 2019, Sikka launched ‘Vianai‘ with an initial funding of $50 million. The startup devised its unique programming language aimed at facilitating the adoption of AI and machine learning techniques by a broader range of developers and organisations.
However, significant changes have occurred since then. OpenAI moved on from being a non-profit organisation to a for-profit entity with a capped return in 2019. Then there was a billion-dollar investment from Microsoft, for which Musk recently took OpenAI to court. The original donors did not get any equity in the company after the Microsoft investment.
On a different note, Altman recently claimed that Musk wanted Tesla to acquire OpenAI and turn it into a for-profit company in 2018, since he thought the company would fail. “He wanted OpenAI to be basically acquired by Tesla in the same way that… or maybe something similar… or maybe something more dramatic than the partnership with Microsoft (sic),” said Altman.
Disagreements occurred, and Musk left.
What’s up with Infosys in GenAI
Sikka’s recognition of OpenAI’s potential at an early stage and his willingness to embrace it underscore his status as a far-sighted visionary. Narayan Murthy, the founder of Infosys, also hailed Sikka for his forward-thinking approach. Infosys could have become the Microsoft of IT consulting if it would have made the deal in 2019.
The current CEO, Salil Parekh, confirmed that Infosys had backed OpenAI through a donation. However, regarding any potential future collaboration with OpenAI, Parekh stated that there were no intentions for investments or engagement in related activities with OpenAI.
In September last year, Infosys had announced a partnership with Microsoft to develop solutions using Infosys Topaz and Azure OpenAI Service jointly and boost enterprise customers’ AI solutions. Parekh had also confirmed that the company was using ChatGPT within the organisation to boost productivity.
During the same month, Infosys also partnered with NVIDIA to boost adoption in enterprise AI. The partnership unveiled plans to establish the NVIDIA Center of Excellence dedicated to training and certifying 50,000 of its employees in NVIDIA AI technology.
Moreover, in December last year, Infosys reported the termination of a $1.5 billion deal by a global client. The announcement came a few months after India’s IT giant disclosed the signing of a $1.5 billion agreement with an unnamed global company in September the same year. There is no definite proof of which company it was that cancelled the deal.
In July, Infosys had also signed a deal with another undisclosed existing client to provide AI services, with the target spend of $2 billion over five years.
Certainly, there’s lots happening to accelerate Infosys’ AI vision under Parekh. But it could have been a lot more beneficial had it been a direct investor in OpenAI, not through Microsoft, or any other company.
The post Infosys’ Biggest GenAI Regret Ever appeared first on Analytics India Magazine.
In his lecture at Oxford University, Geoffrey Hinton, one of three godfathers of AI, said that digital models are already as close to being as good as brains and will eventually get better than brains.
“A large language model has a trillion weights. You have 100 trillion weights. Even if you use 10% of that, you have 10 trillion weights,” said Hinton. He adds that an LLM in its trillion weights knows thousands of times more than we do. “It’s got much more knowledge and that’s partly because it has seen much more data,” he explains, that it might also be because it has much better learning algorithms.
He explains that humans are not optimised for packaging lots of experience into connections. “We are optimised for not having many experiences. You only live for about a billion seconds,” Hinton adds, saying that humans don’t really learn anything after the age of 30. “You have got crazily more parameters than you have got experiences. Our brain is optimised for not having many experiences.”
Recently, Elon Musk also reshared a video from the Joe Rogan podcast with Ray Kurzweil that AI will match the knowledge of any person by 2029. “AI will probably be smarter than any single human next year. By 2029, AI is probably smarter than all humans combined,” said Musk.
Kurzweil explained that people often underestimate the rate at which technology grows. “It actually doubles in fourteen years,” whereas, he said, people think it only grows by 2% every year. Talking about the speed of computers, he said that the calculation speed has increased to 35 billion calculations per second, which is a 24 quadrillion fold increase from 0.00007 calculations per second in 1939.
Hinton, who left Google last year, also has been openly also discussing the threats of AI. He compares the potential risks of AI to the creation of the atomic bomb during World War II, emphasising the dangers of profit-driven AI development that could result in AI-generated content surpassing human-produced content and jeopardising our survival.
The post Geoffrey Hinton Says AI is Very Close to Being as Good as Human Brains appeared first on Analytics India Magazine.
Code completion has been one of the most prominent use cases of large language models (LLMs). GitHub Copilot, the popular AI tool, has been used by over a million developers and 200,000 enterprises.
However, widely-used code generation tools like GitHub CoPilot, AWS CodeWhisperer or Google Duet AI are not open source. Enterprises are unaware of the specific codes on which these models are trained, presenting a significant concern, especially for those in highly scrutinised industries.
Thus, project BigCode, an open scientific collaboration run by Hugging Face and ServiceNow Research, was born. It recently released StarCoder 2, which is trained on a larger dataset (7.5 terabytes) than its predecessors and on 619 programming languages.
StarCoder 2 comes in three sizes – 3-billion, 7-billion and 15-billion-parameter models.
While there are a few open source code LLMs, the StarCoder 2 15-billion model developed by NVIDIA matches and at times even surpasses 33-billion parameter models, like Code Llama, on many evaluations.
How enterprises benefit from open source?
According to Leandro von Werra, machine learning engineer at Hugging Face and co-lead of the BigCode project, StarCoder 2 will empower the developer community to build a wide range of applications more efficiently with full data and training transparency.
Besides the fact that StarCoder 2 is free to use, it also brings in added benefits for developers and enterprises, according to Werra.
“For many companies, using GitHub CoPilot is tricky from a security perspective, because it requires employing the endpoint that CoPilot uses, which is not retained in their environments. You’re sending parts of your code to that endpoint and you have no control over where exactly that code goes.
“Given that code represents a crucial aspect of intellectual property for many companies, we’ve received numerous inquiries requesting an open version to utilise such services securely,” Werra told AIM.
Moreover, enterprises don’t know what codes went into the model during the training process. This lack of transparency poses a potential liability for the enterprise, especially if the model generates copyrighted code.
However, Werra adds that this is a problem which even his team has not been able to solve. “We’re doing licence detection, but it’s not 100% accurate. It’s nearly impossible to do it at that scale 100% correctly, but at least we provide full transparency in what went into it and how we filter data,” he said.
Fine-tuning StarCoder 2
While the above mentioned pointers were from a security perspective, the biggest benefit of StarCoder 2 for enterprises is that they can take the model and fine-tune it with their own enterprise data.
Indeed, many enterprises, for instance, emphasise their unique coding style or internal standards, which may differ from codebases used in training code LLMs.
“By leveraging their own codebase, they streamline processes, avoiding the need for extensive rewriting, such as fixing styles or updating docstrings, often accomplished effortlessly.
Alternatively, they can fine-tune the model for specific use cases, catering to tasks like text-to-SQL code conversion or translating legacy COBOL code to modern languages. This ability to fine-tune models based on their data enables companies to address specialised needs effectively,” Werra said.
For example, while a dedicated model may be more comprehensive for a specific SQL use case, fine-tuning allows for customisation, providing flexibility to tackle various scenarios—a prospect that excites enterprises.
So far, StarCoder 2 is already being used by ServiceNow, which also trained the 3-billion-parameter StarCoder 2 model. Besides, a dozen other enterprises have started leveraging StarCoder 2, according to Werra.
Previously, VMware, an American cloud computing and virtualisation technology company, successfully deployed a fine-tuned version of StarCoder.
Businesses subject to stringent security regulations, such as those in the financial or healthcare sectors, would most likely adopt open-source models. These companies face challenges in sharing data with third parties due to heightened scrutiny.
It is important to note that other code LLMs, like Code Llama, can be fine-tuned. However, Meta has not released the datasets besides stating that it has been trained on widely available public data.
Will enterprises pivot to open-source?
Using open-source technologies comes with its own set of challenges. Despite the promised benefits of StarCoder 2 and the adoption by a handful of enterprises, the question that arises is: will we see a wider adoption by enterprises?
Werra believes that it is probable, as many enterprises initially opt for closed LLMs due to their accessibility and ease of use. However, as companies mature and streamline their use cases, there is a growing desire for models that offer total control. This trend holds true for code LLMs as well.
“Decades ago, software development primarily relied on off-the-shelf solutions. However, the landscape has changed, with many companies, especially IT firms, crafting their own software solutions at the core of their operations.
“Similarly, a parallel trend is emerging with LLMs. While off-the-shelf models serve a broad range of tasks competently, for more specialised or dedicated applications, fine-tuning an open model remains the preferred approach,” Werra said.
Based on open-source principles
The BigCode team has open-sourced the model weights and dataset. “We released Stack V1 a year ago, and now we have released Stack v2,” Werra said.
However, even though the models are supported by an open rail licence, there are some restrictions.
For example, “You can’t use the model to extract Personally Identifiable Information (PII) from the pretrained data or generate potentially malicious code,” Werra warned. Nonetheless, StarCoder2 is available for commercial use.
The post Will StarCoder 2 Win Over Enterprises? appeared first on Analytics India Magazine.
Enterprise automation and software company UiPath recently announced a series of Large Language Models (LLMs) to help enterprises realise the full potential of AI with automation by accessing powerful, specialised AI models tailored to their challenges and most valuable use cases.
The new LLMs- DocPATH and CommPATH, give businesses LLMs that are extensively trained for their specific tasks, document processing and communications.
According to the UiPath, general-purpose GenAI models like GPT-4 struggle to match the performance and accuracy of models specially trained for a specific task.
Instead of relying on imprecise and time-consuming prompt engineering, DocPATH and CommPATH provide businesses with extensive tools to customize AI models to their exact requirements, allowing them to understand any document and a huge variety of message types.
UiPath showcased its latest capabilities at the virtual AI Summit that took place on March 19th, 2024.
The UiPath Business Automation Platform offers end-to-end automation for business processes. There are four key factors that business leaders seeking to embed AI in their automation program must keep top of mind: business context, AI model flexibility, actionability, and trust.
The new AI features of the UiPath Platform address these key areas to ensure customers are equipped with the tools necessary to enhance the performance and accuracy of GenAI models and tools and more easily tackle diverse business challenges with AI and automation.
“Businesses need an assortment of AI models, the best in class for every task, to achieve their full potential. Our new family of UiPath LLMs, along with Context Grounding to optimize GenAI models with business specific data, provide accuracy, consistency, predictability, time to value, and empower customers to transform their business environments with the latest GenAI capabilities on the market,” Graham Sheldon, chief product officer at UiPath said in a press release.
To help businesses use their enterprise data in a safe, reliable, low touch way, UiPath is introducing Context Grounding, a new feature within the UiPath AI Trust Layer that will be entering private preview in April.
Context Grounding makes business data LLM-ready by converting it to an optimised format that can easily be indexed, searched, and injected into prompts to improve GenAI predictions. Context Grounding will enhance all UiPath Gen AI experiences in UiPath Autopilots, GenAI Activities, and intelligent document processing (IDP) products like Document Understanding.
Earlier this year, while speaking at the UiPath DevCon 2024, held in Bengaluru, UiPath co-founder Daniel Dines revealed its UiPath is developing its foundational models.
The post UiPath Unveils New Family of LLMs appeared first on Analytics India Magazine.
With NVIDIA’s announcement of AI Enterprise 5.0 and NVIDIA Inference Microservices at the GTC conference, CEO Jensen Huang plans to begin an era of making enterprise AI deployment easier and more widely applicable than ever before – possibly while changing the main way people interact with computers.
The idea of controlling and programming computers with prompts alone is similar to what Humane has proposed with its prompt-based Ai Pin, but Huang extends it to developers and IT as well as consumers: “The job of the computer is to not require C++ to be useful,” Huang stated during the NVIDIA GTC press Q&A held March 19 in San Jose, California (Figure A).
Figure A
NVIDIA CEO Jensen Huang speaks during a press Q&A during NVIDIA GTC in San Jose, California on March 19. Image: Megan Crouse/TechRepublic
NVIDIA CEO Jensen Huang speaks during a press Q&A during NVIDIA GTC in San Jose, California on March 19. Image: Megan Crouse/TechRepublic
Huang: Prompt engineering is transforming programming
When asked whether programming will remain a useful skill in the age of generative AI prompts, Huang said, “I think that people ought to learn all kinds of skills,” and compared code to juggling, playing piano or learning calculus. However, Huang said, “Programming is not going to be essential for you to be a successful person.”
SEE: Huang announced a wide range of NVIDIA products for data centers, enterprise AI, cryptography and more during the GTC conference keynote. (TechRepublic)
Generative AI, Huang said, is “Closing the technology divide. You don’t have to be a C++ programmer to be successful,” he said. “You just have to be a prompt engineer. And who can’t be a prompt engineer? When my wife talks to me, she’s prompt engineering me. … We all need to learn how to prompt AIs, but that’s no different than learning how to prompt teammates.”
Huang followed up with, “But if somebody wants to learn to do so (program), please do because we’re hiring programmers.”
PREMIUM: Learn how to become a prompt engineer in this TechRepublic Premium download
Prompt engineering is a rapidly changing skill
Will prompt engineering replace traditional programming when it comes to creating generative AI from generative AI as Huang suggested?
“I would not quit my day job just yet to become a prompt engineer,” said Gartner analyst Chirag Dekate in a call to TechRepublic on March 19. “Unfortunately the market is over-correcting.”
And the market is over-correcting to a surge in demand for what prompt engineering used to look like. In the rapidly changing industry, optimizing prompts to get an AI to output the right text may no longer be the way AI prompt engineering is done; instead, prompts may be multimodal.
NIMs are remarkable, Dekate said, because they fit generative AI neatly into the hybrid multicloud context in which many enterprises operate. “What NVIDIA is now building is a foundation for next-generation, AI-native enterprises where everywhere enterprises are going to turn they are going to experience NIM,” he said.
However, NVIDIA may not be the company to make the transformation happen. Dekate pointed to Cognition AI, which last week introduced Devin, its “AI software engineer,” as a sign that the way software engineering is done may change in the future.
No matter whose name ends up on the most common software, Dekate said the way developers interact with generative AI is bound to change quickly.
“The pace of innovation of generative AI continues to accelerate,” said Dekate. “Chances are, we will not be interacting with any of these models using our legacy perceptions. I’m talking about three-month-old or six-month-old technology as legacy. Generative AI changes that fast.”
David Nicholson, chief research officer at The Futurum Group, told TechRepublic by email that in a generative AI future “a facility with human language becomes an important computer science skill.”
“Your degree in English (or) history or law suddenly helps you become a prompt engineer, but an actual computer science minor will never hurt,” Nicholson wrote. “It’s not NVIDIA hype. It’s truly a revolution.”
Disclaimer: NVIDIA paid for my airfare, accommodations and some meals for the NVIDIA GTC event held March 18 – 21 in San Jose, California.
Apple is in talks to build Google’s Gemini engine into the iPhone, reported Bloomberg, citing people familiar with the matter. The two firms are actively in talks for Apple to license Gemini, Google’s collection of generative AI models, to enhance upcoming features in the iPhone software.
Apple plans to introduce fresh functionalities in iOS 18, the upcoming version of the iPhone operating system, using its proprietary AI models. However, these improvements will concentrate on features that work locally on its devices, as opposed to those provided through cloud services.
Reports suggest iOS 18 will incorporate generative AI technology to boost Siri and the Messages app’s capabilities in answering questions and auto-completing sentences. Apple is also reportedly considering generative AI for apps like Apple Music, Pages, Keynote, and Xcode.
Therefore, Apple is looking for a collaborator to handle the complex tasks associated with generative AI, such as generating images and writing essays based on basic prompts. Apple has also had recent discussions with OpenAI and has been considering the use of its model.
Last year, Google introduced Gemini Nano, an LLM specifically built for edge use cases. Prior to partnering with Apple, Google also collaborated with Samsung to introduce Gemini in the Galaxy S24.
Apple Goes Big on Generative AI
Apple recently published a new paper unveiling MM1, a new family of multimodal AI models — with the largest at 30B parameters. Furthermore, it recently quietly acquired an AI startup, Canada’s Darwin AI. The company specialises in machine vision intelligence, smart manufacturing, improving machine learning efficiency, and edge-based intelligence.
Apple may have been slow when it comes to adoption of generative AI when compared to other big tech companies, however the company has been steadily progressing, in a secretive way.
“I won’t delve into specifics, as our policy is to keep development details confidential but rest assured we are heavily invested in this area,” said Apple chief, Tim Cook, during the Q3 earnings call last year. Apple is also planning to bring generative AI to its voice-assistant Siri.
The post After OpenAI, Apple Turns to Google’s Gemini to Integrate Generative AI into iPhone appeared first on Analytics India Magazine.
Palo Alto-based Plume Design, a pioneer in SaaS experience platforms for communication service providers (CSPs), inaugurated its first office in Hyderabad, India. With over 500 employees catering to over 400 CSPs worldwide, this marks Plume’s tenth global office and its first in India.
The move aims to bolster global expansion efforts, particularly with regional CSPs. Led by vice president Shrinivas Bairi, formerly of Intel and Qualcomm, the Hyderabad team will focus on research and development, covering software engineering, data engineering, DevOps, and QA.
The decision to establish a presence in Hyderabad aligns with Plume’s strategic partnership with Reliance Jio to serve over 200 million Indian households. Recognising India’s vibrant business landscape and skilled talent pool, Plume views the country as a hub for software development excellence. The Hyderabad office will complement existing teams, concentrating on software engineering roles spanning cloud, AI, web development, and information security.
“At the end of last year, we announced a partnership with telecom leader Reliance Jio to serve over 200 million homes across India. Opening an office in Hyderabad is part of our commitment to growing this relationship, and further expanding our local market presence is key to our long-term vision,” said Plume’s chief development officer, Kiran Edara.
Plume is also actively hiring for its new Hyderabad office, offering various opportunities. As the creator of the world’s first open, hardware-independent SaaS platform for CSPs, deployed in over 60 million locations globally, Plume empowers CSPs to deliver innovative services for smart homes and small businesses. Their platform provides self-optimising WiFi, cybersecurity, parental controls, and more, backed by robust data- and AI-driven backend applications.
The post Plume Inaugurates First Office in India appeared first on Analytics India Magazine.
Current models of artificial intelligence (AI) aren't ready as instruments for monetary policies, but the technology can lead to human extinction if governments do not intervene with the necessary safeguards, according to new reports. And intervene is exactly what the European Union (EU) did last week.
Also: The 3 biggest risks from generative AI — and how to deal with them
The European Parliament on Wednesday passed into law the EU AI Act, marking the first major wide-reaching AI legislation to be established globally. The European law aims to safeguard against three key risks, including "unacceptable risk" where government-run social scoring indexes such as those used in China are banned.
"The new rules ban certain AI applications that threaten citizens' rights, including biometric categorization systems based on sensitive characteristics and untargeted scraping of facial images from the internet or CCTV footage to create facial recognition databases," the European Parliament said. "Emotion recognition in the workplace and schools, social scoring, predictive policing (when it is based solely on profiling a person or assessing their characteristics), and AI that manipulates human behavior or exploits people's vulnerabilities will also be forbidden."
Applications identified as "high risk", such as resume-scanning tools that rank job applicants, must adhere to specific legal requirements. Applications not listed as high risk or explicitly banned are left largely unregulated.
There are some exemptions for law enforcement, which can use real-time biometric identification systems if "strict safeguards" are met, including limiting their use in time and geographic scope. For instance, these systems can be used to facilitate targeted search of a missing person or to prevent a terrorist attack.
Operators of high-risk AI systems, such as those in critical infrastructures, education, and essential private and public services including healthcare and banking, must assess and mitigate risks as well as maintain use logs and transparency. Other obligations these operators must fulfill include ensuring human oversight and data accuracy.
Also: As AI agents spread, so do the risks, scholars say
Citizens also have the right to submit complaints about AI systems and be given explanations about decisions based on high-risk AI systems that affect their rights.
General-purpose AI systems and the training models on which they are based have to adhere to certain transparency requirements, including complying with EU copyright law and publishing summaries of content used for training. More powerful models that can pose systemic risks will face additional requirements, including performing model evaluations and reporting of incidents.
Furthermore, artificial or manipulated images, audio, and video content, including deepfakes, must be clearly labeled as such.
"AI applications influence what information you see online by predicting what content is engaging to you, capture and analyze data from faces to enforce laws or personalise advertisements, and are used to diagnose and treat cancer," EU said. "In other words, AI affects many parts of your life."
Also: Employees input sensitive data into generative AI tools despite the risks
EU's internal market committee co-rapporteur and Italy's Brando Benifei said: "We finally have the world's first binding law on AI to reduce risks, create opportunities, combat discrimination, and bring transparency. Unacceptable AI practices will be banned in Europe and the rights of workers and citizens will be protected.
Benifei added that an AI Office will be set up to support companies in complying with the rules before they enter into force.
The regulations are subject to a final check by lawyers and a formal endorsement by the European Council. The AI Act will enter into force 20 days after its publication in the official journal and be fully applicable two years after its entry into force, with the exception of bans on prohibited practices, which will apply six months after the entry into force date. Codes of practice also will be enforced nine months after the initial rules kick off, while general-purpose AI rules including governance will take effect a year later. Obligations for high-risk systems will be effective three years after the law enters into force.
A new tool has been developed to guide European small and midsize businesses (SMBs) and startups to understand how they may be affected by the AI Act. The EU AI Act site noted, though, that this tool remains a "work in progress" and recommends organizations seek legal assistance.
Also: AI is supercharging collaboration between developers and business users
"The AI Act ensures Europeans can trust what AI has to offer," the EU said. "While most AI systems pose limited to no risk and can contribute to solving many societal challenges, certain AI systems create risks that we must address to avoid undesirable outcomes. For example, it is often not possible to find out why an AI system has made a decision or prediction and taken a particular action. So, it may become difficult to assess whether someone has been unfairly disadvantaged, such as in a hiring decision or in an application for a public benefit scheme."
The new legislation works to, among others, identify high-risk applications and require a standard assessment before the AI system is put into service or the market.
EU is hoping its AI Act will become a global standard like its General Data Protection Regulation (GDPR).
AI can lead to human extinction without human intervention
In the United States, a new report has called for governmental intervention before AI systems develop into dangerous weapons and lead to "catastrophic" events, including human extinction.
Released by Gladstone AI, the report was commissioned and "produced for review" by the US Department of State, though, its contents do not reflect the views of the government agency, according to the authors.
The report noted the accelerated progress of advanced AI, which has presented both opportunities and new categories of "weapons of mass destruction-like" risks. Such risks have been largely fueled by competition among AI labs to build the most advanced systems capable of achieving human-level and superhuman artificial general intelligence (AGI).
Also: Is humanity really doomed? Consider AI's Achilles heel
These developments are driving risks that are global in scale, have deeply technical origins, and are evolving quickly, Gladstone AI said. "As a result, policymakers face a diminishing opportunity to introduce technically informed safeguards that can balance these considerations and ensure advanced AI is developed and adopted responsibly," it said. "These safeguards are essential to address the critical national security gaps that are rapidly emerging as this technology progresses."
The report pointed to major AI players including Google, OpenAI, and Microsoft, that have acknowledged the potential risks, and noted that the "prospect of inadequate security" at AI labs added to the risk that the "advanced AI systems could be stolen from their US developers and weaponized against US interests".
These leading AI labs also highlighted the possibility of losing control of the AI systems they are developing, which can have "potentially devastating consequences" to global security, Gladstone AI said.
Also: I fell under the spell of an AI psychologist. Then things got a little weird
"Given the growing risk to national security posed by rapidly expanding AI capabilities from weaponization and loss of control, and particularly, the fact that the ongoing proliferation of these capabilities serves to amplify both risks — there is a clear and urgent need for the US government to intervene," the report noted.
It called for an action plan that includes implementing interim safeguards to stabilize advanced AI development, including export controls on the associated supply chain. The US government also should develop basic regulatory oversight and strengthen its capacity for later stages, and move toward a domestic legal regime of responsible AI use, with a new regulatory agency set up to have oversight. This should be later extended to include multilateral and international domains, according to the report.
The regulatory agency should have rule-making and licensing powers to oversee AI development and deployment, Gladstone AI added. A criminal and civil liability regime also should define responsibility for AI-induced damages and determine the extent of culpability for AI accidents and weaponization across all levels of the AI supply chain.
AI is not ready to drive monetary policies
Elsewhere in Singapore, the central bank mulled over the collective failure of global economies to predict the persistence of inflation following the pandemic.
Faced with questions about the effectiveness of existing models, economists were asked if they should be looking at advancements in data analytics and AI technologies to improve their forecasts and models, said Edward S. Robinson, deputy managing director of economic policy and chief economist at Monetary Authority of Singapore (MAS).
Also: Meet Copilot for Finance, Microsoft's latest AI chatbot — here's how to preview it
Traditional big data and machine learning techniques already are widely used in the sector, including central banks that have adopted these in various areas, noted Robinson, who was speaking at the 2024 Advanced Workshop for Central Banks held earlier last week. These include using AI and machine learning for financial supervision and macroeconomic monitoring, where they are used to identify anomalous financial transactions, for instance.
Current AI models, however, are still not ready as instruments for monetary policies, he said.
"A key strength of AI and machine learning modeling approaches in predictive tasks is their ability to let the data flexibly determine the functional form of the model," he explained. This allows the models to capture non-linearities in economic dynamics such that they mimic the judgment of human experts.
Recent advancements in generative AI (GenAI) take this further, with large language models (LLMs) trained on vast volumes of data that can generate alternate scenarios, he said. These specify and simulate basic economic models and surpass human experts at forecasting inflation.
Also: AI adoption and innovation will add trillions of dollars in economic value
The flexibility of LLMs, though, is a drawback, Robinson said. Noting that these AI models can be fragile, he said their output often is sensitive to the choice of the model's parameters or prompts used.
The LLMs also are opaque, he added, making it difficult to parse the underlying drivers of the process being modeled. "Despite their impressive capabilities, current LLMs struggle with logic puzzles and mathematical operations," he said. "[It suggests] they are not yet capable of providing credible explanations for their own predictions."
AI models today lack clarity of structure that allows existing models to be useful to monetary policymakers, he added. Unable to articulate how the economy works or discriminate between competing narratives, AI models cannot yet replace structural models at central banks, he said.
However, preparation is needed for the day GenAI evolves as a GPT, Robinson said.
Astera Labs IPO will reveal how much investors want in on AI
Startups with an AI angle would do well to pay attention
Alex Wilhelm Julie Bort 7 hours
While the technology world breathlessly awaits Reddit’s public debut, another company you might never have heard of is about to go public: Astera Labs. And it may be a more important test of investors’ returning appetite for tech IPOs.
Astera this week announced in a public filing that it’s public debut would be bigger than it initially planned in every way: It will sell more shares — 19.8 million vs. the previous plan of 17.8 million — and at a higher price, expecting to sell at $32 to $34 per share, vs the previous $27 to $30 range. Astera expects to raise $517.6 million at the middle of its raised range, it said, up from $392.4 million. IPO watchers expect it to debut this week.
While Reddit’s IPO could do well from investors looking to buy a well-known social media company that has an interesting, burgeoning AI data business, Astera Labs is an AI hardware story. And no, it’s not taking on Nvida, the American chip giant that created the world’s most in-demand AI chip.
Astera Labs makes connectivity hardware for cloud computing data centers. Because AI requires massive amounts of data moving into, out of and around data centers, Astera has seen recent its revenues bloom. After generating $79.9 million in 2022, revenue swelled 45% in 2023 to $115.8 million.
With 271 mentions of “AI” in its most recent SEC filing, the company is working hard to convince investors that it’s part of the larger artificial intelligence boom.
Just how much AI-juice Astera really has for long-term success is up for debate. Nick Einhorn, vice president of research at Renaissance Capital, a company that tracks the IPO market and offers public-offering focused ETFs, is a touch skeptical. Astera is “not an AI company” Einghorn told TechCrunch. The company, is, however, “benefiting from the trend,” in his view, particularly data center spending driven by AI. So much so, that in 2022, Amazon signed a warrant agreement that allows it to buy just shy of 1.5 million shares, which isn’t proof that Amazon Web Services is a customer, but does hint at it.
Then again, while the company does have an AI story to tell, its rapid recent growth and demonstrated early profitability could be the key drivers to its public-market investor interest.
Companies can grow and make money at the same time
In startup-land, growth and losses often walk hand-in-hand. Startups raise capital from private-market investors, investing the funds into their operations to expand headcount so that they can build, and sell more quickly. Often by the time that a startup reaches the required scale to file for a public offering, it is still unprofitable and not likely to start generating adjusted profits, let alone profit according to more stringent accounting standards, in the near future
Up until the fourth quarter of 2023, Astera Labs appeared to be just that sort of company. It’s business grew rapidly last year, with sticky losses to match.
On its 2022 $79.9 million in revenue, it posted a net loss of $58.3 million; on its 2023 $115.8 million in revenue, net loss tallied $26.3 million. So, on an annual basis, this is far from the kind of profitable company IPO experts say this harsh market requires. Even when the company removed the non-cash costs of paying its workers partially in shares, the company’s adjusted profits were still negative in 2023.
But when we dig in, its financial success becomes more nuanced. In the third quarter of 2023, Astera Labs’ revenue began growing dramatically: from $10.7 million in Q2 2023 to $36.9 million in Q3, and $50.5 million in Q4.
And while that spike in growth is impressive on its own, the company’s profitability picture has also radically improved as 2023 came to a close. After posting a net loss of $20.0 million in Q2 2023, net loss evaporated to a mere $3.1 million in Q3 2023.
And for Q4, Astera Labs swung to a profit: $14.3 million worth of net income.
Einhorn warned that the company’s Q4 2023 results may not augur the company’s new normal. “One of the challenges for companies like this,” he explained, “is that you tend to have a lot of customer concentration and customer buying patterns can be very lumpy.” Good recent quarters do not always imply similar future quarters. Another weakness: in 2023, its biggest three customers represented about 70% of its revenue, Astera disclosed.
Putting it all together: Astera Labs has caught a wave thanks to AI data center spending. Its resulting financial glow-up is impressive, and helps explain why its IPO is is set to occur at a valuation of around $5.2 billion, a healthy lift from of its final private-market price of $3.15 billion.
If the company is able to attract a strong following after its first day of trading, it could wedge the IPO door open for other businesses seeing newfound growth as a by-product of AI. And perhaps that will be enough for more technology offerings to sneak out this year.
Over the past few years, diffusion models have achieved massive success and recognition for image and video generation tasks. Video diffusion models, in particular, have been gaining significant attention due to their ability to produce videos with high coherence as well as fidelity. These models generate high-quality videos by employing an iterative denoising process in their architecture that gradually transforms high-dimensional Gaussian noise into real data.
Stable Diffusion is one of the most representative models for image generative tasks, relying on a Variational AutoEncoder (VAE) to map between the real image and the down-sampled latent features. This allows the model to reduce generative costs, while the cross-attention mechanism in its architecture facilitates text-conditioned image generation. More recently, the Stable Diffusion framework has built the foundation for several plug-and-play adapters to achieve more innovative and effective image or video generation. However, the iterative generative process employed by a majority of video diffusion models makes the image generation process time-consuming and comparatively costly, limiting its applications.
In this article, we will talk about AnimateLCM, a personalized diffusion model with adapters aimed at generating high-fidelity videos with minimal steps and computational costs. The AnimateLCM framework is inspired by the Consistency Model, which accelerates sampling with minimal steps by distilling pre-trained image diffusion models. Furthermore, the successful extension of the Consistency Model, the Latent Consistency Model (LCM), facilitates conditional image generation. Instead of conducting consistency learning directly on the raw video dataset, the AnimateLCM framework proposes using a decoupled consistency learning strategy. This strategy decouples the distillation of motion generation priors and image generation priors, allowing the model to enhance the visual quality of the generated content and improve training efficiency simultaneously. Additionally, the AnimateLCM model proposes training adapters from scratch or adapting existing adapters to its distilled video consistency model. This facilitates the combination of plug-and-play adapters in the family of stable diffusion models to achieve different functions without harming the sample speed.
This article aims to cover the AnimateLCM framework in depth. We explore the mechanism, the methodology, and the architecture of the framework, along with its comparison with state-of-the-art image and video generation frameworks. So, let's get started.
AnimateLCM : Animation of Personalized Diffusion Models
Diffusion models have been the go to framework for image generation and video generation tasks owing to their efficiency and capabilities on generative tasks. A majority of diffusion models rely on an iterative denoising process for image generation that transforms a high dimensional Gaussian noise into real data gradually. Although the method delivers somewhat satisfactory results, the iterative process and the number of iterating samples slows the generation process and also adds to the computational requirements of diffusion models that are much slower than other generative frameworks like GAN or Generative Adversarial Networks. In the past few years, Consistency Models or CMs have been proposed as an alternative to iterative diffusion models to speed up the generation process while keeping the computational requirements constant.
The highlight of consistency models is that they learn consistency mappings that maintain self-consistency of trajectories introduced by the pre-trained diffusion models. The learning process of Consistency Models allows it to generate high-quality images with minimal steps, and also eliminates the need for computation-intensive iterations. Furthermore, the Latent Consistency Model or LCM built on top of the stable diffusion framework can be integrated into the web user interface with the existing adapters to achieve a host of additional functionalities like real time image to image translation. In comparison, although the existing video diffusion models deliver acceptable results, progress is still to be made in the video sample acceleration field, and is of great significance owing to the high video generation computational costs.
That leads us to AnimateLCM, a high fidelity video generation framework that needs a minimal number of steps for the video generation tasks. Following the Latent Consistency Model, AnimateLCM framework treats the reverse diffusion process as solving CFG or Classifier Free Guidance augmented probability flow, and trains the model to predict the solution of such probability flows directly in the latent space. However, instead of conducting consistency learning on raw video data directly that requires high training and computational resources, and often leads to poor quality, the AnimateLCM framework proposes a decoupled consistent learning strategy that decouples the consistency distillation of motion generation and image generation priors.
The AnimateLCM framework first conducts the consistency distillation to adapt the image base diffusion model into the image consistency model, and then conducts 3D inflation to both the image consistency and image diffusion models to accommodate 3D features. Eventually, the AnimateLCM framework obtains the video consistency model by conducting consistency distillation on video data. Furthermore, to alleviate potential feature corruption as a result of the diffusion process, the AnimateLCM framework also proposes to use an initialization strategy. Since the AnimateLCM framework is built on top of the Stable Diffusion framework, it can replace the spatial weights of its trained video consistency model with the publicly available personalized image diffusion weights to achieve innovative generation results.
Additionally, to train specific adapters from scratch or to suit publicly available adapters better, the AnimateLCM framework proposes an effective acceleration strategy for the adapters that do not require training the specific teacher models.
The contributions of the AnimateLCM framework can be very well summarized as: The proposed AnimateLCM framework aims to achieve high quality, fast, and high fidelity video generation, and to achieve this, the AnimateLCM framework proposes a decoupled distillation strategy the decouples the motion and image generation priors resulting in better generation quality, and enhanced training efficiency.
InstantID : Methodology and Architecture
At its core, the InstantID framework draws heavy inspiration from diffusion models and sampling speed strategies. Diffusion models, also known as score-based generative models have demonstrated remarkable image generative capabilities. Under the guidance of score direction, the iterative sampling strategy implemented by diffusion models denoise the noise-corrupted data gradually. The efficiency of diffusion models is one of the major reasons why they are employed by a majority of video diffusion models by training on added temporal layers. On the other hand, sampling speed and sampling acceleration strategies help tackle the slow generation speeds in diffusion models. Distillation based acceleration method tunes the original diffusion weights with a refined architecture or scheduler to enhance the generation speed.
Moving along, the InstantID framework is built on top of the stable diffusion model that allows InstantID to apply relevant notions. The model treats the discrete forward diffusion process as continuous-time Variance Preserving SDE. Furthermore, the stable diffusion model is an extension of DDPM or Denoising Diffusion Probabilistic Model, in which the training data point is perturbed gradually by the discrete Markov chain with a perturbation kennel allowing the distribution of noisy data at different time step to follow the distribution.
To achieve high-fidelity video generation with a minimal number of steps, the AnimateLCM framework tames the stable diffusion-based video models to follow the self-consistency property. The overall training structure of the AnimateLCM framework consists of a decoupled consistency learning strategy for teacher free adaptation and effective consistency learning.
Transition from Diffusion Models to Consistency Models
The AnimateLCM framework introduces its own adaptation of the Stable Diffusion Model or DM to the Consistency Model or CM following the design of the Latent Consistency Model or LCM. It is worth noting that although the stable diffusion models typically predict the noise added to the samples, they are essential sigma-diffusion models. It is in contrast with consistency models that aim to predict the solution to the PF-ODE trajectory directly. Furthermore, in stable diffusion models with certain parameters, it is essential for the model to employ a classifier-free guidance strategy to generate high quality images. The AnimateLCM framework however, employs a classifier-free guidance augmented ODE solver to sample the adjacent pairs in the same trajectories, resulting in better efficiency and enhanced quality. Furthermore, existing models have indicated that the generation quality and training efficiency is influenced heavily by the number of discrete points in the trajectory. Smaller number of discrete points accelerates the training process whereas a higher number of discrete points results in less bias during training.
Decoupled Consistency Learning
For the process of consistency distillation, developers have observed that the data used for training heavily influences the quality of the final generation of the consistency models. However, the major issue with publicly available datasets currently is that often consist of watermark data, or its of low quality, and might contain overly brief or ambiguous captions. Furthermore, training the model directly on large-resolution videos is computationally expensive, and time consuming, making it a non-feasible option for a majority of researchers.
Given the availability of filtered high quality datasets, the AnimateLCM framework proposes to decouple the distillation of the motion priors and image generation priors. To be more specific, the AnimateLCM framework first distills the stable diffusion models into image consistency models with filtered high-quality image text datasets with better resolution. The framework then trains the light LoRA weights at the layers of the stable diffusion model, thus freezing the weights of the stable diffusion model. Once the model tunes the LoRA weights, it works as a versatile acceleration module, and it has demonstrated its compatibility with other personalized models in the stable diffusion communities. For inference, the AnimateLCM framework merges the weights of the LoRA with the original weights without corrupting the inference speed. After the AnimateLCM framework gains the consistency model at the level of image generation, it freezes the weights of the stable diffusion model and LoRA weights on it. Furthermore, the model inflates the 2D convolution kernels to the pseudo-3D kernels to train the consistency models for video generation. The model also adds temporal layers with zero initialization and a block level residual connection. The overall setup helps in assuring that the output of the model will not be influenced when it is trained for the first time. The AnimateLCM framework under the guidance of open sourced video diffusion models trains the temporal layers extended from the stable diffusion models.
It's important to recognize that while spatial LoRA weights are designed to expedite the sampling process without taking temporal modeling into account, and temporal modules are developed through standard diffusion techniques, their direct integration tends to corrupt the representation at the onset of training. This presents significant challenges in effectively and efficiently merging them with minimal conflict. Through empirical research, the AnimateLCM framework has identified a successful initialization approach that not only utilizes the consistency priors from spatial LoRA weights but also mitigates the adverse effects of their direct combination.
At the onset of consistency training, pre-trained spatial LoRA weights are integrated exclusively into the online consistency model, sparing the target consistency model from insertion. This strategy ensures that the target model, serving as the educational guide for the online model, does not generate faulty predictions that could detrimentally affect the online model's learning process. Throughout the training period, the LoRA weights are progressively incorporated into the target consistency model via an exponential moving average (EMA) process, achieving the optimal weight balance after several iterations.
Teacher Free Adaptation
Stable Diffusion models and plug and play adapters often go hand in hand. However, it has been observed that even though the plug and play adapters work to some extent, they tend to lose control in details even when a majority of these adapters are trained with image diffusion models. To counter this issue, the AnimateLCM framework opts for teacher free adaptation, a simple yet effective strategy that either accommodates the existing adapters for better compatibility or trains the adapters from the ground up or. The approach allows the AnimateLCM framework to achieve the controllable video generation and image-to-video generation with a minimal number of steps without requiring teacher models.
AnimateLCM: Experiments and Results
The AnimateLCM framework employs a Stable Diffusion v1-5 as the base model, and implements the DDIM ODE solver for training purposes. The framework also applies the Stable Diffusion v1-5 with open sourced motion weights as the teacher video diffusion model with the experiments being conducted on the WebVid2M dataset without any additional or augmented data. Furthermore, the framework employs the TikTok dataset with BLIP-captioned brief textual prompts for controllable video generation.
Qualitative Results
The following figure demonstrates results of the four-step generation method implemented by the AnimateLCM framework in text-to-video generation, image-to-video generation, and controllable video generation.
As it can be observed, the results delivered by each of them are satisfactory with the generated results demonstrating the ability of the AnimateLCM framework to follow the consistency property even with varying inference steps, maintaining similar motion and style.
Quantitative Results
The following figure illustrates the quantitative results and comparison of the AnimateLCM framework with state of the art DDIM and DPM++ methods.
As it can be observed, the AnimateLCM framework outperforms the existing methods by a significant margin especially in the low step regime ranging from 1 to 4 steps. Furthermore, the AnimateLCM metrics displayed in this comparison are evaluated without using the CFG or classifier free guidance that allows the framework to save nearly 50% of the inference time and inference peak memory cost. Furthermore, to further validate its performance, the spatial weights within the AnimateLCM framework are replaced with a publicly available personalized realistic model that strikes a good balance between fidelity and diversity, that helps in boosting the performance further.
Final Thoughts
In this article, we have talked about AnimateLCM, a personalized diffusion model with adapters that aims to generate high-fidelity videos with minimal steps and computational costs. The AnimateLCM framework is inspired by the Consistency Model that accelerates the sampling with minimal steps by distilling pre-trained image diffusion models, and the successful extension of the Consistency Model, the Latent Consistency Model or LCM that facilitates conditional image generation. Instead of conducting consistency learning on the raw video dataset directly, the AnimateLCM framework proposes to use a decoupled consistency learning strategy that decouples the distillation of motion generation priors and image generation priors, allowing the model to enhance the visual quality of the generated content, and improve the training efficiency simultaneously.
