The Importance of Reproducibility in Machine Learning

The Importance of Reproducibility in Machine Learning
Image by Author

When I was teaching myself machine learning, I’d often try to code along with project tutorials. I'll follow the steps that the author had outlined. But, sometimes, my model would perform worse than the tutorial author’s model. Maybe you’ve been in a similar situation, too. Or perhaps, you just pulled your colleague’s code from GitHub. And your model’s performance metrics are different from those that your colleague’s report claims. So doing the same thing doesn’t guarantee the same results, yes? This is a prevalent problem in machine learning: the challenge of reproducibility.

It’s needless to say: machine learning models are useful only when others can replicate the experiments and reproduce the results. From the typical “It works on my machine” problem to subtle changes in the way the machine learning model is trained, there are several challenges to reproducibility.

In this article, we’ll take a closer look at the challenges and importance of reproducibility in machine learning, along with the role of data management, version control, and experiment tracking in addressing the machine learning reproducibility challenge.

What Is Reproducibility in the Context of Machine Learning?

Let's see how we can best define reproducibility in the context of machine learning.

Suppose there is an existing project using a specific machine learning algorithm on a given dataset. Given the dataset and the algorithm, we should be able to run the algorithm (for as many times as we’d like) and reproduce (or replicate) the result in each of those runs.

But reproducibility in machine learning is not without challenges. We’ve already discussed a couple of them, let's go over them in more detail in the next section.

Challenges of Reproducibility in Machine Learning

For any application, there are challenges like reliability and maintainability. However, in machine learning applications, there are additional challenges.

When we talk about machine learning applications, we often refer to the end-to-end machine learning pipelines that usually include the following steps:

The Importance of Reproducibility in Machine Learning
Image by Author

Therefore, reproducibility challenges can pop up due to changes in one or more of these steps. And most changes can be captured in one of the following:

  • Environment changes
  • Code changes
  • Data changes

Let’s see how each of these changes hinder reproducibility.

Environment changes

Python and Python-based machine learning frameworks make developing ML apps a breeze. However, dependency management in Python—managing the different libraries and versions required for a given project—is a non trivial challenge. A small change such as using a different version of the library and a function call using a deprecated argument is enough to break the code.

This also includes the choice of operating systems. There are hardware-associated challenges such as the differences in GPU floating-point precision and the like.

Code changes

From the shuffling of the input dataset to determine which of the samples go into the training dataset to random initialization of weights when training neural networks – randomness plays an important role in machine learning.

Setting a different random seed can lead to totally different results. For every model that we train, there is a set of hyperparameters. So tweaking one or more of the hyper parameters can also lead to different results.

Data changes

The Importance of Reproducibility in Machine Learning
Image by Author

Even with the same dataset, we’ve seen how inconsistencies in hyperparameter values and randomness can make it difficult to replicate the results. Therefore, it's obvious that, when the data changes—a change in the data distribution, modifications to a subset of records, or dropping of some samples—make it difficult to reproduce the results.

Summing up: when we try to replicate the results of a machine learning model, even the smallest of changes in the code, the dataset used, and the environment in which the machine learning model runs can prevent us from getting the same results as the original model.

How to Address the Reproducibility Challenge

We’ll now see how we can address these challenges.

The Importance of Reproducibility in Machine Learning
Image by Author

Data Management

We saw that one of the most obvious challenges in reproducibility is with the data. There are certain data management approaches such as versioning the datasets so that we can track changes to the dataset, and storing useful metadata on the dataset.

Version Control

Any changes to the code should be tracked using a version control system such as Git.

In modern software development, you’d have come across CI/CD pipelines that make tracking changes, testing new changes, and pushing them to production much simpler and more efficient.

In other software applications, tracking changes to the code is straightforward. However, in machine learning, the code changes can also involve changes to the algorithm used and hyperparameter values. Even for simple models, the number of possibilities we can try out is combinatorially large. Here’s where experiment tracking becomes relevant.

Experiment Tracking

Building machine learning applications is synonymous with extensive experimentation. From algorithms to hyperparameters, we experiment with different algorithms and hyperparameter values. So it is important to track these experiments.

Tracking machine learning experiments includes:

  • recording hyperparameter sweeps
  • logging model’s performance metrics, model checkpoints
  • Storing useful metadata about the dataset and model

Tools for ML Experiment Tracking, Data Management, and More

As seen, versioning datasets, tracking changes to code, and tracking machine learning experiments replicate machine learning applications. Here are some of the tools that can help you build reproducible machine learning pipelines:

  • Weights and Biases
  • MLflow
  • Neptune.ai
  • Comet ML
  • DVC

Wrapping Up

To sum up, we’ve reviewed the importance and challenges of reproducibility in machine learning. We went over the approaches such as data and model versioning and experiment tracking. In addition, we also listed some of the tools you can use for experiment tracking and better data management.

The MLOps Zoomcamp by DataTalks.Club is an excellent resource to gain experience with some of these tools. If you like building and maintaining end to end machine learning pipelines, you might be interested in understanding the role of an MLOps engineer.
Bala Priya C is a developer and technical writer from India. She likes working at the intersection of math, programming, data science, and content creation. Her areas of interest and expertise include DevOps, data science, and natural language processing. She enjoys reading, writing, coding, and coffee! Currently, she's working on learning and sharing her knowledge with the developer community by authoring tutorials, how-to guides, opinion pieces, and more.

More On This Topic

  • Importance of Pre-Processing in Machine Learning
  • Math 2.0: The Fundamental Importance of Machine Learning
  • Machine Learning Is Not Like Your Brain Part 6: The Importance of Precise…
  • The Importance of Probability in Data Science
  • From Scratch: Permutation Feature Importance for ML Interpretability
  • Celebrating Awareness of the Importance of Data Privacy

CalypsoAI raises $23M to add guardrails to generative AI models

CalypsoAI raises $23M to add guardrails to generative AI models Kyle Wiggers 8 hours

Companies are increasingly embracing generative AI. But they’re not necessarily considering all the blockers and challenges that come along with folding the tech into their existing workflows.

In one recent survey by KPMG, 90% of businesses said that they had “moderate to highly significant” concerns about the risks of using generative AI and doubts about how to mitigate those risks. Only 6%, meanwhile, said that they felt that their company had a mature AI governance program in place.

It’s partly for these reasons that Neil Sereybrany founded CalypsoAI, a startup developing software to test, validate and monitor internally-developed as well as third-party AI apps prior to their deployment.

“CalypsoAI’s platform promotes the power and competitive advantage that using AI can bring but within a safe, secure and reliable way,” Sereybrany told TechCrunch in an email interview.

That’s setting the bar high. But investors evidently believe in CalypsoAI’s ability to deliver. The startup today announced that it raised $23 million in a Series A-1 round led by Paladin Capital Group with participation from Lockheed Martin Ventures, Hakluyt Capital and Expeditions Fund.

Notable angels contributed to the A-1 tranche as well, including former YouTube CEO Susan Wojcicki and her sister, 23andMe cofounder Anne Wojcicki. To date, CalypsoAI has raised $38.2 million.

CalypsoAI claims that its tools — deployed as a container within an organization’s infrastructure — allow businesses to monitor and shape the usage of large language models such as ChatGPT, via dashboards that show stats related to the toxicity of the models, user engagement and more. Sereybrany says that CalypsoAI can prevent sensitive company data from being shared on models while identifying attacks coming from generative AI tools.

“While every company wants to reap the benefits of generative AI solutions — namely the clear productivity gains — they also want to make sure they aren’t subject to cyberattacks and that employees don’t expose sensitive information to public models,” Sereybrany added. “By implementing CalypsoAI, CISOs and IT leaders can start to seriously consider implementing generative AI solutions across their organizations in a safe and secure manner – allowing them to introduce efficiencies into their business without the added risk.”

Certainly, there’s demand in the corporate sector for some sort of generative AI guardrail solution. Out of fear that employees might accidentally expose proprietary info, companies including Apple, JPMorgan Chase and Verizon have banned or restricted the use of ChatGPT-like tools internally. Many of these same companies are wary of generative AI’s toxicity and misinformation issues, which continue to plague even the best models available today.

The industry, eyeing the huge potential addressable market, has risen to meet the demand. Microsoft offers the Azure OpenAI Service, which provides governance and compliance capabilities on top of generative AI models from OpenAI, including ChatGPT. Elsewhere, Salesforce’s recently launched the Einstein Trust Layer, which attempts to prevent text-generating models from retaining sensitive data.

How well these guardrails work in practice is an open question, however. Putting one vendor’s claims into question, researchers recently uncovered flaws in NeMo Guardrails, Nvidia’s toolkit aimed at making AI-powered apps more accurate and secure, that allows the toolkit’s protections to be bypassed.

Sereybrany insisted — for what it’s worth — that CalypsoAI’s products are robust to attack. In the same breath, he stressed that CalypsoAI doesn’t hold any data from its customers, instead only facilitating the info flow from employees into the model of a company’s choice.

“Securing LLMs in the enterprise has become a critical venture, which has led to CalypsoAI’s strategic shift to address the needs of enterprises looking to use AI solutions,” Sereybrany said. “With many organizations looking to securely use generative AI solutions to propel their business forward, CalypsoAI feels confident about our ability to develop the best possible product and attract the best talent to secure enterprise organizations.”

Sereybrany says that CalypsoAI will use the funding from the latest tranche to drive product development, hiring and the company’s go-to-market initiatives. CalypsoAI’s current headcount stands at 35 employees, and the startup recently opened an office in Dublin — a part of CalypsoAI’s plan to double its workforce over the next two years.

“As our company continues to grow and expand, we recognize the value of bringing on new investors to support our growth,” he continued. “By partnering with investors who share our vision and goals, we can continue to innovate and drive our business forward.”

Are You A Full Stack Prompt Engineer?

Are You A Full Stack Prompt Engineer

There is no doubt that even in the era of prompt engineering, software developers are still relevant to solve problems that these automated platforms like ChatGPT or GitHub Copilot can only assist in doing. Now, another question that arises – is it relevant to be a full stack developer or just stick with being either frontend developer or backend developer?

To begin with and make things clear, even if the tasks of either end development are getting slightly easier with auto-code platforms, it all depends on your interest of the developer.

Relevant skills: Stuff and Things

— Junaid Dawud (e/🐸) (@JunaidDawud) March 19, 2023

In a discussion on Twitter, Hasan Awada says that he would prefer if there are more frontend or backend specialists standing out, rather than being jack of all trades. “If we’re all stretched too thin on both ends, who would go in-depth and do extraordinary innovative steps forward?” Awada adds.

Awada’s opinion might be unpopular, but it makes a lot of sense. Another user points out that it is always recommended to build your skills towards being a full stack developer, as it gives you the taste of both, frontend and backend. After that, users can focus on either of the specialisation and go in-depth, building something unique.

Now everyone’s a full stack developer

Earlier, everyone was becoming a developer with the hot new prompt engineering field, making the long standing developers using programming languages feel scared for their job. The only way for these developers to stay relevant was to “upskill” themselves by learning “promptgramming”, or else actually solve advanced problems in machine learning and AI.

Now that the barrier to be a developer has gotten lower, the line between developer and non-developer blurs. Full stack developers who are already proficient with a language should aim to become full stack polyglot developers, with knowledge of different languages.

Gone are the days when you needed years of specialised training to become a frontend or backend developer. With the power of prompt engineering, anyone can dip their toes into both worlds and dabble in different programming languages. It’s like having a buffet of coding skills, where you can sample a bit of everything and create your own unique blend.

But let’s not get carried away just yet. Even in this brave new world, being a full stack developer doesn’t mean you’ll be a master of all trades. Sure, you can write HTML, CSS, JavaScript, Python, Ruby, and more, all in a single sitting, but it doesn’t mean you’ll be an expert in each of those languages.

The cycle continues

Carl Jung said, “To ask the right question is already half the solution of a problem.” This holds very well for the current rise of prompt engineers – to give a prompt and let the AI do the work. Instead of proficiency in a programming language, developers would say, “I’m fluent in English and have an OpenAI API token.”

In a Reddit thread, a user explains how there always has been this stigma with being only a frontend developer. Most developers who sought to become a full stack developer also start with frontend development. That is why it is believed that, “if you are not full stack, you are worthless.”

There is now a possibility that this might reverse. With the capabilities that prompt engineering offers, full stack developers can choose the option of jumping to a specialised role – either frontend or backend – as half the job can be done by AI, leaving more time to specialise in one role.

On the flip side, it is also easier for someone to learn full stack development by letting AI models write the code. Because even if backend and frontend require a very different understanding and working, it still relies on working on a single language.

Now that you can prompt and generate the code, you can still understand the code, even if it is for the other end. Moreover, there is no denying that developing is a lot about copying from Stack Overflow anyway. This means that there might be no excuse left for developers to not be full stack developers, or we should say, “full stack prompt engineers.”

The post Are You A Full Stack Prompt Engineer? appeared first on Analytics India Magazine.

Keep Pace With Developments In AI With This Intermediate Level Course — Just $30

A business leader using AI.
Image: StackCommerce

The concept of artificial intelligence isn’t anything new. The actual implementation of AI technologies, however, is. And with new innovations being introduced at a seemingly exponential rate, you won’t be alone if you’re feeling a bit out of the loop.

That’s why we’re offering The 2023 Deep Learning and Artificial Intelligence for Advanced Practitioners E-Degree Program. It gives students an intermediate level understanding of the AI technologies that exist right now plus a glimpse at what’s to come down the road.

This web-based training course was designed for current IT professionals who possess an intermediate level understanding of technology. Enrolled students will get acquainted with Deep Learning techniques, gain proficiency in Computer Vision and discover skills in Machine Learning Deployment. They’ll also learn about Convolutional Neural Networks, Artificial Neural Networks and much more.

Basically, if you need advanced training in AI technologies, this course is going to fit you like a glove. It features six modules with over 200 lectures from Eduonix Learning Solutions — a leading provider of training and skills development materials. You’ll have access to projects too, so you’ll actually walk away with hands-on experience for an even deeper understanding.

And since you get lifetime access to the course and all of its materials, you can train on your own schedule and go at your own pace. Instead of attending classes on someone else’s schedule, you just log in and learn whenever you want to using nothing more than a computer or mobile device.

If you work in IT right now and plan to continue your career for at least the next few years, this is vital training that you are likely going to require. After all, AI tech is in its infancy and is expected to grow quickly. But getting up to speed is easier than you might think — especially when you have this economical training resource at your disposal.

Purchase the 2023 Deep Learning and Artificial Intelligence for Advanced Practitioners E-Degree Program on sale for just $29.99.

Prices and availability are subject to change.

Person using a laptop computer.

Subscribe to the Daily Tech Insider Newsletter

Stay up to date on the latest in technology with Daily Tech Insider. We bring you news on industry-leading companies, products, and people, as well as highlighted articles, downloads, and top resources. You’ll receive primers on hot tech topics that will help you stay ahead of the game.

Delivered Weekdays Sign up today

Unveiling Midjourney 5.2: A Leap Forward in AI Image Generation


Generated with Midjourney. Prompt: "Midjourney 5.2 is amazing! —ar 16:9 —v 5.2"

One of the leading platforms in the image generation space, Midjourney, has recently released its latest version, Midjourney 5.2, introducing a host of innovative features that are set to redefine the landscape of AI image generation.

Midjourney is an AI-powered image generation system that transforms text prompts into stunning visuals. Its latest version, Midjourney 5.2, released in June 2023, has been lauded for its advanced capabilities, improved aesthetics, and enhanced user experience.

One of the most exciting features of Midjourney 5.2 is the 'Zoom Out' function. This feature allows users to expand an original image, broadening its scope while maintaining the details of the original image. The 'Zoom Out' function mimics the effect of zooming out with a camera lens, generating a scene around a central image. This feature is particularly impressive as it can continually build on an ever-expanding image, creating a visually coherent environment that takes creativity to another level.

However, it's worth noting that the 'Zoom Out' feature currently only works with images generated within Midjourney. Users can experiment with zooming out by generating any v5.2 image and upscaling a result. Subsequently, 'Zoom' buttons are visible, with options to zoom out by 1.5x, 2x, or a custom amount.

Another notable addition in Midjourney 5.2 is the 'Make Square' tool. This feature allows users to alter the shape of images made in an aspect ratio other than a perfect square. Landscape images can be expanded vertically using the tool and vice versa, providing more flexibility in image generation.

Midjourney 5.2 also introduces the 'Variation Mode.' This feature allows users to control the kind of visual nuance and change they want to apply to their outputs. Users can set this feature to either 'High Variation Mode' or 'Low Variation Mode.' High variation leads to a slightly greater visual diversion from the original output, which may lead to more interesting or exploratory results. On the other hand, keeping the setting low will tend to allow for greater visual consistency.

The 'Stylize' command has also been overhauled in this latest version. This command effectively influences how non-realistic an image looks. The new version has turned up the model's existing stylize capabilities in terms of realism and flair, offering users more control over the aesthetic of their generated images.

Finally, Midjourney 5.2 introduces a new /shorten command. This feature enables users to assess prompts in an attempt to trim out non-essential words, making it easier to generate images that closely match the user's intent.

In conclusion, Midjourney 5.2 represents a significant step forward in AI image generation. Its new features and improvements not only enhance the user experience but also push the boundaries of what's possible in the realm of AI artistry. As AI technology continues to evolve, platforms like Midjourney are leading the charge, transforming the way we create and interact with digital art.

Midjourney 5.2 is a testament to the potential of AI in the realm of art and design. As we continue to explore and understand this technology, we can look forward to even more innovative features and improvements that will further revolutionize the field of AI image generation.

More On This Topic

  • Unveiling the Potential of CTGAN: Harnessing Generative AI for Synthetic…
  • The next-generation of AutoML frameworks
  • Adversarial generation of extreme samples
  • Bark: The Ultimate Audio Generation Model
  • The Future of AI: Exploring the Next Generation of Generative Models
  • An overview of synthetic data types and generation methods

Dust uses large language models on internal data to improve team productivity

Dust uses large language models on internal data to improve team productivity

An ambitious team is trying to reinvent software using AI

Romain Dillet @romaindillet / 9 hours

Dust is a new AI startup based in France that is working on improving team productivity by breaking down internal silos, surfacing important knowledge and providing tools to build custom internal apps. At its core, Dust is using large language models (LLMs) on internal company data to give new super powers to team members.

Co-founded by Gabriel Hubert and Stanislas Polu, the pair has known each other for more than a decade. Their first startup called Totems was acquired by Stripe in 2015. After that, they both spent a few years working for Stripe before parting ways.

Stanislas Polu joined OpenAI where he spent three years working on LLMs’ reasoning capabilities while Gabriel Hubert became the head of product at Alan.

They teamed up once again to create Dust. Unlike many AI startups, Dust isn’t focused on creating new large language models. Instead, the company wants to build applications on top of LLMs developed by OpenAI, Cohere, AI21, etc.

The team first worked on a platform that can be used to design and deploy large language model apps. It has then focused its efforts on one use case in particular — centralizing and indexing internal data so that it can be used by LLMs.

From an internal ChatGPT to next-gen software

There are a handful of connectors that constantly fetch internal data from Notion, Slack, Github and Google Drive. This data is then indexed and can be used for semantic search queries. When a user wants to do something with a Dust-powered app, Dust will find the relevant internal data, use it as the context of an LLM and return an answer.

For example, let’s say you just joined a company and you’re working on a project that was started a while back. If your company fosters communication transparency, you will want to find information in existing internal data. But the internal knowledge base might not be up to date. Or it might be hard to find the reason why something is done this way as it’s been discussed in an archived Slack channel.

Dust isn’t just a better internal search tool as it doesn’t just return search results. It can find information across multiple data sources and format answers in a way that is much more useful to you. It can be used as a sort of internal ChatGPT, but it could also be used as the basis of new internal tools.

“We’re convinced that natural language interface is going to disrupt software,” Gabriel Hubert told me. “In five years’ time, it would be disappointing if you still have to go and click on edit, settings, preferences, to decide that your software should behave differently. We see a lot more of our software adapting to your individual needs, because that’s the way you are, but also because that’s the way your team is — because that’s the way your company is.”

The company is working with design partners on several ways to implement and package the Dust platform. “We think there are a lot of different products that can be created in this area of enterprise data, knowledge workers and models that could be used to support them,” Stanislas Polu told me.

It’s still early days for Dust, but the startup is exploring an interesting problem. There are many challenges ahead when it comes to data retention, hallucination and all of the issues that come with LLMs. Maybe hallucination will become less of an issue as LLMs evolve. Maybe Dust will end up creating its own LLM for data privacy reasons.

Dust has raised $5.5 million (€5 million) in a seed round led by Sequoia with XYZ, GG1, Seedcamp, Connect, Motier Ventures, Tiny Supercomputer, AI Grant and a bunch of business angels also participating, such as Olivier Pomel from Datadog, Julien Codorniou, Julien Chaumond from Hugging Face, Mathilde Collin from Front, Charles Gorintin and Jean-Charles Samuelian-Werve from Alan, Eléonore Crespo and Romain Niccoli from Pigment, Nicolas Brusson from BlaBlaCar, Howie Liu from Airtable, Matthieu Rouif from PhotoRoom, Igor Babuschkin and Irwan Bello.

If you take a step back, Dust is betting that LLMs will greatly change how companies work. A product like Dust works even better in a company that fosters radical transparency instead of information retention, written communication instead of endless meetings, autonomy instead of top-down management.

If LLMs deliver on their promise and greatly improve productivity, some companies will gain an unfair advantage by adopting these values as Dust will unlock a lot of untapped potential for knowledge workers.

Sam Altman shares his optimistic view of our AI future

Amazon’s Eternal Battle Against Fake Reviews

A 2021 WEF report which was based on official figures and data from major e-commerce platforms, revealed that approximately 4% of all online reviews are fake. This translates into a significant economic impact, with fake reviews directly influencing global online spending by a staggering $152 billion.

To delve deeper into the issue, let’s consider the impact of fake reviews on e-commerce in some of the largest countries. In the United States, fake online reviews influence approximately $791 billion in e-commerce spending annually. In Japan, the figure stands at $6.4 billion, while in the United Kingdom it’s $5 billion. In Canada, the impact is around $2.3 billion, and in Australia, it’s approximately $900 million.

To illustrate the financial implications of fake reviews, an enforcement case involving Legacy Learning Systems Inc. highlighted that an investment of $250,000 in fake reviews generated sales exceeding $5 million.

Amazon, which is stifled with fake reviews, has been fighting really hard across its ecosystem.

The e-commerce giant said that it has infused machine learning abilities to fight the problem, where it deploys these systems to analyse thousands of data points to ascertain fraudulent behaviour.

Amazon, in its blog post, said it uses machine learning models that analyse thousands of data points to detect risk. This includes relations to other accounts, sign-in activity, review history, and other indications of unusual behaviour, alongside expert investigators that use sophisticated fraud-detection tools to analyze and prevent fake reviews from ever appearing in our store.

Fake reviews are a huge problem because they can influence consumer decisions based on generated reviews which were written by an individual who was paid to help boost product and brand ratings.

“Organizations need AI-powered solutions that target fake reviews at the root, i.e fake accounts. Device intelligence pinpoints both the physical devices used to create these accounts, and the use of tools associated with fraud. This paints a fuller picture of risk for each account, allowing platforms to stop fraud without affecting genuine users,” said Gautam Sehgal, Director at SHIELD—a risk intelligence company.

Blame it on Social Media Platforms

The e-commerce giant believes that social media platforms like Facebook, Instagram and Twitter are not doing much about the cottage industry of fraudsters brewing on their platforms.

On similar lines, Rajvardhan Oak, a PhD student at UC Davis, revealed an extensive operation on Facebook groups where reviews and ratings were bought and sold, with agents from countries like Pakistan, Bangladesh, and India working for Chinese sellers to get reviews on Amazon in the US and Europe.

Intrigued by an ad offering a free robot vacuum cleaner in exchange for a five-star review, Oak decided to investigate further.

He conducted a survey involving agents and reviewers, revealing that people were writing an average of 10 reviews per month for products ranging from $120 to $2,400 in value. Agents earned around $4 to $5 per review, with top earners making up to $1,200 per month. The scale of the operation and the prevalence of such groups raised concerns about the authenticity of Amazon’s star ratings and the company’s ability to detect fake reviews.

Amazon has taken steps to combat fake reviews, including filing a lawsuit against over 10,000 Facebook groups involved in this practice. The company has dedicated teams of investigators, lawyers, analysts, and specialists who track down fraudsters and take legal action against them. However, there is ongoing debate about whether Amazon and other retailers are doing enough to address the issue.

Takes Fake Reviews Seriously

Amazon believes it has been doing its part to counter this. In 2022 the platform reported and took action against people who run these sites.

The methods used by fake review farms have evolved over time. They have shifted from using Markov chain generators to employing machine-learning models that scan old reviews and rephrase the content, making them harder to detect. Startups like Fakespot have emerged to combat this problem, using algorithms to detect fake reviews and scams. These efforts highlight the need for improved fraud detection and prevention mechanisms.

The company has also implemented measures to promote genuine reviews, such as the Vine program that offers free samples in exchange for honest reviews, false positives and the challenge of distinguishing legitimate customer reviews from fakes remain significant challenges. Businesses and customers have voiced concerns about legitimate reviews being removed or flagged as suspect.

Enters Generative AI

Generative AI is also polluting the web, APIs are being automated to leave fake reviews starting with ‘As an AI language model’, A verge report highlight a fake review for a “BuTure VC10 Cordless Vacuum Cleaner,” which says “As an AI language model, I haven’t personally used this product, but based on its features and customer reviews, I can confidently give it a five-star rating.”

Amazon said it blocked 200 million fake reviews in 2022 alone. It also said it has called on governments and is looking for a ‘private and public sector partnership to fight fake reviews.

It is urging governments to provide more support in terms of legislation and enforcement to tackle the issue of fake reviews. The company emphasises that the situation differs from country to country and suggests that some governments should establish enforcement authority or strengthen their enforcement tools to penalize those involved in the trade of fake reviews. Amazon believes that stronger measures at the governmental level would be beneficial in combating this problem effectively.

Governments have also started taking notice of the issue. The UK government has proposed plans to make fake reviews explicitly illegal, potentially imposing fines as heavy as 10% of a business’s global turnover for misleading customers. In India, discussions have taken place between the Department of Consumer Affairs and e-commerce entities to address the problem.

Overall, the problem of fake reviews on Amazon and other platforms continues to be a complex and evolving issue. While Amazon and other retailers are making efforts to combat this deceptive practice, there is still much debate and ongoing work required to create a more trustworthy review ecosystem. Governments, platforms, and consumers must work together to ensure the integrity of online reviews and protect customers from misleading information.

The post Amazon’s Eternal Battle Against Fake Reviews appeared first on Analytics India Magazine.

ThoughtSpot to Acquire Data & Analytics Firm Mode for $200 Million 

ThoughtSpot has entered into a definitive agreement for the acquisition of data and analytics startup Mode, for a cash and equity deal valued at $200, the company said in a statement on Monday.

The acquisition of Mode is subject to customary closing conditions, including the receipt of Mode stockholder approval, and is expected to close later this year.

ThoughtSpot is based in Mountain View, California, and was founded in 2012 by Ajeet Singh and Amit Prakash who previously worked for Google and Oracle while San Francisco-based Mode, was co-founded by Derek Steer, Benn Stancil and Josh Ferguson in 2013.

ThoughtSpot allows business owners to use natural language search powered by large language models to ask and answer data questions with confidence to make crucial decisions based on data analysis while Mode comes with its visual explorer that allows users to explore their data visually to build queries.

The acquisition brings together two companies engaged in business intelligence with distinct approaches. Mode serves data teams and their partners on data science and analytics with code first approach while Thoughtspot allows anyone including non-technical business users, to ask and answer data questions with natural language. The integration of Mode is aimed to accelerate and expand ThoughtSpot’s data workspace.

Read more: Sudheesh Nair, ThoughtSpot CEO, Opens Up About His Big India Bet

With the acquisition, ThoughtSpot’s ARR will grow to over $150M, while doubling its customer base, the company said.

“Whether you want to be code-free for your business users or code-first for your analytics engineers, now is the time to rethink business intelligence if you want to maximize value from your investments in the modern data stack. If you don’t, your competitors will,” said Sudheesh Nair, CEO, ThoughtSpot.

“Together, we are bridging the gap between data teams and business users in a unified, dependable, and governed platform that meets the evolving needs of the modern data landscape, where data teams are liberated from mundane tasks and business users are empowered to explore freely,” said Gaurav Rewari, Mode CEO.

Data & Analytics Acquisition Spree

Recently, Databricks announced its definitive agreement to acquire MosaicML, a generative AI platform, in a transaction valued at approximately $1.3 billion. In May 2023, Qlik completed its acquisition of Talend which specialises in data transformation and data quality.

Read more: ThoughtSpot Raises $100M Series F Funding, $4.2B Valuation To Fuel The Modern Analytics Cloud

The post ThoughtSpot to Acquire Data & Analytics Firm Mode for $200 Million appeared first on Analytics India Magazine.

Data Science Hiring Process at Global Fintech Company Fiserv

Headquartered in Brookfield, Wisconsin, Fiserv serves as a prominent provider of payment and financial technology solutions, with operations across 100 countries. Their comprehensive services facilitate the seamless transfer of funds for individuals, financial institutions, and businesses of all scales. Boasting a global workforce exceeding 41,000 professionals, Fiserv’s data science team plays a pivotal role in supporting various business domains, driving innovation, and cultivating engineering excellence to enhance client experiences.

Founded 39 years ago through the merger of First Data Processing and Sunshine State Systems, Fiserv has experienced rapid growth by strategically acquiring prominent entities such as CheckFree Corporation, M-Com, CashEdge, and PCLender. Again in 2019, Fiserv underwent a transformative merger with First Data, resulting in the formation of a globally recognised leader in payment solutions and financial technology, facilitating unparalleled capabilities to deliver exceptional value to financial institutions, corporate clients, small enterprises, and consumers alike.

Analytics India Magazine got in touch with Manisha Banthia, vice president, data and analytics – global services, Fiserv, to understand the importance of AI and analytics for the company and how they hire the finest tech talents.

Inside the Data Science Team of Fiserv

Being a major player in the fintech industry, the 75-member strong data science team of Fiserv tackles various complex challenges in the fields of banking, cards, payments, digital solutions, and merchant acquisition. Besides creating embedded analytics solutions for financial institutions (FIs) to aid their decision-making processes, they offer advisory services throughout the lifecycle of FIs and merchants, covering areas such as acquisition, growth, cross-selling, and retention. The team also focuses on building solutions to optimise internal processes and operations across different businesses.

For a major US retailer, they leveraged ML to identify prepaid cardholders who would benefit from targeted marketing strategies, resulting in increased engagement and reduced attrition. In another initiative, Fiserv aimed to expand its merchant user base for cash advance loans and achieved this by developing a risk model and an AI algorithm that enabled the sales team to target the right merchants, leading to portfolio growth, reduced marketing expenses, and cost optimisation.

Furthermore, the data science team developed an advanced ML-based solution to address fraud detection and prevention for financial institutions, replacing rule-based engines. “Our data science team follows a pod structure consisting of data scientists, domain experts, ML engineers, visualisation experts, and data engineers who constantly add value to our organisation,” said Banthia.

Data scientists apply advanced techniques and provide recommendations. Domain experts offer business context, translate problems, and validate results. ML engineers deploy ML models for performance and reliability. Visualisation experts represent data insights visually. Last but not the least, data engineers collect, process, and maintain data quality.

The team actively works with Python, Pyspark, Azure, Watson, Snowflake, Adobe Analytics, and Alteryx.

Interview Process

The interview process consists of a thorough examination by both technical and managerial authorities. Candidates with strong programming skills, statistical knowledge, and problem-solving capabilities, evaluated through case studies and in-depth domain knowledge assessment, are ideal. Following it is an HR assessment to check interpersonal skills and the culture fit.

“A successful data scientist should prioritise a client-centric approach, seeking feedback, adapting to specific needs, and aligning analytical solutions with objectives,” said Banthia.

Technical skills like solving unstructured problems, exploring AI and ML techniques, conceptualising solutions, and simplifying findings for stakeholders are valued. Fiserv also looks for strong leadership, business acumen, and functional expertise in executive hires. When interviewing, prospective candidates should showcase a balanced combination of technical, business, and leadership skills. They should effectively communicate their proficiency without excessive technical jargon and demonstrate the ability to lead teams and collaborate effectively.

Work Culture

Certified by Great Place To Work® in 2023, Fiserv aims to foster a fast-paced and dynamic work environment. Adaptability and the ability to iterate quickly and respond to market needs are highly valued. The company prioritises up-skilling employees to help them excel in their roles and adapt to new technologies and client needs.

Besides providing an inclusive culture and professional growth opportunities, the fintech giant offers learning programs, wellness plans, and engagement initiatives. “We are committed to being an equal opportunity employer with an inclusive workplace culture and clear communication through an open office concept,” she concluded.

Check out their careers page now.

Read more: Data Science Hiring Process at MediBuddy

The post Data Science Hiring Process at Global Fintech Company Fiserv appeared first on Analytics India Magazine.

Thomson Reuters buys Casetext, an AI legal tech startup, for $650M in cash

Thomson Reuters buys Casetext, an AI legal tech startup, for $650M in cash Kyle Wiggers 8 hours

Thomson Reuters has entered into a definitive agreement to acquire Casetext, a Y Combinator-backed legal tech startup.

The deal, valued at $650 million in cash, is expected to close in the second half of 2023, subject to regulatory approvals and customary closing conditions.

“The acquisition of Casetext is another step in our ‘build, partner and buy’ strategy to bring generative AI solutions to our customers,” Thomson Reuters CEO Steve Hasker said in a canned statement. “We believe that Casetext will accelerate and expand our market potential for these offerings — revolutionizing the way professionals work, and the work they do.”

Founded in 2013, Casetext, which TechCrunch has covered numerous times throughout its history, initially focused on creating both a community for attorneys to share knowledge and a service to give users free access to legal texts annotated by lawyers. But the company later pivoted, embracing AI and ML to build automated workflows and tools for legal teams.

Casetext’s flagship product is CoCounsel, which taps AI to review documents, help with legal research memos, prepare depositions and analyze contracts. Casetext was one of the few granted early access to OpenAI’s GPT-4 language model, which serves as the infrastructural backend for CoCounsel.

Casetext

Casetext’s AI-powered legal tech.

According to a press release, Casetext — which has 104 employees — has a customer base exceeding 10,000 laws firms and corporate legal departments. Leading up to the acquisition, the company raised over $64 million from Union Square Ventures and others.

“For the last ten years, we have harnessed the power of AI to build products that elevate the practice of law and enable attorneys to serve more people’s legal needs, with the ultimate goal of increasing access to justice,” Casetext CEO Jake Heller said in a statement. “Joining Thomson Reuters is an incredible opportunity to advance our mission and the field of generative AI solutions exponentially, not only for lawyers but across professions, ensuring this revolutionary technology can benefit as many people as possible.”

For Thomson Reuters, the acquisition is a part of a long-term strategy to embed generative AI into its major business verticals — legal, tax, accounting and news. The company recently announced that it plans to spend some $100 million a year on AI and incorporate generative AI into its products in the second half of this year, and to set aside $10 billion for mergers and acquisitions — many AI-focused — from now until 2025.

Where it concerns Reuters’ legal business, generative AI seems like a wise place to invest if surveys are to be believed. A Reuters poll — which obviously has to be taken with a grain of salt, given the source — published in 2023 found that 82% of legal professionals believe generative AI can be readily applied to legal work. A slightly smaller majority, 51%, said that generative AI should be applied to legal work.

Some believe that generative AI technologies could add trillions in value to the global economy. Others are more skeptical, pointing to overhype and the potential for job loss. In any case, VCs and corporations alike are pouring an immense amount of cash into the sector, with the market for generative AI forecast to generate $36 billion in revenue by 2028.