Looks like OpenAI has had enough. And now, it wants to give it back to Meta’s Llama 2 and its very own partner Microsoft who thought it could play OpenAI by forging multiple partnerships with Meta and Databricks.
OpenAI recently announced that fine-tuning for GPT-3.5 Turbo is now available and fine-tuning for GPT-4 is coming this fall. In their blog post, OpenAI stated that a finely-tuned variant of GPT-3.4 Turbo has the potential to achieve, and in some cases even surpass, the capabilities of the base GPT-4 model on certain narrow tasks.
The company has stated that the fine-tuning of GPT 3.5 Turbo is suitable specifically for businesses and developers to customize the model depending upon their use case as it lets them train the model on company’s data and run it at scale.
With this development, OpenAI has shown that it does care about enterprises. The creator of ChatGPT came with distinct reasons why businesses should adopt GPT-3.5 Turbo API. OpenAI elaborated how fine-tuning is a great way to hone the qualitative feel of the model output such as its tone, so it better fits the voice of businesses’ brands.
All of OpenAI’s efforts sound great, but the question is, will fine-tuning for GPT-3.5 Turbo effectively addresses the issues of cost and security for enterprises.
Fine-tuning is essential for broader AI application adoption but I think @OpenAI got this wrong.
1. Data privacy: No company will feel comfortable uploading their data to fine-tune the model.
2. ROI vs Cost: Companies/individuals are paying for training and inference while…— Dr Ahmed Zaidi (@_ahmedzaidi) August 23, 2023
Breaking Down the Cost
With GPT-3.5 Turbo API, OpenAI has made genuine efforts to cut down on the prices as compared to GPT-4 API. Moreover, the upcoming integration of fine-tuning is expected to lead to even lower costs. Fine-tuning on GPT-3.5 Turbo helps users make prompts shorter, which means using fewer words to get the same results. This can lower the overall cost of using the API. OpenAI found that early testers were able to make prompts much shorter, up to 90% less, by tweaking how they instructed the model. This not only made the API calls faster but also saved money.
According to OpenAI’s blog, fine-tuning expenses are divided into two categories: the initial training cost and the usage cost. The training cost is $0.008 for every 1,000 tokens, while the input usage cost is $0.012 per 1,000 tokens, and the output usage cost is $0.016 per 1,000 tokens. For example, if a fine-tuning task involves the GPT-3.5-Turbo model, utilizing a training file of 100,000 tokens, and undergoes training for 3 epochs, the anticipated cost would amount to $2.40.
What’s important to remember is not to mix up fine-tuning costs with general API expenses. Specifically for the GPT-3.5 Turbo API, the charges are $0.003 per 1,000 tokens for input and $0.004 per 1,000 tokens for output with a 16K context length.
However, it’s still uncertain if it can compete with Llama 2 in terms of pricing. Several users on Hacker News expressed that GPT-3.5 Turbo is better than Llama 2. “The Llama 2 70B performance is probably between GPT-3.5 and GPT-4. But running it personally isn’t cheap. The cheapest I found is about $4/hr to run the whole thing. I only spend around $3 on average a month on GPT-3.5 API for my personal stuff,” a Hacker News user said.
The new update can handle 4k tokens, double of the previous fine-tuned models. OpenAI said that fine-tuning is the most powerful when combined with other techniques like prompt engineering, information retrieval, and function calling.
One of the users of X praising availability of GPT-3.5 fine-tuning said that a fine-tuned GPT-3.5-Turbo has trivial training cost and inference is about 1/2 of GPT-4. “I can see this being the best play in a wide variety of scenarios, especially low-intelligence agentic flows w/ idiosyncratic tool usage and low latency requirements.”
OpenAI Cares About Data Privacy
Enterprises possess valuable data that they handle with great care and are cautious about sharing with external entities. After the fine-tuning announcement was made for GPT 3.5 Turbo, some users on social media platforms questioned whether OpenAI would use the data that the enterprises provide for training. To this, OpenAI’s Logan Kilpatrick replied, “No data is used for training, it’s the same across all our endpoints.” He corroborated the statement with an updated blog on OpenAI’s API data privacy measures.
Glad to hear it then
. Will OpenAI you use the training set people provide? Is it same ToC as using the chat completion API that no data will be used for training?
—
Lihu (@Ntrovis) August 23, 2023
A few days back, Sam Altman had also clarified on X saying that OpenAI does not use any API data to train its models putting all speculations that arose regarding security issues to rest.
Despite all the claims by OpenAI, users are still sceptical about the security of data on the platform. They would rather trust well-established players such as IBM, Azure, AWS, and Databricks as these platforms offer a range of LLMs and provide customers with the capability to train and personalise these models on their respective platforms. It’s clear that if OpenAI plans to establish its position in the enterprise segment, it must establish itself against these companies.
Interestingly, Microsoft’s Azure OpenAI service also permits customers to customise their models using fine-tune on their own datasets. It remains to be seen whether enterprises will opt to approach OpenAI directly or choose the route through Azure.
The post OpenAI Trying Hard to Woo Enterprises appeared first on Analytics India Magazine.
