3 big challenges of commercial LLMs

Large language models (LLMs) can understand and generate human language. These generative AI tools are powerful and popular, with over 90% of retail and ecommerce leaders reporting using them to assist with work tasks, according to a recent Future Commerce report.

For example, LLMs can generate different versions of a product description for different types of customers—such as those interested in sustainability, price or style—helping ecommerce businesses personalize their engagement and therefore drive more revenue. LLMs received a $10.5 billion valuation in 2022, and experts predict their valuation will reach $40.8 billion by 2029.

However, stakeholders must address three crucial barriers before businesses can widely adopt LLMs: the high cost of LLM development and training, the lack of pricing transparency, and the impact of open-source LLMs on commercial LLMs. The hype around LLMs is real, but only companies with large cash balances can afford to run them—even at a loss—in the early stages.

Challenge 1: The high cost of development and training

The financial frontier of LLM development and training has made it difficult for businesses to justify the investment. LLMs require massive amounts of data and computing power to train—an incredibly expensive line item on an operations budget. For example, GPT-3, a popular LLM, cost OpenAI over $4.6 million to train.

Historically, deploying and training LLMs have been costly, requiring specialized hardware and software. A basic in-house deployment could cost around $60,000 over five years, but this may not be scalable or performant enough for some applications. A more scalable deployment could cost closer to $95,000. Additionally, there are expenses associated with hiring data scientists and support staff, building a suitable execution environment, and maintaining the LLM over time. All of this requires stakeholder approval for the project’s full scope to avoid unexpected long-term costs.

Today, the cost of deploying and training LLMs is still high but is becoming more affordable and accessible. Smaller companies running on streamlined budgets have struggled to access LLMs, but companies like OpenAI have made it more affordable and accessible by providing a software-as-a-service (SaaS) version of their APIs. This means companies don’t need to buy and maintain their own hardware and software to use these powerful language models. They can simply subscribe to a service and access the APIs online.

Training is also less expensive now because most companies use “fine-tuning” instead of training from the ground up. Fine-tuning is a technique that allows companies to train large language foundation models on their own data, which is much cheaper than training from scratch. Fine-tuning only requires training the LLM on new data rather than training the entire model from the beginning. In this scenario, the LLM already understands basic language patterns, saving companies time and money.

The high cost of LLM development and training has presented a barrier to entry for many small and medium-sized businesses. As a result, LLM adoption slowed for all but large companies with the capital to invest heavily in this technology. However, deploying and training LLMs is becoming more affordable and accessible thanks to companies like OpenAI and strategies like fine-tuning.

Challenge 2: The lack of pricing transparency

The lack of pricing transparency can still be a challenge for small and medium-sized businesses (SMBs) to acquire LLMs, even with the current pricing models of pay-per-query for independent software vendors (ISVs) and subscription systems for end users. LLM pricing can vary depending on several factors, such as the size and complexity of the model, the amount of data it was trained on, and the specific features it offers. These aspects can make it difficult for SMBs to compare the pricing of different LLM providers and choose the best option for their needs.

Some LLM providers may not even disclose their pricing up front, making it challenging for SMBs to accurately budget for an LLM before signing a contract. And even with pay-per-query and subscription pricing models, business owners may still find LLMs prohibitive, especially small businesses with limited budgets.

Other hurdles SMBs may face when acquiring LLMs include:

• Difficulty understanding the pricing of LLMs: LLMs can be complex and opaque for SMBs without AI and machine learning expertise, making it challenging to shop for the best fit.
• Hidden costs: Some LLM providers may charge hidden fees, such as setup, maintenance, and overage charges.
• Long-term contracts: Some LLM providers force SMBs into long-term contracts, which can be financially risky for businesses that can’t afford sustained LLM usage.

LLM pricing is unstandardized and unpredictable, but one thing is certain: its high cost. The current circumstances create a barrier to entry for small and medium-sized ecommerce businesses, ultimately holding back industry innovation.

Challenge 3: The impact of open-source LLMs

Open-source LLMs, such as Llama 2 and Megatron-Turing NLG, can democratize access to this powerful technology and make it more accessible. However, if open-source LLMs become successful, they could present substantial obstacles for companies seeking to commercialize them.

Open-source LLMs present a dual challenge to commercialization of LLMs. First, they offer a cost-free alternative, enabling businesses to opt for open-source solutions rather than paying for commercial models. Second, open-source LLMs serve as a breeding ground for developing new applications and services that directly compete with commercial LLM offerings. Think AI-driven chatbots, translation tools, and code-generation tools.

Open-source software has a proven track record of success in other industries. For example, the open-source Linux operating system and the Apache web server have become dominant players in their respective markets. A growing community of developers and researchers is generating new ideas and innovations at breakneck speed. The cost of computing power is also steadily decreasing, making LLM business usage more affordable.

One caveat: Open-source LLMs lack standardization, making it difficult for businesses to choose the right LLM for their needs and integrate it into their existing systems. Because open-source LLMs are typically not supported by commercial vendors, businesses need to have in-house expertise or to partner with a third-party provider for upkeep and ongoing support.

The promise of open-source LLMs

Despite these issues, open-source LLMs could help fuel innovation and economic growth—but it will take time to get there because fostering community-driven development and addressing critical ethical and privacy concerns require careful planning, collaboration, and iterative refinements. Businesses already are using open-source LLMs for a number of applications:

• Customer service chatbots that provide 24/7 support and answer customer questions quickly and accurately.
• Marketing campaigns that generate personalized marketing copy and target ads to specific audiences more effectively.
• Product design and development systems that generate new product ideas and improvements.
• Code generation that saves developers time while improving code quality.
• Product recommendation engines that enhance shopping experiences and increase sales.

Commercialized LLMs hold immense promise, especially as companies help combat traditionally high costs with SaaS API solutions. However, the challenges of cost, pricing transparency, and the rise of open-source alternatives continue to underscore the need for concerted efforts to drive innovation and accessibility in the LLM landscape.

Xun Wang is CTO at Bloomreach.

—

Generative AI Insights provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss the challenges and opportunities of generative artificial intelligence. The selection is wide-ranging, from technology deep dives to case studies to expert opinion, but also subjective, based on our judgment of which topics and treatments will best serve InfoWorld’s technically sophisticated audience. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Contact doug_dineley@foundryco.com.