Anyone building a competitive edge using state-of-the-art (SOTA) models is destined to fail.
Despite the buzz and news releases, you shouldn’t run to build with the latest models.
Yes, they are fun to play with. No, you should not use them for serious work.
Here are three reasons why (how so very LLM-esque of me!):
First, SOTA models are the most expensive and the slowest. This means the moment someone figures out how to do the same thing with a cheaper model, or a faster one, you’ve lost your advantage. Jeff Bezos was famous for saying that “Your margin is my opportunity”. Margins exists both in terms of token costs and UX (in this case latency).
The best operators know that the best inference is the one that happens for free on the user’s device with near-zero latency.
Second, the areas where SOTA models offer improved functionality are, by definition, the bleeding edge. But this edge is brittle. This is where things break. You are now including failures into your operational domain. You might be able to make a case this is manageable. But whatever you do to fix it, it will quickly be undone by the next model update.
The best operators know that reliability is not found at the bleeding edge.
Third, organizations reach for SOTA models for the most complex workflows - presumably to augment or replace humans. LLMs in enterprise may be new, but history keeps teaching the same lesson about automation. It may at first seem straightforward to automate the visible but then soon discover that the real challenges lie in the context around it. But LLMs can be every more damaging to an organization because they can proliferate quickly and are non-deterministic.
The best operators know that an organization’s best human judgment becomes inaccessible once you automate away the context it relied on.
SOTA models won’t help you create a lasting advantage. They cost more, are slower and invite use cases that are detrimental to an organization’s long term health.