At plenio, one pipeline has done its job reliably for over a year. Several stages, multiple steps, parts of it running on frontier models from OpenAI and Anthropic. It works. We're rebuilding it anyway.
That sounds like a contradiction. Why replace something that runs? The honest answer has little to do with an acute problem and a lot to do with the maths you have to do for the years ahead. If you run your core functions permanently through a frontier lab's API, you pay a price you don't control. And you hand over data whose path, in the worst case, you can no longer trace.
Why we're rebuilding at plenio
The trigger wasn't an exploding bill. It was a strategic call. We don't want to make ourselves dependent, and we don't want to pay a cloud tax.
Token prices at the big labs are cheap today, in some cases suspiciously cheap. Look at the current finances of the major frontier labs and it's obvious these prices aren't fixed. They can rise. If they do, you're at their mercy, because your own product logic hangs on an API you neither own nor can negotiate.
With a fine-tuned open-source model, the relationship flips. You host it yourself, run inference on your own server, and keep full control over your data. For Swiss companies that's often the deciding argument, ahead of price. An open-source model can run locally, with a Swiss host, with a European host. The data never has to leave the country. Go the cloud route and, if you're not careful, it ends up somewhere in Silicon Valley.
«If we make ourselves dependent outside of Silicon Valley, we're at their mercy when token prices go up. With an open-source model you become completely independent, you can host it yourself, and you have full control over data privacy.»
Jakob KayaCEO
The distillation principle: a year of data that now saves work
We're not starting from zero. At plenio we worked with frontier models for a year and captured their outputs the whole time. Those historical outputs are our training foundation.
This is called distillation. You take the results an expensive frontier model has already produced and use them to train a smaller open-source model. Your own model then takes over the task, and the frontier model becomes redundant for that step. We're not swapping out the pipeline all at once, we're doing it step by step. Every step a trained model can handle drops off the running token bill.
How much this saves in the end, we can't yet back with our own numbers at plenio, because we're not live in production. What we expect is up to 90 percent lower operating cost compared to token prices. That order of magnitude isn't wishful thinking. It's the range reported again and again across the industry for comparable cases, where a specialised, self-hosted model replaces a generic frontier call. I'm writing that as an expectation, not a measured result. The hard numbers come once we're live.
What a client needs for this, and what they don't
The concern I hear most often: do you need to bring huge, clean volumes of data for a project like this? Volume matters, but it's rarely the obstacle people fear.
As a rough guide, think thousands of examples, ideally toward a hundred thousand, and it usually starts to make sense from a few thousand. Quantity isn't the whole story. Quality counts too. Sensible data you can actually work with beats a large, noisy collection.
And if a client doesn't have that data foundation yet? No reason to walk away. You build it, the way we did at plenio: first generate the data with frontier models, then train your own model on it. That costs tokens during the build phase, but once you have it, you have it. For quality control we use human-in-the-loop with confidence scoring. Anything the model classifies with high confidence passes through. Anything flagged with low confidence gets checked by a person by hand. You end up with a training foundation you can trust, without signing off every single data point manually.
Which open-source model we use in the end, and what hardware it needs, we decide case by case, depending on the use case and the data that has to be processed. The model size determines the hardware, not the other way around.
At plenio we're finishing this right now, and Headlinq is next in line. For us this is no longer an experiment. It's the standard route for anyone who wants to run their AI functions for the long term instead of renting them indefinitely.
