Cutting Costs by Over 90% by Replacing Google APIs With an LLM-Powered Microservice

Thu, 28 Aug 2025 10:14:00 +0000
Scala, Microservices, LLMs, Google Apps

We replaced interactions for two separate Google services with a single, small LLM-powered microservice using Gemini AI (also from Google). The result? Costs were cut by over 90% and improved flexibility with just a couple of days work.

Introduction

During a routine GCP billing account check for our client, I paid special attention to some out-of-the-box APIs provided by Google, which we use for language translations (Cloud Translation API) and sentiment analysis (Cloud Natural Language API). Because of our latest changes in some of the business logic, as well as some upcoming features we planned to implement over the summer, it was expected that the cost of those services would increase heavily, and I needed to estimate by how much to better prepare for the product discussions with the so-called ‘business’.

We originally used two Google APIs in a simple workflow: translate user text to English and then run sentiment checks. The translation step existed only because the sentiment API didn’t support the text’s original language. If the text looked suspicious (offensive, profane, or unnaturally polished), we flagged it for a human review team.

Looking ahead, we planned to use the Translation API in many more ways, and the number of interactions with those services would grow exponentially. We wanted to enhance the UX with translations into 12 languages and keep sentiment analysis. That meant calling the API thousands of times a day just to get the translations (we cache results and only refresh when the source text changes).

While analysing current and future call volumes, we realized a single LLM-based service could replace both APIs. I needed to do a quick Proof-of-Concept (PoC) and check how much that would be if we tried to get the same functionality from simple LLM calls through the API.

Business requirements for our new PoC microservice were simple:

support 12 languages,

support sentiment analysis (ideally as a single call instead of two),

compare costs versus the existing setup.

Background & Previous Setup

The architecture before migration was trivial. In one of our microservices, we received a message containing the text to be checked, executed the call to the Translation Google API to get the text in English, and then called the Cloud Natural Language API twice to get different results for different sentiment analyses we were interested in.

With the new features, we needed to translate text into 11 languages, detect the source language when unknown, and continue running sentiment analysis.

That means we needed to multiply the number of requests to the API we already had, multiple times, but that was not all. The new features we were introducing over the summer incorporated changes which were causing many more (hundreds of times more) pieces of text to be pre-translated. Overall, API usage was set to grow by hundreds-fold in the near future.

Before estimating how much we will pay for the services with the current setup or how much that will be if we use LLM, we could do a simple test:...