Personal project, 2026
CallScope
An event-driven backend that scores contact-center conversations after they end: how the customer felt, whether that improved during the call, and how empathetic the agent was. A Spring Boot service runs the workflow off an SQS queue, and a FastAPI service runs Hugging Face models on the CPU.

- Spring Boot runs the workflow and FastAPI runs the models, with an SQS queue between them that absorbs bursts and retries failures.
- Real Hugging Face models on the CPU: RoBERTa for customer sentiment and zero-shot BART for agent empathy, rolled up into three scores from -100 to +100.
- Duplicate deliveries are skipped, broken events end up in a dead-letter queue, and a live dashboard shows every message on its way through.
What it is
When a contact-center conversation ends, CallScope scores it three ways: how the customer felt, whether that got better or worse during the call, and how empathetic the agent was. Each finished conversation arrives as an event on a queue. A Spring Boot service picks it up and owns the workflow, and a FastAPI service runs two Hugging Face models over the turns. The results land in a store and on a live dashboard, which can also send a batch with a duplicate and a broken event mixed in, to watch the failure paths as they happen.
I built it in 2026 as a personal project, to see what it takes to run ML models inside an event-driven backend rather than in a notebook. Everything runs locally and for free with Docker Compose: real models on the CPU, and ElasticMQ, which speaks the SQS API, instead of an AWS account.
How a conversation flows
- Queue. The contact-center platform, here a simulator or the dashboard’s button, sends a
conversation.completedevent with every turn of the call. - Check. conversation-service receives it and validates it against the contract. A conversation that is already DONE is skipped, and any other is marked PENDING.
- Analyze. It posts the turns to analysis-service, which makes one batched model call per speaker: sentiment for the customer’s turns, empathy for the agent’s.
- Save, then confirm. The result is saved as DONE, and only then is the message deleted from the queue. If anything fails, the record is saved as FAILED with the reason, and the message stays in the queue.
The queue owns the retries. A received message stays hidden for 20 seconds. If it isn’t deleted by then, SQS hands it out again, and after three failed tries it moves to a dead-letter queue, where a person can inspect it. Nothing is lost, and the service needs no retry loop of its own.
Why two services. The ML ecosystem lives in Python, while a long-running, stateful workflow fits a JVM service. With a queue between them, conversations can arrive in bursts, and each side can scale on its own.
One contract on both sides. The Pydantic models in analysis-service define the API, and both services test themselves against the same example messages, so neither side can drift from them without failing CI.
From labels to scores
Each turn’s label counts as +1, 0 or -1, times the model’s confidence. The turns then roll up into three scores from -100 to +100:
- Customer sentiment: the average over the customer’s turns. The last 15% count double, because how a call ends matters most.
- Sentiment trend: the second half of the customer’s turns minus the first half. Positive means the customer ended happier than they started.
- Agent empathy: the average over the agent’s turns.
A score is left empty when there’s nothing to measure, such as a call with no agent turns.
A model for each question. Customer turns go to twitter-roberta-base-sentiment-latest, which was trained on short, informal text close to chat and speech-to-text. Empathy has no model off the shelf, and the sentiment model calls every apology negative: “I’m so sorry about that, let me fix it right away” came out NEGATIVE 0.80, which would score the most caring agents worst. So agent turns go to bart-large-mnli as zero-shot classification, which asks whether each reply is “empathetic and caring”, “neutral” or “dismissive or unhelpful”.
Tested on hard sentences. Before trusting the models, I ran them on turns picked to be hard: negation, sarcasm, one-word answers and scripted apologies. Sentiment got 9 of 11 right and empathy 10 of 10. The misses showed where the limits are: low-confidence answers are unstable, since a single full stop flipped one label, and frustration without negative words (“It’s been two weeks!”) reads as neutral to a model that sees one turn at a time.

Details that matter
Timeouts that fit together. The call to analysis-service gives up after 10 seconds, well inside the queue’s 20-second visibility timeout. If it waited longer, SQS would hand the same message to a second worker while the first was still waiting for an answer.
Four at a time, on purpose. The models share the CPU, so taking more messages at once makes each one slower. In a burst of 20 on a laptop, 10 at a time took about 4.4 seconds per conversation, and after a night idle pushed one call past the timeout. 4 at a time took 2.1 seconds each and finished the burst only about 25% later. The rest wait in the queue, which is what the queue is for.
Duplicates are skipped. SQS may deliver a message more than once, so a conversation that is already DONE is not analyzed again.
Broken events don’t block the line. An event that breaks the contract, such as one with no turns, can never succeed. Validation rejects it before any work is done, and after its three tries it rests in the dead-letter queue, about a minute later, while healthy messages keep flowing.
Failures say why. A read timeout arrives wrapped in an exception about content types. The FAILED record stores the innermost cause instead, SocketTimeoutException: Read timed out, so the dashboard shows the real reason.
Long turns are cut, not fatal. The sentiment model reads at most 512 tokens, and its tokenizer doesn’t declare that limit, so the classifier passes it explicitly and a long turn is truncated instead of failing the whole conversation.
It recovers by itself. With analysis-service stopped and started again, a conversation that failed in between was analyzed on its next try, with no one touching it.
Testing
GitHub Actions runs both services’ tests on every push, without Docker, a queue or the models: each service replaces its neighbors with fakes, so CI takes about a minute.
- analysis-service (pytest, mypy): the scoring math on hand-worked examples, the analyzer with keyword classifiers in place of the models, the API’s validation and the shared contract. The models sit behind a
TextClassifierinterface, so fakes and real models are interchangeable. A second set of tests loads the real models when they’re installed, including an apology that must come out empathetic and a turn past the 512-token limit. - conversation-service (JUnit): every path through the processor (a new conversation ends DONE, a duplicate is skipped, a failure is stored with its root cause and rethrown so SQS retries, and the retry ends DONE), the listener’s validation, the store and the HTTP API.
Designed, not built
Left out to keep the scope small, but planned so they fit in:
- A durable store. DynamoDB behind the existing
ConversationStoreinterface, with a conditional write that saves PENDING only if no record exists. That atomic claim also covers the one duplicate case a read-then-write check can’t: two copies arriving at the same moment both see no record, so both are analyzed. - Running on AWS. Both services as containers on ECS, real SQS with only the endpoint changed, and consumers that scale with the queue’s depth.
- Integration tests against a real queue with Testcontainers.