Observability is meant to give software teams confidence. It should make it easier to understand what is happening inside complex systems, not harder. But for many organisations, especially those working across multiple environments, observability becomes an unintentional tangle of tools, dashboards, and alerts. When Steve Foster joined MadeCurious as our Cloud Engineering Specialist, he inherited exactly that.
The story that follows is not about chasing a new tool for the sake of it. It is about untangling years of accumulated complexity and finding a calmer, more deliberate way to understand the systems we support. OpenTelemetry became central to that shift, but the real change came from stepping back and rethinking how we wanted observability to work.
The Hidden Complexity Behind Our Early Observability Setup
Because MadeCurious works across many partner environments, each with its own priorities and history, the monitoring setup had evolved in pockets rather than as a single system.
For example, most environments had their own CloudWatch accounts. Each account had its own dashboards, its own alarms, and its own way of measuring health. Alongside CloudWatch, we were also using a range of other tools, think Pingdom, Uptime Robot, Raygun, Report URI, and occasionally DataDog. Each tool absolutely had value - the challenge was that we lacked a unified way to derive the right insights across the whole estate.
The effect was easy to feel and hard to ignore. Alerts popping up constantly, often duplicated across tools. Understanding an issue meant time lost jumping between dashboards, comparing timestamps and thresholds, and trying to align inconsistent signals. There was no single view of truth, and no shared way for developers to navigate the noise. As the number of partners grew, the mental load grew with it.
We knew we needed to revisit the structure and intent behind our choices and make life easier for our engineers.
Reframing the Problem: What Good Observability Should Look Like
Rather than reaching immediately for a new tool, we started by asking a simple question: what should observability look like for us? We mapped the problem, the desired outcomes, the trade-offs to consider, and the indicators that would show we were making progress.
The conversations were straightforward but important. We wanted a single place to understand how our systems behaved. We wanted fewer dashboards, fewer tools, and less noise. We wanted consistency across partner environments. And we wanted developers to be able to spend more time solving meaningful problems instead of piecing together fragments of data.
This clarity shaped everything that followed.
Exploring the Options: What We Looked for in an Observability Platform
With the problem well defined, we began assessing the platforms that could support a unified approach. We compared New Relic, Honeycomb, and Elastic Cloud. All three were capable, and all three could work in certain situations. We needed something that would work across many situations.
Elastic stood out. It provided unified observability across applications and infrastructure, combining logs, metrics, application traces and user experience data into a single, integrated platform. This consolidation allows for powerful, cross-referenced analysis, enabling our teams to move from detecting issues to understanding root causes quickly and efficiently.
Elastic felt like a platform we could build on, not just a tool we could plug in.
Why OpenTelemetry Became the Standard That Made Everything Click
OpenTelemetry is an open, vendor-neutral standard for generating and collecting telemetry data. It gives teams one way to instrument applications, regardless of where that data goes.
For us, that neutrality mattered. It meant:
- We could instrument code once
- We could send telemetry wherever we needed it
- We weren’t locked into any vendor
- We could build one consistent approach across all environments
OpenTelemetry didn’t just simplify the technical work. It removed an entire category of complexity around maintaining different agents, different formats, and different patterns across clients.
It gave us the foundation we had been missing.
The Journey to Building a Unified Observability Pipeline
The shift did not happen overnight. We experimented with shared CloudWatch accounts, metric streams, X-Ray traces, Firehose pipelines, and various combinations of them. Some approaches worked for individual environments but struggled when applied across organisations. Others worked technically but weren’t cost-effective at scale.
OpenTelemetry changed the direction. It let us build a centralised pipeline while still respecting the boundaries of each partner environment. We began deploying OpenTelemetry collectors, configuring exporters, and connecting everything to Elastic.
Piece by piece, the picture became clearer. Instead of a scattered collection of tools, we had a single path for telemetry to travel through.
How OpenTelemetry and Elastic Changed Our Day-to-Day Workflows
The impact was noticeable almost immediately. Developers no longer had to navigate a maze of dashboards to understand what was happening. Logs, metrics, and traces appeared together. Alerts were cleaner and more meaningful. Debugging became faster because the data told a more coherent story.
Costs also decreased. Retiring redundant dashboards and removing overlapping tools meant we were no longer paying for complexity we didn’t want. But even more valuable was the reduction in mental overhead. When engineers know exactly where to look, everything moves more smoothly.
Perhaps most importantly, the entire team could now deepen their understanding of one solid observability approach, rather than spreading their attention thin across a long list of inconsistent systems.
The Practical Improvements We Saw After the Shift
The benefits were not abstract; they showed up in our daily work.
We could onboard new partner environments far more easily because the pattern was already established. Alerts became something to pay attention to again, rather than noise to filter out. When issues arose, the path to root cause became shorter and clearer. And with a single place to understand the state of our systems, conversations within the team - and with partners - became more grounded.
Observability turned from a burden into a capability.
What This Journey Taught Us About Modern Observability
A few lessons stood out clearly. Defining the problem well at the start prevented unnecessary complexity later. Experimentation wasn’t optional - the constraints of real-world environments only reveal themselves through trying, adjusting, and trying again. And observability isn’t a one-off project. It evolves. The goal is to build a foundation that is adaptable, not fragile.
OpenTelemetry and Elastic gave us that base, but the refinement continues. That’s the nature of observability work. It grows with the systems it supports.
Where We’re Taking Our Observability Practice Next
We’re continuing to expand our OpenTelemetry coverage, introduce new data sources, and refine our dashboards. The difference now is that we no longer feel like we’re patching over gaps. We’re shaping observability deliberately, using open standards and a stack we trust.
The move from monitoring chaos to meaningful observability took time, but the shift has been transformative. We now have a calmer, clearer, and more resilient way of understanding our systems - and a foundation that supports the way we work: curious, thoughtful, and committed to quality.
