Three ideas from systems engineering that changed how I size a pipeline

This year I started a master's in engineering management. Its first course, Systems Validation & Testing, is a systems engineering course, and I went in expecting vocabulary. In the first few weeks, that did arrive, in the form of V-models, verification matrices and a great many diagrams. But three ideas followed me back to my desk and changed how I write integration specs, and none of them are about diagrams.

I want to write those three down, because each one corrected a habit I had carried for years without noticing it.

A requirement is not a requirement until a number is attached

In more than one specification, I have written the sentence "the sync shall be near real-time," and at the time I thought I was describing a requirement.

In systems engineering terms, that sentence is a goal, and it becomes a design-to criterion only once a measurable bound is attached to it. The distinction sounds like pedantry until an engineer notices what it costs downstream. At 30 seconds and at 15 minutes, "near real-time" describes two different systems: the first subscribes to a change feed, and the second polls on a schedule. Those two designs differ in cost, in failure modes, and in who gets paged at 3am.

So the habit I changed is a small one, and in two requests this year it has already paid for itself. When an integration request arrives without a number, I no longer estimate it. I send back one question — what is the longest delay that still works for you — and I wait. The answer comes back within a day, and in about half of those cases the tolerance is looser than I would have assumed, which means the cheaper design was available from the start.

Serial dependencies multiply, and the contract sets the ceiling

For the second idea, I have to admit an arithmetic I got wrong for years.

Consider a chain of 4 services, each promising 99.975% availability. Multiplied together they give about 99.9%, which is the number everyone wants to put on the slide. I had done that arithmetic before and felt competent doing it, but I had not once looked at where each of those 4 numbers came from.

One of them was a contracted external dependency, and the vendor's own agreement guaranteed 99.9%. Therefore the chain could not beat 99.9% whatever I built around it, because that single link already consumed the entire budget by itself. No amount of retry logic, caching or redundancy on my side moves a number that a contract has fixed.

For me, the practical change is the order of operations. I now start an availability conversation by reading the weakest vendor contract in the chain, and only then do I look at the architecture. If the business needs 99.99% and a vendor has sold us 99.9%, that is a procurement problem wearing an engineering costume, and it should go back to the people who can renegotiate it.

Allocate before you measure

Of the three, this is the one I still find hardest to apply.

When someone asks how fast a pipeline is, the instinct of most engineers is to measure it: you instrument the hops, you add up the latencies, and you report the total. That feels rigorous, however it runs backwards. What you have produced is a description of what you happen to have built, and there is a strong pull to then call that number the target.

Allocation runs the other way. You take the top-level figure the business actually needs, you divide it into a budget for each stage before measuring anything, and then you measure to find out which stage is over its budget. A 10-second end-to-end target might allocate as 2 seconds to extract, 3 to transform, 4 to load and 1 of slack. When the transform comes back at 7 seconds, you have not just found a slow stage; you have found a stage that is 4 seconds over a number someone agreed to.

For four years of tuning queries, I had been doing the first version and calling it analysis. The second version is barely harder, and it converts a performance discussion into a design conversation, because a component that cannot meet its allocation is telling you the architecture is wrong rather than the SQL.

What I think the degree is actually for

None of these three ideas is complicated. An engineer could read all three in an afternoon, and I suspect a good number of people arrive at them alone, after enough incidents.

What the course gave me was a reason to slow down and write out the reasoning behind habits I already had. I think that is the real value, and it is not the credential. For the past few years, most of my technical learning happened at 11pm against a deadline, where the goal is to make the thing work and the understanding arrives later, if at all. A structured course inverts that, and the inversion is uncomfortable in a useful way.

I'm not through my first course yet, so treat that as a first impression and not a conclusion. Ask me again after the fourth one.