The orders/updated firehose

I only listened when I was the one talking

In the last post I described setting up Shopify order fulfillment for a former colleague’s store — the undocumented partner API, the offshore go-live, the request to test through production. This is the part of the story I’m less proud of.

Confession: I turned off orders/updated

When I designed the system I subscribed to the webhook topics that seemed obviously necessary: order created, order updated. I designed an idempotent store with retry logic to make a durable handoff to product manufacturing. During development and test phases I felt I had a tight reliable system with zero holes.

Here’s why it was tidy, as best I can reconstruct it: I only started my local listener when I was ready to submit an order from the test store. Listen, fire the order, verify, stop. Any edits I made to orders outside those windows happened while nothing was listening. Did the test store send me updates outside of my expected happy path? I really don’t know. My dev workflow was structurally incapable of showing me the noise, because I only listened when I was the one talking.

Production listens all the time. The moment I started monitoring live traffic for the store my system heard everything. I learned what orders/updated actually means. It doesn’t mean “something you care about changed.” It means anything changed: a fulfillment status ticked over, a note was edited, a tag was added, a payment state shuffled internally. One real order produced a flurry of updates, each one a fresh chance for my idempotency logic to misclassify a duplicate as new. If only my dev environment listened all the time.

My heart sank and I scrambled for a solution.

So I did the thing you do at 11 p.m. with production live: I scaled the subscription back to orders/create only, and promised myself I’d deal with the noise properly later. The pipeline stayed accurate. The promise stayed unkept.

The actual problem: every event goes to one handler at full volume

The mistake wasn’t subscribing to orders/updated. There is real value in a system that can monitor updates and self-correct when/if an order changes. The mistake was an architecture where every event Shopify emits lands on one handler at full fidelity, and that handler was not smart enough to sort signal from noise. In product manufacturing a classification bug becomes a fulfillment bug.

What I wanted was a layer in front: something that captures everything (so no data is lost), lets me see the noise before writing code against it, and forwards each downstream service only what it needs, in the shape it needs. A capture endpoint is also always listening. One could point the dev store at one on day one, and every event fires into it whether or not your local handler is running. The firehose would have shown up in week one of development instead of night one of production.

Shopify has since given developers includeFields and delivery filters, and you should use them. They are precision tools you can use to optimize your system and receive only what you care about. But you may only care after you've seen your own traffic. I needed something that showed me everything first at design time, or I needed the capability to filter without intrusive hacks to my final system. With FlurryPORT, if you filter out too much you could change your filter and replay it again.

Transform in flight, route per target

That’s the other half of FlurryPORT. Captures can be reshaped with a JSONata transformation and fanned out. The same webhook can go to multiple endpoints of your application, each receiving the payload as-is or reshaped by its own JSONata transformation. One webhook in, a differently-shaped copy out to each target that needs it. In my fulfillment setup, that looks like:

• orders/create → transformed to the manufacturer’s schema → fulfillment endpoint.

• orders/updated → stripped to the three fields I actually cared about → a separate tracking service that updates status and nothing else. Or, just sit and record each orders/updated notification and leave the collection of notifications as data for the next iteration of the project.

And it solves the test-traffic problem from my previous article. When you replay a captured production order at a test pipeline, the transformation can rewrite it in flight by injecting a test flag, zeroing the quantity, rewriting the SKU to a sandbox value. Mutate a real order so that if a replay somehow reached a manufacture queue it would carry ample evidence it was a test.

One more layer of protection worth mentioning: FlurryPORT validates inbound signatures before storing anything, rejecting invalid requests with a 401 and flagging the attempt. A public webhook endpoint that triggers manufacturing is exactly the endpoint you want verifying every sender.

The lesson I’d spill for anyone building fulfillment on webhooks: subscribe wide, but never let raw provider events touch the system that spends money. Capture first, transform deliberately, route narrowly.

Want to watch your own firehose before deciding what deserves to drink from it? Paste an endpoint from flurryport.dev into your provider’s webhook settings — no account, nothing to install, captures live for about an hour. The free tier includes a transformation, enough to put one between your events and anything that spends money.