Overview — Start here

Something went wrong this morning — and your pipeline didn't notice.

Three vendors. Daily feeds. One bad file. Here is exactly what happened.

What happened this morning

A DHL engineer upgraded their system on Tuesday

As part of the upgrade, a field called status_code was renamed to delivery_status_code in their export. The DHL engineer was doing their job correctly. They had a ticket, a review, a working system. They had no way of knowing GFS's pipeline depended on that field name.

The file arrived at 08:11 this morning — and GFS's pipeline ran green

No error. No alert. The pipeline dashboard showed a green tick. But status_code was now NULL for all 1,847 rows in that file. Delivery ETA calculations were running on missing data.

GFS's ops team found out on Thursday — via a customer complaint

A customer called to say their shipment tracking hadn't updated in three days. By the time the team traced it back to the DHL file, the data had already been used in three reports and two decisions.

The vendor was not careless. GFS's engineers did nothing wrong. There was simply no system between the vendor and the warehouse that checked the file before it landed.

Why does this happen?

4–6 hrs

Average time to find the root cause

After a bad vendor file lands, this is how long it takes a data engineer to trace the failure. The pipeline is green. The data is wrong. The investigation is manual.

Wakefield Research · 2023

44%

Of a data engineer's week spent on pipeline maintenance

Nearly half their time is reactive firefighting — not building. Bad vendor data is a large contributor to that number.

Wakefield Research · 2023

5–7 days

Per year lost to unnoticed API/feed changes

Organisations without proactive third-party monitoring spend this reconciling data gaps caused by vendor changes they never caught.

Gartner · 2024

0
Tools that check vendor files before they reach your warehouse
Infrastructure monitoring catches server failures. Data observability catches anomalies in your warehouse. Neither one sits before ingestion and checks the contract.
This is the gap Velraq fills

Most failures announce themselves. This one doesn't.

Infrastructure failure

Throws an error. You know in minutes.

Visible

Vendor data failure

No error. Pipeline runs green. Data is silently wrong. Nobody knows until a report breaks or a customer complains.

Silent

By the time someone notices, the damage is already done — and usually it has been done for days.

What this looks like on a Monday morning

MONDAY 09:00 — GFS OPS TEAM

The carrier rate sheet from Freightline loaded with a blank price_gbp column. 1,200 shipment quotes went out at £0. Finance found out at end of day reconciliation.

MONDAY 09:00 — GFS DATA TEAM

The route_manifest file from RouteIQ switched date format overnight — from ISO 8601 to DD/MM/YYYY. ETA calculations were off by months. The team spent 6 hours tracing it.

MONDAY 09:00 — GFS CDO

DHL's shipment_status feed had been missing status_code for 3 days before it was caught. Three reports sent to the board contained wrong delivery performance figures. The data was already in decisions.

None of these were the vendor's fault. None were GFS's fault. There was no gate. Now there is.

What Velraq does

Intercept

Every vendor file hits Velraq before your warehouse. SFTP drop, API call, webhook — all intercepted at the edge.

Check

Validated against a contract you define once. Rules run in under 2 seconds. No manual review needed.

Quarantine or Pass

Clean file → straight to your warehouse. Bad file → quarantine + Slack alert with exact diagnosis. Warehouse clean.

How it flows

What changes for GFS

	Before Velraq	With Velraq
When bad data is caught	When a dashboard breaks (days)	Pre-ingestion, under 2 seconds
Who finds out first	Whoever runs the broken report	Slack alert, #data-ops, instantly
What the diagnosis is	Unknown — manual investigation	Exact field, rule, row count in alert
Warehouse state	Bad data ingested	Never touched
Vendor accountability	No record	Audit log + vendor scorecard

7 steps · ~5 minutes · GFS data already loaded

GFS has three active vendor feeds. Each one has a contract and an enforcement mode — switch the mode without touching your pipeline.

The DHL feed at the bottom is the one that fired this morning. Click it to see what happened.

Step 1 of 6 — Vendor Feeds

Vendor Feeds

3 active feeds · Last checked 4 minutes ago

These are the three feeds GFS depends on every day. One is already in trouble — click it to see what Velraq caught.

Feed Name

Vendor

Protocol

Format

Send Cycle

Mode

Status

carrier_rates

Freightline UK

SFTP

CSV

Daily 06:00 UTC

ALERT

PASS

Connection

endpoint	sftp.freightline.co.uk
auth	SSH key (rotated 30d)
format	CSV · UTF-8
schedule	Daily 06:00 UTC
last file	carrier_rates_20260511.csv
last received	Today 06:02:14

Enforcement mode

Slack alerts

Auto-retry

Email alerts

route_manifest

RouteIQ Ltd

SFTP

CSV

Twice daily

ALERT

WARNING · 2

Connection

endpoint	sftp.routeiq.com
auth	Password + IP allowlist
format	CSV · UTF-8
schedule	07:00 & 19:00 UTC
last file	route_manifest_20260511_070314.csv
last received	Today 07:03:14

Enforcement mode

Slack alerts

Auto-retry

Email alerts

shipment_status

DHL Express

API

JSON

Realtime

ALERT

QUARANTINED

Connection

endpoint	api.dhl.com/shipments/v3
auth	OAuth2 · client credentials
format	JSON
schedule	Webhook · realtime
last file	shipment_status_20260511_081102.json
last received	Today 08:11:02

Email alert recipients

These addresses receive an email alert on every quarantine event.

Enforcement mode

Slack alerts

Auto-retry

Email alerts

Vendor reliability — last 30 days

98

Freightline UK

31 / 31 files clean this month

85

RouteIQ Ltd

28 / 31 — 3 format warnings

72

DHL Express

1 quarantine event this month

Feed name

Vendor

Protocol

Format

A contract is the ruleset Velraq checks every file against. You set it once. Velraq enforces it on every file, forever.

The DHL contract has 7 rules. This morning's file broke 2 of them. The rules are plain English — no code required.

Step 2 of 6 — Contracts

Contracts

One contract per feed · Version controlled · Editable any time

shipment_status_v3

carrier_rates_v2

route_manifest_v1

shipment_status_v3

DHL Express · 7 rules · 2 violations this morning

FAILING

Required Fields

shipment_idOK

carrier_codeOK

status_code

MISSING — field absent from payload

FAIL

destination_postcodeOK

estimated_deliveryOK

Quality Rules

row_count ≥ 500PASS (1,847)

null_rate(status_code) ≤ 5%

FAIL — 100% null (1,847 / 1,847 rows)

FAIL

duplicate_rate(shipment_id) ≤ 0.1%PASS (0.0%)

# shipment_status_v3 — DHL Express
contract: shipment_status_v3
vendor: DHL Express
format: json

required_fields:
  - shipment_id
  - carrier_code
  - status_code        # MISSING in today's file
  - destination_postcode
  - estimated_delivery

quality_rules:
  - name: row_count
    min: 500
  - name: null_rate
    field: status_code
    max: 0.05      # FAIL — got 1.00
  - name: duplicate_rate
    field: shipment_id
    max: 0.001

on_failure:
  - quarantine_file
  - slack_alert: #data-ops
  - email_alert

First-time setup — one manual step required

Velraq auto-scans a sample file and generates a contract draft. A data engineer reviews and activates it. After that — fully automated forever. The review step is intentional: a contract is a business decision, not just a technical one.

carrier_rates_v2

Freightline UK · 6 rules · 0 violations

PASSING

Required Fields

rate_idOK

origin_postcodeOK

destination_postcodeOK

price_gbpOK

Quality Rules

row_count ≥ 1000PASS (1,204)

null_rate(price_gbp) = 0%PASS (0.0%)

# carrier_rates_v2 — Freightline UK
contract: carrier_rates_v2
vendor: Freightline UK
format: csv

required_fields:
  - rate_id
  - origin_postcode
  - destination_postcode
  - price_gbp

quality_rules:
  - name: row_count
    min: 1000
  - name: null_rate
    field: price_gbp
    max: 0.0

on_failure:
  - quarantine_file
  - slack_alert: #data-ops

route_manifest_v1

RouteIQ Ltd · 5 rules · 3 format warnings (last 30d)

WARNING

Required Fields

route_idOK

depot_codeOK

eta_iso8601OK

Quality Rules

row_count ≥ 200PASS (412)

eta_iso8601 format = ISO 8601

WARNING — DD/MM/YYYY format detected

WARN

# route_manifest_v1 — RouteIQ Ltd
contract: route_manifest_v1
vendor: RouteIQ Ltd
format: csv

required_fields:
  - route_id
  - depot_code
  - eta_iso8601

quality_rules:
  - name: row_count
    min: 200
  - name: date_format
    field: eta_iso8601
    expects: ISO 8601

on_failure:
  - quarantine_file
  - slack_alert: #data-ops

At 08:11 this morning, DHL sent a file with 1,847 rows. The status_code field was completely absent.

Watch what Velraq did in the next 2 seconds.

Step 3 of 6 — Enforcement

Enforcement — Live simulation

DHL Express · shipment_status · Today 08:11:02

Simulate: DHL file arrives

Press the button to replay the intercept that happened this morning.

This file would have hit your warehouse 11 minutes ago. It didn't.

Warehouse protected

1,847 rows never touched your PostgreSQL tables. Your pipeline ran normally on yesterday's clean data.

File in quarantine

/quarantine/shipment_status/20260511/shipment_status_20260511_081102.json — diagnosis attached. Review in Step 4.

Team notified

Slack alert fired to #data-ops in < 3 seconds. James Morrow, Sarah Chen notified. Full diagnosis in the alert — no investigation needed.

Without Velraq

This file would have ingested into shipment_status table. status_code = NULL for 1,847 rows. Your logistics ops team would find out when the delivery ETA reports break.

With Velraq

File intercepted at 08:11:02. Warehouse clean. Team notified with exact diagnosis. Vendor contacted to resend corrected file.

The quarantine event fires a Slack alert in under 3 seconds. Sarah Chen already knows — before she opened her laptop.

The alert contains the exact diagnosis. No investigation needed. Your warehouse was never touched.

Step 4 of 6 — Alerts & Resolution

Quarantine Queue

1 file awaiting resolution · Warehouse not affected

shipment_status_20260511_081102.json

DHL Express · 1,847 rows · received 08:11:02

2 VIOLATIONS

FIELD MISSING — status_code Required field absent from payload. Contract: shipment_status_v3. All 1,847 rows affected.

QUALITY RULE FAILED — null_rate(status_code) Observed: 100% null. Limit: 5%. Likely cause: vendor renamed field in upstream system.

Email draft — to DHL Express integration team

To: integration@dhl.com
From: data-ops@gfs.co.uk
Subject: shipment_status feed — status_code field missing (file 20260511_081102)

Hi DHL Integration team,

We've intercepted today's shipment_status feed at our ingestion gateway. The
required field status_code is absent from all 1,847 rows in the payload
delivered at 08:11:02 UTC. This violates our shipment_status_v3 contract.

We suspect this is related to a recent change on your side — possibly a
rename to delivery_status_code. Could you confirm and re-send the file with
the original schema, or let us know if the field has been intentionally
renamed so we can update our contract on our side?

The file is held in quarantine. It has not been ingested into our warehouse.

Thanks,
GFS Data Ops

Slack alert — #data-ops

# data-ops

4 members

V

VelraqAPP08:11 AM

Quarantine event — DHL Express · shipment_status

File: shipment_status_20260511_081102.json

Rows: 1,847

Contract: shipment_status_v3

Violation 1: required field status_code missing

Violation 2: null_rate(status_code) = 100% (limit 5%)

Warehouse: UNTOUCHED

File held at: /quarantine/shipment_status/20260511/

SC

Sarah Chen08:13 AM

On it — I'll contact DHL and request a corrected file. Pipeline is fine, running on yesterday's data.

Older quarantine events

route_manifest_20260510_143022.csv

RouteIQ Ltd · 412 rows · received yesterday 14:30

FORMAT WARNING

eta_iso8601 field arrived in DD/MM/YYYY format instead of ISO 8601. Held pending review.

route_manifest_20260508_083015.csv

RouteIQ Ltd · 388 rows · received 08 May 08:30

FORMAT WARNING

Same date-format violation.

Same vendor · same violation · 3rd time this month → Vendor scorecard: 85/100

Every quarantine event, every check, every resolution is logged here. This record cannot be edited or deleted.

This is the audit trail that proves your team responded correctly — before anyone asks.

Step 5 of 6 — Audit Log

Audit Log

247 events · Append-only · Cannot be edited or deleted

Chain verified · SHA-256

Timestamp	Event	Feed	Detail	Actor	Hash
Today 08:11:02	QUARANTINE	shipment_status	2 violations · 1,847 rows	velraq/gateway	abc1...
event_id: evt_4c8a91e2b6f3d7 prev_hash: 9f3a82bc41d56e0712... event_hash: abc1d4f5e6789012ab34cd56ef7890fa12bc34de56fa78901234567890abcdef
Today 08:11:05	ALERT_SENT	shipment_status	#data-ops notified	velraq/alerts	def2...
prev_hash: abc1d4f5e6789012ab34cd56... event_hash: def2e5f6a78901234bc56de78f901a3bc45de67fa89...
Yesterday 14:30	FORMAT_WARN	route_manifest	DD/MM/YYYY — 412 rows	velraq/gateway	ghi3...
prev_hash: def2e5f6a78901234bc56de78... event_hash: ghi3f6a789012345cd67ef89a0b1c2d3e4...
08 May 09:02	QUARANTINE	route_manifest	format violation	velraq/gateway	jkl4...
prev_hash: ghi3f6a789012345cd67ef89... event_hash: jkl4a78901234567de89fa0b1...
08 May 09:05	ALERT_SENT	route_manifest	#data-ops notified	velraq/alerts	mno5...
prev_hash: jkl4a78901234567de89fa0b1... event_hash: mno5b8901234567ef9a0b1c2d...
01 May 08:02	PASS	carrier_rates	1,204 rows · clean	velraq/gateway	pqr6...
prev_hash: mno5b8901234567ef9a0b1c2d... event_hash: pqr6c901234567fa0b1c2d3e4...

Hash chain

event_id: evt_4c8a91e2b6f3d6

prev_hash: 8e2971ab3...

event_hash: 9f3a82bc41d56e07127f...

→

event_id: evt_4c8a91e2b6f3d7

prev_hash: 9f3a82bc41d56e07127f...

event_hash: abc1d4f5e6789012ab34...

This is what Velraq prevented for GFS in the last 30 days.

Every number here is based on your feeds and published industry data. Hover any stat to see the source.

Step 6 of 6 — Impact & ROI

What Velraq prevented for GFS

Last 30 days · Based on your feeds and published industry data

ROI Summary

Vendor Scorecards

3

Quarantine events this month

1 DHL · 2 RouteIQ

244

Hours of engineering time not spent debugging

Based on Wakefield avg: 4–6 hrs per incident × 3 incidents × number of engineers involved (estimated)

~60 hrs

Total engineering hours saved this month

3 incidents × 20hrs average trace time (Wakefield Research / Monte Carlo 2023)

~£3,300

Estimated cost of those incidents avoided

60 hrs × £55/hr average UK data engineer rate (industry benchmark)

How we calculated this

3 quarantine events this month

× 20 hrs average trace time per incident (Wakefield Research 2023 — data engineers spend 44% of time on pipeline maintenance)

× £55/hr UK data engineer day rate (industry benchmark)

= £3,300 estimated cost of incidents Velraq intercepted

Sources: Wakefield Research 2023 · Monte Carlo State of Data Quality 2026 · UK IT salary benchmarks

Incident breakdown

Date	Feed	Vendor	Rows affected	Issue	Outcome
Today 08:11	shipment_status	DHL Express	1,847	status_code missing	QUARANTINED
Yesterday 14:30	route_manifest	RouteIQ Ltd	412	date format wrong	QUARANTINED
08 May 08:30	route_manifest	RouteIQ Ltd	388	date format wrong	QUARANTINED

What does a vendor data incident actually cost your team?

We will calculate it together using your numbers — your feeds, your engineer rate, your incident frequency. Not a sales call. A 20-minute conversation about how your vendor data actually flows.

No pricing discussion on the first call · No commitment required · 60-day free pilot available after

adwait_kumbhare@velraq.com · velraq.com

Something went wrong this morning — and your pipeline didn't notice.

What happened this morning

Why does this happen?

Most failures announce themselves. This one doesn't.

Infrastructure failure

Vendor data failure

What this looks like on a Monday morning

What Velraq does

How it flows

What changes for GFS

Vendor Feeds

Vendor reliability — last 30 days

Freightline UK

RouteIQ Ltd

DHL Express

Contracts

First-time setup — one manual step required

Enforcement — Live simulation

Simulate: DHL file arrives

Warehouse protected

File in quarantine

Team notified

Without Velraq

With Velraq

Quarantine Queue

Slack alert — #data-ops

Older quarantine events

Audit Log

Hash chain

What Velraq prevented for GFS

How we calculated this

Incident breakdown

Vendor reliability — 30-day scorecard

Freightline UK

RouteIQ Ltd

DHL Express

What does a vendor data incident actually cost your team?