Bringing data in
Runs, history and replay
Every connector keeps a record of what it did. You find it on the connector’s page, which you open from Ingest. Everyone in the organisation can read it, except the bodies of payloads, which are for owners.
Past runs
Past runs lists every run, newest first, 50 at a time. Select Show older for the next 50.
| Column | Meaning |
|---|---|
| When | When the run started, such as “5 minutes ago”. Hover for the exact time. |
| How it ran | Fetched (on its schedule, or Run now), Received (a payload was sent), Uploaded (a file) or Replayed (a kept payload was routed again). |
| Took | How long it took. |
| Outcome | Landed, Nothing new or Failed. A failure shows its reason. |
| Rows in | Rows applied. |
| Refused | Rows refused. |
| Tables | The tables it wrote, each a link. |
- Landed means the data was kept and routed.
- Nothing new means a fetch ran and found the same answer as last time, so nothing was routed.
- Failed means nothing reached a table. A fetch that failed before anything landed is listed too.
- Checks in a row that found nothing new, or that failed for the same reason, are shown as one line, such as “Nothing new, 1,000 checks, 9:00 am to 4:40 pm”.
- A replay is one run for each payload routed again.
A run shows counts, table names and a failure’s reason. It never shows a payload or a value from one.
What has landed
What has landed lists the payloads the connector took, newest first, 100 at a time. Select Load more for more. Each shows when it arrived, its size and, for each route, what it did:
orders: 3 applied, 0 unchanged, 1 refused, 1 gone, 2 out of date
- applied: rows added or changed.
- unchanged: rows already stored the same.
- refused: rows that didn’t fit. Open Refused rows to see the first 10 as “Row 4, price: not_minor_units”, and how many more there were.
- gone: rows marked gone by a Replace everything route.
- out of date: rows left alone because the stored row was newer.
- “no records”: the route’s records path isn’t in the payload.
A payload that couldn’t be read, or a route that failed, shows its error.
An owner can select Show body to read a payload as it was stored, with card numbers already masked. If the payload is no longer stored, the page says “That payload is no longer stored.”
Versions
Saving a change to a connector makes a new version. Versions lists every version, newest first, with when it was saved and by whom. Select Show changes to see what changed from the version before: lines added, and lines removed. Select Show document to read the whole document of that version. The first version says “The first version: nothing came before it.”
The next run uses the newest version. Data that arrived earlier was read with the version that was current then.
Replay
Because Mestra keeps every payload as it arrived, you can read them again with the connector’s current version. Use it after you fix a connector, for example after you correct a column’s type, so the rows that were refused are taken.
- Open the connector and select Replay. An owner only.
- Mestra asks “Replay N payloads through the current version? Rows already written are updated, not duplicated.” N is every payload the connector has landed.
- Select Replay. Mestra routes each payload again, oldest first, and says “Replayed N payloads.”
A replay is done with the current version, so replaying twice at one version changes nothing the second time. A replay shows in Past runs as Replayed. A fetch connector’s next run uses the new version. To re-read what already arrived, replay.
Problems with a run
| What you see | What to do |
|---|---|
| Failing with a reason | See Fetch data on a schedule or Errors and refusals. |
| Rows refused | Open Refused rows. Fix the data at its source, or change the column’s type or the connector. Then replay. |
routing_failed |
A route couldn’t write its table. The payload is kept. Fix the connector, then replay. |
Busy (429) on a replay |
Mestra can’t take the work just now. Nothing was changed. Try again after the Retry-After seconds. |