Skip to content

Bringing data in

Connectors

A connector brings data into Topos and builds your tables. Each connector has one way of getting data and one or more routes. A route picks records out of what arrived and writes them into one table. See Routes, modes and columns.

The three types

Type How data arrives When it runs Page
Receive Your system sends data to an address Mestra gives you, with a key. As it arrives. Send data to Mestra
Fetch Mestra reads JSON or CSV from a web address on an interval you choose. On the interval, or when you select Run now. Fetch data on a schedule
Upload You upload a spreadsheet or CSV file and it becomes a table. When the file is uploaded. Upload a file

The connectors you can add are:

Connector Type What it is
Custom Receive Your system sends data to an address Mestra gives you, with a key. You choose the tables.
Point of sale Receive Your till sends its menu and sales to an address Mestra gives you, into two ready-made tables. See Templates.
REST Fetch Mestra reads JSON or CSV from a web address on an interval you choose.
Spreadsheet or CSV Upload You upload a spreadsheet or CSV file and choose its columns.

If you need a connector that isn’t listed, the add flow links to Support.

The Ingest page

Ingest lists your connectors in one table.

Column What it shows
Connector Its name. Select it to open it.
Type Receive, Fetch or Upload.
Schedule Every 15 minutes and so on for a fetch (hover to see when the next run is due), As it arrives for a receive connector, By hand for an upload.
Health Healthy, Failing, Overdue or Never run. A failing connector shows its error below.
Last run When it last ran and what it did, such as “120 rows” or Nothing new or Failed.
Builds The tables it fills, each a link.

An owner sees Add a connector. Everyone else sees the list.

How health is judged

Health Meaning
Failing The last run failed. The label says for how long, such as “Failing for 3 hours”, and the error is shown. Tables keep the data from the last good run.
Overdue A fetch connector is more than one of its own intervals past the time its next run was due, and it isn’t running now.
Never run It has never run, landed anything or failed.
Healthy None of the above.

The home page counts failing and overdue connectors under Ingest and lists each under needs a look.

Add a connector

An owner selects Add a connector. There are five steps, and nothing is saved until the last.

  1. Type. Choose Receive, Fetch or Upload.
  2. Connector. Choose one. A search box shows when there are six or more of a type.
  3. Configure.
    • For Custom and REST, Mestra shows the data as a table you can shape. For a push connector, paste a sample of what your system sends. For a fetch connector, enter the address and select Pull the data, or paste a sample instead. Name each column, choose its type, tick Key and Required, and remove columns you don’t want. If the data holds several lists of records, choose which list under Where the records are. For several tables from one payload, or to read only some records, select Set it up by hand.
    • For Point of sale, the tables are ready made. There is nothing to map.
    • For Spreadsheet or CSV, choose a file and shape its columns.
  4. Test. Try the connector once. A fetch connector fetches the address and shows the rows it would make and the ones it would refuse. A receive connector tests a pasted sample. Nothing is saved. You can go on without testing.
  5. Save and schedule. Name it. A fetch connector asks for Every, in minutes or hours. A receive connector makes a first key. Save connector finishes.

If you leave with something entered, Mestra asks “Leave without saving?”. Choosing a different connector after entering something asks “Start again?”.

The steps are at /ingest?add=1, and Back keeps what you entered.

A connector’s page

Select a connector to open it. The page has these parts.

  • Where it goes: Send to (the address, for a receive connector), Fetches (the address and its interval, for a fetch connector), Writes (its tables) and Version.
  • Downstream: a picture of what depends on the connector: its tables, the mappings that read them, the model types those fill, and the decisions that read them. If nothing depends on it yet, the page says so.
  • Past runs and What has landed: see Runs, history and replay.
  • Keys (receive connectors, owners only): see Send data to Mestra.
  • Versions: see Runs, history and replay.

If the last run failed, a note says “Last run failed:” with the reason, and when the tables last held good data.

An owner also has these buttons.

Button What it does
Run now Fetch connectors only. Fetches the address now, whether or not it is due, and says “Fetched.”
Replay Routes everything that has landed again with the current version. See Runs, history and replay.
Edit Opens the connector to change it. Saving makes a new version. Upload connectors can’t be edited.
Delete Forgets the connector’s address, keys, settings and saved versions. Its tables are kept if they hold data, and so is what has landed.

Connector keys

A connector has a key, the short name in its address, such as pos. It uses lower-case letters, digits and underscores, starts with a letter, and can’t be changed after the connector is made. A connector’s name is up to 80 characters and can change.

Saving a connector sends a document of at most 64 KiB.