Poya Shad

It's Never Just One Integration

Thoughts from a developer working in IoT

“It’s just one integration.”

It was never just one.

The old me would open the System X API docs and start coding.

  • Figure out where to store the credentials and how to access them in a secure way.
  • Build the integration.
  • Write the tests.
  • Make it performant and stable.
  • Add fallbacks. Done.

LGTM, ship it!

Then in 6 months I’d do it all again for System Y.

Now I start with a different question.

What is an integration?

If I strip away the specific API, every integration I have built in the IoT space has some mix of these four attributes.

Attribute The question The usual answers
Authentication How do we prove who we are? Username + password, API key, OAuth2
Inbound How does their data reach us? We subscribe (MQTT, Kafka, WebSocket), we poll on a schedule, they push to our webhook
Outbound How does our data reach them? We publish and they subscribe, we push to their endpoint, they poll our API
Actions What can we ask their system to do? Request/response: read, create, update, delete

Three typical systems mapped onto the model:

Auth Inbound Outbound Actions
Air quality service API key Poll every 15 min - -
Thermostat service OAuth2 Webhook - Yes
LoRaWAN network server Username + password Subscribe (MQTT uplinks) Publish (MQTT downlinks) -

Every attribute is nullable

Start with the air quality service. It needs an API key and polling. Build those two adapters. Leave the rest empty.

Then we decide to integrate with a thermostat service. It needs OAuth2, a webhook and actions. Three new adapters.

public class Integration
{
    public required string Id { get; init; }
    public required string Kind { get; init; }
    public IAuthentication? Auth { get; init; }
    public IInbound? Inbound { get; init; }
    public IOutbound? Outbound { get; init; }
    public IActions? Actions { get; init; }
}

That ? makes the attributes nullable. An integration is whatever subset of the four it needs.

One message shape

Every inbound adapter, no matter the transport, hands your platform the same thing:

public record InboundMessage(
    string IntegrationId,
    DateTimeOffset Timestamp,
    IReadOnlyDictionary<string, object> Values);

A polling adapter looks like this:

public class PollingInbound(TimeSpan interval) : IInbound
{
    public async Task StartAsync(Func<InboundMessage, Task> onMessage, CancellationToken ct)
    {
        using var timer = new PeriodicTimer(interval);
        while (await timer.WaitForNextTickAsync(ct))
        {
            // fetch, map to InboundMessage, call onMessage
        }
    }
}

Downstream, nothing knows whether a message came from a broker, a poll or a webhook.

Integrations are usually declarative, not procedural

Registering one looks like this in a test:

var airQuality = new Integration
{
    Id = "air-quality",
    Kind = "air-quality-service",
    Auth = new ApiKeyAuth("air-quality"),
    Inbound = new PollingInbound(TimeSpan.FromMinutes(15)),
};

Need historic readings too? Add an Actions adapter. Nothing else changes.

var airQuality = new Integration
{
    Id = "air-quality",
    Kind = "air-quality-service",
    Auth = new ApiKeyAuth("air-quality"),
    Inbound = new PollingInbound(TimeSpan.FromMinutes(15)),
    Actions = new RestActions("https://api.example.com"),
};

Nullable has a price. A caller can’t assume an integration supports actions, so it checks first:

if (airQuality.Actions is { } actions)
{
    var readings = await actions.ExecuteAsync(
        new ActionRequest("get-history", new { from = "2026-03-01", to = "2026-03-31" }),
        ct);
}

In production, none of this is hardcoded. Each integration is a row in the database.

erDiagram
    integration_kinds ||--o{ integrations : "is a"
    integrations ||--o{ inbound_channels : "receives through"
    integrations ||--o{ outbound_channels : "sends through"

    integration_kinds {
        string kind PK "air-quality-service, thermostat-service, lorawan"
        string auth_kind "api-key, oauth2, basic"
        string[] inbound_kinds "subscribe, poll, webhook"
        string[] outbound_kinds "publish, push, api"
        string[] actions "get-history, set-temperature"
    }

    integrations {
        string id PK "air-quality"
        string kind FK
        json config "base URL, settings"
        string secret_ref "name in the secret store, never the secret"
        string status "active, paused, failed"
    }

    inbound_channels {
        string id PK
        string integration_id FK
        string kind "poll"
        json settings "interval: 15m"
        timestamp last_message_at
    }

    outbound_channels {
        string id PK
        string integration_id FK
        string kind "push"
        json settings "target, format"
    }

The kinds table is the catalog. It says which adapters exist in code, so an integration can never be configured with something the platform can’t run.

Actions have no table of their own. They are single requests, not flows, so the catalog only needs to list them.

The database also allows several channels in each direction, where the class above has one. The class is the simplest shape.

Credentials are a reference to a secret store, never the secret itself.

A new instance of a known kind is a row. A new kind needs code only where the system needs a set of procedural actions: its login flow, its message envelope, its actions.

You can push more of that into config. A generic kind like mqtt-json would onboard most simple systems with no code at all. Its config holds a topic pattern, the envelope (where in each message the id, the values and the timestamp sit) and the mapping (which fields become values, and in which unit). Login, topic, envelope, mapping. Done.

How far can you take integrations as data?

Further than I first thought, but not all the way. Give the generic kind a small vocabulary for actions too: URL template, method, headers, where in the response the result sits, and a plain login with a static key or a simple token fetch. That usually covers most vendor APIs.

Where it stops is anything with state and steps. You can express that in config, but then you have built an interpreter for your own language, with no types, no tests on the adapter, no debugger and a new attack surface.

The line is between declarative and procedural. Anything you can express as values belongs in the database. Anything that needs a sequence of steps with state is code.

flowchart LR
    Q["Can it be written as<br/>fields and values?"]
    Q -- yes --> D["<b>Declarative</b><br/>A row in the database"]
    Q -- "no: it needs <i>then, if, wait,<br/>retry, remember</i>" --> P["<b>Procedural</b><br/>Code in the adapter"]

    D --> D1["Broker, URL, credentials reference"]
    D --> D2["Topic, poll interval"]
    D --> D3["Envelope: where id, values<br/>and timestamp sit"]
    D --> D4["Mapping: which fields<br/>become values, in which unit"]

    P --> P1["Login that caches a token<br/>and retries when it is rejected"]
    P --> P2["Device list: page with a cursor,<br/>cache, filter locally"]
    P --> P3["Setup in the other system,<br/>and teardown when save fails"]
    P --> P4["Backfill of history"]

    style D fill:#e3f3f0,stroke:#0f766e,color:#0f3d39
    style P fill:#fbeedb,stroke:#b45309,color:#4a2305
    style D1 fill:#ffffff,stroke:#0f766e,color:#0f3d39
    style D2 fill:#ffffff,stroke:#0f766e,color:#0f3d39
    style D3 fill:#ffffff,stroke:#0f766e,color:#0f3d39
    style D4 fill:#ffffff,stroke:#0f766e,color:#0f3d39
    style P1 fill:#ffffff,stroke:#b45309,color:#4a2305
    style P2 fill:#ffffff,stroke:#b45309,color:#4a2305
    style P3 fill:#ffffff,stroke:#b45309,color:#4a2305
    style P4 fill:#ffffff,stroke:#b45309,color:#4a2305

None of this is new

When I researched this post, I found a book called Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf.

The book came out in 2003. I was ten years old. I started programming in 2016 and spent years building one integration at a time, what two people had already had figured out.

Where I got burned

  • Test the connection separately from saving it. If the user clicks Save and the credentials are wrong, nothing should have been persisted. Test, then save, as two steps.
  • Clean up on failure. If saving an integration creates things in the other system, a failed save has to remove them. Otherwise you leak half-configured accounts on their side.
  • Avoid deleting or changing what’s not yours. Your integration owns its own channel and its own login in the other system. (Usually)
  • Every adapter reports its health. Each adapter reports whether it is connected and when it last received something.