Refinement

GTM engineers, this one is for you. Your product already tells you what a contact did: time on site, repeat visits, a trial. Refinement tells you who they are.

refineContact() asks an enrichment provider (Apollo, or whatever you plug in) about a contact you already know, and lands the answer on the contact's own canonical fields: title, seniority, company, company_employees, company_industry. It fills the gaps and never overwrites what you already set. Because it writes through the ingestion pipeline, bucket membership re-evaluates the instant the traits land.

So a bucket built from behaviour is narrowed by fit the moment the company data arrives, and the two together become a ranked list of people worth a call. That is the whole feature. No new primitive to learn: buckets are still buckets, properties are still properties, and the score is a function you write.

Refinement is inert until you configure a provider. With no APOLLO_API_KEY (or another provider registered), refineContact() returns { status: "skipped", reason: "no_provider" } and costs nothing. Existing apps are unaffected.

The loop

behaviour lands a contact in a bucket
  → bucket.on("enter") calls refineContact()
  → canonical traits land via ingestEvent (fill-if-absent)
  → bucket membership re-evaluates immediately
  → your scoring function blends fit with behaviour
  → the score lands via ingestEvent
  → a score bucket flips, and you notify sales

Each hop is something you already know how to write.

How it fits the rest

The loop is not a silo. refineContact() emits a contact.refined event, and a qualifying score flips a bucket.entered one. Both are ordinary outbound events, so a defineDestination pushes your ranked list to a CRM for your reps, one destination away. The company it lands is the key to model that contact's whole account as a group and aim a lifecycle journey at the company's problems, not just the person's. Wire analytics and every step mirrors to PostHog. That is the inbound loop closed, in TypeScript you can read.

Quick example

// src/buckets/gtm-high-intent.ts
import { days } from "@hogsend/core";
import { defineBucket, refineContact } from "@hogsend/engine";
import { Events } from "../journeys/constants/index.js";

export const gtmHighIntent = defineBucket({
  meta: {
    id: "gtm-high-intent",
    name: "GTM high intent",
    enabled: true,
    timeBased: true,
    // Every entry can spend a vendor lookup, so bound how often one contact
    // can re-enter.
    entryLimit: "once_per_period",
    entryPeriod: days(30),
    criteria: (b) =>
      b.all(
        b.event(Events.KEY_ACTION).within(days(30)).atLeast(5),
        b.event(Events.PAID_FEATURE_ATTEMPTED).within(days(30)).atLeast(1),
      ),
  },
}).on("enter", async (user, ctx) => {
  const result = await ctx.once("refine", () =>
    refineContact({ userId: user.id, email: user.email }),
  );

  // It never throws. A skip is a normal outcome, not an error.
  if (result.status === "skipped") {
    await ctx.checkpoint(`refine-skipped:${result.reason}`);
  }
});

Usage alone is rarely intent. Pairing repeated key actions with a commercial signal (someone hit a paywall, someone got a result worth a number) is what makes the bucket mean something a salesperson would act on.

Why a bucket reaction and not a journey? A journey would need its own trigger event, its own entry guards, and its own copy of this bucket's criteria: three copies of one rule. The reaction inherits the bucket's membership decision, so the criteria live in exactly one place, and the enter transition is already the moment you mean. A reaction handler runs inside a real, replayable journey run, so ctx.once and every durable primitive behave exactly as they do in a hand-written journey.

What lands on the contact

Eleven canonical, flat fact properties — the fields your contacts already carry — plus one nested enrichment object for provenance:

title              company            (company name)
seniority          company_domain
department         company_industry
linkedin_url       company_employees   (number)
country            company_revenue     (number)
                   company_country

enrichment = { provider, at }          (provenance, nested)

The facts land fill-if-absent: a value you already set from a first-party source is never overwritten, so a paid lookup only fills what was missing. The enrichment object records which provider answered and when — it is provenance, not a fact, so it nests under one key instead of cluttering the top level with two more.

Query the facts like any other property:

export const enterpriseFit = defineBucket({
  meta: {
    id: "enterprise-fit",
    name: "Enterprise fit",
    enabled: true,
    criteria: (b) =>
      b.all(
        b.prop("company_employees").gte(200),
        b.prop("seniority").eq("vp"),
      ),
  },
});

Three rules here have silent failure modes, so they are worth stating plainly:

• Facts are flat, never dotted. Write b.prop("seniority"). A dotted b.prop("company.industry") resolves to nothing, so the condition is simply never true and nothing tells you. That is why the facts are flat: conditions read one top-level key, so provenance — which conditions never read — is the only thing that nests.
• Numbers must be real JSON numbers. Conditions do no coercion, so the string "250" never matches gte(100). The two numeric traits are written as numbers. If you add your own, keep them numeric.
• First-party data wins. Refinement fills gaps, it does not correct you. If you want the vendor's value for a field, don't also set it yourself.

Every return status

refineContact() never throws. A vendor outage will not take down the journey run that called it. Branch on the status instead.

cached means this lookup key was already paid for, not this contact already has the answer. The stored answer is landed on whichever contact asked. That is what makes a shared company domain worth caching: the first person at acme.com pays for the company data, and every colleague after them gets it free.

Install the provider package

The key on its own does nothing. Install the provider into your app:

pnpm add @hogsend/plugin-apollo

It has to be a direct dependency of your app. The engine lists it under optionalDependencies, and that is not enough: your app bundles the engine, so the engine's lazy import("@hogsend/plugin-apollo") resolves against your node_modules, where an engine-only optional dependency is never linked.

You do not need to import or wire anything. With the package installed and APOLLO_API_KEY set, the engine registers the provider itself. If you set the key without the package, the boot log tells you precisely that, and refinement stays inert.

Spending money on purpose

Every lookup costs. Four environment variables control it.

Set ENRICHMENT_MONTHLY_LOOKUPS deliberately in every environment. The default is uncapped, which is the right default for a feature that is off until you configure it and the wrong one to leave in production by accident.

Note the failure mode when a cap is exhausted: refinement silently stops enriching. It looks like the feature does nothing rather than like an error. If you run capped, alert on budget_exceeded.

Everything is recorded in an enrichment_lookups ledger, one row per provider and lookup key, which serves as the cache, the negative cache, and the exactly-once guarantee at the same time. The cap counts provider calls rather than rows, so a force: true refresh loop cannot spend past it.

Scoring

The score is plain TypeScript in your app. It is not an engine primitive and it is not a config screen, because scoring weights are business logic that changes weekly and a UI for them is a worse version of a function.

export function computeGtmScore(input: GtmScoreInput): number {
  const fit = fitPoints(input);           // from the canonical fit traits
  const behaviour = behaviourPoints(input) * recencyFactor(input.daysSinceLastActivity);
  return Math.max(0, Math.min(100, fit + Math.round(behaviour)));
}

Two rules make this work, and the first is not a matter of taste.

Recompute, never increment

There is no SQL-side + in a property write. A read-modify-write increment silently loses every concurrent update, and nothing surfaces the loss.

So make the score a pure function of current state. Same input, same number, forever. That dissolves the concurrency problem, makes decay trivial, and is replay-safe by construction. Pass recency in as an argument rather than reading the clock inside the function, or you cannot test it.

Write through ingestEvent

ingestEvent is the only write path that re-runs bucket membership. A direct database update writes the property and moves nothing.

This bites hardest on a pure property bucket:

export const gtmQualified = defineBucket({
  meta: {
    id: "gtm-qualified",
    name: "GTM qualified",
    enabled: true,
    criteria: (b) => b.prop("gtmScore").gte(20),
  },
});

This bucket is not time-based, which is deliberate. The reconcile cron skips non-time-based buckets, so ingest is the only thing that will ever evaluate it. There is no backstop. Write the score any other way and this bucket freezes forever with no error to notice.

The honest cost: one user_events row per contact whose score changed, per run. Skip contacts whose recomputed score equals the stored one and a stable base costs close to nothing. The rows you do pay for are unavoidable, because ingest is the only thing that moves membership.

Two traps in a nightly recompute

If you batch the recompute across your whole contact base, two mistakes are easy to make and both look fine in testing.

Termination. A recompute does not shrink its own work set. A scored contact still matches "every contact", so a naive LIMIT 250 predicate re-selects the same first page forever. Use a keyset cursor on contacts.id and advance it to the last row you saw.

The self-feeding metric. If the job writes an event and its recency input is an unfiltered MAX(occurred_at), that write resets the contact's own recency and inflates the next run's decay multiplier. Filter the recency aggregate to the events the score actually reads. A metric that feeds a computation whose output resets that metric never settles.

A complete, commented implementation ships in the scaffold at src/workflows/gtm-score.ts.

Ranking: who do I call today

Enriching and scoring a contact still does not tell you who to call. That is what ordering is for.

GET /v1/admin/contacts?orderBy=property&orderProperty=gtmScore&orderDir=desc

Non-numeric and absent values sort last, so one junk value in one contact's property bag cannot take the endpoint down.

For a large contacts table, add the expression index:

CREATE INDEX CONCURRENTLY contacts_gtm_score_idx
  ON contacts (((properties ->> 'gtmScore')::numeric))
  WHERE jsonb_typeof(properties -> 'gtmScore') = 'number';

The WHERE clause is not optional. Without it the expression is evaluated for every row, and the first contact whose gtmScore is not a number fails the cast and breaks every write to the contacts table.

Bringing your own provider

Apollo is the reference, not a requirement. A provider is a dumb wire: it queries a vendor and normalises the response. Caching, budget, preferences and the database all stay in the engine.

import { defineEnrichmentProvider } from "@hogsend/core";

export const createAcmeProvider = (opts: { apiKey: string; fetch?: typeof fetch }) =>
  defineEnrichmentProvider({
    meta: { id: "acme", name: "Acme Data" },
    capabilities: { personLookup: true, companyLookup: true },
    async enrichPerson(query) {
      // Call the vendor, return a normalised EnrichmentResult.
      // Omit absent fields; never map a vendor null into a written null.
    },
  });

Take fetch as an injectable option. Every test then drives recorded fixtures through it, so your suite needs no network and no API key.

Mirror packages/plugin-apollo for the package shape. Its README documents the traps a real vendor contract holds: array-vs-string fields, a bare domain field next to a full-URL one, and independently nullable person and company links.

What refinement is not

• Not a defineSignal primitive. A bucket already is one.
• Not a traits primitive. Contact properties are it.
• Not account rollup. Groups exist, but journeys stay person-scoped for now.
• Not an outbound sender. Refinement writes properties; what you do with them is a journey.

Related

• Buckets for membership, reactions, and the criteria builder
• Conditions for the property operators
• Events for the ingestion pipeline the writes flow through
• Plugins for the provider package shape