Skip to content
demuddler

Validation hooks

Check imported values against your own data, such as customer IDs in your database. The check runs on your page, so the values do not reach Demuddler.

A validation hook is a function on your page that checks values. A custom rule in your schema names the hook. The importer sends the values of the matched column to your page, and the hook returns the values that fail. The hook can ask your own API, for example whether a customer ID exists.

The values go from the importer frame to your page. Your page decides where they go next. They do not reach Demuddler servers.

This example checks each customer ID with your API. It stops the request after 5 seconds, or earlier if the importer stops waiting.

import "@demuddler/importer";
import type { ValidationHooks } from "@demuddler/importer";
const importer = document.querySelector("demuddler-importer");
if (!importer) throw new Error("The page has no importer element.");
importer.schema = {
fields: [
{
key: "customer_id",
label: "Customer ID",
dtype: "text",
rules: [{ type: "required" }, { type: "custom", hook: "known-customer" }],
},
{ key: "seats", label: "Seats", dtype: "int64" },
],
};
const validationHooks: ValidationHooks = {
"known-customer": async (values, { signal }) => {
// Your own endpoint, on your own origin, with your own sign-in.
const response = await fetch("/api/customers/lookup", {
method: "POST",
credentials: "same-origin",
headers: { "content-type": "application/json" },
body: JSON.stringify({ ids: values }),
signal: AbortSignal.any([signal, AbortSignal.timeout(5000)]),
});
if (!response.ok) {
throw new Error(`The customer lookup failed with status ${response.status}.`);
}
const { known } = (await response.json()) as { known: string[] };
const found = new Set(known);
return values
.filter((id) => !found.has(id))
.map((id) => ({ value: id, message: `No customer has the ID ${id}.` }));
},
};
importer.validationHooks = validationHooks;

If the request fails or takes too long, the hook throws. The importer tries the call again. If every try fails, the rule reports that it cannot run, with the error message. A value that the hook did not check never passes.

In React, pass the same object as the validationHooks prop:

import { DemuddlerImporter } from "@demuddler/importer/react";
import type { ImporterSchema, ValidationHooks } from "@demuddler/importer/react";
const schema: ImporterSchema = {
fields: [{ key: "sku", label: "SKU", rules: [{ type: "custom", hook: "known-sku" }] }],
};
const validationHooks: ValidationHooks = {
"known-sku": async (values, { signal }) => {
const response = await fetch("/api/products/unknown-skus", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ skus: values }),
signal,
});
if (!response.ok) throw new Error("The product lookup failed.");
return (await response.json()) as string[];
},
};
export function ProductImport() {
return <DemuddlerImporter schema={schema} validationHooks={validationHooks} />;
}

If your application already checks a value with Zod, Joi or valibot, a hook can run the same check. The schema of the importer stays plain data, and the library runs only on your page. Each example checks discount codes, such as SAVE-2024, and returns each value that fails with the message of the library.

Zod:

import { z } from "zod";
const DiscountCode = z.string().regex(/^[A-Z]{4}-\d{4}$/, "Use the form ABCD-1234.");
importer.validationHooks = {
"discount-code": (values: string[]) =>
values.flatMap((value) => {
const result = DiscountCode.safeParse(value);
return result.success ? [] : [{ value, message: result.error.issues[0].message }];
}),
};

Joi:

import Joi from "joi";
const DiscountCode = Joi.string()
.pattern(/^[A-Z]{4}-\d{4}$/)
.messages({ "string.pattern.base": "Use the form ABCD-1234." });
importer.validationHooks = {
"discount-code": (values: string[]) =>
values.flatMap((value) => {
const { error } = DiscountCode.validate(value);
return error ? [{ value, message: error.message }] : [];
}),
};

valibot:

import * as v from "valibot";
const DiscountCode = v.pipe(v.string(), v.regex(/^[A-Z]{4}-\d{4}$/, "Use the form ABCD-1234."));
importer.validationHooks = {
"discount-code": (values: string[]) =>
values.flatMap((value) => {
const result = v.safeParse(DiscountCode, value);
return result.success ? [] : [{ value, message: result.issues[0].message }];
}),
};

The field names the hook in its rules: { key: "discount", label: "Discount code", rules: [{ type: "custom", hook: "discount-code" }] }.

To stop a user from importing a record that your system already holds, refuse the values that you find. The importer then shows those rows as problems.

import type { ValidationHooks } from "@demuddler/importer";
const validationHooks: ValidationHooks = {
"new-email": async (values, { signal }) => {
const response = await fetch("/api/contacts/existing", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ emails: values }),
signal,
});
if (!response.ok) throw new Error("The contact lookup failed.");
const { existing } = (await response.json()) as { existing: string[] };
return existing.map((email) => ({ value: email, message: "This contact already exists." }));
},
};

Name the hook in the rules of the field, as { type: "custom", hook: "new-email" }. The endpoint must return each email as your page sent it, also when it ignores case.

This hook finds exact values only. To update existing records on your backend, do not refuse them. Deliver every row, and let your backend insert or update each row by its key.

Item Meaning
values The different values of the matched column, as the importer shows them. Empty cells are not sent. Use required for them.
context.field The key of the field whose rule names the hook.
context.hook The hook name from the rule.
context.signal An AbortSignal that aborts when the importer stops waiting. Pass it to fetch.
Return value An array of the values that fail. Each item is a value, or { value, message }. A value that you do not return passes.

The importer shows your message in place of its own message for that value.

The importer compares values as exact text. If your data ignores case or spaces, normalise each value in your hook before you look it up. Return the value as you received it.

A hook can serve more than one field. Use context.field when the fields need different checks.

Limit Value When it is passed
Values in one call 1,000 Larger columns use more calls.
Calls waiting at once 4 The next call starts when one answers.
Time for one call 10 seconds The call is tried again.
Tries of one call 3, after waits of 0.5 and 2 seconds The rule cannot run.
Distinct values for one rule 100,000 The rule cannot run, and no value is sent.
Characters in one value 1,000 That value is not sent, and the rule reports it.
Characters in a message 200 The rest of the message is dropped.

A column with a million rows and a few hundred distinct IDs needs one call.

A refused value is a rule failure in each row that holds it. The importer shows it with the other problems, and the filters and fixes work with it. The validate event reports it with the rule name custom. A delivery that reads with skipInvalid: true leaves those rows out.

Completion and downloads wait until every hook answers.

After an edit, the importer sends only the new values. It keeps the earlier answers while the file is open. A new file starts with no answers.

A custom rule reports once, with the reason, that it cannot run when:

  • Your page registers no hook with its name.
  • One call fails three times: the hook throws, rejects or takes longer than 10 seconds. Make your hook safe to call again with the same values.
  • The hook returns something other than an array of values that it received. The importer does not try that call again.
  • The column has more than 100,000 distinct values.

A rule that cannot run did not pass. It marks no row, so skipInvalid does not remove rows for it. Values that the hook refused before the failure still count. While the file is open, the importer does not call the hook again for that rule.

If you set validationHooks with a different set of names, the importer checks every value again. New functions with the same names apply only to new values, so a React render does not repeat the calls.

If your own data changes while a file is open, call importer.recheckValidationHooks(), for example after a user adds a customer. The importer drops every earlier answer and sends every value to the hooks again.

Your server must still check the rows that it accepts. See Validation does not silently discard rows.