@flatfile/plugin-record-hook

A plugin for running custom logic on individual data records in Flatfile.

@flatfile/plugin-record-hook

The @flatfile/plugin-record-hook plugin offers a convenient way to execute custom logic on individual data records within Flatfile. By setting up an event listener for the commit:created event, this plugin seamlessly integrates with the data processing flow.

Event Type:

listener.on('commit:created')

Parameters

sheetSlug · string

The sheetSlug parameter is the slug of the sheet you want to listen to.

callback · function

The callback parameter takes a function that will be run on the record or records.

options.chunkSize · number · default: "10_000"

The chunkSize parameter allows you to specify the quantity of records to in each chunk.

options.parallel · number · default: "1"

The parallel parameter allows you to specify the number of chunks to process in parallel.

Imported NPM Packages

• @flatfile/api@1.5.13+ provides the API calls.
• @flatfile/hooks@1.3.1+ provides the record type.
• @flatfile/listener@0.3.15+ provides the listener and event types.
• @flatfile/util-common@0.1.1 provides utility functions for asynchronously batching.

Usage

install

npm i @flatfile/plugin-record-hook @flatfile/hooks

Import

bulkRecordHook

import { FlatfileRecord, bulkRecordHook } from "@flatfile/plugin-record-hook";
import { FlatfileEvent, FlatfileListener } from "@flatfile/listener";

Pass bulkRecordHook or recordHook to a Flatfile data listener and provide a function to run when data is added or updated.

Listen for data changes

Set up a listener to configure Flatfile and respond to data Events. Then use this plugin to set up a hook that responds to data changes.

bulkRecordHook js

import { bulkRecordHook } from "@flatfile/plugin-record-hook";

export default async function (listener) {
  listener.use(
    bulkRecordHook("my-sheet", (records) => {
      return records.map((r) => {
        //do your work here
        return r;
      });
    })
  );
}

Additional Options

bulkRecordHook can accept additional properties. Props will be passed along to the transformer.

bulkRecordHook js

import { bulkRecordHook } from "@flatfile/plugin-record-hook";

export default async function (listener) {
  listener.use(
    bulkRecordHook("my-sheet", (records) => {
      return records.map((r) => {
        //do your work here
        return r;
      });
    }),
    { chunkSize: 100, parallel: 2 }
  );
}

Flexible Options

chunkSize · number · default: "10_000"

Define how many records you want to process in each batch. This allows you to balance efficiency and resource utilization based on your specific use case.

parallel · number · default: "1"

Choose whether the records should be processed in parallel. This enables you to optimize the execution time when dealing with large datasets.

Example Usage

This example sets up a record hook using listener.use to modify records in the “my-sheet” sheet.

When a record is processed by the hook, it checks if an email address is missing, empty, or invalid, and if so, it logs corresponding error messages and adds them to a form validation context (if the r object is related to form validation). This helps ensure that only valid email addresses are accepted in the application.

In the bulkRecordHook example, it passes a chunkSize of 100 and asks the hooks to run 2 at a time via the parallel property.

bulkRecordHook js

import { bulkRecordHook } from "@flatfile/plugin-record-hook";

export default async function (listener) {
  listener.use(
    bulkRecordHook(
      "my-sheet",
      (records) => {
        return records.map((r) => {
          const email = r.get("email") as string;
          if (!email) {
            console.log("Email is required");
            r.addError("email", "Email is required");
          }
          const validEmailAddress = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
          if (email !== null && !validEmailAddress.test(email)) {
            console.log("Invalid email address");
            r.addError("email", "Invalid email address");
          }
          return r;
        });
      },
      { chunkSize: 100, parallel: 2 }
    )
  );
}

See the code

@flatfile/plugin-record-hook