Flatfile's Data Hooks are a useful data healing element to re-format, validate and/or correct data automatically during the import without the user having to correct manually. When used properly, they can be used for things like automatically reformatting area codes or country codes, removing special characters, validating emails against external data, and really anything else you can code up. Data Hooks are the most powerful data healing element to date within Flatfile. There are two hooks that are available, field hooks (AKA column hooks) and record hooks (AKA row hooks).
Before beginning, there are a couple considerations to be made when choosing which type of hook to use and how to use it, and those revolve around the order and event flow of the hooks. Below is a helpful diagram that shows the general flow for Flatfile, but it should be noted that
registerFieldHook()
registerRecordHook()
FlatfileImporter.registerFieldHook(field: string, values => { // function block })
Field hooks run validation on a particular field (column) of data at the beginning of the matching stage. These hooks are run before record hooks and will only run once during the import process. These are best used to bulk edit a field or use an outside data source for validation. For example, say you want to verify the email addresses in a file are not already in your database, you could grab all the values in the column within a field hook, send them to your server and validate against your server and send back an error message with any that already exist in your system to display for the user.
In order to use field hooks, you call the field hook with
FlatfileImporter.registerFieldHook("field_name", callback => {})
Name | |
---|---|
John Doe | john@doe.com |
Jane Doe | jane@doe.com |
Steve Smith | steve@something.com |
Wayne Jones | wayne@something.com |
In the above scenario, let's say we are sending the values to our server and returning an error for any emails that are already in the database. Let's assume that John and Steve's emails are already in the system
With all that in mind, let's visualize the above within the context of making a server call. Quick note: while it's not required to use async/await with data hooks, we recommend using it when working with an outside data source/API call. Note: For visual purposes, we included the
value
With external data example:
In the below examples, you'll see us add a zero to the beginning of each value. Please notice in this instance that we do not use the async/await syntax.
Field Hooks additional notes:
value
info
level
.registerFieldHook()
.registerFieldHook()
FlatfileImporter.registerRecordHook((record, index, mode) => { // function block})
Record hooks run validation on each record (row) of data and return the record with new data and/or error messaging for the user. These hooks run on
init
change
record
index
mode
record
record.fieldName
index
mode
init
change
These hooks can be used in conjunction with other validators (like regex) or can also be used to replace some of the regex validators and pre-format errors instead of having the user do it. They can be used to reformat, replace and validate data accuracy on init and change of a record during the "review" step.
Here are some examples of using the hooks. For context, here is a configuration we can use for all these hooks with the commented out section being where you would put your hooks.
Single field validation example - zip code re-formatting (using the above example)
Multi-field validation example
Cross-field validation example - if city and state aren't present, then zip code is required
Filtering event with mode - call mode and do something on "change" only - also do something on init only