Fileset

Remote Folder

By specifying a Remote Folder name, a folder by this name will be created or found on the FileCatalyst Server and all files will be transferred to and from that folder.

Enable Dynamic Folders

Selecting this will force the HotFolder to dynamically replace tags in the Remote Folder path when uploading. It supports the following tags:

• <%HOSTNAME%> is replaced with the hostname of the machine running the HotFolder.

• <%TIME%> is replaced with the time in the format: HH.mm.ss

• <%DATE%> is replaced with the date in the format: yyyy.MM.DD

File Filter

FileCatalyst HotFolder tasks allow you to define a filter which can be used to include or exclude particular files from the transfer. This filter is applied at the beginning of the transfer and can be configured through a wide range of available options that allow you to tune your fileset as needed.

Filter Phrase:

This option allows you to define a text phase that will filter any files in the HotFolder directory or on the Server that match the expression. To disable file filters entirely, leave this option blank.

Filter Mode:

This option allows you to define the type of filter that you wish to use. The available filter modes are as follows:

• Exclude Filter: Will exclude any files that match the given wildcard expression. The format for the expression is a comma-delimited list of files names, using a standard wildcard (“*”) character.

• Include Filter: Will only include files that match the given wildcard expression. The format for the expression is a comma-delimited list of files names, using a standard wildcard (“*”) character.

• Regular Expression Filter: This option allows you to build a complex regular expression to select which file(s) you want to include in a transfer.

For additional documentation to how Java implements regular expressions, the following sources may be used:

• https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html

• http://www.sitepoint.com/article/java-regex-api-explained/

Construct	Matches
x	The character x
\\	The backslash character
\0n	The character with octal value 0n (0 <= n <= 7)
\0nn	The character with octal value 0nn (0 <= n <= 7)
\0mnn	The character with octal value 0mnn (0 <= m <= 3, 0 <= n <= 7)
\xhh	The character with hexadecimal value 0xhh
\uhhhh	The character with hexadecimal value 0xhhhh
\t	The tab character ('\u0009')
\n	The newline (line feed) character ('\u000A')
\r	The carriage-return character ('\u000D')
\f	The form-feed character ('\u000C')
\a	The alert (bell) character ('\u0007')
\e	The escape character ('\u001B')
\cx	The control character corresponding to x

Character classes

[abc]	a, b, or c (simple class)
[^abc]	Any character except a, b, or c (negation)
[a-zA-Z]	a through z or A through Z, inclusive (range)
[a-d[m-p]]	a through d, or m through p: [a-dm-p] (union)
[a-z&&[def]]	d, e, or f (intersection)
[a-z&&[^bc]]	a through z, except for b and c: [ad-z] (subtraction)
[a-z&&[^m-p]]	a through z, and not m through p: [a-lq-z](subtraction)

Predefined character classes

.	Any character (may or may not match line terminators)
\d	A digit: [0-9]
\D	A non-digit: [^0-9]
\s	A white space character: [ \t\n\x0B\f\r]
\S	A non-whitespace character: [^\s]
\w	A word character: [a-zA-Z_0-9]
\W	A non-word character: [^\w]

POSIX character classes (US-ASCII only)

\p{Lower}	A lower-case alphabetic character: [a-z]
\p{Upper}	An upper-case alphabetic character:[A-Z]
\p{ASCII}	All ASCII:[\x00-\x7F]
\p{Alpha}	An alphabetic character:[\p{Lower}\p{Upper}]
\p{Digit}	A decimal digit: [0-9]
\p{Alnum}	An alphanumeric character:[\p{Alpha}\p{Digit}]
\p{Punct}	Punctuation: One of !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~
\p{Graph}	A visible character: [\p{Alnum}\p{Punct}]
\p{Print}	A printable character: [\p{Graph}]
\p{Blank}	A space or a tab: [ \t]
\p{Cntrl}	A control character: [\x00-\x1F\x7F]
\p{XDigit}	A hexadecimal digit: [0-9a-fA-F]
\p{Space}	A whitespace character: [ \t\n\x0B\f\r]

Classes for Unicode blocks and categories

\p{InGreek}	A character in the Greek block (simple block)
\p{Lu}	An uppercase letter (simple category)
\p{Sc}	A currency symbol
\P{InGreek}	Any character except one in the Greek block (negation)
[\p{L}&&[^\p{Lu}]]	Any letter except an uppercase letter (subtraction)

Boundary matchers

^	The beginning of a line
$	The end of a line
\b	A word boundary
\B	A non-word boundary
\A	The beginning of the input
\G	The end of the previous match
\Z	The end of the input but for the final terminator, if any
\z	The end of the input

Greedy quantifiers

X?	X, once or not at all
X*	X, zero or more times
X+	X, one or more times
X{n}	X, exactly n times
X{n,}	X, at least n times
X{n,m}	X, at least n but not more than m times

Reluctant quantifiers

X??	X, once or not at all
X*?	X, zero or more times
X+?	X, one or more times
X{n}?	X, exactly n times
X{n,}?	X, at least n times
X{n,m}?	X, at least n but not more than m times

Possessive quantifiers

X?+	X, once or not at all
X*+	X, zero or more times
X++	X, one or more times
X{n}+	X, exactly n times
X{n,}+	X, at least n times
X{n,m}+	X, at least n but not more than m times

Logical operators

XY	X followed by Y
X|Y	Either X or Y
(X)	X, as a capturing group

Filter Examples

Here are some example settings that can be used to help filter out necessary files in a transfer:

1. Mode: Exclude

Target: File name

Phrase: *.pdf

Result of settings:

Single phrase filter that transfers all files that do not end with the '.pdf' extension

2. Mode: Exclude

Target: File path

Phrase: folder1/**/*.docx,folder3/**/*.png

Result of settings:

Two phrase filter that transfers all files expect '.docx' files found in folder1, and '.png' files found in folder3

3. Mode: Include

Target: File name

Phrase: *.mov,*.mp4

Result of settings:

Two phrase filter that transfers all files that end with the '.mov' or '.mp4' extensions

4. Mode: Include

Target: File path

Phrase: **/videoProduction/**/4K Footage/**/*.mkv

Result of settings:

Single phrase filter that transfers all '.mkv' files that exist within the '4K Footage' sub-folders contained inside the 'videoProduction' parent folder.

5. Mode: Reg Expr

Target: File name

Phrase: ".*?\.dat$"

Regular Expression Definitions:

Expression Section	Description
".*?"	Any character, 0, 1, or multiple times with reluctant matching
"\."	Followed by a period (you must escape the dot character or it will be treated as a wildcard)
"dat"	The exact characters 'dat' all lowercase
"$"	Must be at end of the line

Result of settings:

Filter that transfer only files that end with ".dat". Windows wildcard equivalent of "*.dat"

6. Mode: Reg Expr

Target: File name

Phrase: ".*?[0-9]+?.*?\.(mp3|wav)$"

Regular Expression Definitions:

Expression Section	Description
".*?"	Any character, 0, 1, or multiple times with reluctant matching
"[0-9]+?"	Any numerical digit one or more times with reluctant matching
".*?"	Any character, 0, 1, or multiple times with reluctant matching
"\."	Followed by a period (you must escape the dot character or it will be treated as a wildcard)
"(mp3|wav)"	Characters may match exactly "mp3" OR "wav"
"$"	Must be at end of the line

Result of settings:

Filter that transfer sound files (.mp3 or .wav) with numbers in the title

7. Mode: Reg Expr

Target: File name

Phrase: "^.*\.(?!tmp$)[^.]+$"

Regular Expression Definitions:

Expression Section	Description
"^.*"	Matches starting characters, 0, 1, or multiple times.
"\."	Followed by a period (you must escape the dot character or it will be treated as a wildcard)
"(?!tmp$)"	Negative lookahead that excludes files with the extension "tmp"
"[^.]+"	Match everything not eliminated by the previous negative lookahead.
"$"	Must be at end of the line

Result of settings:

Filter that transfer everything except temporary files ending in ".tmp"

8. Mode: Reg Expr

Target: File path

Phrase: "^.*\.(?!(html|js|png)$)[^.]+$"

Regular Expression Definitions:

Expression Section	Description
"^.*"	Matches starting characters, 0, 1, or multiple times.
"\."	Followed by a period (you must escape the dot character or it will be treated as a wildcard)
"(?!(html|js|png)$)"	Negative lookahead that excludes extensions that match "html", "js", or "png"
"[^.]+"	Match everything not eliminated by the previous negative lookahead.
"$"	Must be at end of the line

Result of settings:

Filter that transfers everything except files ending in ".html", ".js", or ".png"

Filter Target

This option allows you to specify the type of item that the file filter will apply to. The currently available options are as follows:

• Directory and file name: Applies the file filter to both file names and directory names.

  • Note 1: If a directory does not match the current filter settings, its contents will not be searched for other files that would have otherwise matched.

• File name: Applies the file filter to only file names. All directories that are found by the filter are automatically explored for other files that may match the current filter settings

• File path: Applies the file filter to the file path for a given file.

  • Note 1: Path-based filtering is relative to the source location currently defined by the transfer's current direction. If the task has an upload direction, then all path-based filter checks will be made relative to the current HotFolder location. If the task is a download, then all path-based filter calls will be made within the context of the currently defined remote folder. If no remote folder is currently defined, then the path based filter calls will be relative to the Server's home directory.

  • Note 2: If the task is a download task and the filter phrase starts with a leading slash, then all path based filter checks will be made within the context of the site's established home directory.

Transfer Files Based on Size

This option allows you to specify whether the HotFolder should transfer all files or only files within the specified size limits.

Transfer Files Newer than X Days

This option allows you to specify that only files newer than the given value (in days) will be transferred. This is based on the current modification time of the file, not when it was placed into the folder.

Filter Test

The filter tester will allow you to view the files that will be transferred with the given filters before applying them to the task, this is especially useful when experimenting with complicated regular expressions. Using the "Test" button will populate the text field with all of the files that will be allowed under the given filter, "Apply" will save your settings to the "Fileset" panel but will still need to be applied via the "Fileset" "Apply" button.

Transfer Cache

The file transfer cache is a history of all the files you have uploaded/downloaded from the server for this task. This is used to indicate whether a file has changed on the server and has been flagged for upload or download again. File changes are tracked using the size and the last modified timestamp. These values are updated when a change is detected and the file is redownloaded. Thus, even if you remove the files from the local directory, they will not be downloaded again unless the cache is cleared. When this cache is deleted, it will re-download all files from the server. In the case of upload tasks, only files that have been changed locally will be uploaded again.

The cache database directory is saved in the install directory as .fcdb by default. To save it somewhere else, specify the path in fchf.conf file with the following parameter key:

FC.config.derby.root=CACHEPATH

where CACHEPATH is replaced with the path of your target directory.

Clear Cache

Clears the current transfer cache entries. Files will be re-transferred even if they have previously been transferred.

Generate Cache

Generates the transfer cache without performing a transfer. This option is useful if both source and destination are already in sync. Cache generation occurs upon next task execution.

Remove File

When clicked, the user will have the ability to select a file for removal from the transfer cache. A browse dialog is presented showing the files in the source folder for this task. When a file is selected, if it is present in the transfer cache, it will be removed. If there was an error removing the file from the cache (i.e. it was not present), an error will be shown.

Always transfer directory structure (even if empty)

Empty directories are not transferred unless this is selected. If a filter hides all files in a directory, it is considered empty.

Transfer in priority sequence

This will apply a order the sequence of file transfers.

Oldest

Transfers files by last modified from oldest to newest.

Newest

Transfers files by last modified from newest to oldest.

Largest

Transfers files by size from largest to smallest.

Smallest

Transfers files by size from smallest to largest.

Synchronization Options

Delete files on destination not existing on source.

This option forces the source (HotFolder on uploads, Server on downloads) to make sure that both directories contain exactly the same fileset. This includes deletion of files which do not appear on the source.

File details (send details for Upload tasks, get details for Download tasks).

These flags are for a special configuration where two HotFolders want to synchronize files with each other with a server in between acting as a transit point (small temporary cache, not permanent file storage).

By enabling this feature, the source HotFolder will forward file listing information from the source in addition to the files themselves, allowing the destination HotFolder to download both the files and the listing and take appropriate actions.

The following options should be enabled to get this configuration setup:

On the Source HotFolder:

• "Transfer Cache" should be enabled (ensuring the same file is not sent up twice on the server)

• "Send details for Remote HotFolder synchronization" should be enabled.

On the Destination HotFolder:

• "Get remote HotFolder synchronization details" should be enabled.

• In Post Task tab, "Delete source file(s)" should be enabled.

This ensures that the destination HotFolder remain in-synch (Source HotFolder acting as the master repository), and that even through a Server acting as an intermediary, file creating and deletion events will be synchronized to the destination HotFolder.

Delete files on Destination that do not exist at source

When this item is checked, any files that exists on the destination that does not exist at the source will be deleted. In this way, the destination folder will always be kept in sync with the source folder because any new files added will be set to the destination, while anything removed will be deleted.

File Priority Settings

If this option is enabled, FileCatalyst allows you to specify which files should be transferred first. Depending on the number of files to be transferred, this setting may cause a delay before the actual transfer begins. This is a result of the file being sorted into the proper sequence.