Backblaze B2

This section assumes that the FileCatalyst server will be installed in close proximity (same geographic region) to your Backblaze B2 storage for best possible performance. All that is required is that the machine have HTTP access to the public internet.

Version Support

The following table shows which versions of FileCatalyst officially support the Backblaze B2 EFS driver.

Version Compatible Supported
3.5 - 3.7.2 Yes No
3.7.3+ Yes Yes

Prerequisites

  • The Access Key used to define a B2 file system must have the ability to list buckets. This allows the file system to be verified and for a bucket to be selected during setup.

  • Ensure that the server you install FileCatalyst server on has sufficient bandwidth for your tests.

  • Ensure your Firewall has the appropriate ports open. TCP port 21, 8000-8999, UDP ports 8000-8999 are the minimum ports to transfer files. Optionally, port 990 (SSL) and port 22 (SFTP) may be opened if you have enabled them. If you require remote administration, TCP port 12400 should be open. Note that these are all default ports. The ports you open should reflect any changes you have made to these defaults.

  • For best performance, run your FileCatalyst Server should be in the same region as your B2 storage. Try to keep network latency as low as possible.

Getting Started

  1. Install FileCatalyst Server 3.7.3 or higher.

  2. On first run, install the provided license key from your sales rep.

  3. Modify the IP masquerade address (Under Advanced/Network) to match your FileCatalyst Server Public IP address.

  4. On File Systems tab, click "New."

  5. Select "Backblaze B2" from the Type drop-down menu.

  6. Enter your Access Key, Secret Key, and enter "api.backblaze.com" for the Endpoint.

  7. Click the Test button to ensure that the credentials are entered correctly.

  8. Click the Save button.

  9. Make sure the B2 File System appears in the list.

  10. Create a new user or Edit an existing user. Under the Home Directory section (located on Account Information panel for New user, or Account Settings when editing an existing user) the File System type drop down should have an option "Backblaze B2". Select it and click Browse.

  11. Select a home directory like you normally would. Note that the top level directories are the B2 buckets. User accounts will only have READ access at the top level, and will have access to all buckets if this is set as the home directory.

You have now successfully setup the FileCatalyst Server to connect to your B2 storage.

URL Composition

When manually defining an Backblaze B2 EFS URL (i.e. not using the Browse button, or when no static File System has previously been defined) as part of a User home directory or Virtual File/Folder path, the following URL composition should be followed.

Scheme: b2

Default Endpoint: api.backblaze.com

Full Syntax: b2://<access key>:<secret key>@api.backblaze.com/<bucket>/<path>

NOTE: The access key and secret key are only required when when no File System is defined for the file system you are attempting to connect to.

Limitations

The following features are NOT supported in FileCatalyst client applications when connecting to a user account backed by Backblaze B2. Features marked with an asterisk (*) will be implemented in a future release.

  • Keep File Permissions (OSX/Linux Only).

  • Keep file modification timestamp.

  • Transfer in priority sequence - works only by creation date not modified date.

  • Creation/Modification time for directories always show (in listings) as current date since they are virtual objects.

  • HTTP uploads are not supported on B2.

  • Incremental/Deltas is not supported on B2.

  • Also see Common EFS Limitations.

Notes

Your FileCatalyst Server sees B2 as a complete file system. However in the background, it is all using the B2 REST API, hence all communication is done with HTTP. This means reading and writing is not going to be as fast as a local hard drive. There are several performance limitations when dealing with B2. Here is an idea of what you can expect:

  1. Large individual file uploads - Uploads will be fast, up to 1 Gbps write speed for individual files. Files are uploaded in chunks (using parallel HTTP uploads) in the background, and then need to be assembled by B2 at completion. This can cause a pause at the end of each transferred file. The larger the file the longer the pause, though it should not be more than a few seconds for files under 2 GB.

  2. Large individual file downloads - Downloads will be fast, up to 1 Gbps write speed for individual files. No pause after download like there is with uploads.

  3. Upload large groups of smaller files - Large groups of small files will upload more slowly than individual files, but multi-client compensates for this just like on local file systems because several files are in transit at one time. See additional note for Multi-Client.

  4. Download large groups of smaller files - Large groups of small files will download more slowly than individual files, but multi-client compensates for this just like on local file systems because several files are in transit at one time. See additional note for Multi-Client.

  5. Multi-client transfers - When executing multi-client transfers, the FileCatalyst client may try to initiate many requests in quick succession. If the External File System is unable to process the rapid influx of requests, this may result in some requests being rejected or the UI becoming unresponsive.

  6. Renaming files - Renaming is inconsistent in B2, and may not work for files larger than 20MB. It is recommended that features that use rename (i.e. transfer with temporary file name) be disabled when using B2 storage.

  7. Renaming folders - Renaming folders is not possible in FileCatalyst as they are virtual, and would require every file under them to be renamed. As per the previous limitation (renaming files), this would be extremely slow. For this reason renaming folders is currently not implemented.

  8. MD5 checksums - Calculating MD5 checksums requires all bytes of the file to be read by the FileCatalyst Server. Since the files are on B2 the entire file is transferred from B2 your FileCatalyst Server to calculate the checksum. This means MD5 calculations will take longer than they would on a local file system.

  9. TCP downloads are slow - FileCatalyst uses HTTP GET with a "range" header in the background when it performs read operations on objects. Because the TCP transfer buffers are small (only 16KB) these read operations are not very efficient, and hence downloads are slow. This will be addressed in a future release.

  10. LINK feature will not work if user is on the root path - FileCatalyst LINK requires the ability to create a directory FC_OUTBOX to store files uploaded to the server. If the user home directory is stored in the root path of the Backblaze B2 account (as opposed to inside a defined bucket), the server will not be able to create the required folder to temporarily hold files. It is recommended that the user home directory be defined inside an existing bucket.