Transfer Adapters

Git LFS servers and clients can implement and negotiate different transfer adapters. Typically, Git LFS will only define a basic transfer mode and support that. basic is simple and efficient for direct-to-storage uploads for backends that support uploading using a single PUT request.

basic Transfer Mode

External Storage basic transfer adapter

The basic_external transfer adapter is designed to facilitate LFS basic mode transfers (the default transfer mode of Git LFS) for setups in which the storage backends supports communicating directly with the Git LFS client. That is, files will be uploaded or downloaded directly from a storage service that supports HTTP PUT / GET based access, without passing through Giftless. With this adapter, Giftless will not handle any file transfers - it will only be responsible for providing the client with access to storage.

This transfer adapter works with storage adapters implementing the ExternalStorage storage interface - typically these are Cloud storage service based backends.

Streaming basic transfer adapter

The basic_streaming transfer adapter facilitates LFS basic mode transfers in which Giftless also handles object upload, download and verification requests directly. This is less scalable and typically less performant than the basic_external adapter, as all data and potentially long-running HTTP requests must be passed through Giftless and its Python runtime. However, in some situations this may be preferable to direct-to-storage HTTP requests.

basic_streaming supports local storage, and also streaming requests from some Cloud storage service backends such as Azure and Google Cloud - although these tend to also support the basic_external transfer adapter.

Multipart Transfer Mode

To support more complex, and especially multi-part uploads (uploads done using more than one HTTP request, each with a different part of a large file) directly to backends that support that, Giftless adds support for a non-standard multipart-basic transfer mode.

NOTE: basic-multipart is a non-standard transfer mode, and will not be supported by most Git LFS clients; For a Python implementation of a Git LFS client library that does, see giftless-client.

Enabling Multipart Transfer Mode

You can enable multipart transfers by adding the following lines to your Giftless config file:

TRANSFER_ADAPTERS:
  # Add the following lines:
  multipart-basic:
    factory: giftless.transfer.multipart:factory
    options:
      storage_class: giftless.storage.azure:AzureBlobsStorage
      storage_options:
        connection_string: "somesecretconnectionstringhere"
        container_name: my-multipart-storage

You must specify a storage_class that supports multipart transfers (implements the MultipartStorage interface). Currently, these are:

• giftless.storage.azure:AzureBlobsStorage - Azure Blob Storage

The following additional options are available for multipart-basic transfer adapter:

• action_lifetime - The maximal lifetime in seconds for signed multipart actions; Because multipart uploads tend to be of very large files and can easily take hours to complete, we recommend setting this to a few hours; The default is 6 hours.

• max_part_size - Maximal length in bytes of a single part upload. The default is 10MB.

See the specific storage adapter for additional backend-specific configuration options to be added under storage_options.