ownCloud
Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage
  1. ownCloud
  2. /
  3. Services
  4. /
  5. Thumbnails
Edit page

Thumbnails

Abstract

The thumbnails service provides methods to generate thumbnails for various files and resolutions based on requests. It retrieves the sources at the location where the user files are stored and saves the thumbnails where system files are stored. Those locations have defaults but can be manually defined via environment variables.

Table of Contents

File Locations Overview

The relevant environment variables defining file locations are:

  • (1) OCIS_BASE_DATA_PATH
  • (2) STORAGE_USERS_OCIS_ROOT
  • (3) THUMBNAILS_FILESYSTEMSTORAGE_ROOT

(1) … Having a default set by the Infinite Scale code, but if defined, used as base path for other services. (2) … Source files, defaults to (1) plus path component, but can be freely defined if required. (3) … Target files, defaults to (1) plus path component, but can be freely defined if required.

For details and defaults for these environment variables see the ocis admin documentation.

Thumbnail Location

It may be beneficial to define the location of the thumbnails to be other than the default (with system files). This is due the fact that storing thumbnails can consume a lot of space over time which not necessarily needs to reside on the same partition or mount or expensive drives.

Thumbnail Source File Types

Thumbnails can be generated from the following source file types:

  • png
  • jpg
  • gif
  • tiff
  • bmp
  • txt

The thumbnail service retrieves source files using the information provided by the backend. The Linux backend identifies source files usually based on the extension.

If a file type was not properly assigned or the type identification failed, thumbnail generation will fail and an error will be logged.

Thumbnail Target File Types

Thumbnails can either be generated as png, jpg or gif files. These types are hardcoded and no other types can be requested. A requestor, like another service or a client, can request one of the available types to be generated. If more than one type is required, each type must be requested individually.

Thumbnail Query String Parameters

Clients can request thumbnail previews for files by adding ?preview=1 to the file URL. Requests for files with no thumbnail available respond with HTTP status 404.

The following query parameters are supported:

Parameter Required Default Value Description
preview YES 1 generates preview
x YES first x-value configured in THUMBNAILS_RESOLUTIONS horizontal target size
y YES first y-value configured in THUMBNAILS_RESOLUTIONS vertical target size
scalingup NO 0 prevents up-scaling of small images
a NO 1 aspect ratio
c NO Caching string Clients should send the etag, so they get a fresh thumbnail after a file change
processor NO resize for gifs and thumbnail for all others preferred thumbnail processor

Thumbnail Resolution

Various resolutions can be defined via THUMBNAILS_RESOLUTIONS. A requestor can request any arbitrary resolution and the thumbnail service will use the one closest to the requested resolution. If more than one resolution is required, each resolution must be requested individually.

Example:

Requested: 18x12
Available: 30x20, 15x10, 9x6
Returned: 15x10

Thumbnail Processors

Normally, an image might get cropped when creating a preview, depending on the aspect ratio of the original image. This can have negative impacts on previews as only a part of the image will be shown. When using an optional processor in the request, cropping can be avoided by defining on how the preview image generation will be done. The following processors are available:

  • resize resizes the image to the specified width and height and returns the transformed image. If one of width or height is 0, the image aspect ratio is preserved.
  • fit scales down the image to fit the specified maximum width and height and returns the transformed image.
  • fill: creates an image with the specified dimensions and fills it with the scaled source image. To achieve the correct aspect ratio without stretching, the source image will be cropped.
  • thumbnail scales the image up or down, crops it to the specified width and height and returns the transformed image.

To apply one of those, a query parameter has to be added to the request, like ?processor=fit. If no query parameter or processor is added, the default behaviour applies which is resize for gifs and thumbnail for all others.

Deleting Thumbnails

As of now, there is no automated thumbnail deletion. This is especially true when a source file gets deleted or moved. This situation will be solved at a later stage. For the time being, if you run short on physical thumbnails space, you have to manually delete the thumbnail store to free space. Thumbnails will then be recreated on request.

Memory Considerations

Since source files need to be loaded into memory when generating thumbnails, large source files could potentially crash this service if there is insufficient memory available. For bigger instances when using container orchestration deployment methods, this service can be dedicated to its own server(s) with more memory. To have more control over memory (and CPU) consumption the maximum number of concurrent requests can be limited by setting the environment variable THUMBNAILS_MAX_CONCURRENT_REQUESTS. The default value is 0 which does not apply any restrictions to the number of concurrent requests. As soon as the number of concurrent requests is reached any further request will be responded with 429/Too Many Requests and the client can retry at a later point in time.

Thumbnails and SecureView

If a resource is shared using SecureView, the share reciever will get a 403 (forbidden) response when requesting a thumbnail. The requesting client needs to decide what to show and usually a placeholder thumbnail is used.

Using libvips for Thumbnail Generation

To improve performance and to support a wider range of images formats, the thumbnails service is able to utilize the libvips library for thumbnail generation. Support for libvips needs to be enabled at buildtime and has a couple of implications:

  • With libvips support enabled, it is not possible to create a statically linked ocis binary.
  • Therefore, the libvips shared libraries need to be available at runtime in the same release that was used to build the ocis binary.
  • When using the ocis docker images, the libvips shared libraries are included in the image and are correctly embedded.

Support of libvips is disabled by default. To enable it, make sure libvips and its buildtime dependencies are installed in your build environment. For macOS users, add the build time dependencies via:

brew install vips pkg-config
export PKG_CONFIG_PATH="/usr/local/opt/libffi/lib/pkgconfig"

Then you just need to set the ENABLE_VIPS variable on the make command:

make -C ocis build ENABLE_VIPS=1

Or include the enable_vips build tag in the go build command:

go build -tags enable_vips -o ocis -o bin/ocis ./cmd/ocis

When building a docker image using the Dockerfile in the top-level directory of ocis, libvips support is enabled and the libvips shared libraries are included in the resulting docker image.

Example Yaml Config 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

# Autogenerated
# Filename: thumbnails-config-example.yaml

tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""
log:
  level: ""
  pretty: false
  color: false
  file: ""
debug:
  addr: 127.0.0.1:9189
  token: ""
  pprof: false
  zpages: false
grpc:
  addr: 127.0.0.1:9185
  tls: null
  max_concurrent_requests: 0
http:
  addr: 127.0.0.1:9186
  tls:
    enabled: false
    cert: ""
    key: ""
  root: /thumbnails
  cors:
    allow_origins:
    - '*'
    allow_methods:
    - GET
    - POST
    - PUT
    - PATCH
    - DELETE
    - OPTIONS
    allow_headers:
    - Authorization
    - Origin
    - Content-Type
    - Accept
    - X-Requested-With
    - X-Request-Id
    - Cache-Control
    allow_credentials: true
grpc_client_tls: null
thumbnail:
  resolutions:
  - 16x16
  - 32x32
  - 64x64
  - 128x128
  - 1080x1920
  - 1920x1080
  - 2160x3840
  - 3840x2160
  - 4320x7680
  - 7680x4320
  filesystem_storage:
    root_directory: /var/lib/ocis/thumbnails
  webdav_allow_insecure: false
  cs3_allow_insecure: false
  reva_gateway: com.owncloud.api.gateway
  font_map_file: ""
  transfer_secret: ""
  data_endpoint: http://127.0.0.1:9186/thumbnails/data
  max_input_width: 7680
  max_input_height: 7680
  max_input_image_file_size: 50MB

Environment Variables

Name Type Default Value Description
OCIS_TRACING_ENABLED
THUMBNAILS_TRACING_ENABLED		 bool false Activates tracing.
OCIS_TRACING_TYPE
THUMBNAILS_TRACING_TYPE		 string The type of tracing. Defaults to ‘’, which is the same as ‘jaeger’. Allowed tracing types are ‘jaeger’ and ’’ as of now.
OCIS_TRACING_ENDPOINT
THUMBNAILS_TRACING_ENDPOINT		 string The endpoint of the tracing agent.
OCIS_TRACING_COLLECTOR
THUMBNAILS_TRACING_COLLECTOR		 string The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.
OCIS_LOG_LEVEL
THUMBNAILS_LOG_LEVEL		 string The log level. Valid values are: ‘panic’, ‘fatal’, ’error’, ‘warn’, ‘info’, ‘debug’, ’trace’.
OCIS_LOG_PRETTY
THUMBNAILS_LOG_PRETTY		 bool false Activates pretty log output.
OCIS_LOG_COLOR
THUMBNAILS_LOG_COLOR		 bool false Activates colorized log output.
OCIS_LOG_FILE
THUMBNAILS_LOG_FILE		 string The path to the log file. Activates logging to this file if set.
THUMBNAILS_DEBUG_ADDR string 127.0.0.1:9189 Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.
THUMBNAILS_DEBUG_TOKEN string Token to secure the metrics endpoint.
THUMBNAILS_DEBUG_PPROF bool false Enables pprof, which can be used for profiling.
THUMBNAILS_DEBUG_ZPAGES bool false Enables zpages, which can be used for collecting and viewing in-memory traces.
THUMBNAILS_GRPC_ADDR string 127.0.0.1:9185 The bind address of the GRPC service.
THUMBNAILS_MAX_CONCURRENT_REQUESTS int 0 Number of maximum concurrent thumbnail requests. Default is 0 which is unlimited.
THUMBNAILS_HTTP_ADDR string 127.0.0.1:9186 The bind address of the HTTP service.
OCIS_HTTP_TLS_ENABLED bool false Activates TLS for the http based services using the server certifcate and key configured via OCIS_HTTP_TLS_CERTIFICATE and OCIS_HTTP_TLS_KEY. If OCIS_HTTP_TLS_CERTIFICATE is not set a temporary server certificate is generated - to be used with PROXY_INSECURE_BACKEND=true.
OCIS_HTTP_TLS_CERTIFICATE string Path/File name of the TLS server certificate (in PEM format) for the http services.
OCIS_HTTP_TLS_KEY string Path/File name for the TLS certificate key (in PEM format) for the server certificate to use for the http services.
THUMBNAILS_HTTP_ROOT string /thumbnails Subdirectory that serves as the root for this HTTP service.
OCIS_CORS_ALLOW_ORIGINS
THUMBNAILS_CORS_ALLOW_ORIGINS		 []string [*] A list of allowed CORS origins. See following chapter for more details: Access-Control-Allow-Origin at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin. See the Environment Variable Types description for more details.
OCIS_CORS_ALLOW_METHODS
THUMBNAILS_CORS_ALLOW_METHODS		 []string [GET POST PUT PATCH DELETE OPTIONS] A list of allowed CORS methods. See following chapter for more details: Access-Control-Request-Method at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Request-Method. See the Environment Variable Types description for more details.
OCIS_CORS_ALLOW_HEADERS
THUMBNAILS_CORS_ALLOW_HEADERS		 []string [Authorization Origin Content-Type Accept X-Requested-With X-Request-Id Cache-Control] A list of allowed CORS headers. See following chapter for more details: Access-Control-Request-Headers at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Request-Headers. See the Environment Variable Types description for more details.
OCIS_CORS_ALLOW_CREDENTIALS
THUMBNAILS_CORS_ALLOW_CREDENTIALS		 bool true Allow credentials for CORS.See following chapter for more details: Access-Control-Allow-Credentials at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Credentials.
THUMBNAILS_RESOLUTIONS []string [16x16 32x32 64x64 128x128 1080x1920 1920x1080 2160x3840 3840x2160 4320x7680 7680x4320] The supported list of target resolutions in the format WidthxHeight like 32x32. You can define any resolution as required. See the Environment Variable Types description for more details.
THUMBNAILS_FILESYSTEMSTORAGE_ROOT string /var/lib/ocis/thumbnails The directory where the filesystem storage will store the thumbnails. If not defined, the root directory derives from $OCIS_BASE_DATA_PATH/thumbnails.
OCIS_INSECURE
THUMBNAILS_WEBDAVSOURCE_INSECURE		 bool false Ignore untrusted SSL certificates when connecting to the webdav source.
OCIS_INSECURE
THUMBNAILS_CS3SOURCE_INSECURE		 bool false Ignore untrusted SSL certificates when connecting to the CS3 source.
OCIS_REVA_GATEWAY string com.owncloud.api.gateway CS3 gateway used to look up user metadata
THUMBNAILS_TXT_FONTMAP_FILE string The path to a font file for txt thumbnails.
THUMBNAILS_TRANSFER_TOKEN string The secret to sign JWT to download the actual thumbnail file.
THUMBNAILS_DATA_ENDPOINT string http://127.0.0.1:9186/thumbnails/data The HTTP endpoint where the actual thumbnail file can be downloaded.
THUMBNAILS_MAX_INPUT_WIDTH int 7680 The maximum width of an input image which is being processed.
THUMBNAILS_MAX_INPUT_HEIGHT int 7680 The maximum height of an input image which is being processed.
THUMBNAILS_MAX_INPUT_IMAGE_FILE_SIZE string 50MB The maximum file size of an input image which is being processed. Usable common abbreviations: [KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB, EB, EiB], example: 2GB.
gdoc_arrow_left_alt Service Configuration Service Configuration gdoc_arrow_right_alt