Gateway

Abstract

The gateway service is responsible for passing requests to the storage providers. Other services never talk to the storage providers directly but will always send their requests via the gateway service.

Caching
Service Endpoints
Storage Registry
Example Yaml Config

Caching

The gateway service is using caching as it is highly frequented with the same requests. As of now it uses two different caches:

the provider cache is caching requests to list or get storage providers.
the create home cache is caching requests to create personal spaces (as they only need to be executed once).

Both caches can be configured via the OCIS_CACHE_* envvars (or GATEWAY_PROVIDER_CACHE_* and GATEWAY_CREATE_HOME_CACHE_* respectively). See the envvar section for details.

Use OCIS_CACHE_STORE (GATEWAY_PROVIDER_CACHE_STORE, GATEWAY_CREATE_HOME_CACHE_STORE) to define the type of cache to use:

memory: Basic in-memory store and the default.
redis-sentinel: Stores data in a configured Redis Sentinel cluster.
nats-js-kv: Stores data using key-value-store feature of nats jetstream
noop: Stores nothing. Useful for testing. Not recommended in production environments.

Other store types may work but are not supported currently.

Note: The service can only be scaled if not using memory store and the stores are configured identically over all instances!

Note that if you have used one of the deprecated stores, you should reconfigure to one of the supported ones as the deprecated stores will be removed in a later version.

Store specific notes:

When using redis-sentinel, the Redis master to use is configured via e.g. OCIS_CACHE_STORE_NODES in the form of <sentinel-host>:<sentinel-port>/<redis-master> like 10.10.0.200:26379/mymaster.
When using nats-js-kv it is recommended to set OCIS_CACHE_STORE_NODES to the same value as OCIS_EVENTS_ENDPOINT. That way the cache uses the same nats instance as the event bus.
When using the nats-js-kv store, it is possible to set OCIS_CACHE_DISABLE_PERSISTENCE to instruct nats to not persist cache data on disc.

Service Endpoints

IMPORTANT
This functionality is currently highly experimental and intended for testing only! There are known bugs that need to be sorted out like not removing sockets when a service ends.

The gateway acts as a proxy for other CS3 services. As such it has to forward requests to a lot of services and needs to establish connections by looking up the IP address using the service registry. Instead of using the service registry each endpoint can also be configured to use the grpc dns:// or kubernetes:// URLs, which might be useful when running in kubernetes.

For a local single node deployment you might want to use unix: sockets as shown below. Using unix sockets will reduce the amount of service lookups and omit the TCP stack. For now, this is experimental and the services do not delete the socket on shutdown. PRs welcome.

The scheme for this setup is the following. Note that there is, except storage, always a service and a gateway envvar triple:

envvar	default	alternative
OCIS_GRPC_PROTOCOL or `<service>`_GRPC_PROTOCOL	tcp	unix
`<service>`_GRPC_ADDR	127.0.0.1:`<port>`	/var/run/ocis/`<service>`.sock
GATEWAY_`<service>`_ENDPOINT	com.owncloud.api.`<service>`	unix:/var/run/ocis/`<service>`.sock dns: … kubernetes: …

USERS_GRPC_PROTOCOL=unix"
USERS_GRPC_ADDR=/var/run/ocis/users.sock"
GATEWAY_USERS_ENDPOINT=unix:/var/run/ocis/users.sock"

GROUPS_GRPC_PROTOCOL=unix"
GROUPS_GRPC_ADDR=/var/run/ocis/groups.sock"
GATEWAY_GROUPS_ENDPOINT=unix:/var/run/ocis/groups.sock"

AUTH_APP_GRPC_PROTOCOL=unix"
AUTH_APP_GRPC_ADDR=/var/run/ocis/auth-app.sock"
GATEWAY_AUTH_APP_ENDPOINT=unix:/var/run/ocis/auth-app.sock"

AUTH_BASIC_GRPC_PROTOCOL=unix"
AUTH_BASIC_GRPC_ADDR=/var/run/ocis/auth-basic.sock"
GATEWAY_AUTH_BASIC_ENDPOINT=unix:/var/run/ocis/auth-basic.sock"

AUTH_MACHINE_GRPC_PROTOCOL=unix"
AUTH_MACHINE_GRPC_ADDR=/var/run/ocis/auth-machine.sock"
GATEWAY_AUTH_MACHINE_ENDPOINT=unix:/var/run/ocis/auth-machine.sock"

AUTH_SERVICE_GRPC_PROTOCOL=unix"
AUTH_SERVICE_GRPC_ADDR=/var/run/ocis/auth-service.sock"
GATEWAY_AUTH_SERVICE_ENDPOINT=unix:/var/run/ocis/auth-service.sock"

STORAGE_PUBLIC_LINK_GRPC_PROTOCOL=unix"
STORAGE_PUBLIC_LINK_GRPC_ADDR=/var/run/ocis/storage-public-link.sock"
GATEWAY_STORAGE_PUBLIC_LINK_ENDPOINT=unix:/var/run/ocis/storage-public-link.sock"

STORAGE_USERS_GRPC_PROTOCOL=unix"
STORAGE_USERS_GRPC_ADDR=/var/run/ocis/storage-users.sock"
GATEWAY_STORAGE_USERS_ENDPOINT=unix:/var/run/ocis/storage-users.sock"
// graph sometimes bypasses the gateway so we need to configure the socket here as wel
GRAPH_SPACES_STORAGE_USERS_ADDRESS=unix:/var/run/ocis/storage-users.sock"

STORAGE_SHARES_GRPC_PROTOCOL=unix"
STORAGE_SHARES_GRPC_ADDR=/var/run/ocis/storage-shares.sock"
GATEWAY_STORAGE_SHARES_ENDPOINT=unix:/var/run/ocis/storage-shares.sock"

APP_REGISTRY_GRPC_PROTOCOL=unix"
APP_REGISTRY_GRPC_ADDR=/var/run/ocis/app-registry.sock"
GATEWAY_APP_REGISTRY_ENDPOINT=unix:/var/run/ocis/app-registry.sock"

OCM_GRPC_PROTOCOL=unix"
OCM_GRPC_ADDR=/var/run/ocis/ocm.sock"
GATEWAY_OCM_ENDPOINT=unix:/var/run/ocis/ocm.sock"

// storage related
SETTINGS_STORAGE_GATEWAY_GRPC_ADDR="unix:/var/run/ocis/storage-system.sock"
SETTINGS_STORAGE_GRPC_ADDR="unix:/var/run/ocis/storage-system.sock"
STORAGE_SYSTEM_GRPC_PROTOCOL="unix"
STORAGE_SYSTEM_GRPC_ADDR="/var/run/ocis/storage-system.sock"
SHARING_USER_CS3_PROVIDER_ADDR="unix:/var/run/ocis/storage-system.sock"
SHARING_USER_JSONCS3_PROVIDER_ADDR="unix:/var/run/ocis/storage-system.sock"
SHARING_PUBLIC_CS3_PROVIDER_ADDR="unix:/var/run/ocis/storage-system.sock"
SHARING_PUBLIC_JSONCS3_PROVIDER_ADDR="unix:/var/run/ocis/storage-system.sock"

Storage Registry

In order to add another storage provider the CS3 storage registry that is running as part of the CS3 gateway hes to be made aware of it. The easiest cleanest way to do it is to set GATEWAY_STORAGE_REGISTRY_CONFIG_JSON=/path/to/storages.json and list all storage providers like this:

{
  "com.owncloud.api.storage-users": {
    "providerid": "{storage-users-mount-uuid}",
    "spaces": {
      "personal": {
        "mount_point":   "/users",
        "path_template": "/users/{{.Space.Owner.Id.OpaqueId}}"
      },
      "project": {
        "mount_point":   "/projects",
        "path_template": "/projects/{{.Space.Name}}"
      }
    }
  },
  "com.owncloud.api.storage-shares": {
    "providerid": "a0ca6a90-a365-4782-871e-d44447bbc668",
    "spaces": {
      "virtual": {
        "mount_point": "/users/{{.CurrentUser.Id.OpaqueId}}/Shares"
      },
      "grant": {
        "mount_point": "."
      },
      "mountpoint": {
        "mount_point":   "/users/{{.CurrentUser.Id.OpaqueId}}/Shares",
        "path_template": "/users/{{.CurrentUser.Id.OpaqueId}}/Shares/{{.Space.Name}}"
      }
    }
  },
  "com.owncloud.api.storage-publiclink": {
    "providerid": "7993447f-687f-490d-875c-ac95e89a62a4",
    "spaces": {
      "grant": {
        "mount_point": "."
      },
      "mountpoint": {
        "mount_point":   "/public",
        "path_template": "/public/{{.Space.Root.OpaqueId}}"
      }
    }
  },
  "com.owncloud.api.ocm": {
    "providerid": "89f37a33-858b-45fa-8890-a1f2b27d90e1",
    "spaces": {
      "grant": {
        "mount_point": "."
      },
      "mountpoint": {
        "mount_point":   "/ocm",
        "path_template": "/ocm/{{.Space.Root.OpaqueId}}"
      }
    }
  },
  "com.owncloud.api.storage-hello": {
    "providerid": "hello-storage-id",
    "spaces": {
      "project": {
        "mount_point":   "/hello",
        "path_template": "/hello/{{.Space.Name}}"
      }
    }
  }
}

In the above replace {storage-users-mount-uuid} with the mount UUID that was generated for the storage-users service. You can find it in the config.yaml generated on by ocis init. The last entry com.owncloud.api.storage-hello and its providerid "hello-storage-id" are an example for in additional storage provider, in this case running hellofs, an example minimal storage driver.

Example Yaml Config

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72


# Autogenerated
# Filename: gateway-config-example.yaml

tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""
log:
  level: ""
  pretty: false
  color: false
  file: ""
debug:
  addr: 127.0.0.1:9143
  token: ""
  pprof: false
  zpages: false
grpc:
  addr: 127.0.0.1:9142
  tls: null
  protocol: tcp
token_manager:
  jwt_secret: ""
reva:
  address: com.owncloud.api.gateway
  tls:
    mode: ""
    cacert: ""
skip_user_groups_in_token: false
commit_share_to_storage_grant: true
share_folder_name: Shares
disable_home_creation_on_login: true
transfer_secret: ""
transfer_expires: 86400
cache:
  provider_cache_store: noop
  provider_cache_nodes:
  - 127.0.0.1:9233
  provider_cache_database: cache-providers
  provider_cache_ttl: 5m0s
  provider_cache_disable_persistence: false
  provider_cache_auth_username: ""
  provider_cache_auth_password: ""
  create_home_cache_store: memory
  create_home_cache_nodes:
  - 127.0.0.1:9233
  create_home_cache_database: cache-createhome
  create_home_cache_ttl: 5m0s
  create_home_cache_disable_persistence: false
  create_home_cache_auth_username: ""
  create_home_cache_auth_password: ""
frontend_public_url: https://localhost:9200
users_endpoint: com.owncloud.api.users
groups_endpoint: com.owncloud.api.groups
permissions_endpoint: com.owncloud.api.settings
sharing_endpoint: com.owncloud.api.sharing
auth_app_endpoint: com.owncloud.api.auth-app
auth_basic_endpoint: com.owncloud.api.auth-basic
auth_bearer_endpoint: ""
auth_machine_endpoint: com.owncloud.api.auth-machine
auth_service_endpoint: com.owncloud.api.auth-service
storage_public_link_endpoint: com.owncloud.api.storage-publiclink
storage_users_endpoint: com.owncloud.api.storage-users
storage_shares_endpoint: com.owncloud.api.storage-shares
app_registry_endpoint: com.owncloud.api.app-registry
ocm_endpoint: com.owncloud.api.ocm
storage_registry:
  driver: spaces
  rules: []
  json: ""
  storage_users_mount_id: ""

Environment Variables

Name	Type	Default Value	Description
OCIS_TRACING_ENABLED GATEWAY_TRACING_ENABLED	bool	false	Activates tracing.
OCIS_TRACING_TYPE GATEWAY_TRACING_TYPE	string		The type of tracing. Defaults to ‘’, which is the same as ‘jaeger’. Allowed tracing types are ‘jaeger’ and ’’ as of now.
OCIS_TRACING_ENDPOINT GATEWAY_TRACING_ENDPOINT	string		The endpoint of the tracing agent.
OCIS_TRACING_COLLECTOR GATEWAY_TRACING_COLLECTOR	string		The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.
OCIS_LOG_LEVEL GATEWAY_LOG_LEVEL	string		The log level. Valid values are: ‘panic’, ‘fatal’, ’error’, ‘warn’, ‘info’, ‘debug’, ’trace’.
OCIS_LOG_PRETTY GATEWAY_LOG_PRETTY	bool	false	Activates pretty log output.
OCIS_LOG_COLOR GATEWAY_LOG_COLOR	bool	false	Activates colorized log output.
OCIS_LOG_FILE GATEWAY_LOG_FILE	string		The path to the log file. Activates logging to this file if set.
GATEWAY_DEBUG_ADDR	string	127.0.0.1:9143	Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.
GATEWAY_DEBUG_TOKEN	string		Token to secure the metrics endpoint.
GATEWAY_DEBUG_PPROF	bool	false	Enables pprof, which can be used for profiling.
GATEWAY_DEBUG_ZPAGES	bool	false	Enables zpages, which can be used for collecting and viewing in-memory traces.
OCIS_GATEWAY_GRPC_ADDR GATEWAY_GRPC_ADDR	string	127.0.0.1:9142	The bind address of the GRPC service.
OCIS_GRPC_PROTOCOL GATEWAY_GRPC_PROTOCOL	string	tcp	The transport protocol of the GRPC service.
OCIS_JWT_SECRET GATEWAY_JWT_SECRET	string		The secret to mint and validate jwt tokens.
OCIS_REVA_GATEWAY	string	com.owncloud.api.gateway	The CS3 gateway endpoint.
OCIS_GRPC_CLIENT_TLS_MODE	string		TLS mode for grpc connection to the go-micro based grpc services. Possible values are ‘off’, ‘insecure’ and ‘on’. ‘off’: disables transport security for the clients. ‘insecure’ allows using transport security, but disables certificate verification (to be used with the autogenerated self-signed certificates). ‘on’ enables transport security, including server certificate verification.
OCIS_GRPC_CLIENT_TLS_CACERT	string		Path/File name for the root CA certificate (in PEM format) used to validate TLS server certificates of the go-micro based grpc services.
GATEWAY_SKIP_USER_GROUPS_IN_TOKEN	bool	false	Disables the loading of user’s group memberships from the reva access token.
GATEWAY_COMMIT_SHARE_TO_STORAGE_GRANT	bool	true	Commit shares to storage grants. This grants access to shared resources for the share receiver directly on the storage.
GATEWAY_SHARE_FOLDER_NAME	string	Shares	Name of the share folder in users’ home space.
GATEWAY_DISABLE_HOME_CREATION_ON_LOGIN	bool	true	Disable creation of the home space on login.
OCIS_TRANSFER_SECRET	string		The storage transfer secret.
GATEWAY_TRANSFER_EXPIRES	int	86400	Expiry for the gateway tokens.
OCIS_CACHE_STORE GATEWAY_PROVIDER_CACHE_STORE	string	noop	The type of the cache store. Supported values are: ‘memory’, ‘redis-sentinel’, ’nats-js-kv’, ’noop’. See the text description for details.
OCIS_CACHE_STORE_NODES GATEWAY_PROVIDER_CACHE_STORE_NODES	[]string	[127.0.0.1:9233]	A list of nodes to access the configured store. This has no effect when ‘memory’ store is configured. Note that the behaviour how nodes are used is dependent on the library of the configured store. See the Environment Variable Types description for more details.
OCIS_CACHE_DATABASE	string	cache-providers	The database name the configured store should use.
OCIS_CACHE_TTL GATEWAY_PROVIDER_CACHE_TTL	Duration	5m0s	Default time to live for user info in the cache. Only applied when access tokens has no expiration. See the Environment Variable Types description for more details.
OCIS_CACHE_DISABLE_PERSISTENCE GATEWAY_PROVIDER_CACHE_DISABLE_PERSISTENCE	bool	false	Disables persistence of the provider cache. Only applies when store type ’nats-js-kv’ is configured. Defaults to false.
OCIS_CACHE_AUTH_USERNAME GATEWAY_PROVIDER_CACHE_AUTH_USERNAME	string		The username to use for authentication. Only applies when store type ’nats-js-kv’ is configured.
OCIS_CACHE_AUTH_PASSWORD GATEWAY_PROVIDER_CACHE_AUTH_PASSWORD	string		The password to use for authentication. Only applies when store type ’nats-js-kv’ is configured.
OCIS_CACHE_STORE GATEWAY_CREATE_HOME_CACHE_STORE	string	memory	The type of the cache store. Supported values are: ‘memory’, ‘redis-sentinel’, ’nats-js-kv’, ’noop’. See the text description for details.
OCIS_CACHE_STORE_NODES GATEWAY_CREATE_HOME_CACHE_STORE_NODES	[]string	[127.0.0.1:9233]	A list of nodes to access the configured store. This has no effect when ‘memory’ store is configured. Note that the behaviour how nodes are used is dependent on the library of the configured store. See the Environment Variable Types description for more details.
OCIS_CACHE_DATABASE	string	cache-createhome	The database name the configured store should use.
OCIS_CACHE_TTL GATEWAY_CREATE_HOME_CACHE_TTL	Duration	5m0s	Default time to live for user info in the cache. Only applied when access tokens has no expiration. See the Environment Variable Types description for more details.
OCIS_CACHE_DISABLE_PERSISTENCE GATEWAY_CREATE_HOME_CACHE_DISABLE_PERSISTENCE	bool	false	Disables persistence of the create home cache. Only applies when store type ’nats-js-kv’ is configured. Defaults to false.
OCIS_CACHE_AUTH_USERNAME GATEWAY_CREATE_HOME_CACHE_AUTH_USERNAME	string		The username to use for authentication. Only applies when store type ’nats-js-kv’ is configured.
OCIS_CACHE_AUTH_PASSWORD GATEWAY_CREATE_HOME_CACHE_AUTH_PASSWORD	string		The password to use for authentication. Only applies when store type ’nats-js-kv’ is configured.
OCIS_URL GATEWAY_FRONTEND_PUBLIC_URL	string	https://localhost:9200	The public facing URL of the oCIS frontend.
GATEWAY_USERS_ENDPOINT	string	com.owncloud.api.users	The endpoint of the users service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_GROUPS_ENDPOINT	string	com.owncloud.api.groups	The endpoint of the groups service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_PERMISSIONS_ENDPOINT	string	com.owncloud.api.settings	The endpoint of the permissions service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_SHARING_ENDPOINT	string	com.owncloud.api.sharing	The endpoint of the shares service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_AUTH_APP_ENDPOINT	string	com.owncloud.api.auth-app	The endpoint of the auth-app service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_AUTH_BASIC_ENDPOINT	string	com.owncloud.api.auth-basic	The endpoint of the auth-basic service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_AUTH_BEARER_ENDPOINT	string		The endpoint of the auth-bearer service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_AUTH_MACHINE_ENDPOINT	string	com.owncloud.api.auth-machine	The endpoint of the auth-machine service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_AUTH_SERVICE_ENDPOINT	string	com.owncloud.api.auth-service	The endpoint of the auth-service service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_STORAGE_PUBLIC_LINK_ENDPOINT	string	com.owncloud.api.storage-publiclink	The endpoint of the storage-publiclink service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_STORAGE_USERS_ENDPOINT	string	com.owncloud.api.storage-users	The endpoint of the storage-users service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_STORAGE_SHARES_ENDPOINT	string	com.owncloud.api.storage-shares	The endpoint of the storage-shares service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_APP_REGISTRY_ENDPOINT	string	com.owncloud.api.app-registry	The endpoint of the app-registry service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_OCM_ENDPOINT	string	com.owncloud.api.ocm	The endpoint of the ocm service. Can take a service name or a gRPC URI with the dns, kubernetes or unix protocol.
GATEWAY_STORAGE_REGISTRY_DRIVER	string	spaces	The driver name of the storage registry to use.
GATEWAY_STORAGE_REGISTRY_RULES	[]string	[]	The rules for the storage registry. See the Environment Variable Types description for more details.
GATEWAY_STORAGE_REGISTRY_CONFIG_JSON	string		Additional configuration for the storage registry in json format.
GATEWAY_STORAGE_USERS_MOUNT_ID	string		Mount ID of this storage. Admins can set the ID for the storage in this config option manually which is then used to reference the storage. Any reasonable long string is possible, preferably this would be an UUIDv4 format.