ownCloud / oCIS - ownCloud Infinite Scale Edit this page

oCIS - ownCloud Infinite Scale

ownCloud Infinite Scale

Welcome to oCIS, the modern file-sync and share platform, which is based on our knowledge and experience with the PHP based ownCloud server.

oCIS microservice runtime

The oCIS runtime allows us to dynamically manage services running in a single process. We use suture to create a supervisor tree that starts each service in a dedicated goroutine. By default oCIS will start all built-in oCIS extensions in a single process. Individual services can be moved to other nodes to scale-out and meet specific performance requirements. A go-micro based registry allows services in multiple nodes to form a distributed microservice architecture.

oCIS extensions

Every oCIS extension uses ocis-pkg, which implements the go-micro interfaces for servers to register and clients to lookup nodes with a service registry. We are following the 12 Factor methodology with oCIS. The uniformity of services also allows us to use the same command, logging and configuration mechanism. Configurations are forwarded from the oCIS runtime to the individual extensions.

go-micro

While the go-micro framework provides abstractions as well as implementations for the different components in a microservice architecture, it uses a more developer focused runtime philosophy: It is used to download services from a repo, compile them on the fly and start them as individual processes. For oCIS we decided to use a more admin friendly runtime: You can download a single binary and start the contained oCIS extensions with a single bin/ocis server. This also makes packaging easier.

We use ocis-pkg to configure the default implementations for the go-micro grpc server, client and mdns registry, swapping them out as needed, eg. to use the kubernetes registry plugin.

REVA

A lot of embedded services in oCIS are built upon the REVA runtime. We decided to bundle some of the CS3 services to logically group them. A home storage provider, which is dealing with metadata, and the corresponding data provider, which is dealing with up and download, are one example. The frontend with the oc flavoured webdav, ocs handlers and a datagateway are another.

Protocol driven development

Interacting with oCIS involves a multitude af APIs. The server and all clients rely on OpenID Connect for authentication. The embedded konnectd can be replaced with any other OpenID Connect Identity Provider. Clients use the WebDAV based oc sync protocol to manage files and folders, ocs to manage shares and TUS to upload files in a resumable way. On the server side REVA is the reference implementation of the CS3 apis which is defined using protobuf. By embedding glauth, oCIS provides a read-only LDAP interface to make accounts, including guests available to firewalls and other systems. In the future, we are looking into the Microsoft Graph API, which is based on odata, as a well defined REST/JSON dialect for the existing endpoints.

Acceptance test suite

We run a huge test suite, which originated in ownCloud 10 and continues to grow. A detailed description can be found in the developer docs for testing.

Architecture Overview

Running bin/ocis server will start the below services, all of which can be scaled and deployed on a single node or in a cloud native environment, as needed.

Storage
Storage
ocis
ocis
example extension
example extension
hello
hello
backend
backend
staticweb ui
staticweb ui
identity management
identity management
accounts
accounts
glauth (ldap)
glauth (ldap)
proxy

part
reverse proxy
 
part
api gateway
proxy...
Persistency Layer
Persistency Layer
Stateless
Stateless
API Gateway / BFF
API Gateway / BFF
Edge / Static
Edge / St...
storage-users
storage-users
userprovider
(ldap)
userprovider...
storage-users
storage-users
storageprovider
storageprovider
dataprovider
dataprovider
storage-authbearer
storage-authbearer
authprovider
(oidc)
authprovider...
storage-gateway
storage-gateway
gateway
gateway
authregistry
authregistry
storageregistry
storageregistry
web
(aka phoenix)
web...
storage-authbasic
storage-authbasic
authprovider
(basic)
authprovider...
settings
settings
store
store
Consumers
Consumers
load balancer
load balancer
CDN
CDN
storage-frontend
storage-frontend
ocs
ocs
ocdav
ocdav
ocm
ocm
datagateway
datagateway
graph
graph
graph-explorer
graph-explorer
ocs
ocs
webdav
webdav
onlyoffice
onlyoffice
registry
mdns / etcd
registry...
Stateful
Stateful
JSON
files
JSON...
bleve
index
bleve...
ocis
driver
ocis...
ocis
driver
ocis...
single
JSON
file
single...
single
JSON
file
single...
DNS
DNS
thumbnails
thumbnails
storage-publiclink
storage-publiclink
publicstorageprovider
publicstorageprovider
authprovider
(publicshares)
authprovider...
idp
idp
konnectd
konnectd
login ui
login ui
storage-meta
storage-meta
storageprovider
storageprovider
dataprovider
dataprovider
storage-home
storage-home
storageprovider
storageprovider
dataprovider
dataprovider
storage-sharing
storage-sharing
usershareprovider
usershareprovider
publicshareprovider
publicshareprovider
ocis
ocis
reva
reva
konnectd
konnectd
storage
storage
glauth
glauth
glauth
glauth
provided by
provided by
Viewer does not support full SVG 1.1