Spaces
The OData specification allows for a mirage of ways of addressing an entity. We will support addressing a Drive entity by its unique identifier, which is the one the graph-api returns when listing spaces, and its format is:
{
"id": "1284d238-aa92-42ce-bdc4-0b0000009157!b6e2c9cc-9dbe-42f0-b522-4f2d3e175e9c"
}
This is an extract of an element of the list spaces response. An entire object has the following shape:
{
"driveType": "project",
"id": "1284d238-aa92-42ce-bdc4-0b0000009157!b6e2c9cc-9dbe-42f0-b522-4f2d3e175e9c",
"lastModifiedDateTime": "2021-10-07T11:06:43.245418+02:00",
"name": "marketing",
"owner": {
"user": {
"id": "ddc2004c-0977-11eb-9d3f-a793888cd0f8"
}
},
"quota": {
"total": 65536
},
"root": {
"id": "1284d238-aa92-42ce-bdc4-0b0000009157!b6e2c9cc-9dbe-42f0-b522-4f2d3e175e9c",
"webDavUrl": "https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157!b6e2c9cc-9dbe-42f0-b522-4f2d3e175e9c"
}
}
Having introduced the above, one can refer to a Drive with the following URL format:
'https://localhost:9200/graph/v1.0/drives/1284d238-aa92-42ce-bdc4-0b0000009157!07c26b3a-9944-4f2b-ab33-b0b326fc7570
Updating an entity attribute:
curl -X PATCH 'https://localhost:9200/graph/v1.0/drives/1284d238-aa92-42ce-bdc4-0b0000009157!07c26b3a-9944-4f2b-ab33-b0b326fc7570' -d '{"name":"42"}' -v
The previous URL resource path segment (1284d238-aa92-42ce-bdc4-0b0000009157!07c26b3a-9944-4f2b-ab33-b0b326fc7570) is parsed and handed over to the storage registry in order to apply the patch changes in the body, in this case update the space name attribute to 42. Since space names are not unique we only support addressing them by their unique identifiers, any other query would render too ambiguous and explode in complexity.
Since every space is the root of a webdav directory, following some conventions we can make use of this to set a default storage description and image. In order to do so, every space is created with a hidden .space folder at its root, which can be used to store such data.
curl -k -X PUT https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157\!07c26b3a-9944-4f2b-ab33-b0b326fc7570/.space/description.md -d "Add a description to your spaces" -u admin:admin
Verify the description was updated:
❯ curl -k https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157\!07c26b3a-9944-4f2b-ab33-b0b326fc7570/.space/description.md -u admin:admin
Add a description to your spaces
This feature makes use of the internal storage layout and is completely abstracted from the end user.
Spaces capacity (quota) is independent of the Storage quota. As a Space admin you can set the quota for all users of a space, and as such, there are no limitations and is up to the admin to make a correct use of this.
It is possible to have a space quota greater than the storage quota. A Space may also have “infinite” quota, meaning a single space without quota can occupy the entirety of a disk.
Creating a Space with a quota of 10 bytes:
curl -k -X POST 'https://localhost:9200/graph/v1.0/drives' -u admin:admin -d '{"name":"marketing", "quota": {"total": 10}}' -v
/var/tmp/ocis/storage/users
├── blobs
├── nodes
│ ├── 627981c2-2a71-4adf-b680-177e245afdda
│ ├── 9541e7c3-8fda-4b49-b697-e7e51457cf5a
│ ├── b5692345-108d-4b80-9747-3a7e9739ad57
│ └── root
│ ├── 118351d7-67a4-4cdf-b495-6093d1e572ed -> ../627981c2-2a71-4adf-b680-177e245afdda
│ └── ddc2004c-0977-11eb-9d3f-a793888cd0f8 -> ../b5692345-108d-4b80-9747-3a7e9739ad57
├── spaces
│ ├── personal
│ │ └── b5692345-108d-4b80-9747-3a7e9739ad57 -> ../../nodes/b5692345-108d-4b80-9747-3a7e9739ad57
│ ├── project
│ │ └── 627981c2-2a71-4adf-b680-177e245afdda -> ../../nodes/627981c2-2a71-4adf-b680-177e245afdda
│ └── share
├── trash
└── uploads
Verify the new space has 10 bytes, and none of it is used:
{
"driveType": "project",
"id": "1284d238-aa92-42ce-bdc4-0b0000009157!627981c2-2a71-4adf-b680-177e245afdda",
"lastModifiedDateTime": "2021-10-15T11:16:26.029188+02:00",
"name": "marketing",
"owner": {
"user": {
"id": "ddc2004c-0977-11eb-9d3f-a793888cd0f8"
}
},
"quota": {
"remaining": 10,
"total": 10,
"used": 0
},
"root": {
"id": "1284d238-aa92-42ce-bdc4-0b0000009157!627981c2-2a71-4adf-b680-177e245afdda",
"webDavUrl": "https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157!627981c2-2a71-4adf-b680-177e245afdda"
}
}
Upload a 6 bytes file:
curl -k -X PUT https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157\!627981c2-2a71-4adf-b680-177e245afdda/6bytes.txt -d "012345" -u admin:admin -v
Query the quota again:
{
"quota": {
"remaining": 4,
"total": 10,
"used": 6
}
}
Now attempt to upload 5 bytes to the space:
curl -k -X PUT https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157\!627981c2-2a71-4adf-b680-177e245afdda/5bytes.txt -d "01234" -u admin:admin -v
The request will fail with 507 Insufficient Storage:
HTTP/1.1 507 Insufficient Storage
< Access-Control-Allow-Origin: *
< Content-Length: 0
< Content-Security-Policy: default-src 'none';
< Date: Fri, 15 Oct 2021 09:24:46 GMT
< Vary: Origin
< X-Content-Type-Options: nosniff
< X-Download-Options: noopen
< X-Frame-Options: SAMEORIGIN
< X-Permitted-Cross-Domain-Policies: none
< X-Robots-Tag: none
< X-Xss-Protection: 1; mode=block
<
* Connection #0 to host localhost left intact
* Closing connection 0
- If a Space quota is updated to unlimited, the upper limit is the entire available space on disk
The current implementation in oCIS might not yet fully reflect this concept. Feel free to add links to ADRs, PRs and Issues in short warning boxes like this.
A storage space is a logical concept. It organizes a set of resources in a hierarchical tree. It has a single owner (user or group),
a quota, permissions and is identified by a storage space id.
Examples would be every user’s personal storage space, project storage spaces or group storage spaces. While they all serve different purposes and may or may not have workflows like antivirus scanning enabled, we need a way to identify and manage these subtrees in a generic way. By creating a dedicated concept for them this becomes easier and literally makes the codebase cleaner. A storage Spaces Registry then allows listing the capabilities of storage spaces, e.g. free space, quota, owner, syncable, root etag, upload workflow steps, …
Finally, a logical storage space id is not tied to a specific spaces provider. If the storage driver supports it, we can import existing files including their file id, which makes it possible to move storage spaces between spaces providers to implement storage classes, e.g. with or without archival, workflows, on SSDs or HDDs.
To be clarified: we are aware that storage spaces may be too ‘heavyweight’ for ad hoc sharing with groups. That being said, there is no technical reason why group shares should not be treated like storage spaces that users can provision themselves. They would share the quota with the users home or personal storage space and the share initiator would be the sole owner. Technically, the mechanism of treating a share like a new storage space would be the same. This obviously also extends to user shares and even file individual shares that would be wrapped in a virtual collection. It would also become possible to share collections of arbitrary files in a single storage space, e.g. the ten best pictures from a large album.
We can implement ListStorageSpaces by either
- iterating over the root of the storage and treating every folder following the
<user_layout>as ahomestorage space, - iterating over the root of the storage and treating every folder following a new
<project_layout>as aprojectstorage space, or - iterating over the root of the storage and treating every folder following a generic
<layout>as a storage space for a configurable space type, or - we allow configuring a map of
space typetolayout(based on the CreateStorageSpaceRequest) which would allow things like
home=/var/lib/ocis/storage/home/{{substr 0 1 .Owner.Username}}/{{.Owner.Username}}
spaces=/spaces/var/lib/ocis/storage/projects/{{.Name}}