Antivirus

Abstract

The antivirus service is responsible for scanning files for viruses.

Memory Considerations
Configuration
Operation Modes
- Postprocessing
- Scaling in Kubernetes
Example Yaml Config

Memory Considerations

The antivirus service can consume considerably amounts of memory. This is relevant to provide or define sufficient memory for the deployment selected. To avoid out of memory (OOM) situations, the following equation gives a rough overview based on experiences made. The memory calculation comes without any guarantee, is intended as overview only and subject of change.

memory limit = max file size x workers x factor 8 - 14

With: ANTIVIRUS_WORKERS == 1

 50MB file --> factor 14   --> 700MB memory
844MB file --> factor  8,3 -->   7GB memory

Configuration

Antivirus Scanner Type

The antivirus service currently supports ICAP and ClamAV as antivirus scanners. The ANTIVIRUS_SCANNER_TYPE environment variable is used to select the scanner. The detailed configuration for each scanner heavily depends on the scanner type selected. See the environment variables for more details.

For icap, only scanners using the X-Infection-Found header are currently supported.
For clamav only local sockets can currently be configured.

Maximum Scan Size

Several factors can make it necessary to limit the maximum filesize the antivirus service will use for scanning. Use the ANTIVIRUS_MAX_SCAN_SIZE environment variable to scan only a given amount of bytes. Obviously, it is recommended to scan the whole file, but several factors like scanner type and version, bandwidth, performance issues, etc. might make a limit necessary.

IMPORTANT

Streaming of files to the virus scan service still needs to be implemented. To prevent OOM errors ANTIVIRUS_MAX_SCAN_SIZE needs to be set lower than available ram.

Antivirus Workers

The number of concurrent scans can be increased by setting ANTIVIRUS_WORKERS. Be aware that this will also increase memory usage.

Infected File Handling

The antivirus service allows three different ways of handling infected files. Those can be set via the ANTIVIRUS_INFECTED_FILE_HANDLING environment variable:

delete: (default): Infected files will be deleted immediately, further postprocessing is cancelled.
abort: (advanced option): Infected files will be kept, further postprocessing is cancelled. Files can be manually retrieved and inspected by an admin. To identify the file for further investigation, the antivirus service logs the abort/infected state including the file ID. The file is located in the storage/users/uploads folder of the ocis data directory and persists until it is manually deleted by the admin via the Manage Unfinished Uploads command.
continue: (obviously not recommended): Infected files will be marked via metadata as infected but postprocessing continues normally. Note: Infected Files are moved to their final destination and therefore not prevented from download which includes the risk of spreading viruses.

In all cases, a log entry is added declaring the infection and handling method and a notification via the userlog service sent.

Scanner Inaccessibility

In case a scanner is not accessible by the antivirus service like a network outage, service outage or hardware outage, the antivirus service uses the abort case for further processing, independent of the actual setting made. In any case, an error is logged noting the inaccessibility of the scanner used.

Operation Modes

The antivirus service can scan files during postprocessing. on demand scanning is currently not available and might be added in a future release.

Postprocessing

The antivirus service will scan files during postprocessing. It listens for a postprocessing step called virusscan. This step can be added in the environment variable POSTPROCESSING_STEPS. Read the documentation of the postprocessing service for more details.

The number of concurrent scans can be increased by setting ANTIVIRUS_WORKERS, but be aware that this will also increase the memory usage.

Scaling in Kubernetes

In kubernetes, ANTIVIRUS_WORKERS and ANTIVIRUS_MAX_SCAN_SIZE can be used to trigger the horizontal pod autoscaler by requesting a memory size that is below ANTIVIRUS_MAX_SCAN_SIZE. Keep in mind that ANTIVIRUS_MAX_SCAN_SIZE amount of memory might be held by ANTIVIRUS_WORKERS number of go routines.

Example Yaml Config

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38


# Autogenerated
# Filename: antivirus-config-example.yaml

file: ""
log:
  level: ""
  pretty: false
  color: false
  file: ""
debug:
  addr: 127.0.0.1:9277
  token: ""
  pprof: false
  zpages: false
tracing:
  enabled: false
  type: ""
  endpoint: ""
  collector: ""
infected-file-handling: delete
events:
  endpoint: 127.0.0.1:9233
  cluster: ocis-cluster
  tls_insecure: false
  tls_root_ca_certificate: ""
  enable_tls: false
  username: ""
  password: ""
workers: 10
scanner:
  type: clamav
  clamav:
    socket: /run/clamav/clamd.ctl
  icap:
    scan_timeout: 5m0s
    url: icap://127.0.0.1:1344
    service: avscan
max-scan-size: ""

Environment Variables

Name	Type	Default Value	Description
OCIS_LOG_LEVEL ANTIVIRUS_LOG_LEVEL	string		The log level. Valid values are: ‘panic’, ‘fatal’, ’error’, ‘warn’, ‘info’, ‘debug’, ’trace’.
OCIS_LOG_PRETTY ANTIVIRUS_LOG_PRETTY	bool	false	Activates pretty log output.
OCIS_LOG_COLOR ANTIVIRUS_LOG_COLOR	bool	false	Activates colorized log output.
OCIS_LOG_FILE ANTIVIRUS_LOG_FILE	string		The path to the log file. Activates logging to this file if set.
ANTIVIRUS_DEBUG_ADDR	string	127.0.0.1:9277	Bind address of the debug server, where metrics, health, config and debug endpoints will be exposed.
ANTIVIRUS_DEBUG_TOKEN	string		Token to secure the metrics endpoint.
ANTIVIRUS_DEBUG_PPROF	bool	false	Enables pprof, which can be used for profiling.
ANTIVIRUS_DEBUG_ZPAGES	bool	false	Enables zpages, which can be used for collecting and viewing in-memory traces.
OCIS_TRACING_ENABLED ANTIVIRUS_TRACING_ENABLED	bool	false	Activates tracing.
OCIS_TRACING_TYPE ANTIVIRUS_TRACING_TYPE	string		The type of tracing. Defaults to ‘’, which is the same as ‘jaeger’. Allowed tracing types are ‘jaeger’ and ’’ as of now.
OCIS_TRACING_ENDPOINT ANTIVIRUS_TRACING_ENDPOINT	string		The endpoint of the tracing agent.
OCIS_TRACING_COLLECTOR ANTIVIRUS_TRACING_COLLECTOR	string		The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.
ANTIVIRUS_INFECTED_FILE_HANDLING	string	delete	Defines the behaviour when a virus has been found. Supported options are: ‘delete’, ‘continue’ and ‘abort ‘. Delete will delete the file. Continue will mark the file as infected but continues further processing. Abort will keep the file in the uploads folder for further admin inspection and will not move it to its final destination.
OCIS_EVENTS_ENDPOINT ANTIVIRUS_EVENTS_ENDPOINT	string	127.0.0.1:9233	The address of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture.
OCIS_EVENTS_CLUSTER ANTIVIRUS_EVENTS_CLUSTER	string	ocis-cluster	The clusterID of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture. Mandatory when using NATS as event system.
OCIS_INSECURE ANTIVIRUS_EVENTS_TLS_INSECURE	bool	false	Whether to verify the server TLS certificates.
OCIS_EVENTS_TLS_ROOT_CA_CERTIFICATE ANTIVIRUS_EVENTS_TLS_ROOT_CA_CERTIFICATE	string		The root CA certificate used to validate the server’s TLS certificate. If provided ANTIVIRUS_EVENTS_TLS_INSECURE will be seen as false.
OCIS_EVENTS_ENABLE_TLS ANTIVIRUS_EVENTS_ENABLE_TLS	bool	false	Enable TLS for the connection to the events broker. The events broker is the ocis service which receives and delivers events between the services.
OCIS_EVENTS_AUTH_USERNAME ANTIVIRUS_EVENTS_AUTH_USERNAME	string		The username to authenticate with the events broker. The events broker is the ocis service which receives and delivers events between the services.
OCIS_EVENTS_AUTH_PASSWORD ANTIVIRUS_EVENTS_AUTH_PASSWORD	string		The password to authenticate with the events broker. The events broker is the ocis service which receives and delivers events between the services.
ANTIVIRUS_WORKERS	int	10	The number of concurrent go routines that fetch events from the event queue.
ANTIVIRUS_SCANNER_TYPE	string	clamav	The antivirus scanner to use. Supported values are ‘clamav’ and ‘icap’.
ANTIVIRUS_CLAMAV_SOCKET	string	/run/clamav/clamd.ctl	The socket clamav is running on. Note the default value is an example which needs adaption according your OS.
ANTIVIRUS_ICAP_SCAN_TIMEOUT	Duration	5m0s	Scan timeout for the ICAP client. Defaults to ‘5m’ (5 minutes). See the Environment Variable Types description for more details.
ANTIVIRUS_ICAP_URL	string	icap://127.0.0.1:1344	URL of the ICAP server.
ANTIVIRUS_ICAP_SERVICE	string	avscan	The name of the ICAP service.
ANTIVIRUS_MAX_SCAN_SIZE	string		The maximum scan size the virus scanner can handle. Only this many bytes of a file will be scanned. 0 means unlimited and is the default. Usable common abbreviations: [KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB, EB, EiB], example: 2GB.
ANTIVIRUS_DEBUG_SCAN_OUTCOME	string		A predefined outcome for virus scanning, FOR DEBUG PURPOSES ONLY! (example values: ‘found,infected’)