fluent-bit-go-azblob

README

fluent-bit Azure Blob output plugin

Azure Blob output plugin buffers logs in local file and upload them to Azure Blob periodically. This plugin is porting from fluent-bit-go-s3.

Usage

$ fluent-bit -e /path/to/built/out_azblob.so -c fluent-bit.conf

Prerequisites

• Go 1.15+
• gcc (for cgo)
• make

Build

$ make

Configuration Options

Example:

Add this section to fluent-bit.conf:

[OUTPUT]
    Name                     azblob
    Match                    *
    Azure_Storage_Account    teststorageaccount
    Azure_Storage_Access_Key dGVzYWNjZXNzdGtleQo=
    Azure_Container          testcontainer
    Batch_Retry_Limit        3

Key	Description	Default value
Azure_Storage_Account (Required)	Your Azure Storage Account Name.	""
Azure_Storage_SAS (Required*)	Your Azure Storage SAS Signature. Required if Azure_Storage_Access_Key is empty.	""
Azure_Storage_Access_Key (Required*)	Your Azure Storage Access Key. Required if Azure_Storage_SAS is empty.	""
Azure_Container (Required)	Azure Storage Container name.	""
Auto_Create_Container	Create container automatically.	false
Store_As	Archive format on Azure Storage. You can use following types: text/gzip	gzip
Path	Path prefix of the files on Azure Storage.	""
Azure_Object_Key_Format	The format of Azure Storage object keys. You can use several built-in variables: %{path}/%{time_slice}/%{uuid}/%{hostname}/%{file_extension}	%{path}%{time_slice}_%{uuid}.%{file_extension}
Time_Slice_Format	Format of the time used as the file name. See: Golang Time Format	2006010215-04
Batch_Wait	Time to wait before send a log batch to Azure Blob in seconds.	5
Batch_Size	Log batch size to send a log batch to Azure Blob.	32k
Batch_Retry_Limit	When Batch_Retry_Limit is set to empty, means that there is not limit for the number of retries that the plugin can do.
Time_Zone	Specify TZInfo based region (e.g. Asia/Taipei).	""
Logging	Specify Log Level. See: logrus logging levels	info

Useful links

• fluent-bit-go