azlogs

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 7, 2024 License: MIT Imports: 13 Imported by: 0

README

Azure Monitor Query client module for Go

The Azure Monitor Query client module is used to execute read-only queries against Azure Monitor's log data platform.

Logs collects and organizes log and performance data from monitored resources. Data from different sources such as platform logs from Azure services, log and performance data from virtual machines agents, and usage and performance data from apps can be consolidated into a single Azure Log Analytics workspace. The various data types can be analyzed together using the Kusto Query Language. See the Kusto to SQL cheat sheet for more information.

Source code | Package (pkg.go.dev) | REST API documentation | Product documentation | Samples

Getting started

Prerequisites
Install the packages

Install the azlogs and azidentity modules with go get:

go get github.com/Azure/azure-sdk-for-go/sdk/monitor/query/azlogs
go get github.com/Azure/azure-sdk-for-go/sdk/azidentity

The azidentity module is used for Azure Active Directory authentication during client construction.

Authentication

An authenticated client object is required to execute a query. The examples demonstrate using azidentity.NewDefaultAzureCredential to authenticate; however, the client accepts any azidentity credential. See the azidentity documentation for more information about other credential types.

The clients default to the Azure public cloud. For other cloud configurations, see the cloud package documentation.

Create a client

Example client.

Key concepts

Timespan

It's best practice to always query with a timespan (type TimeInterval) to prevent excessive queries of the entire logs data set. Log queries use the ISO8601 Time Interval Standard. All time should be represented in UTC. If the timespan is included in both the Kusto query string and Timespan field, the timespan is the intersection of the two values.

Use the NewTimeInterval() method for easy creation.

Logs query rate limits and throttling

The Log Analytics service applies throttling when the request rate is too high. Limits, such as the maximum number of rows returned, are also applied on the Kusto queries. For more information, see Query API.

If you're executing a batch logs query, a throttled request will return a ErrorInfo object. That object's code value will be ThrottledError.

Advanced logs queries
Query multiple workspaces

To run the same query against multiple Log Analytics workspaces, add the additional workspace ID strings to the AdditionalWorkspaces slice in the QueryBody struct.

When multiple workspaces are included in the query, the logs in the result table are not grouped according to the workspace from which they were retrieved.

Increase wait time, include statistics, include render (visualization)

The LogsQueryOptions type is used for advanced logs options.

  • By default, your query will run for up to three minutes. To increase the default timeout, set LogsQueryOptions.Wait to the desired number of seconds. The maximum wait time is 10 minutes (600 seconds).

  • To get logs query execution statistics, such as CPU and memory consumption, set LogsQueryOptions.Statistics to true.

  • To get visualization data for logs queries, set LogsQueryOptions.Visualization to true.

azlogs.QueryWorkspaceOptions{
			Options: &azlogs.LogsQueryOptions{
				Statistics:    to.Ptr(true),
				Visualization: to.Ptr(true),
				Wait:          to.Ptr(600),
			},
		}

Examples

Get started with our examples.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information, see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Documentation

Index

Examples

Constants

View Source 
const ServiceNameLogs cloud.ServiceName = "query/azlogs"

Cloud Service Names for Monitor Query Logs, used to identify the respective cloud.ServiceConfiguration

Variables

This section is empty.

Functions

This section is empty.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client contains the methods for the Logs group. Don't use this type directly, use a constructor function instead.

func NewClient 

func NewClient(credential azcore.TokenCredential, options *ClientOptions) (*Client, error)

NewClient creates a client that accesses Azure Monitor logs data.

Example 
package main

import (
	"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	"github.com/Azure/azure-sdk-for-go/sdk/monitor/query/azlogs"
)

func main() {
	cred, err := azidentity.NewDefaultAzureCredential(nil)
	if err != nil {
		//TODO: handle error
	}

	client, err := azlogs.NewClient(cred, nil)
	if err != nil {
		//TODO: handle error
	}
	_ = client
}

Output:

func (*Client) QueryResource 

func (client *Client) QueryResource(ctx context.Context, resourceID string, body QueryBody, options *QueryResourceOptions) (QueryResourceResponse, error)

QueryResource - Executes an Analytics query for data in the context of a resource. Here [https://learn.microsoft.com/azure/azure-monitor/logs/api/azure-resource-queries] is an example for using POST with an Analytics query. If the operation fails it returns an *azcore.ResponseError type.

Generated from API version 2022-10-27

Example 
package main

import (
	"context"
	"fmt"
	"time"

	"github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
	"github.com/Azure/azure-sdk-for-go/sdk/monitor/query/azlogs"
)

var client azlogs.Client

func main() {
	// Instead of requiring a Log Analytics workspace,
	// QueryResource allows users to query logs directly from an Azure resource through a resource ID.

	// To find the resource ID:
	// 1. Navigate to your resource's page in the Azure portal.
	// 2. From the **Overview** blade, select the **JSON View** link.
	// 3. In the resulting JSON, copy the value of the `id` property.

	resourceID := "/subscriptions/fajfkx93-c1d8-40ad-9cce-e49c10ca8qe6/resourceGroups/testgroup/providers/Microsoft.Storage/storageAccounts/mystorageacount" // example resource ID

	res, err := client.QueryResource(
		context.TODO(),
		resourceID,
		azlogs.QueryBody{
			Query:    to.Ptr("StorageBlobLogs | where TimeGenerated > ago(3d)"), // example Kusto query
			Timespan: to.Ptr(azlogs.NewTimeInterval(time.Date(2022, 12, 25, 0, 0, 0, 0, time.UTC), time.Date(2022, 12, 25, 12, 0, 0, 0, time.UTC))),
		},
		nil)
	if err != nil {
		//TODO: handle error
	}
	if res.Error != nil {
		//TODO: handle partial error
	}

	// Print Rows
	for _, table := range res.Tables {
		for _, row := range table.Rows {
			fmt.Println(row)
		}
	}
}

Output:

func (*Client) QueryWorkspace 

func (client *Client) QueryWorkspace(ctx context.Context, workspaceID string, body QueryBody, options *QueryWorkspaceOptions) (QueryWorkspaceResponse, error)

QueryWorkspace - Executes an Analytics query for data. Here [https://learn.microsoft.com/azure/azure-monitor/logs/api/request-format] is an example for using POST with an Analytics query. If the operation fails it returns an *azcore.ResponseError type.

Generated from API version 2022-10-27

Example 
package main

import (
	"context"
	"fmt"
	"time"

	"github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
	"github.com/Azure/azure-sdk-for-go/sdk/monitor/query/azlogs"
)

var client azlogs.Client

func main() {
	// QueryWorkspace allows users to query log data.

	// A workspace ID is required to query logs. To find the workspace ID:
	// 1. If not already made, create a Log Analytics workspace (https://learn.microsoft.com/azure/azure-monitor/logs/quick-create-workspace).
	// 2. Navigate to your workspace's page in the Azure portal.
	// 3. From the **Overview** blade, copy the value of the ***Workspace ID*** property.

	workspaceID := "g4d1e129-fb1e-4b0a-b234-250abc987ea65" // example Azure Log Analytics Workspace ID

	res, err := client.QueryWorkspace(
		context.TODO(),
		workspaceID,
		azlogs.QueryBody{
			Query:    to.Ptr("AzureActivity | top 10 by TimeGenerated"), // example Kusto query
			Timespan: to.Ptr(azlogs.NewTimeInterval(time.Date(2022, 12, 25, 0, 0, 0, 0, time.UTC), time.Date(2022, 12, 25, 12, 0, 0, 0, time.UTC))),
		},
		nil)
	if err != nil {
		//TODO: handle error
	}
	if res.Error != nil {
		//TODO: handle partial error
	}

	// Print Rows
	for _, table := range res.Tables {
		for _, row := range table.Rows {
			fmt.Println(row)
		}
	}
}

Output:
Example (Second) 
// `QueryWorkspace` also has more advanced options, including querying multiple workspaces
// and LogsQueryOptions (including statistics and visualization information and increasing default timeout).

// When multiple workspaces are included in the query, the logs in the result table are not grouped
// according to the workspace from which it was retrieved.
workspaceID1 := "g4d1e129-fb1e-4b0a-b234-250abc987ea65" // example Azure Log Analytics Workspace ID
workspaceID2 := "h4bc4471-2e8c-4b1c-8f47-12b9a4d5ac71"
additionalWorkspaces := []string{workspaceID2}

// Advanced query options
// Setting Statistics to true returns stats information in Results.Statistics
// Setting Visualization to true returns visualization information in Results.Visualization
options := &azlogs.QueryWorkspaceOptions{
	Options: &azlogs.LogsQueryOptions{
		Statistics:    to.Ptr(true),
		Visualization: to.Ptr(true),
		Wait:          to.Ptr(600),
	},
}

res, err := client.QueryWorkspace(
	context.TODO(),
	workspaceID1,
	azlogs.QueryBody{
		Query:                to.Ptr(query),
		Timespan:             to.Ptr(azlogs.NewTimeInterval(time.Date(2022, 12, 25, 0, 0, 0, 0, time.UTC), time.Date(2022, 12, 25, 12, 0, 0, 0, time.UTC))),
		AdditionalWorkspaces: additionalWorkspaces,
	},
	options)
if err != nil {
	//TODO: handle error
}
if res.Error != nil {
	//TODO: handle partial error
}

// Example of converting table data into a slice of structs.
// Query results are returned in Table Rows and are of type any.
// Type assertion is required to access the underlying value of each index in a Row.
var QueryResults []queryResult
for _, table := range res.Tables {
	QueryResults = make([]queryResult, len(table.Rows))
	for index, row := range table.Rows {
		QueryResults[index] = queryResult{
			Bool:   row[0].(bool),
			Long:   int64(row[1].(float64)),
			Double: float64(row[2].(float64)),
			String: row[3].(string),
		}
	}
}

fmt.Println(QueryResults)

// Print out Statistics
fmt.Printf("Statistics: %s", string(res.Statistics))

// Print out Visualization information
fmt.Printf("Visualization: %s", string(res.Visualization))

Output:

type ClientOptions 

type ClientOptions struct {
	azcore.ClientOptions
}

ClientOptions contains optional settings for Client.

type Column 

type Column struct {
	// The name of this column.
	Name *string

	// The data type of this column.
	Type *LogsColumnType
}

Column - A column in a table.

func (Column) MarshalJSON 

func (c Column) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaller interface for type Column.

func (*Column) UnmarshalJSON 

func (c *Column) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaller interface for type Column.

type ErrorInfo 

type ErrorInfo struct {
	// REQUIRED; A machine readable error code.
	Code string
	// contains filtered or unexported fields
}

ErrorInfo - The code and message for an error.

func (*ErrorInfo) Error 

func (e *ErrorInfo) Error() string

Error implements a custom error for type ErrorInfo.

func (*ErrorInfo) UnmarshalJSON 

func (e *ErrorInfo) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaller interface for type ErrorInfo.

type LogsColumnType 

type LogsColumnType string

LogsColumnType - The data type of this column. 

const (
	LogsColumnTypeBool     LogsColumnType = "bool"
	LogsColumnTypeDatetime LogsColumnType = "datetime"
	LogsColumnTypeDecimal  LogsColumnType = "decimal"
	LogsColumnTypeDynamic  LogsColumnType = "dynamic"
	LogsColumnTypeGUID     LogsColumnType = "guid"
	LogsColumnTypeInt      LogsColumnType = "int"
	LogsColumnTypeLong     LogsColumnType = "long"
	LogsColumnTypeReal     LogsColumnType = "real"
	LogsColumnTypeString   LogsColumnType = "string"
	LogsColumnTypeTimespan LogsColumnType = "timespan"
)

func PossibleLogsColumnTypeValues 

func PossibleLogsColumnTypeValues() []LogsColumnType

PossibleLogsColumnTypeValues returns the possible values for the LogsColumnType const type.

type LogsQueryOptions 

type LogsQueryOptions struct {
	// Set Statistics to true to get logs query execution statistics,
	// such as CPU and memory consumption. Defaults to false.
	Statistics *bool

	// Set Visualization to true to get visualization
	// data for logs queries. Defaults to false.
	Visualization *bool

	// By default, the Azure Monitor Query service will run your
	// query for up to three minutes. To increase the default timeout,
	// set Wait to desired number of seconds.
	// Max wait time the service will allow is ten minutes (600 seconds).
	Wait *int
}

LogsQueryOptions sets server timeout, query statistics and visualization information

type QueryBody 

type QueryBody struct {
	// REQUIRED; The query to execute.
	Query *string

	// A list of workspaces to query in addition to the primary workspace.
	AdditionalWorkspaces []string

	// Optional. The timespan over which to query data. This is an ISO8601 time period value. This timespan is applied in addition
	// to any that are specified in the query expression.
	Timespan *TimeInterval
}

QueryBody - The Analytics query. Learn more about the Analytics query syntax [https://azure.microsoft.com/documentation/articles/app-insights-analytics-reference/]

func (QueryBody) MarshalJSON 

func (q QueryBody) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaller interface for type QueryBody.

func (*QueryBody) UnmarshalJSON 

func (q *QueryBody) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaller interface for type QueryBody.

type QueryResourceOptions 

type QueryResourceOptions struct {
	// Optional. The prefer header to set server timeout, query statistics and visualization information.
	Options *LogsQueryOptions
}

QueryResourceOptions contains the optional parameters for the Client.QueryResource method.

type QueryResourceResponse 

type QueryResourceResponse struct {
	// Contains the tables, columns & rows resulting from a query.
	QueryResults
}

QueryResourceResponse contains the response from method Client.QueryResource.

type QueryResults 

type QueryResults struct {
	// REQUIRED; The results of the query in tabular format.
	Tables []Table

	// The code and message for an error.
	Error *ErrorInfo

	// Statistics represented in JSON format.
	Statistics []byte

	// Visualization data in JSON format.
	Visualization []byte
}

QueryResults - Contains the tables, columns & rows resulting from a query.

func (QueryResults) MarshalJSON 

func (q QueryResults) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaller interface for type QueryResults.

func (*QueryResults) UnmarshalJSON 

func (q *QueryResults) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaller interface for type QueryResults.

type QueryWorkspaceOptions 

type QueryWorkspaceOptions struct {
	// Optional. The prefer header to set server timeout, query statistics and visualization information.
	Options *LogsQueryOptions
}

QueryWorkspaceOptions contains the optional parameters for the Client.QueryWorkspace method.

type QueryWorkspaceResponse 

type QueryWorkspaceResponse struct {
	// Contains the tables, columns & rows resulting from a query.
	QueryResults
}

QueryWorkspaceResponse contains the response from method Client.QueryWorkspace.

type Row 

type Row []any

Row of data in a table, types of data used by service specified in LogsColumnType

type Table 

type Table struct {
	// REQUIRED; The list of columns in this table.
	Columns []Column

	// REQUIRED; The name of the table.
	Name *string

	// REQUIRED; The resulting rows from this query.
	Rows []Row
}

Table - Contains the columns and rows for one table in a query response.

func (Table) MarshalJSON 

func (t Table) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaller interface for type Table.

func (*Table) UnmarshalJSON 

func (t *Table) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaller interface for type Table.

type TimeInterval 

type TimeInterval string

TimeInterval specifies the time range over which to query. Use NewTimeInterval() for help formatting. Follows the ISO8601 time interval standard with most common format being startISOTime/endISOTime. ISO8601 durations also supported (ex "PT2H" for last two hours). Use UTC for all times.

func NewTimeInterval 

func NewTimeInterval(start time.Time, end time.Time) TimeInterval

NewTimeInterval creates a TimeInterval for use in a query. Use UTC for start and end times.

func (TimeInterval) Values 

func (i TimeInterval) Values() (time.Time, time.Time, error)

Values returns the interval's start and end times if it's in the format startISOTime/endISOTime, else it will return an error.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.