tcobject

package
v53.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 27, 2023 License: MPL-2.0 Imports: 20 Imported by: 0

Documentation

Overview

This package was generated from the schema defined at /references/object/v1/api.json The object service provides HTTP-accessible storage for large blobs of data.

Objects can be uploaded and downloaded, with the object data flowing directly from the storage "backend" to the caller, and not directly via this service. Once uploaded, objects are immutable until their expiration time.

See:

How to use this package

First create an Object object: 

object := tcobject.New(nil)

and then call one or more of object's methods, e.g.: 

err := object.Ping(.....)

handling any errors... 

if err != nil {
	// handle error...
}

Taskcluster Schema

The source code of this go package was auto-generated from the API definition at <rootUrl>/references/object/v1/api.json together with the input and output schemas it references,

Index

Constants

View Source 
const (
	DataInlineMaxSize = 8192
)

Variables

View Source 
var ACCEPTABLE_HASHES = []string{
	"sha256",
	"sha512",
}

The subset of hashes supported by hashing{read,write}stream which are "accepted" as per the object service's schemas.

Functions

This section is empty.

Types

type CreateUploadRequest 

type CreateUploadRequest struct {

	// Date at which this entry expires from the object table.  The expiration cannot
	// be changed in subsequent calls to the same method.
	Expires tcclient.Time `json:"expires"`

	// Hashes of the content of this object.  These values will be verified by
	// well-behaved downloaders.  The format is `{alogrithm: value}`.
	//
	// Multiple calls to `createUpload` or `finishUpload` for the same object
	// can specify additional hashes, but existing hashes cannot be changed;
	// this allows "amending" an upload with hashes after the data has been
	// transferred, for example.
	//
	// At least one non-deprecated algorithm must be included, preferably the
	// most advanced (SHA512).  Deprecated algorithms may also be included.
	//
	// Defined properties:
	//
	//  struct {
	//
	//  	// Syntax:     ^[a-z0-9]{64}$
	//  	//
	//		//  	SHA256 string `json:"sha256,omitempty"`
	//
	//  	// Syntax:     ^[a-z0-9]{128}$
	//  	//
	//		//  	SHA512 string `json:"sha512,omitempty"`
	//  }
	//
	// Additional properties allowed
	Hashes json.RawMessage `json:"hashes,omitempty"`

	// Project identifier.
	//
	// Syntax:     ^([a-zA-Z0-9._/-]*)$
	// Min length: 1
	// Max length: 500
	ProjectID string `json:"projectId"`

	// Upload methods, with details, that the caller is prepared to execute.  If
	// this object is empty, then the server will reject the request but still
	// create the upload with the given `uploadId`, `projectId`, and `expires`,
	// so any subsequent calls must share those values.  The server may choose
	// any of the proposed methods at its discretion.
	ProposedUploadMethods ProposedUploadMethods `json:"proposedUploadMethods"`

	// Unique identifier for this upload.   Once an object is created with an uploadId,
	// uploads of the same object with different uploadIds will be rejected.  Callers
	// should pass a randomly-generated slugid here.
	//
	// Syntax:     ^[A-Za-z0-9_-]{8}[Q-T][A-Za-z0-9_-][CGKOSWaeimquy26-][A-Za-z0-9_-]{10}[AQgw]$
	UploadID string `json:"uploadId"`
}

A request to begin an upload, containing proposed upload methods to which the server may agree to or reject.

type CreateUploadResponse 

type CreateUploadResponse struct {
	Expires tcclient.Time `json:"expires"`

	// Syntax:     ^([a-zA-Z0-9._/-]*)$
	// Min length: 1
	// Max length: 500
	ProjectID string `json:"projectId"`

	// Syntax:     ^[A-Za-z0-9_-]{8}[Q-T][A-Za-z0-9_-][CGKOSWaeimquy26-][A-Za-z0-9_-]{10}[AQgw]$
	UploadID string `json:"uploadId"`

	// The selected upload method, from those contained in the request.  At most one
	// property will be set, indicating the selected method.  If no properties are set,
	// then none of the proposed methods were selected.
	UploadMethod SelectedUploadMethodOrNone `json:"uploadMethod"`
}

A response from the `createUpload` method, either agreeing to an upload method or rejecting the proposal.

type DataInlineUploadRequest 

type DataInlineUploadRequest struct {

	// Content-type to be returned when downloading this data
	ContentType string `json:"contentType"`

	// Base64-encoded byte data, with decoded size at most 8k.
	//
	// Max length: 10926
	ObjectData string `json:"objectData"`
}

Upload data included directly in the request. The data has a fixed maximum length, so this should be used only for value that are known to be of constant, fairly small size to avoid surprises as the payload grows. In general, this is useful for testing and for metadata objects such as separate cryptographic signatures.

type DownloadObjectRequest 

type DownloadObjectRequest struct {

	// Download methods that the caller can suport, together with parameters for each method.
	// The server will choose one method and make the corresponding response.
	AcceptDownloadMethods SupportedDownloadMethods `json:"acceptDownloadMethods"`
}

See [Download Methods](https://docs.taskcluster.net/docs/docs/reference/platform/object/download-methods) for details.

type DownloadObjectResponse 

type DownloadObjectResponse json.RawMessage

See [Download Methods](https://docs.taskcluster.net/docs/docs/reference/platform/object/download-methods) for details.

One of:

  • SimpleDownloadResponse
  • GetURLDownloadResponse

func (*DownloadObjectResponse) MarshalJSON 

func (this *DownloadObjectResponse) MarshalJSON() ([]byte, error)

MarshalJSON calls json.RawMessage method of the same name. Required since DownloadObjectResponse is of type json.RawMessage...

func (*DownloadObjectResponse) UnmarshalJSON 

func (this *DownloadObjectResponse) UnmarshalJSON(data []byte) error

UnmarshalJSON is a copy of the json.RawMessage implementation.

type FinishUploadRequest 

type FinishUploadRequest struct {

	// Hashes of the content of this object.  These values will be verified by
	// well-behaved downloaders.  The format is `{alogrithm: value}`.
	//
	// Multiple calls to `createUpload` or `finishUpload` for the same object
	// can specify additional hashes, but existing hashes cannot be changed;
	// this allows "amending" an upload with hashes after the data has been
	// transferred, for example.
	//
	// At least one non-deprecated algorithm must be included, preferably the
	// most advanced (SHA512).  Deprecated algorithms may also be included.
	//
	// Defined properties:
	//
	//  struct {
	//
	//  	// Syntax:     ^[a-z0-9]{64}$
	//  	//
	//		//  	SHA256 string `json:"sha256,omitempty"`
	//
	//  	// Syntax:     ^[a-z0-9]{128}$
	//  	//
	//		//  	SHA512 string `json:"sha512,omitempty"`
	//  }
	//
	// Additional properties allowed
	Hashes json.RawMessage `json:"hashes,omitempty"`

	// Project identifier.
	//
	// Syntax:     ^([a-zA-Z0-9._/-]*)$
	// Min length: 1
	// Max length: 500
	ProjectID string `json:"projectId"`

	// Unique identifier for this upload.
	//
	// Syntax:     ^[A-Za-z0-9_-]{8}[Q-T][A-Za-z0-9_-][CGKOSWaeimquy26-][A-Za-z0-9_-]{10}[AQgw]$
	UploadID string `json:"uploadId"`
}

type GetURLDownloadResponse 

type GetURLDownloadResponse struct {

	// The time after which `url` is no longer valid.
	// If the client wishes to begin an HTTP GET request after this time, it should first call `startDownload` again to get a fresh URL.
	Expires tcclient.Time `json:"expires"`

	// Hashes of the content of this object.  The caller should verify all
	// hashes present for recognized algorithms, and verify that at least one
	// non-deprecated hash is present.
	//
	// Defined properties:
	//
	//  struct {
	//
	//  	// Syntax:     ^[a-z0-9]{64}$
	//  	//
	//		//  	SHA256 string `json:"sha256,omitempty"`
	//
	//  	// Syntax:     ^[a-z0-9]{128}$
	//  	//
	//		//  	SHA512 string `json:"sha512,omitempty"`
	//  }
	//
	// Additional properties allowed
	Hashes json.RawMessage `json:"hashes"`

	// Constant value: "getUrl"
	Method string `json:"method"`

	// The URL to which the client should make a GET request.
	URL string `json:"url"`
}

This download method returns a URL from which the data may be fetched with an HTTP GET request.

The client should begin a GET request as soon as possible after receiving the reponse. The server will respond with a 200 OK containing the data, or with a 4xx or 5xx error response. It will _not_ redirect to another URL (3xx), and the client should not follow any such redirects.

The client can use standard Range requests to download portions of the object or to resume an interrupted download. Per the HTTP standard, servers may return more data than requested by the Range header.

If retrying a failed or interrupted download, and the `expires` field is in the past, the client should call `startDownload` again to get an updated `url`. The client can assume that the object data and `hashes` will be the same for all calls to `startDownload`.

The client can use standard Accept-Encoding headers to indicate the encodings it can accept. However, in a deviation from standard HTTP, the client _must_ accept at least `identity` and `gzip` encodings. If the HTTP response has a `Content-Encoding` header, the client should decode the body before verifying its hashes and returning it to the application.

The client _must_ verify that the resulting data matches the supplied hashes. The object service does not, itself, validate object data and relies on clients to do so.

type HTTPRetryError 

type HTTPRetryError = internal.HTTPRetryError

type Object 

type Object tcclient.Client

func New 

func New(credentials *tcclient.Credentials, rootURL string) *Object

New returns an Object client, configured to run against production. Pass in nil credentials to create a client without authentication. The returned client is mutable, so returned settings can be altered. 

object := tcobject.New(
    nil,                                      // client without authentication
    "http://localhost:1234/my/taskcluster",   // taskcluster hosted at this root URL on local machine
)
err := object.Ping(.....)                     // for example, call the Ping(.....) API endpoint (described further down)...
if err != nil {
	// handle errors...
}

func NewFromEnv 

func NewFromEnv() *Object

NewFromEnv returns an *Object configured from environment variables.

The root URL is taken from TASKCLUSTER_PROXY_URL if set to a non-empty string, otherwise from TASKCLUSTER_ROOT_URL if set, otherwise the empty string.

The credentials are taken from environment variables: 

TASKCLUSTER_CLIENT_ID
TASKCLUSTER_ACCESS_TOKEN
TASKCLUSTER_CERTIFICATE

If TASKCLUSTER_CLIENT_ID is empty/unset, authentication will be disabled.

func (*Object) CreateUpload 

func (object *Object) CreateUpload(name string, payload *CreateUploadRequest) (*CreateUploadResponse, error)

Create a new object by initiating upload of its data.

This endpoint implements negotiation of upload methods. It can be called multiple times if necessary, either to propose new upload methods or to renew credentials for an already-agreed upload.

The `name` parameter can contain any printable ASCII character (0x20 - 0x7e). The `uploadId` must be supplied by the caller, and any attempts to upload an object with the same name but a different `uploadId` will fail. Thus the first call to this method establishes the `uploadId` for the object, and as long as that value is kept secret, no other caller can upload an object of that name, regardless of scopes. Object expiration cannot be changed after the initial call, either. It is possible to call this method with no proposed upload methods, which has the effect of "locking in" the `expiration`, `projectId`, and `uploadId` properties and any supplied hashes.

Unfinished uploads expire after 1 day.

Required scopes: 

object:upload:<projectId>:<name>

See #createUpload

func (*Object) Download 

func (object *Object) Download(name string) error

Get the data in an object directly. This method does not return a JSON body, but redirects to a location that will serve the object content directly.

URLs for this endpoint, perhaps with attached authentication (`?bewit=..`), are typically used for downloads of objects by simple HTTP clients such as web browsers, curl, or wget.

This method is limited by the common capabilities of HTTP, so it may not be the most efficient, resilient, or featureful way to retrieve an artifact. Situations where such functionality is required should ues the `startDownload` API endpoint.

See [Simple Downloads](https://docs.taskcluster.net/docs/reference/platform/object/simple-downloads) for more detail.

Required scopes: 

object:download:<name>

See #download

func (*Object) DownloadToBuf 

func (object *Object) DownloadToBuf(name string) (buf []byte, contentType string, contentLength int64, err error)

DownloadToBuf is a convenience method to download an object to an in-memory byte slice. Returns the object itself, the Content-Type and Content-Length of the downloaded object.

func (*Object) DownloadToFile 

func (object *Object) DownloadToFile(name string, filepath string) (contentType string, contentLength int64, err error)

DownloadToFile is a convenience method to download an object to a file. The file is overwritten if it already exists. Returns the Content-Type and Content-Length of the downloaded object.

func (*Object) DownloadToWriteSeeker 

func (object *Object) DownloadToWriteSeeker(name string, writeSeeker io.WriteSeeker) (contentType string, contentLength int64, err error)

DownloadToWriteSeeker downloads the named object from the object service and writes it to writeSeeker, retrying if intermittent errors occur. Returns the Content-Type and Content-Length of the downloaded object.

func (*Object) Download_SignedURL 

func (object *Object) Download_SignedURL(name string, duration time.Duration) (*url.URL, error)

Returns a signed URL for Download, valid for the specified duration.

Required scopes: 

object:download:<name>

See Download for more details.

func (*Object) FinishUpload 

func (object *Object) FinishUpload(name string, payload *FinishUploadRequest) error

This endpoint marks an upload as complete. This indicates that all data has been transmitted to the backend. After this call, no further calls to `uploadObject` are allowed, and downloads of the object may begin. This method is idempotent, but will fail if given an incorrect uploadId for an unfinished upload.

It is possible to finish an upload with no hashes specified via either `startUpload` or `finishUpload`. However, many clients will refuse to download an object with no hashes. The utility methods included with the client libraries always include hashes as of version 44.0.0.

Note that, once `finishUpload` is complete, the object is considered immutable.

Required scopes: 

object:upload:<projectId>:<name>

See #finishUpload

func (*Object) Heartbeat 

func (object *Object) Heartbeat() error

Respond with a service heartbeat.

This endpoint is used to check on backing services this service depends on.

See #heartbeat

func (*Object) Lbheartbeat 

func (object *Object) Lbheartbeat() error

Respond without doing anything. This endpoint is used to check that the service is up.

See #lbheartbeat

func (*Object) Object 

func (object *Object) Object(name string) (*ObjectMetadata, error)

Get the metadata for the named object. This metadata is not sufficient to get the object's content; for that use `startDownload`.

Required scopes: 

object:download:<name>

See #object

func (*Object) Object_SignedURL 

func (object *Object) Object_SignedURL(name string, duration time.Duration) (*url.URL, error)

Returns a signed URL for Object, valid for the specified duration.

Required scopes: 

object:download:<name>

See Object for more details.

func (*Object) Ping 

func (object *Object) Ping() error

Respond without doing anything. This endpoint is used to check that the service is up.

See #ping

func (*Object) StartDownload 

func (object *Object) StartDownload(name string, payload *DownloadObjectRequest) (*DownloadObjectResponse, error)

Start the process of downloading an object's data. Call this endpoint with a list of acceptable download methods, and the server will select a method and return the corresponding payload.

Returns a 406 error if none of the given download methods are available.

See [Download Methods](https://docs.taskcluster.net/docs/reference/platform/object/download-methods) for more detail.

Required scopes: 

object:download:<name>

See #startDownload

func (*Object) UploadFromBuf 

func (object *Object) UploadFromBuf(projectID string, name string, contentType string, expires time.Time, uploadID string, buf []byte) (err error)

UploadFromBuf is a convenience method to publish an Object to the Object Service with content buf and given projectID, name, contentType, expires, and uploadID.

func (*Object) UploadFromFile 

func (object *Object) UploadFromFile(projectID string, name string, contentType string, expires time.Time, uploadID string, filepath string) (err error)

UploadFromFile is a convenience method to publish an Object to the Object Server from file with path filepath and given projectID, name, contentType, expires, and uploadID.

func (*Object) UploadFromReadSeeker 

func (object *Object) UploadFromReadSeeker(projectID string, name string, contentType string, contentLength int64, expires time.Time, uploadID string, readSeeker io.ReadSeeker) (err error)

UploadFromReadSeeker publishes an Object to the Object Service, with given name, projectID, contentType, contentLength, expiry, and uploadID, with the object content read from readSeeker. The value of contentLength is not validated prior to upload.

func (*Object) Version 

func (object *Object) Version() error

Respond with the JSON version object. https://github.com/mozilla-services/Dockerflow/blob/main/docs/version_object.md

See #version

type ObjectContentHashes 

type ObjectContentHashes json.RawMessage

Hashes of the content of this object. These values will be verified by well-behaved downloaders. The format is `{alogrithm: value}`.

Multiple calls to `createUpload` or `finishUpload` for the same object can specify additional hashes, but existing hashes cannot be changed; this allows "amending" an upload with hashes after the data has been transferred, for example.

At least one non-deprecated algorithm must be included, preferably the most advanced (SHA512). Deprecated algorithms may also be included.

Defined properties: 

 struct {

 	// Syntax:     ^[a-z0-9]{64}$
 	//
	//  	SHA256 string `json:"sha256,omitempty"`

 	// Syntax:     ^[a-z0-9]{128}$
 	//
	//  	SHA512 string `json:"sha512,omitempty"`
 }

Additional properties allowed

func (*ObjectContentHashes) MarshalJSON 

func (this *ObjectContentHashes) MarshalJSON() ([]byte, error)

MarshalJSON calls json.RawMessage method of the same name. Required since ObjectContentHashes is of type json.RawMessage...

func (*ObjectContentHashes) UnmarshalJSON 

func (this *ObjectContentHashes) UnmarshalJSON(data []byte) error

UnmarshalJSON is a copy of the json.RawMessage implementation.

type ObjectContentHashesForDownload 

type ObjectContentHashesForDownload json.RawMessage

Hashes of the content of this object. The caller should verify all hashes present for recognized algorithms, and verify that at least one non-deprecated hash is present.

Defined properties: 

 struct {

 	// Syntax:     ^[a-z0-9]{64}$
 	//
	//  	SHA256 string `json:"sha256,omitempty"`

 	// Syntax:     ^[a-z0-9]{128}$
 	//
	//  	SHA512 string `json:"sha512,omitempty"`
 }

Additional properties allowed

func (*ObjectContentHashesForDownload) MarshalJSON 

func (this *ObjectContentHashesForDownload) MarshalJSON() ([]byte, error)

MarshalJSON calls json.RawMessage method of the same name. Required since ObjectContentHashesForDownload is of type json.RawMessage...

func (*ObjectContentHashesForDownload) UnmarshalJSON 

func (this *ObjectContentHashesForDownload) UnmarshalJSON(data []byte) error

UnmarshalJSON is a copy of the json.RawMessage implementation.

type ObjectMetadata 

type ObjectMetadata struct {
	Expires tcclient.Time `json:"expires"`

	// Hashes of the content of this object.  The caller should verify all
	// hashes present for recognized algorithms, and verify that at least one
	// non-deprecated hash is present.
	//
	// Defined properties:
	//
	//  struct {
	//
	//  	// Syntax:     ^[a-z0-9]{64}$
	//  	//
	//		//  	SHA256 string `json:"sha256,omitempty"`
	//
	//  	// Syntax:     ^[a-z0-9]{128}$
	//  	//
	//		//  	SHA512 string `json:"sha512,omitempty"`
	//  }
	//
	// Additional properties allowed
	Hashes json.RawMessage `json:"hashes"`

	// Syntax:     ^([a-zA-Z0-9._/-]*)$
	// Min length: 1
	// Max length: 500
	ProjectID string `json:"projectId"`
}

Metadata about an object.

type ProposedUploadMethods 

type ProposedUploadMethods struct {

	// Upload data included directly in the request.  The data has a fixed maximum length, so this should
	// be used only for value that are known to be of constant, fairly small size to avoid surprises as
	// the payload grows. In general, this is useful for testing and for metadata objects such as
	// separate cryptographic signatures.
	DataInline DataInlineUploadRequest `json:"dataInline,omitempty"`

	// Request a URL to which a PUT request can be made.
	PutURL PutURLUploadRequest `json:"putUrl,omitempty"`
}

Upload methods, with details, that the caller is prepared to execute. If this object is empty, then the server will reject the request but still create the upload with the given `uploadId`, `projectId`, and `expires`, so any subsequent calls must share those values. The server may choose any of the proposed methods at its discretion.

type PutURLUploadRequest 

type PutURLUploadRequest struct {

	// Length, in bytes, of the uploaded data.
	ContentLength int64 `json:"contentLength"`

	// Content-type of the data to be uploaded.
	ContentType string `json:"contentType"`
}

Request a URL to which a PUT request can be made.

type PutURLUploadResponse 

type PutURLUploadResponse struct {

	// Expiration time for the URL.  After this time, the client must
	// call `createUpload` again to get a fresh URL.
	Expires tcclient.Time `json:"expires"`

	// Headers which must be included with the PUT request.  In many
	// cases, these are included in a signature embedded in the URL,
	// and must be provided verbatim.
	//
	// The `Content-Length` header may be included here.  Many HTTP client
	// libraries will also set this directly when the length is known.  In
	// this case, the values should be identical, and the header should only
	// be specified once.
	//
	// Map entries:
	Headers map[string]string `json:"headers"`

	// URL to which a PUT request should be made.
	URL string `json:"url"`
}

Response containing a URL to which to PUT the data.

type SelectedUploadMethodOrNone 

type SelectedUploadMethodOrNone struct {

	// Indication that the data has been uploaded.
	//
	// Constant value: %!q(bool=true)
	DataInline bool `json:"dataInline,omitempty"`

	// Response containing a URL to which to PUT the data.
	PutURL PutURLUploadResponse `json:"putUrl,omitempty"`
}

The selected upload method, from those contained in the request. At most one property will be set, indicating the selected method. If no properties are set, then none of the proposed methods were selected.

type SimpleDownloadResponse 

type SimpleDownloadResponse struct {

	// Constant value: "simple"
	Method string `json:"method"`

	URL string `json:"url"`
}

A simple download returns a URL to which the caller should make a GET request. See [Simple Downloads](https://docs.taskcluster.net/docs/docs/reference/platform/object/simple-downloads) for details.

type SupportedDownloadMethods 

type SupportedDownloadMethods struct {

	// Constant value: %!q(bool=true)
	GetURL bool `json:"getUrl,omitempty"`

	// Constant value: %!q(bool=true)
	Simple bool `json:"simple,omitempty"`
}

Download methods that the caller can suport, together with parameters for each method. The server will choose one method and make the corresponding response.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.