Documentation

Overview

    Package txnBuf contains a transaction buffer filter for the datastore service.

    By default, datastore transactions take a snapshot of the entity group as soon as you Get or Put into it. All subsequent Get (and query) operations reflect the state of the ORIGINAL transaction snapshot, regardless of any Put/Delete operations you've done since the beginning of the transaction.

    If you've installed this transaction buffer, then:

    - All mutations will be reflected in all read operations (see LIMITATIONS).
      Without the buffer, read operations always observe the state of the
      entity group(s) at the time that the transaction started.
    
    - All mutation operations will be buffered until the close of the
      transaction. This can help reduce the transaction size, and thus avoid
      the transaction size limit (currently 10MB). Multiple puts to the same
      entity will not increase the transaction size multiple times.
    
    - Transactions inside of an existing transaction will add to their outer
      transaction if they don't cause the outer transaction to exceed its
      size budget.
    
    - If an inner transaction would cause the OUTERMOST transaction to exceed
      the appengine-imposed 10MB transaction size limit, an error will be
      returned from the inner transaction, instead of adding it into the
      outermost transaction. This only applies to the first level of inner
      transactions, and does not apply to recursive transactions. The reason
      for this is that it's entirely feasible for inner transactions to
      temporarially exceed the limit, but still only commit an outer
      transaction which is under the limit. An example of this would be having
      one inner-inner transaction add a lot of large entities and then having
      a subsequent inner-inner transaction delete some of those entities.
    

    LIMITATIONS (only inside of a transaction)

    - KeysOnly/Projection/Count queries are supported, but may incur additional
      costs.
    
      These query types are implemented via projection queries, but will
      project all order-by fields in addition to any specified in the original
      query.
    
    - Distinct Projection queries do all 'distinct' deduplication in-memory.
      This could make them substantially more expensive than their native
      equivalent.
    
    - Metadata entities (e.g. `__entity_group__`) will reflect their values as
      they were at the beginning of the transaction, and will not increment
      as you write inside of the transaction.
    
    - Query cursors are not supported. Since the cursor format for the
      in-memory datastore implementation isn't compatible with the production
      cursors, it would be pretty tricky to make it so that cursors were
      viable outside the transaction as well as inside of it while also having
      it accurately reflect the 'merged' query results.
    
    - No parallel access* to datastore while in a transaction; all nested
      operations are serialized. This is done for simplicity and correctness.
    
      * The exception is that callbacks inside of
      a Run/GetMulti/DeleteMulti/PutMulti query MAY read/write the current
      transaction. Modifications to the datastore during query executions will
      not affect the query results (e.g. the query has snapshot consistency
      from the moment that it begins iteration). Note, however, that datastore
      operations within the callback are still synchronized. This behavior is
      so that the user is not forced to buffer all of the query results before
      doing work with them, but can treat the query like a stream of events,
      if they so choose.
    
    - The changing of namespace inside of a transaction is undefined... This is
      just generally a terrible idea anyway, but I thought it was worth
      mentioning.
    
    - Currently, the soft transactions are not directly accessible using the
      CurrentTransaction interface; it returns the wrapped datastore's
      transaction. While this is still correct, it could definitely be made
      more useful by adding transaction buffer metadata to the returned
      object.
    

    Index

    Constants

    View Source
    const DefaultSizeBudget = int64((10 * 1000 * 1000) * 0.95)

      DefaultSizeBudget is the size budget for the root transaction.

      Because our estimation algorithm isn't entirely correct, we take 5% off the limit for encoding and estimate inaccuracies.

      10MB taken on 2015/09/24: https://cloud.google.com/appengine/docs/go/datastore/#Go_Quotas_and_limits

      View Source
      const DefaultWriteCountBudget = 500

        DefaultWriteCountBudget is the maximum number of entities that can be written in a single call.

        This is not known to be documented, and has instead been extracted from a datastore error message.

        View Source
        const XGTransactionGroupLimit = 25

          XGTransactionGroupLimit is the number of transaction groups to allow in an XG transaction.

          25 taken on 2015/09/24: https://cloud.google.com/appengine/docs/go/datastore/transactions#Go_What_can_be_done_in_a_transaction

          Variables

          View Source
          var ErrTooManyRoots = errors.New(
          	"operating on too many entity groups in nested transaction")

            ErrTooManyRoots is returned when executing an operation which would cause the transaction to exceed it's allotted number of entity groups.

            View Source
            var ErrTransactionTooLarge = errors.New(
            	"applying the transaction would make the parent transaction too large")

              ErrTransactionTooLarge is returned when applying an inner transaction would cause an outer transaction to become too large.

              Functions

              func FilterRDS

              func FilterRDS(c context.Context) context.Context

                FilterRDS installs a transaction buffer datastore filter in the context.

                Types

                This section is empty.