README

Gosip - SharePoint authentication, HTTP client & fluent API wrapper for Go (Golang)

Build Status Go Report Card GoDoc License codecov FOSSA Status Mentioned in Awesome Go

Gosip

Main features

  • Unattended authentication using different strategies.
  • Fluent API syntax for SharePoint object model.
  • Simplified API consumption (REST, CSOM, SOAP).
  • SharePoint-aware embedded features (retries, header presets, error handling).
Supported SharePoint versions
  • SharePoint Online (SPO)
  • On-Premises (2019/2016/2013)
Supported auth strategies
  • SharePoint Online:

    • Azure Certificate (App Only) πŸ”—
    • Azure Username/Password πŸ”—
    • SAML based with user credentials
    • Add-In only permissions
    • ADFS user credentials (automatically detects in SAML strategy)
    • On-Demand auth πŸ”—
    • Azure AD Device flow πŸ”—
  • SharePoint On-Premises 2019/2016/2013:

    • User credentials (NTLM)
    • ADFS user credentials (ADFS, WAP -> Basic/NTLM, WAP -> ADFS)
    • Behind a reverse proxy (Forefront TMG, WAP -> Basic/NTLM, WAP -> ADFS)
    • Form-based authentication (FBA)
    • On-Demand auth πŸ”—

Installation

go get github.com/koltyakov/gosip

Usage insights

1. Understand SharePoint environment type and authentication strategy.

Let's assume it's SharePoint Online and Add-In Only permissions. Then strategy "github.com/koltyakov/gosip/auth/addin" subpackage should be used.

package main

import (
	"github.com/koltyakov/gosip"
	"github.com/koltyakov/gosip/api"
	strategy "github.com/koltyakov/gosip/auth/addin"
)

2. Initiate an authentication object.

auth := &strategy.AuthCnfg{
	SiteURL:      os.Getenv("SPAUTH_SITEURL"),
	ClientID:     os.Getenv("SPAUTH_CLIENTID"),
	ClientSecret: os.Getenv("SPAUTH_CLIENTSECRET"),
}

AuthCnfg from different strategies contains different options relevant for a specified auth type.

The authentication options can be provided explicitly or can be read from a configuration file.

configPath := "./config/private.json"
auth := &strategy.AuthCnfg{}

err := auth.ReadConfig(configPath)
if err != nil {
	fmt.Printf("Unable to get config: %v\n", err)
	return
}

3. Bind auth client with Fluent API.

client := &gosip.SPClient{AuthCnfg: auth}

sp := api.NewSP(client)

res, err := sp.Web().Select("Title").Get()
if err != nil {
	fmt.Println(err)
}

fmt.Printf("%s\n", res.Data().Title)

Usage samples

Fluent API client

Fluent API gives a simple way of constructing API endpoint calls with IntelliSense and chainable syntax.

Fluent Sample

package main

import (
	"encoding/json"
	"fmt"
	"log"

	"github.com/koltyakov/gosip"
	"github.com/koltyakov/gosip/api"
	strategy "github.com/koltyakov/gosip/auth/addin"
)

func main() {
	// Getting auth params and client
	client, err := getAuthClient()
	if err != nil {
		log.Fatalln(err)
	}

	// Binding SharePoint API
	sp := api.NewSP(client)

	// Custom headers
	headers := map[string]string{
		"Accept": "application/json;odata=minimalmetadata",
		"Accept-Language": "de-DE,de;q=0.9",
	}
	config := &api.RequestConfig{Headers: headers}

	// Chainable request sample
	data, err := sp.Conf(config).Web().Lists().Select("Id,Title").Get()
	if err != nil {
		log.Fatalln(err)
	}

	// Response object unmarshalling (struct depends on OData mode and API method)
	res := &struct {
		Value []struct {
			ID    string `json:"Id"`
			Title string `json:"Title"`
		} `json:"value"`
	}{}

	if err := json.Unmarshal(data, &res); err != nil {
		log.Fatalf("unable to parse the response: %v", err)
	}

	for _, list := range res.Value {
		fmt.Printf("%+v\n", list)
	}

}

func getAuthClient() (*gosip.SPClient, error) {
	configPath := "./config/private.spo-addin.json"
	auth := &strategy.AuthCnfg{}
	if err := auth.ReadConfig(configPath); err != nil {
		return nil, fmt.Errorf("unable to get config: %v", err)
	}
	return &gosip.SPClient{AuthCnfg: auth}, nil
}
Generic HTTP client helper

Provides generic GET/POST helpers for REST operations, reducing the amount of http.NewRequest scaffolded code, can be used for custom or not covered with Fluent API endpoints.

package main

import (
	"fmt"
	"log"

	"github.com/koltyakov/gosip"
	"github.com/koltyakov/gosip/api"
	strategy "github.com/koltyakov/gosip/auth/ntlm"
)

func main() {
	configPath := "./config/private.ntlm.json"
	auth := &strategy.AuthCnfg{}

	if err := auth.ReadConfig(configPath); err != nil {
		log.Fatalf("unable to get config: %v\n", err)
	}

	sp := api.NewHTTPClient(&gosip.SPClient{AuthCnfg: auth})

	endpoint := auth.GetSiteURL() + "/_api/web?$select=Title"

	data, err := sp.Get(endpoint, nil)
	if err != nil {
		log.Fatalf("%v\n", err)
	}

	// sp.Post(endpoint, body, nil) // generic POST
	// sp.Delete(endpoint, nil) // generic DELETE helper crafts "X-Http-Method"="DELETE" header
	// sp.Update(endpoint, nil) // generic UPDATE helper crafts "X-Http-Method"="MERGE" header
	// sp.ProcessQuery(endpoint, body) // CSOM helper (client.svc/ProcessQuery)

	fmt.Printf("response: %s\n", data)
}
Low-level HTTP client usage

Low-lever SharePoint-aware HTTP client from github.com/koltyakov/gosip package for custom or not covered with a Fluent API client endpoints with granular control for an HTTP request, response, and http.Client parameters. The client is used internally but rarely required in consumer code.

client := &gosip.SPClient{AuthCnfg: auth}

var req *http.Request
// Initiate API request
// ...

resp, err := client.Execute(req)
if err != nil {
	fmt.Printf("Unable to request api: %v", err)
	return
}

SPClient has Execute method which is a wrapper function injecting SharePoint authentication and ending up calling http.Client's Do method.

Authentication strategies

Auth strategy should be selected corresponding to your SharePoint environment and its configuration.

Import path strategy "github.com/koltyakov/gosip/auth/{strategy}". Where /{strategy} stands for a strategy auth package.

/{strategy} SPO On-Prem Credentials sample(s)
AAD /azurecert βœ… ❌ details
AAD /azurecreds βœ… ❌ details
AAD /device βœ… ❌ details
/saml βœ… ❌ sample
/addin βœ… ❌ sample
/ntlm ❌ βœ… sample
/adfs βœ… βœ… spo, on-prem, on-prem (wap)
/fba ❌ βœ… sample
/tmg ❌ βœ… sample

JSON and struct representations are different in terms of language notations. So credentials parameters names in private.json files and declared as structs initiators vary.

SAML Auth (SharePoint Online user credentials authentication)

This authentication option uses Microsoft Online Security Token Service https://login.microsoftonline.com/extSTS.srf and SAML tokens in order to obtain an authentication cookie.

// AuthCnfg - SAML auth config structure
type AuthCnfg struct {
	// SPSite or SPWeb URL, which is the context target for the API calls
	SiteURL string `json:"siteUrl"`
	// Username for SharePoint Online, for example `[user]@[company].onmicrosoft.com`
	Username string `json:"username"`
	// User or App password
	Password string `json:"password"`
}
AddIn Only Auth

This type of authentication uses AddIn Only policy and OAuth bearer tokens for authenticating HTTP requests.

// AuthCnfg - AddIn Only auth config structure
type AuthCnfg struct {
	// SPSite or SPWeb URL, which is the context target for the API calls
	SiteURL string `json:"siteUrl"`
	// Client ID obtained when registering the AddIn
	ClientID string `json:"clientId"`
	// Client Secret obtained when registering the AddIn
	ClientSecret string `json:"clientSecret"`
	// Your SharePoint Online tenant ID (optional)
	Realm string `json:"realm"`
}

Realm can be left empty or filled in, which will add small performance improvement. The easiest way to find the tenant is to open SharePoint Online site collection, click Site Settings -> Site App Permissions. Taking any random app, the tenant ID (realm) is the GUID part after the @.

See more details of AddIn Configuration and Permissions.

NTLM Auth (NTLM handshake)

This type of authentication uses an HTTP NTLM handshake to obtain an authentication header.

// AuthCnfg - NTML auth config structure
type AuthCnfg struct {
	// SPSite or SPWeb URL, which is the context target for the API calls
	SiteURL  string `json:"siteUrl"`
	Domain   string `json:"domain"`   // AD domain name
	Username string `json:"username"` // AD user name
	Password string `json:"password"` // AD user password
}

Gosip uses github.com/Azure/go-ntlmssp NTLM negotiator, however, a custom one also can be provided in case of demand.

ADFS Auth (user credentials authentication)
// AuthCnfg - ADFS auth config structure
type AuthCnfg struct {
	// SPSite or SPWeb URL, which is the context target for the API calls
	SiteURL      string `json:"siteUrl"`
	Username     string `json:"username"`
	Password     string `json:"password"`
	// Following are not required for SPO
	Domain       string `json:"domain"`
	RelyingParty string `json:"relyingParty"`
	AdfsURL      string `json:"adfsUrl"`
	AdfsCookie   string `json:"adfsCookie"`
}

See more details ADFS user credentials authentication.

Gosip's ADFS also supports a scenario of ADFS or NTML behind WAP (Web Application Proxy) which adds additional auth flow and EdgeAccessCookie involved into play.

FBA/TMG Auth (Form-based authentication)

FBA - Form-based authentication for SharePoint On-Premises.

TMG - Microsoft Forefront Threat Management Gateway, currently is legacy but was a popular way of exposing SharePoint into the external world back in the days.

// AuthCnfg - FBA/TMG auth config structure
type AuthCnfg struct {
	// SPSite or SPWeb URL, which is the context target for the API calls
	SiteURL string `json:"siteUrl"`
	// Username for SharePoint On-Prem, format depends in FBA/TMG settings,
	// can include domain or doesn't
	Username string `json:"username"`
	// User password
	Password string `json:"password"`
}

Secrets encoding

When storing credential in local private.json files, which can be handy in local development scenarios, we strongly recommend to encode secrets such as password or clientSecret using cpass. Class converts a secret to an encrypted representation, which can only be decrypted on the same machine where it was generated. That reduces accidental leaks, e.g. together with git commits.

Reference

Many auth flows have been "copied" from node-sp-auth library (used as a blueprint), which we intensively use in Node.js ecosystem for years.

Fluent API and wrapper syntax are inspired by PnPjs, which is also the first-class citizen on almost all our Node.js and front-end projects with SharePoint involved.

πŸ“š Documentation

πŸ“¦ Samples

License

FOSSA Status

Expand β–Ύ Collapse β–΄

Documentation

Overview ΒΆ

    Package gosip is pure Go library for dealing with SharePoint unattended authentication and API consumption.

    It supports a variety of different authentication strategies such as:

    - SAML based with user credentials
    - Add-in only permissions
    - Azure AD auth strategies (via extensions https://go.spflow.com/auth/custom-auth)
    - ADFS user credentials
    - NTLM/NTLM v2 windows auth
    - Auth to SharePoint behind a reverse proxy (TMG, WAP)
    - Form-based authentication (FBA)
    - Web login/On-Demand auth (via extension https://go.spflow.com/auth/custom-auth/on-demand)
    

    Amongst supported platform versions are:

    - SharePoint Online (SPO)
    - On-Premise: 2019, 2016, and 2013
    

    Index ΒΆ

    Constants ΒΆ

    This section is empty.

    Variables ΒΆ

    This section is empty.

    Functions ΒΆ

    func GetDigest ΒΆ

    func GetDigest(context context.Context, client *SPClient) (string, error)

      GetDigest retrieves and caches SharePoint API X-RequestDigest value

      Types ΒΆ

      type AuthCnfg ΒΆ

      type AuthCnfg interface {
      	// Authentication initializer (token/cookie/header, expiration, error)
      	// to support cabability for exposing tokens for external tools
      	// e.g. as of this sample project https://github.com/koltyakov/spvault
      	GetAuth() (string, int64, error)
      
      	// Authentication middleware fabric
      	// applyes round tripper or enriches requests with authentication and metadata
      	SetAuth(req *http.Request, client *SPClient) error
      
      	ParseConfig(jsonConf []byte) error  // Parses credentials from a provided JSON byte array content
      	ReadConfig(configPath string) error // Reads credentials from storage
      
      	GetSiteURL() string  // SiteURL getter method
      	GetStrategy() string // Strategy code getter
      
      }

        AuthCnfg is an abstract auth config interface, allows different authentications strategies' dependency injection

        type HookEvent ΒΆ

        type HookEvent struct {
        	Request    *http.Request
        	StartedAt  time.Time
        	StatusCode int
        	Error      error
        }

          HookEvent hook event parameters struct

          type HookHandlers ΒΆ

          type HookHandlers struct {
          	OnError    func(event *HookEvent) // when error appeared
          	OnRetry    func(event *HookEvent) // before retry request
          	OnRequest  func(event *HookEvent) // before request is sent
          	OnResponse func(event *HookEvent) // after response is received
          }

            HookHandlers struct to configure events handlers

            type SPClient ΒΆ

            type SPClient struct {
            	http.Client
            	AuthCnfg   AuthCnfg // authentication configuration interface
            	ConfigPath string   // private.json location path, optional when AuthCnfg is provided with creds explicitly
            
            	RetryPolicies map[int]int   // allows redefining error state requests retry policies
            	Hooks         *HookHandlers // hook handlers definition
            }

              SPClient : SharePoint HTTP client struct

              func (*SPClient) Execute ΒΆ

              func (c *SPClient) Execute(req *http.Request) (*http.Response, error)

                Execute : SharePoint HTTP client is a wrapper for standard http.Client' `Do` method, injects authorization tokens, etc.

                Directories

                Path Synopsis
                Package api represents Fluent API for SharePoint object model
                Package api represents Fluent API for SharePoint object model
                auth
                addin
                Package addin implements AddIn Only Auth This type of authentication uses AddIn Only policy and OAuth bearer tokens for authenticating HTTP requests.
                Package addin implements AddIn Only Auth This type of authentication uses AddIn Only policy and OAuth bearer tokens for authenticating HTTP requests.
                adfs
                Package adfs implements ADFS Auth (user credentials authentication) Amongst supported platform versions are: - SharePoint Online (SPO) - On-Premise: 2019, 2016, and 2013
                Package adfs implements ADFS Auth (user credentials authentication) Amongst supported platform versions are: - SharePoint Online (SPO) - On-Premise: 2019, 2016, and 2013
                anon
                Package anon provides anonymous "strategy" no auth mechanisms are applied to the requests
                Package anon provides anonymous "strategy" no auth mechanisms are applied to the requests
                fba
                Package fba implements FBA (Form-based authentication) Amongst supported platform versions are: - On-Premise: 2019, 2016, and 2013
                Package fba implements FBA (Form-based authentication) Amongst supported platform versions are: - On-Premise: 2019, 2016, and 2013
                ntlm
                Package ntlm implements NTLM Auth (NTLM handshake) This type of authentication uses HTTP NTLM handshake in order to obtain authentication header.
                Package ntlm implements NTLM Auth (NTLM handshake) This type of authentication uses HTTP NTLM handshake in order to obtain authentication header.
                saml
                Package saml implements SAML Auth (SharePoint Online user credentials authentication) This authentication option uses Microsoft Online Security Token Service `https://login.microsoftonline.com/extSTS.srf` and SAML tokens in order to obtain authentication cookie.
                Package saml implements SAML Auth (SharePoint Online user credentials authentication) This authentication option uses Microsoft Online Security Token Service `https://login.microsoftonline.com/extSTS.srf` and SAML tokens in order to obtain authentication cookie.
                tmg
                Package tmg implements FBA authentication behind TMG (Microsoft Forefront Threat Management Gateway) Currently is legacy but was a popular way of exposing SharePoint into external world back in the days.
                Package tmg implements FBA authentication behind TMG (Microsoft Forefront Threat Management Gateway) Currently is legacy but was a popular way of exposing SharePoint into external world back in the days.
                cmd
                Package csom helps building CSOM XML requests
                Package csom helps building CSOM XML requests
                test