dlp_snippets

README

Google Cloud Data Loss Prevention (DLP) API: Go Samples

The Data Loss Prevention API provides programmatic access to a powerful detection engine for personally identifiable information and other privacy-sensitive data in unstructured data streams.

Table of Contents

• Before you begin
• Samples
  • De-identify
  • Inspect
  • Metadata
  • Redact
  • Risk Analysis
  • Templates
  • Triggers

Before you begin

Before running the samples, make sure you've:

1. Enabled the DLP API.
2. Enabled the PubSub API.
3. Set up authentication.
4. Installed the sample application by running:

   go get -u github.com/GoogleCloudPlatform/golang-samples/dlp
   

Samples

Usage: ./dlp -project <my-project> [options] subcommand [args]

Options:
  -bytesType value
    	Bytes type of input file for inspectFile and redactImage [IMAGE_SVG, TEXT_UTF8, BYTES_TYPE_UNSPECIFIED, IMAGE_JPEG, IMAGE_BMP, IMAGE_PNG] (default BYTES_TYPE_UNSPECIFIED)
  -includeQuote
    	Include a quote of findings for inspect* (default false)
  -infoTypes string
    	Info types to inspect*, redactImage, createTrigger, and createInspectTemplate (default "PHONE_NUMBER,EMAIL_ADDRESS,CREDIT_CARD_NUMBER,US_SOCIAL_SECURITY_NUMBER")
  -languageCode string
    	Language code for infoTypes (default "en-US")
  -maxFindings int
    	Number of results for inspect*, createTrigger, and createInspectTemplate (default 0 (no limit))
  -minLikelihood value
    	Minimum likelihood value for inspect*, redactImage, createTrigger, and createInspectTemplate [LIKELY, VERY_LIKELY, LIKELIHOOD_UNSPECIFIED, VERY_UNLIKELY, UNLIKELY, POSSIBLE] (default LIKELIHOOD_UNSPECIFIED)
  -project string
    	GCloud project ID (required)

Subcommands and their args are described below.

De-identify

View the source code.

Usage:

go build
./dlp -project <project> [options] dateShift <string>
./dlp -project <project> [options] fpe <string> <wrappedKeyFileName> <cryptoKeyname> <surrogateInfoType>
./dlp -project <project> [options] mask <string>
./dlp -project <project> [options] reidentifyFPE <string> <wrappedKeyFileName> <cryptoKeyname> <surrogateInfoType>

Examples:

./dlp -project my-project dateShift "My birthday is January 1, 1970"
./dlp -project my-project mask "My SSN is 111222333"
ENC=$(./dlp -project my-project fpe "My SSN is 111222333" key.enc projects/my-project/locations/global/keyRings/my-key-ring/cryptoKeys/my-key randomstring)
./dlp -project my-project reidentifyFPE "$ENC" key.enc projects/my-project/locations/global/keyRings/my-key-ring/cryptoKeys/my-key randomstring

For more information, see https://cloud.google.com/dlp/docs.

Inspect

View the source code.

Usage:

go build
./dlp -project <project> [options] inspect <string>
./dlp -project <project> [options] inspectBigquery <pubSubTopic> <pubSubSub> <dataProject> <datasetID> <tableID>
./dlp -project <project> [options] inspectDatastore <pubSubTopic> <pubSubSub> <dataProject> <namespaceID> <kind>
./dlp -project <project> [options] inspectFile <filename>
./dlp -project <project> [options] inspectGCSFile <pubSubTopic> <pubSubSub> <bucketName> <fileName>

Examples:

./dlp -project my-project inspect "My SSN is 111222333 and my phone number is (123) 456-7890"
./dlp -project my-project inspectBigquery inspect-topic inspect-sub dataProject datasetID tableID
./dlp -project my-project inspectDatastore inspect-topic inspect-sub my-data-project my-namespace my-kind
./dlp -project my-project inspectFile my-file
./dlp -project my-project inspectGCSFile inspect-topic inspect-sub my-bucket my-file

For more information, see https://cloud.google.com/dlp/docs.

Jobs

View the source code.

Usage:

go build
./dlp -project <project> [options] deleteJob <jobID>
./dlp -project <project> [options] listJobs <filter> <jobType>

Examples:

./dlp -project my-project inspect "My SSN is 111222333 and my phone number is (123) 456-7890"
    ./dlp -project my-project deleteJob /projects/my-project/dlpJobs/my-job
    ./dlp -project my-project listJobs "" ""

For more information, see https://cloud.google.com/dlp/docs.

Metadata

View the source code.

Usage:

go build
./dlp -project <project> [options] infoTypes <filter>

Examples:

./dlp -project my-project infoTypes supported_by=INSPECT

For more information, see https://cloud.google.com/dlp/docs.

Redact

View the source code.

Usage:

go build
./dlp -project <project> [options] redactImage <inputPath> <outputPath>

Examples:

./dlp -project my-project -bytesType IMAGE_PNG redactImage input.png output.png

For more information, see https://cloud.google.com/dlp/docs.

Risk Analysis

View the source code.

Usage:

go build
./dlp -project <project> [options] riskCategorical <dataProject> <pubSubTopic> <pubSubSub> <datasetID> <tableID> <columnName>
./dlp -project <project> [options] riskKAnonymity  <dataProject> <pubSubTopic> <pubSubSub> <datasetID> <tableID> <column,names>
./dlp -project <project> [options] riskKMap        <dataProject> <pubSubTopic> <pubSubSub> <datasetID> <tableID> <region> <column,names>
./dlp -project <project> [options] riskLDiversity  <dataProject> <pubSubTopic> <pubSubSub> <datasetID> <tableID> <sensitiveAttribute> <column,names>
./dlp -project <project> [options] riskNumerical   <dataProject> <pubSubTopic> <pubSubSub> <datasetID> <tableID> <columnName>

Examples:

./dlp -project my-project riskNumerical   bigquery-public-data risk-topic risk-sub nhtsa_traffic_fatalities accident_2015 state_number
./dlp -project my-project riskCategorical bigquery-public-data risk-topic risk-sub nhtsa_traffic_fatalities accident_2015 state_number
./dlp -project my-project riskKAnonymity  bigquery-public-data risk-topic risk-sub nhtsa_traffic_fatalities accident_2015 state_number,county
./dlp -project my-project riskLDiversity  bigquery-public-data risk-topic risk-sub nhtsa_traffic_fatalities accident_2015 city state_number,county
./dlp -project my-project riskKMap        bigquery-public-data risk-topic risk-sub san_francisco bikeshare_trips US zip_code

For more information, see https://cloud.google.com/dlp/docs.

Templates

View the source code.

Usage:

go build
./dlp -project <project> [options] createInspectTemplate <templateID> <displayName> <description>
./dlp -project <project> [options] deleteInspectTemplate <fullTemplateID>
./dlp -project <project> [options] listInspectTemplates

Examples:

./dlp -project my-project createInspectTemplate my-template "My Template" "My template description"
./dlp -project my-project deleteInspectTemplate projects/my-project/inspectTemplates/my-template
./dlp -project my-project listInspectTemplates

For more information, see https://cloud.google.com/dlp/docs.

Triggers

View the source code.

Usage:

./dlp -project <project> [options] createTrigger <triggerID> <displayName> <description> <bucketName>
./dlp -project <project> [options] deleteTrigger <fullTriggerID>
./dlp -project <project> [options] listTriggers

Examples:

./dlp -project my-project createTrigger my-trigger "My Trigger" "My trigger description" my-bucket
./dlp -project my-project deleteTrigger projects/my-project/jobTriggers/my-trigger
./dlp -project my-project listTriggers

Documentation

Overview

dlp is an example of using the DLP API.