search

package module
v0.0.0-...-ec93f51 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 25, 2024 License: MIT Imports: 8 Imported by: 18

README

Google Search Results GoLang API

test

This Golang package enables to scrape and parse results from Google, Bing, Baidu, Yahoo, Yandex, Ebay, Google Schoolar and more using SerpApi.

This project is an implementation of SerpApi in Golang 1.12 There is no dependency for this project.

This Go module is meant to scrape and parse Google results using SerpApi. The following services are provided:

SerpApi provides a script builder to get you started quickly.

An implementation example is provided here.

demo/demo.go

You take a look to our test.

test/google_seach_results_test.go

The full documentation is available here.

Installation

Go 1.10+ must be already installed.

go get -u github.com/serpapi/google-search-results-golang

Quick start

package main

import (
	"fmt"
	g "github.com/serpapi/google-search-results-golang"
)

parameter := map[string]string{
    "q":            "Coffee",
    "location":     "Portland"
}

query := NewGoogleSearch(parameter, api)
// Many search engine available: bing, yahoo, baidu, googemaps, googleproduct, googlescholar, ebay, walmart, youtube..

rsp, err := query.json()
results := rsp["organic_results"].([]interface{})
first_result := results[0].(map[string]interface{})
fmt.Println(ref["title"].(string))

This example runs a search about "coffee" using your secret api key.

The SerpApi service (backend)

  • searches on Google using the search: q = "coffee"
  • parses the messy HTML responses
  • return a standardizes JSON response
  • Format the request
  • Execute GET http request against SerpApi service
  • Parse JSON response into a deep hash Et voila..

Alternatively, you can search:

  • Bing using NewBingSearch method
  • Baidu using NewBaiduSearch method

See the playground to generate your code.

Example

Search API capability
parameter = {
  "q": "query",
  "google_domain": "Google Domain",
  "location": "Location Requested",
  "device": device,
  "hl": "Google UI Language",
  "gl": "Google Country",
  "safe": "Safe Search Flag",
  "num": "Number of Results",
  "start": "Pagination Offset",
  "api_key": "Your SERP API Key",
  "tbm": "nws|isch|shop",
  "tbs": "custom to be search criteria",
  "async": true|false,  // allow async 
  "output": "json|html" // output format
}

// api_key from https://serpapi.com/dashboard
api_key := "your personal API key"

// set search engine: google|yahoo|bing|ebay|yandex
engine := "yahoo"

// define the search search
search := NewSearch(engine, parameter, api_key)

// override an existing parameter
search.parameter["location"] = "Portland,Oregon,United States"

// search format return as raw html
data, err := search.GetHTML()

// search format returns a json
data, err := search.GetJSON()

(the full documentation)[https://serpapi.com/search-api]

Full example: [https://github.com/serpapi/google-search-results-golang/blob/master/demo/demo.go]

see below for more hands on examples.

Location API
var locationList SerpResponseArray
var err error
locationList, err = search.GetLocation("Austin", 3)

if err != nil {
  log.Println(err)
}
log.Println(locationList)

rsp contains the first 3 location matching Austin (Texas, Texas, Rochester)

Search Archive API

Run a search then get search result from the archive using the search archive API.

parameter := map[string]string{
  "q":        "Coffee",
  "location": "Portland"
  }

search := NewGoogleSearch(parameter, "your user key")
rsp, err := search.GetJSON()

if err != nil {
  log.Println("unexpected error", err)
}

searchID := rsp["search_metadata"].(map[string]interface{})["id"].(string)
searchArchive, err := search.GetSearchArchive(searchID)
if err != nil {
  log.Println(err)
  return
}

searchIDArchive := searchArchive["search_metadata"].(map[string]interface{})["id"].(string)
if searchIDArchive != searchID {
  log.Println("search_metadata.id do not match", searchIDArchive, searchID)
}

log.Println(searchIDArchive)

it prints the search ID from the archive.

Account API
var data SearchResult
var err error
data, err = search.GetAccount()

if err != nil {
  log.Println(err)
  return
}
log.Println(data)

data contains the account information.

Example by specification

We love true open source, continuous integration and Test Drive Development (TDD). We are using "go test" to test our infrastructure around the clock to achieve the best QoS (Quality Of Service).

The directory test/ includes specification/examples.

To run the test from bash using make

export API_KEY="your secret key"
make test
Error management

This library follows the basic error management solution provided by Go. A simple error is returned in case something goes wrong. The error wraps a simple error message.

Change log

  • 3.2
    • add naver search
    • add apple store search
  • 3.1
    • Add home depot search engine
  • 3.0
    • Naming convention change.
    • Rename Client to Search
    • Fix lint issue
    • Add walmart and youtube
  • 2.1
    • Add support for Yandex, Ebay, Yahoo
    • create HTTP search only once per SerpApiClient
  • 2.0 Rewrite fully the implementation to be more scalable in order to support multiple engines.
  • 1.3 Add support for Bing and Baidu
  • 1.2 Export NewGoogleSearch outside of the package.

Conclusion

SerpApi supports mutiple search engines and subservices all available for this Golang search.

For example: Using Google search. To enable a type of search, the field tbm (to be matched) must be set to:

  • isch: Google Images API.
  • nws: Google News API.
  • shop: Google Shopping API.
  • any other Google service should work out of the box.
  • (no tbm parameter): regular Google search.

The field tbs allows to customize the search even more.

The full documentation is available here.

Contributing

Contributions are welcome, feel to submit a pull request!

To run the tests:

make test

Documentation

Index

Constants

View Source 
const (
	// Current version
	VERSION = "3.2.0"
)

Variables

This section is empty.

Functions

This section is empty.

Types 

type Search struct {
	Engine     string
	Parameter  map[string]string
	ApiKey     string
	HttpSearch *http.Client
}

Search holds query

func NewAppleStoreSearch 

func NewAppleStoreSearch(parameter map[string]string, apiKey string) Search

NewAppleStoreSearch creates search for Apple store (itunes.apple.com)

func NewBaiduSearch 

func NewBaiduSearch(parameter map[string]string, apiKey string) Search

NewBaiduSearch creates search for baidu

func NewBingSearch 

func NewBingSearch(parameter map[string]string, apiKey string) Search

NewBingSearch creates search for bing

func NewEbaySearch 

func NewEbaySearch(parameter map[string]string, apiKey string) Search

NewEbaySearch creates search for ebay

func NewGoogleMapsSearch 

func NewGoogleMapsSearch(parameter map[string]string, apiKey string) Search

NewGoogleMapsSearch creates search for google_maps

func NewGoogleProductSearch 

func NewGoogleProductSearch(parameter map[string]string, apiKey string) Search

NewGoogleProductSearch creates search for google_product

func NewGoogleScholarSearch 

func NewGoogleScholarSearch(parameter map[string]string, apiKey string) Search

NewGoogleScholarSearch creates search for google_product

func NewGoogleSearch 

func NewGoogleSearch(parameter map[string]string, apiKey string) Search

NewGoogleSearch creates search for google

func NewHomeDepotSearch 

func NewHomeDepotSearch(parameter map[string]string, apiKey string) Search

NewHomeDepotSearch creates search for ebay

func NewNaverSearch 

func NewNaverSearch(parameter map[string]string, apiKey string) Search

NewNaverSearch creates search for Naver search engine

func NewSearch 

func NewSearch(engine string, parameter map[string]string, apiKey string) Search

NewSearch create generic search which support any search engine

func NewWalmartSearch 

func NewWalmartSearch(parameter map[string]string, apiKey string) Search

NewWalmartSearch creates search for ebay

func NewYahooSearch 

func NewYahooSearch(parameter map[string]string, apiKey string) Search

NewYahooSearch creates search for yahoo

func NewYandexSearch 

func NewYandexSearch(parameter map[string]string, apiKey string) Search

NewYandexSearch creates search for yandex

func NewYoutubeSearch 

func NewYoutubeSearch(parameter map[string]string, apiKey string) Search

NewYoutubeSearch creates search for ebay

func (*Search) GetAccount 

func (search *Search) GetAccount() (SearchResult, error)

GetAccount return account information

func (*Search) GetHTML 

func (search *Search) GetHTML() (*string, error)

GetHTML returns html as a string

func (*Search) GetJSON 

func (search *Search) GetJSON() (SearchResult, error)

GetJSON returns SearchResult containing

func (*Search) GetLocation 

func (search *Search) GetLocation(location string, limit int) (SearchResultArray, error)

GetLocation returns the standardize location takes location and limit as input.

func (*Search) GetSearchArchive 

func (search *Search) GetSearchArchive(searchID string) (SearchResult, error)

GetSearchArchive retrieve search from the archive using the Search Archive API

func (*Search) SetApiKey 

func (search *Search) SetApiKey(key string)

SetApiKey globaly set api_key

type SearchResult 

type SearchResult map[string]interface{}

SearchResult holds response

type SearchResultArray 

type SearchResultArray []interface{}

SearchResultArray hold response array

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.