urlbisect

package module
Version: v0.0.0-...-98f50cc Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 21, 2021 License: MIT Imports: 8 Imported by: 0

README

urlbisect

For URLs with (gapless) autoincrement ids, find the upper bound using binary search. Reports the last successful id and URL.

Installation

$ go get github.com/miku/urlbisect/...

Usage

$ urlbisect -u "https://freidok.uni-freiburg.de/data/@" -f 0 -t 500000
169476 https://freidok.uni-freiburg.de/data/169476

Verbose.

$ urlbisect -v -u "https://freidok.uni-freiburg.de/data/@" -f 0 -t 500000
2020/11/11 21:53:47 [404] https://freidok.uni-freiburg.de/data/250000
2020/11/11 21:53:47 [200] https://freidok.uni-freiburg.de/data/125000
2020/11/11 21:53:47 [404] https://freidok.uni-freiburg.de/data/187500
2020/11/11 21:53:47 [200] https://freidok.uni-freiburg.de/data/156250
2020/11/11 21:53:47 [404] https://freidok.uni-freiburg.de/data/171875
2020/11/11 21:53:47 [200] https://freidok.uni-freiburg.de/data/164063
2020/11/11 21:53:48 [200] https://freidok.uni-freiburg.de/data/167969
2020/11/11 21:53:48 [404] https://freidok.uni-freiburg.de/data/169922
2020/11/11 21:53:48 [200] https://freidok.uni-freiburg.de/data/168946
2020/11/11 21:53:48 [200] https://freidok.uni-freiburg.de/data/169434
2020/11/11 21:53:48 [404] https://freidok.uni-freiburg.de/data/169678
2020/11/11 21:53:48 [404] https://freidok.uni-freiburg.de/data/169556
2020/11/11 21:53:48 [404] https://freidok.uni-freiburg.de/data/169495
2020/11/11 21:53:48 [200] https://freidok.uni-freiburg.de/data/169465
2020/11/11 21:53:48 [404] https://freidok.uni-freiburg.de/data/169480
2020/11/11 21:53:48 [200] https://freidok.uni-freiburg.de/data/169473
2020/11/11 21:53:48 [404] https://freidok.uni-freiburg.de/data/169477
2020/11/11 21:53:48 [200] https://freidok.uni-freiburg.de/data/169475
2020/11/11 21:53:48 [200] https://freidok.uni-freiburg.de/data/169476
169476 https://freidok.uni-freiburg.de/data/169476

Another example.

$ urlbisect -v -u https://www.semanticscholar.org/author/@ -f 0 -t 500000000
2020/11/11 22:15:22 [404] https://www.semanticscholar.org/author/250000000
2020/11/11 22:15:22 [404] https://www.semanticscholar.org/author/125000000
2020/11/11 22:15:23 [404] https://www.semanticscholar.org/author/62500000
2020/11/11 22:15:24 [200] https://www.semanticscholar.org/author/31250000
2020/11/11 22:15:25 [200] https://www.semanticscholar.org/author/46875000
2020/11/11 22:15:25 [404] https://www.semanticscholar.org/author/54687500
2020/11/11 22:15:27 [200] https://www.semanticscholar.org/author/50781250
2020/11/11 22:15:27 [404] https://www.semanticscholar.org/author/52734375
2020/11/11 22:15:27 [404] https://www.semanticscholar.org/author/51757813
2020/11/11 22:15:28 [404] https://www.semanticscholar.org/author/51269532
2020/11/11 22:15:29 [200] https://www.semanticscholar.org/author/51025391
2020/11/11 22:15:29 [200] https://www.semanticscholar.org/author/51147462
2020/11/11 22:15:30 [200] https://www.semanticscholar.org/author/51208497
2020/11/11 22:15:31 [200] https://www.semanticscholar.org/author/51239015
2020/11/11 22:15:31 [404] https://www.semanticscholar.org/author/51254274
2020/11/11 22:15:31 [404] https://www.semanticscholar.org/author/51246645
2020/11/11 22:15:32 [200] https://www.semanticscholar.org/author/51242830
2020/11/11 22:15:32 [200] https://www.semanticscholar.org/author/51244738
2020/11/11 22:15:32 [404] https://www.semanticscholar.org/author/51245692
2020/11/11 22:15:33 [200] https://www.semanticscholar.org/author/51245215
2020/11/11 22:15:34 [200] https://www.semanticscholar.org/author/51245454
2020/11/11 22:15:34 [404] https://www.semanticscholar.org/author/51245573
2020/11/11 22:15:35 [200] https://www.semanticscholar.org/author/51245514
2020/11/11 22:15:35 [404] https://www.semanticscholar.org/author/51245544
2020/11/11 22:15:35 [404] https://www.semanticscholar.org/author/51245529
2020/11/11 22:15:36 [200] https://www.semanticscholar.org/author/51245522
2020/11/11 22:15:36 [404] https://www.semanticscholar.org/author/51245526
2020/11/11 22:15:37 [200] https://www.semanticscholar.org/author/51245524
2020/11/11 22:15:37 [200] https://www.semanticscholar.org/author/51245525
51245525 https://www.semanticscholar.org/author/51245525

Example sites

Observations

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Bisect

func Bisect(base, placeholder, indicate404 string, redirect404 bool, min, max int) (int, error)

Bisect runs binary search on a template url with autoincrement numbers and will report back the last number that did yield a 200 OK.

func ScanHandle

func ScanHandle(min, max int, w io.Writer) error

Types

This section is empty.

Source Files

Directories

Path Synopsis
cmd
urlbisect
For URLs with gapless integer identifiers, urlbisect can return the highest number that returns a successful HTTP status code.
For URLs with gapless integer identifiers, urlbisect can return the highest number that returns a successful HTTP status code.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL