Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Do ¶
func Do(sfs RemoteFiles, workers int, dPath string) error
Do downloads and processes the sirene files
func FileTypeName ¶
FileTypeName returns the string representation of a FileType
func NewDateSirene ¶
Types ¶
type CSVImport ¶
type CSVImport struct { Path string File *os.File Kind FileType ZipName string ProgressChan chan *progress.Progress }
CSVImport is a struct helper to import a CSV file to the database It implementes the pgx.Copy interface
type RemoteFile ¶
type RemoteFile struct { Checksum api.Checksum URL string FileName string Path string Type FileType YearDay int OnDisk bool ExtractedFiles []string ProgressChan chan *progress.Progress }
RemoteFile is a struct that adds and remove some fields from a Resource struct and actually keep only useful fields
func NewFromResource ¶
func NewFromResource(r api.Resource, dPath string) (*RemoteFile, error)
NewFromResource takes an Resource and transforms it to a gouv_sirene.RemoteFile
func (*RemoteFile) CalculateChecksum ¶
func (rf *RemoteFile) CalculateChecksum() (string, error)
CalculateChecksum generates the checksum of the file using the hasher type as defined in the Checksum.Type field
func (*RemoteFile) ChecksumMatch ¶
func (rf *RemoteFile) ChecksumMatch() (bool, error)
ChecksumMatch calculates the checksum of the file on disk and checks if it matches the checksum defined in the Checksum.Value field
func (*RemoteFile) Download ¶
func (rf *RemoteFile) Download(dPath string) error
Download will download the file
func (*RemoteFile) Unzip ¶
func (rf *RemoteFile) Unzip(dPath string) error
Unzip will un-compress a zip archive moving all files and folders to an output directory
type RemoteFiles ¶
type RemoteFiles []*RemoteFile
RemoteFiles is a slice of pointers to RemoteFile
func GrabLatestFull ¶
func GrabLatestFull(dPath string) (RemoteFiles, error)
GrabLatestFull retrieves all the files that needs to be downloaded and applied to the database in inverse order (stock first, then each daily file)
func (RemoteFiles) String ¶
func (r RemoteFiles) String() string