hadoopconf

Documentation

Overview

Package hadoopconf provides utilities for reading and parsing Hadoop's xml configuration files.

Index

• type HadoopConf
  • func Load(path string) (HadoopConf, error)
  • func LoadFromEnvironment() (HadoopConf, error)
  • func (conf HadoopConf) Namenodes() []string

Types

type HadoopConf

type HadoopConf map[string]string

HadoopConf represents a map of all the key value configutation pairs found in a user's hadoop configuration files.

func Load

func Load(path string) (HadoopConf, error)

Load returns a HadoopConf object representing configuration from the specified path. It will parse core-site.xml, hdfs-site.xml, and mapred-site.xml.

If no configuration files could be found, Load returns a nil map. If the configuration files exist but there was an error opening or parsing them, that is returned as well.

func LoadFromEnvironment

func LoadFromEnvironment() (HadoopConf, error)

LoadFromEnvironment tries to locate the Hadoop configuration files based on the environment, and returns a HadoopConf object representing the parsed configuration. If the HADOOP_CONF_DIR environment variable is specified, it uses that, or if HADOOP_HOME is specified, it uses $HADOOP_HOME/conf.

If no configuration can be found, it returns a nil map. If the configuration files exist but there was an error opening or parsing them, that is returned as well.

func (HadoopConf) Namenodes

func (conf HadoopConf) Namenodes() []string

Namenodes returns the namenode hosts present in the configuration. The returned slice will be sorted and deduped. The values are loaded from fs.defaultFS (or the deprecated fs.default.name), or fields beginning with dfs.namenode.rpc-address.

To handle 'logical' clusters Namenodes will not return any cluster names found in dfs.ha.namenodes.<clustername> properties.

If no namenode addresses can befound, Namenodes returns a nil slice.