To send messages from AxoSyslog to HDFS, complete the following steps.
If you want to use the Java-based modules of AxoSyslog (for example, the Elasticsearch, HDFS, or Kafka destinations), you must compile AxoSyslog with Java support.
Download and install the Java Runtime Environment (JRE), 1.7 (or newer). You can use OpenJDK or Oracle JDK, other implementations are not tested.
Install gradle version 2.2.1 or newer.
LD_LIBRARY_PATHto include the
libjvm.sofile, for example:
Note that many platforms have a simplified links for Java libraries. Use the simplified path if available. If you use a startup script to start AxoSyslog set
LD_LIBRARY_PATHin the script as well.
If you are behind an HTTP proxy, create a
modules/java-modules/directory. Set the proxy parameters in the file. For details, see The Gradle User Guide.
Download the Hadoop Distributed File System (HDFS) libraries (version 2.x) from http://hadoop.apache.org/releases.html.
Extract the HDFS libraries into a temporary directory, then collect the various
.jarfiles into a single directory (for example,
/opt/hadoop/lib/) where AxoSyslog can access them. You must specify this directory in the AxoSyslog configuration file. The files are located in the various
libdirectories under the
share/directory of the Hadoop release package. (For example, in Hadoop 2.7, required files are
hdfs/lib/\*, but this may change between Hadoop releases, so it is easier to copy every
.jarfile into a single directory.