Friday, May 22, 2015

RHive Setup and Sample Code

Congiguration:

install.packages("Rserve")
install.packages("rJava")
install.packages("RHive")
library(Rserve)
library(rJava)
library(RHive)
Sys.setenv(HADOOP_HOME="/home/ravi/apache/hadoop-1.0.4")
Sys.setenv(HIVE_HOME="/home/ravi/apache/hive-0.12.0/")
Sys.setenv()
rhive.env()
rhive.init()
rhive.connect(hiveServer2=FALSE)


If rJava does'nt get installed properly try below command

sudo apt-get install r-cran-rjava
 
sudo updatedb && locate libjvm.so
 
and then try  :

install.packages("rJava")

After that if error is:

Error: java.io.IOException: Mkdirs failed to create file:/home/rhive/lib/2.0-0.2


Sys.setenv("RHIVE_FS_HOME"="your RHive installation directory here e.g. /home/rhive")
This needs to be local directory on the node with hive installed, create one if doesnt exist. The user created (rstudio) have chown -R rights on this local directory.

For other erroes visit the below link :
http://dailyitsolutions.blogspot.in/2014_12_01_archive.html

Problem 2:
Rhive works well with simple queriess but does not work with complex queries or queries with aggregate SQL functions.
We get the following error code on R console(RStudio)
Error: java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask 
 OR
Following exception on Hiveserver2 console
org.apache.hadoop.security.AccessControlException: Permission denied: user=anonymous, access=EXECUTE, inode="/tmp/hadoop-yarn/staging/anonymous/.staging":trendwise:supergroup:drwx------

Solution:
To resolve the above problem, we need to change the permission mode of the /tmp folder. To do so type,
$hadoop fs -chmod -R 777 /tmp/hadoop-yarn

 Sample Code:


library(Rserve)
library(RHive)
Sys.setenv(HADOOP_HOME="/usr/local/hadoop")
Sys.setenv(HIVE_HOME="/usr/local/hive")
Sys.setenv("RHIVE_FS_HOME"="/home/trendwise/R/x86_64-pc-linux-gnu-library/3.2/RHive")
rhive.env()
rhive.init()
rhive.connect(hiveServer2=TRUE)
rhive.query("use database_name")
rhive.query("select * from table");





No comments:

Post a Comment