1.Hive的安装

    • 安装JDK-略过
    • 安装Hadoop-略过
    • 安装Mysql-略过

    2安装Hive

    2. hadoop@Master:~$ sudo cp -R apache-hive-1.1.1-bin /usr/local/hive
    3. hadoop@Master:~$ sudo chmod -R 775 /usr/local/hive/
    4. hadoop@Master:~$ sudo chown hadoop:hadoop /usr/local/hive/
    5. #修改/etc/profile加入HIVE_HOME的变量
    6. export HIVE_HOME=/usr/local/hive
    7. export PATH=$PATH:$HIVE_HOME/bin
    8. export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:/usr/local/hive/lib
    10. #修改hive/conf下的几个template模板并重命名为其他
    11. cp hive-env.sh.template hive-env.sh
    12. cp hive-default.xml.template hive-site.xml
    14. #配置hive-env.sh文件，指定HADOOP_HOME
    15. HADOOP_HOME=/usr/local/hadoop
    17. #修改hive-site.xml文件，指定MySQL数据库驱动、数据库名、用户名及密码，修改的内容如下所示
    18. <property>
    19.   <name>javax.jdo.option.ConnectionURL</name>
    20.   <value>jdbc:mysql://192.168.1.178:3306/hive?createDatabaseIfNotExist=true</value>
    21.   <description>JDBC connect string for a JDBC metastore</description>
    22. </property>
    23. <property>
    24.   <name>javax.jdo.option.ConnectionDriverName</name>
    25.   <description>Driver class name for a JDBC metastore</description>
    26. </property>
    27. <property>
    28.   <name>javax.jdo.option.ConnectionUserName</name>
    29.   <value>hive</value>
    30.   <description>username to use against metastore database</description>
    31. </property>
    32. <property>
    34.   <value>hive</value>
    35.   <description>password to use against metastore database</description>
    36. </property>
    38. 其中：
    39. javax.jdo.option.ConnectionURL参数指定的是Hive连接数据库的连接字符串；
    40. javax.jdo.option.ConnectionDriverName参数指定的是驱动的类入口名称；
    41. javax.jdo.option.ConnectionUserName参数指定了数据库的用户名；
    42. javax.jdo.option.ConnectionPassword参数指定了数据库的密码。
    1. export JAVA_HOME=/usr/lib/jvm
    2. export HADOOP_HOME=/usr/local/hadoop
    3. export HIVE_HOME=/usr/local/hive

    4下载mysql-connector-java-5.1.27-bin.jar文件，并放到$HIVE_HOME/lib目录下

    1. hadoop fs -mkdir /tmp
    2. hadoop fs -mkdir /user/hive/warehouse
    3. hadoop fs -chmod g+w /tmp
    4. hadoop fs -chmod g+w /user/hive/warehouse

    6启动hadoop。进入hive shell，输入一些命令查看

    1. hive
    2. show databases;
    3. show tables;

    Hive使用实例

    1查询示例

    1. hive> SHOW TABLES;
    2. OK
    3. testuser
    4. Time taken: 0.707 seconds, Fetched: 1 row(s)
    6. hive> DESC testuser;
    7. OK
    8. id                      int                                         
    9. username                string                                      
    10. hive> SELECT * from testuser limit 10;
    11. OK
    12. 1    sssss
    14. Time taken: 0.865 seconds, Fetched: 2 row(s)
    15. hive> 
    17. hive> select count(1) from testuser;
    18. Query ID = hadoop_20160205004747_9d84aaca-887a-43a0-bad9-eddefe4e2219
    19. Total jobs = 1
    20. Launching Job 1 out of 1
    21. Number of reduce tasks determined at compile time: 1
    22. In order to change the average load for a reducer (in bytes):
    23.   set hive.exec.reducers.bytes.per.reducer=<number>
    24. In order to limit the maximum number of reducers:
    25.   set hive.exec.reducers.max=<number>
    26. In order to set a constant number of reducers:
    27.   set mapreduce.job.reduces=<number>
    28. Starting Job = job_1454604205731_0001, Tracking URL = http://Master:8088/proxy/application_1454604205731_0001/
    29. Kill Command = /usr/local/hadoop/bin/hadoop job  -kill job_1454604205731_0001
    30. Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
    31. 2016-02-05 00:48:11,942 Stage-1 map = 0%,  reduce = 0%
    32. 2016-02-05 00:48:19,561 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.38 sec
    33. 2016-02-05 00:48:28,208 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 2.77 sec
    34. MapReduce Total cumulative CPU time: 2 seconds 770 msec
    35. Ended Job = job_1454604205731_0001
    36. MapReduce Jobs Launched: 
    37. Stage-Stage-1: Map: 1  Reduce: 1   Cumulative CPU: 2.77 sec   HDFS Read: 6532 HDFS Write: 2 SUCCESS
    38. Total MapReduce CPU Time Spent: 2 seconds 770 msec
    39. OK
    40. 2
    41. Time taken: 35.423 seconds, Fetched: 1 row(s)

    通过这些消息,可以知道该查询生成了一个Mapreduce作业,Hive之美在于用户根本不需要知道MapReduce的存在,用户所需关心的,仅仅是使用一种类似于SQL的语言.

    1. 多次重复实现大量数据插入
    3. hive> insert overwrite table testuser
    4.     > select id,count(id)