用newLISP读取Hive的元数据

用newLISP读取Hive的元数据

思想是通过调用hive -e命令，然后解析返回的结果。

下面的hive.lsp文件是一个例子，提供了读取本地hive数据库名称，表名，表结构和根据hive表的创建语句生成对应的MySQL的创建语句。

最新代码参考我在gitlab.com上的开源项目：https://gitlab.com/newlisp/armory.git

;; hive.lsp

;; @module hive
;; @description hive module provides some features for manuplating local hive database
;; @note must make sure the hive env are correct
;; @location hive.lsp
;; @version 0.1.0
;; @author Dean Chen
;; @example
;; (load "/opt/newlisp_util/env.lsp")
;; (HIVE:show-dbs)
;; (HIVE:show-tables db-name)
;; (HIVE:show-create-table-statment db-name table-name)

(context 'HIVE)

;; @syntax (HIVE:show-dbs)
(define (show-dbs)
  (exec (format "hive -e 'show databases' 2>&1")))

;; @syntax (HIVE:show-tables db-name)
(define (show-tables db-name)
  (exec (format "hive -e 'use %s; show tables' 2>&1" db-name)))

;; @syntax (HIVE:describe-table db-name table-name)
(define (describe-table db-name table-name)
  (exec (format "hive -e 'use %s; describe %s' 2>&1" db-name table-name)))

;; @syntax (HIVE:show-create-table db-name table-name)
(define (show-create-table db-name table-name)
  (exec (format "hive -e 'use %s; show create table %s' 2>&1" db-name table-name)))

;; @syntax (HIVE:gen-mysql-create-table hive-db-name hive-table-name)
;; @note this method use list index to locate CREATE EXTERNAL statement
;; it could be improved later for getting more compatibilty
(define (gen-mysql-create-table db-name table-name)
  (set 's-list (show-create-table db-name table-name))
  (set 'header (replace "CREATE EXTERNAL" (s-list 6) "CREATE"))
  (set 'mysql-create-table-statement "")
  (set 'mysql-create-table-statement (append mysql-create-table-statement header))
  (set 'body (slice s-list 7))
  (catch (dolist (field body)
    (if (regex "PARTITIONED BY" field)
        (begin
  (set 'sub-length (- (length mysql-create-table-statement) 1))
  (set 'mysql-create-table-statement (slice mysql-create-table-statement 0 sub-length))
  (throw mysql-create-table-statement))
      (begin
      (set 'column-list (parse field ","))
      (dolist (column column-list)
  (set 'column-name (parse (trim column) " "))
  (if column-name
      (begin
      (set 'mysql-create-table-statement (append mysql-create-table-statement (column-name 0)))
      (set 'mysql-create-table-statement (append mysql-create-table-statement " "))
      (set 'mysql-create-table-statement (append mysql-create-table-statement (replace "string" (column-name 1) "varchar(1024)")))
      (set 'mysql-create-table-statement (append mysql-create-table-statement ","))
      )
    )))))))

newLISP 的详细介绍：请点这里
newLISP 的下载地址：请点这里

为Emacs配置newLISP开发环境

newLISP做GitLab系统备份

newLISP 遍历目录树，清理编译目录

本文永久更新链接地址：