Hive 메트릭

Hive 메트릭 (Hive Metrics)

Hive가 수집하는 메트릭은 HiveServer2 Web UI의 "Metrics Dump" 탭을 통해 확인할 수 있어요.

매트릭 덤프는 JMX를 통해 사용 가능한 모든 메트릭을 JSON으로 인코딩해 표시합니다.

대안으로, 해당 metric reporter를 활성화하면 메트릭을 HDFS에 직접 쓰거나, HS2 인스턴스가 실행 중인 로컬 파일 시스템의 JSON 파일에 쓰거나, 콘솔에 출력할 수 있어요. 기본적으로는 JMX reporter와 JSON 파일 reporter만 활성화되어 있습니다.

출처: 문서

본문

수집되는 메트릭

다음 메트릭들이 포함돼요 (괄호는 도입된 Hive 버전).

• jvm.pause.info-threshold (Hive 2.0.0)
• jvm.pause.warn-threshold (Hive 2.0.0)
• jvm.pause.extraSleepTime (Hive 2.0.0)
• open_connections (Hive 2.0.0)
• open_operations (Hive 2.0.0)
• cumulative_connection_count (Hive 2.1.0)
• metastore_hive_locks (Hive 2.0.0)
• zookeeper_hive_sharedlocks (Hive 2.0.0)
• zookeeper_hive_exclusivelocks (Hive 2.0.0)
• zookeeper_hive_semisharedlocks (Hive 2.0.0)
• exec_async_queue_size (Hive 2.0.0)
• exec_async_pool_size (Hive 2.0.0)
• HiveServer2 operations (Hive 2.0.0)
• HiveServer2 operations completed (Hive 2.0.0)
• SQL operations (Hive 2.1.0)
• SQL operations completed (Hive 2.1.0)
• init_total_count_dbs (Hive 2.1.0)
• init_total_count_tables (Hive 2.1.0)
• init_total_count_partitions (Hive 2.1.0)
• create_total_count_dbs (Hive 2.1.0)
• create_total_count_tables (Hive 2.1.0)
• create_total_count_partitions (Hive 2.1.0)
• delete_total_count_dbs (Hive 2.1.0)
• delete_total_count_tables (Hive 2.1.0)
• delete_total_count_partitions (Hive 2.1.0)
• directsql_errors (Hive 2.1.0)
• waiting_compile_ops (Hive 2.2.0)
• hive_mapred_tasks (Hive 2.2.0)
• hive_spark_tasks (Hive 2.2.0)
• hive_tez_tasks (Hive 2.2.0)
• acquireReadWriteLocks (Hive 0.8.0)
• compile (Hive 0.8.0)
• doAuthorization (Hive 0.8.0)
• Driver.execute (Hive 0.8.0)
• releaseLocks (Hive 0.8.0)
• prune-listing (Hive 0.8.0)
• partition-retrieving (Hive 0.8.0)
• PreHook (Hive 0.8.0)
• PostHook (Hive 0.8.0)
• FailureHook (Hive 0.8.0)
• parse (Hive 0.12.0)
• semanticAnalyze (Hive 0.12.0)
• getInputSummary (Hive 0.12.0)
• getSplits (Hive 0.12.0)
• runTasks (Hive 0.12.0)
• serializePlan (Hive 0.12.0)
• deserializePlan (Hive 0.12.0)
• clonePlan (Hive 0.12.0)
• task (Hive 0.12.0)
• optimizer (Hive 2.0.0)
• Driver.run (Hive 0.9.0)
• TezCompiler (Hive 2.1.0)
• TezSubmitToRunningDag (Hive 0.13.0)
• TezBuildDag (Hive 0.13.0)
• TezSubmitDag (Hive 0.13.0)
• TezRunDag (Hive 0.13.0)
• TezCreateVertex (Hive 0.13.0)
• TezRunVertex (Hive 0.13.0)
• TezInitializeProcessor (Hive 0.13.0)
• TezRunProcessor (Hive 0.13.0)
• TezInitializeOperators (Hive 0.13.0)
• LoadHashtable (Hive 0.13.0)
• SparkSubmitToRunning (Hive 1.1.0)
• SparkBuildPlan (Hive 1.1.0)
• SparkBuildRDDGraph (Hive 1.1.0)
• SparkSubmitJob (Hive 1.1.0)
• SparkRunJob (Hive 1.1.0)
• SparkCreateTran (Hive 1.1.0)
• SparkRunStage (Hive 1.1.0)
• SparkInitializeOperators (Hive 1.1.0)
• SparkGenerateTaskTree (Hive 1.1.0)
• SparkFlushHashTable (Hive 1.1.0)
• SparkOptimizeOperatorTree (Hive 1.1.0)
• SparkOptimizeTaskTree (Hive 1.1.0)
• hs2_open_sessions (Hive 2.2.0)
• hs2_active_sessions (Hive 2.2.0)
• hs2_abandoned_sessions (Hive 2.2.0)
• hs2_avg_open_session_time (Hive 2.2.0)
• hs2_avg_active_session_time (Hive 2.2.0)
• hs2_submitted_queries (Hive 2.2.0)
• hs2_compiling_queries (Hive 2.2.0)
• hs2_executing_queries (Hive 2.2.0)
• hs2_failed_queries (Hive 2.2.0)
• hs2_succeeded_queries (Hive 2.2.0)
• GarbageCollectorMetricSet (Hive 1.3.0) - GarbageCollectorMXBeans의 속성 게시
• MemoryUsageGaugeSet (Hive 1.3.0) - MemoryMXBeans의 속성 게시
• ThreadStatesGaugeSet (Hive 1.3.0) - ThreadMXBeans의 속성 게시
• ClassLoadingGaugeSet (Hive 1.3.0) - ClassLoadingMXBeans의 속성 게시
• BufferPoolMetricSet (Hive 1.3.0) - BufferPool JMX bean의 속성 게시
• compaction_num_working (Hive 4.0.0)
• compaction_num_initiated (Hive 4.0.0)
• compaction_num_failed (Hive 4.0.0)
• compaction_num_succeeded (Hive 4.0.0)
• compaction_num_did_not_initiate (Hive 4.0.0)
• compaction_num_ready_for_cleaning (Hive 4.0.0)
• compaction_oldest_enqueue_age_in_sec (Hive 4.0.0)
• api_compaction_initiator_cycle (Hive 4.0.0)
• api_compaction_cleaner_cycle_minor (Hive 4.0.0)
• api_compaction_worker_cycle_minor (Hive 4.0.0)
• num_aborted_transactions (Hive 4.0.0)
• oldest_aborted_txn_id (Hive 4.0.0)
• oldest_aborted_txn_age_in_sec (Hive 4.0.0)
• total_num_aborted_transactions (Hive 4.0.0)
• total_num_committed_transactions (Hive 4.0.0)
• total_num_timed_out_transactions (Hive 4.0.0)
• num_locks (Hive 4.0.0)
• oldest_lock_age_in_sec (Hive 4.0.0)
• compaction_num_txn_to_writeid (Hive 4.0.0)
• compaction_num_completed_txn_components (Hive 4.0.0)
• compaction_num_initiators (Hive 4.0.0)
• compaction_num_workers (Hive 4.0.0)
• compaction_num_initiator_versions (Hive 4.0.0)
• compaction_num_worker_versions (Hive 4.0.0)
• oldest_open_repl_txn_id (Hive 4.0.0)
• oldest_open_non_repl_txn_id (Hive 4.0.0)
• oldest_open_repl_txn_age_in_sec (Hive 4.0.0)
• oldest_open_non_repl_txn_age_in_sec (Hive 4.0.0)
• tables_with_x_aborted_transactions (Hive 4.0.0)
• num_writes_to_disabled_compaction_table (Hive 4.0.0)
• oldest_ready_for_cleaning_age_in_sec (Hive 4.0.0)
• compaction_num_active_deltas (Hive 4.0.0)
• compaction_num_small_deltas (Hive 4.0.0)
• compaction_initiator_failure_counter (Hive 4.0.0)
• compaction_cleaner_failure_counter (Hive 4.0.0)
• compaction_initiator_cycle_duration (Hive 4.0.0)
• compaction_cleaner_cycle_duration (Hive 4.0.0)
• compaction_oldest_working_age_in_sec (Hive 4.0.0)
• compaction_oldest_cleaning_age_in_sec (Hive 4.0.0)
• compaction_num_obsolete_deltas (Hive 4.0.0)

메트릭 관련 구성 속성은 Metrics 문서에서 확인할 수 있어요.

HiveServer2에 대한 더 자세한 내용은 HiveServer2 Overview를 참고하세요.

더 알아보기 (Learn more)

Hive 메트릭은 HiveServer2 Web UI의 Metrics Dump 탭에서 JSON으로 확인하거나, reporter를 통해 HDFS·로컬 파일·콘솔로 내보낼 수 있어요. 세션, 쿼리, 컴팩션, 트랜잭션 등 다양한 상태 지표를 버전별로 수집하므로 운영·모니터링에 활용하기 좋습니다.