Hadoop plugin for HQ
Hadoop简介
Hadoop 是一个能够对大量数据进行分布式处理的软件框架。但是 Hadoop 是以一种可靠、高效、可伸缩的方式进行处理的。Hadoop 是可靠的,因为它假设计算元素和存储会失败,因此它维护多个工作数据副本,确保能够针对失败的节点重新分布处理。Hadoop 是高效的,因为它以并行的方式工作,通过并行处理加快处理速度。Hadoop 还是可伸缩的,能够处理 PB 级数据。此外,Hadoop 依赖于社区服务器,因此它的成本比较低,任何人都可以使用。
Hadoop是项目的总称,起源于作者儿子的一只玩具大象的名字。主要是由HDFS、MapReduce和Hbase组成。
HDFS是Google File System(GFS)的开源实现。
MapReduce是Google MapReduce的开源实现。
HBase是Google BigTable的开源实现
开源的Hyperic HQ软件可以对Hadoop DataNode server,Name Node server,Job Tracker等 实现监控。
Hadoop数据节点服务器
日志追踪支持: Yes
配置追踪支持: Yes
定制属性支持:
Name | Description |
---|---|
VmVersion | Vm Version |
VmVendor | Vm Vendor |
VmName | Vm Name |
支持的控制操作: None
支持的度量参数:
Name | Alias | Units | Category | Default On | Default Interval |
---|---|---|---|---|---|
Process Virtual Memory Size | ProcessVirtualMemorySize | B | UTILIZATION | false | 5 min |
Process Resident Memory Size | ProcessResidentMemorySize | B | UTILIZATION | true | 5 min |
Process Page Faults | ProcessPageFaults | none | UTILIZATION | false | 10 min |
Process Page Faults per Minute | ProcessPageFaults1m | none | UTILIZATION | false | 10 min |
Process Cpu System Time | ProcessCpuSystemTime | ms | UTILIZATION | false | 10 min |
Process Cpu System Time per Minute | ProcessCpuSystemTime1m | ms | UTILIZATION | false | 10 min |
Process Cpu User Time | ProcessCpuUserTime | ms | UTILIZATION | false | 10 min |
Process Cpu User Time per Minute | ProcessCpuUserTime1m | ms | UTILIZATION | false | 10 min |
Process Cpu Total Time | ProcessCpuTotalTime | ms | UTILIZATION | false | 10 min |
Process Cpu Total Time per Minute | ProcessCpuTotalTime1m | ms | UTILIZATION | false | 10 min |
Process Cpu Usage | ProcessCpuUsage | percentage | UTILIZATION | true | 5 min |
Process Start Time | ProcessStartTime | epoch-millis | AVAILABILITY | false | 5 min |
Process Open File Descriptors | ProcessOpenFileDescriptors | none | UTILIZATION | false | 5 min |
Process Threads | ProcessThreads | none | UTILIZATION | false | 5 min |
Availability | Availability | percentage | AVAILABILITY | true | 5 min |
Heap Memory Free | HeapMemoryFree | B | UTILIZATION | true | 5 min |
Thread Count | ThreadCount | none | UTILIZATION | true | 5 min |
Total HotSpot Compilation Time | TotalHotSpotCompilationTime | ms | PERFORMANCE | true | 5 min |
配置帮助:
一般日志和配置追踪属性
启用配置追踪 - 检查启用配置追踪
配置文件 - 逗号分隔的配置文件跟踪列表。相对文件是指相对于$ {安装路径}
启用日志追踪 - 检查启用日志追踪
跟踪事件日志级别 - 仅跟踪时间级别等于或大于这个级别.顺序是: [Error, Warn, Info, Debug]
映射:
FATAL,ERROR -> Error
WARN -> Warn
INFO -> Info
DEBUG -> Debug
登录模式匹配 - 包括匹配给定的正则表达式的消息。给定的模式可以是日志消息或正则表达式的子串。请参见:java.util.regex.Pattern。
登录模式排除 - 排除匹配给定的正则表达式的消息。
日志文件 - 逗号分隔的日志文件跟踪列表。相对文件是指相对于$ {安装路径}。
Hadoop数据节点FS数据集服务
日志跟踪技术支持:No
配置跟踪支持:No
支持自定义属性:No
支持的控制动作:None
支持的指标:
Name | Alias | Units | Category | Default On | Default Interval |
---|---|---|---|---|---|
Availability | Availability | percentage | AVAILABILITY | true | 10 min |
Remaining | Remaining | B | UTILIZATION | true | 5 min |
Capacity | Capacity | B | UTILIZATION | true | 5 min |
DfsUsed | DfsUsed | B | UTILIZATION | true | 5 min |
Hadoop数据节点RPC服务
日志追踪支持: No
配置追踪支持: No
定制属性支持: None
支持的控制操作: None
支持的参数:
Name | Alias | Units | Category | Default On | Default Interval |
---|---|---|---|---|---|
Availability | Availability | percentage | AVAILABILITY | true | 10 min |
Num Open Connections | NumOpenConnections | none | UTILIZATION | true | 5 min |
Call Queue Len | CallQueueLen | none | UTILIZATION | false | 5 min |
Rpc Ops Avg Processing Time | RpcOpsAvgProcessingTime | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Processing Time Max | RpcOpsAvgProcessingTimeMax | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Processing Time Min | RpcOpsAvgProcessingTimeMin | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Queue Time | RpcOpsAvgQueueTime | ms | UTILIZATION | true | 5 min |
Rpc Ops Avg Queue Time Max | RpcOpsAvgQueueTimeMax | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Queue Time Min | RpcOpsAvgQueueTimeMin | ms | UTILIZATION | false | 5 min |
Rpc Ops Number | RpcOpsNumber | none | UTILIZATION | false | 5 min |
Hadoop数据节点统计服务
日志追踪支持: No
配置追踪支持: No
定制属性支持: None
支持的控制操作: None
支持的参数:
Name | Alias | Units | Category | Default On | Default Interval |
---|---|---|---|---|---|
Availability | Availability | percentage | AVAILABILITY | true | 10 min |
Bytes Read | BytesRead | B | UTILIZATION | false | 10 min |
Bytes Read per Minute | BytesRead1m | B | UTILIZATION | true | 10 min |
Bytes Written | BytesWritten | B | UTILIZATION | false | 10 min |
Bytes Written per Minute | BytesWritten1m | B | UTILIZATION | true | 10 min |
Blocks Read | BlocksRead | none | UTILIZATION | false | 10 min |
Blocks Read per Minute | BlocksRead1m | none | UTILIZATION | false | 10 min |
Blocks Removed | BlocksRemoved | none | UTILIZATION | false | 10 min |
Blocks Removed per Minute | BlocksRemoved1m | none | UTILIZATION | false | 10 min |
Blocks Replicated | BlocksReplicated | none | UTILIZATION | false | 10 min |
Blocks Replicated per Minute | BlocksReplicated1m | none | UTILIZATION | false | 10 min |
Blocks Written | BlocksWritten | none | UTILIZATION | false | 10 min |
Blocks Written per Minute | BlocksWritten1m | none | UTILIZATION | false | 10 min |
Block Verification Failures | BlockVerificationFailures | none | UTILIZATION | false | 10 min |
Block Verification Failures per Minute | BlockVerificationFailures1m | none | UTILIZATION | false | 10 min |
Blocks Verified | BlocksVerified | none | UTILIZATION | false | 10 min |
Blocks Verified per Minute | BlocksVerified1m | none | UTILIZATION | false | 10 min |
Reads From Local Client | ReadsFromLocalClient | none | UTILIZATION | false | 10 min |
Reads From Local Client per Minute | ReadsFromLocalClient1m | none | UTILIZATION | false | 10 min |
Reads From Remote Client | ReadsFromRemoteClient | none | UTILIZATION | false | 10 min |
Reads From Remote Client per Minute | ReadsFromRemoteClient1m | none | UTILIZATION | false | 10 min |
Writes From Local Client | WritesFromLocalClient | none | UTILIZATION | false | 10 min |
Writes From Local Client per Minute | WritesFromLocalClient1m | none | UTILIZATION | false | 10 min |
Writes From Remote Client | WritesFromRemoteClient | none | UTILIZATION | false | 10 min |
Writes From Remote Client per Minute | WritesFromRemoteClient1m | none | UTILIZATION | false | 10 min |
Read Block Op Average Time | ReadBlockOpAverageTime | none | UTILIZATION | false | 5 min |
Read Block Op Max Time | ReadBlockOpMaxTime | none | UTILIZATION | false | 5 min |
Read Block Op Min Time | ReadBlockOpMinTime | none | UTILIZATION | false | 5 min |
Read Block Op Num | ReadBlockOpNum | none | UTILIZATION | false | 5 min |
Read Metadata Op Average Time | ReadMetadataOpAverageTime | none | UTILIZATION | false | 5 min |
Read Metadata Op Max Time | ReadMetadataOpMaxTime | none | UTILIZATION | false | 5 min |
Read Metadata Op Min Time | ReadMetadataOpMinTime | none | UTILIZATION | false | 5 min |
Read Metadata Op Num | ReadMetadataOpNum | none | UTILIZATION | false | 5 min |
Block Checksum Op Average Time | BlockChecksumOpAverageTime | none | UTILIZATION | false | 5 min |
Block Checksum Op Max Time | BlockChecksumOpMaxTime | none | UTILIZATION | false | 5 min |
Block Checksum Op Min Time | BlockChecksumOpMinTime | none | UTILIZATION | false | 5 min |
Block Checksum Op Num | BlockChecksumOpNum | none | UTILIZATION | false | 5 min |
Replace Block Op Average Time | ReplaceBlockOpAverageTime | none | UTILIZATION | false | 5 min |
Replace Block Op Max Time | ReplaceBlockOpMaxTime | none | UTILIZATION | false | 5 min |
Replace Block Op Min Time | ReplaceBlockOpMinTime | none | UTILIZATION | false | 5 min |
Replace Block Op Num | ReplaceBlockOpNum | none | UTILIZATION | false | 5 min |
Write Block Op Average Time | WriteBlockOpAverageTime | none | UTILIZATION | false | 5 min |
Write Block Op Max Time | WriteBlockOpMaxTime | none | UTILIZATION | false | 5 min |
Write Block Op Min Time | WriteBlockOpMinTime | none | UTILIZATION | false | 5 min |
Write Block Op Num | WriteBlockOpNum | none | UTILIZATION | false | 5 min |
Copy Block Op Average Time | CopyBlockOpAverageTime | none | UTILIZATION | false | 5 min |
Copy Block Op Max Time | CopyBlockOpMaxTime | none | UTILIZATION | false | 5 min |
Copy Block Op Min Time | CopyBlockOpMinTime | none | UTILIZATION | false | 5 min |
Copy Block Op Num | CopyBlockOpNum | none | UTILIZATION | false | 5 min |
Block Reports Average Time | BlockReportsAverageTime | none | UTILIZATION | false | 5 min |
Block Reports Max Time | BlockReportsMaxTime | none | UTILIZATION | false | 5 min |
Block Reports Min Time | BlockReportsMinTime | none | UTILIZATION | false | 5 min |
Block Reports Num | BlockReportsNum | none | UTILIZATION | false | 5 min |
Heartbeats Average Time | HeartbeatsAverageTime | none | UTILIZATION | false | 5 min |
Heartbeats Max Time | HeartbeatsMaxTime | none | UTILIZATION | false | 5 min |
Heartbeats Min Time | HeartbeatsMinTime | none | UTILIZATION | false | 5 min |
Heartbeats Num | HeartbeatsNum | none | UTILIZATION | false | 5 min |
Hadoop Job Tracker服务器
日志追踪支持: Yes
配置追踪支持: Yes
定制属性支持:
Name | Description |
---|---|
VmVersion | Vm Version |
VmVendor | Vm Vendor |
VmName | Vm Name |
支持的控制操作: None
支持的参数:
Name | Alias | Units | Category | Default On | Default Interval |
---|---|---|---|---|---|
Process Virtual Memory Size | ProcessVirtualMemorySize | B | UTILIZATION | false | 5 min |
Process Resident Memory Size | ProcessResidentMemorySize | B | UTILIZATION | true | 5 min |
Process Page Faults | ProcessPageFaults | none | UTILIZATION | false | 10 min |
Process Page Faults per Minute | ProcessPageFaults1m | none | UTILIZATION | false | 10 min |
Process Cpu System Time | ProcessCpuSystemTime | ms | UTILIZATION | false | 10 min |
Process Cpu System Time per Minute | ProcessCpuSystemTime1m | ms | UTILIZATION | false | 10 min |
Process Cpu User Time | ProcessCpuUserTime | ms | UTILIZATION | false | 10 min |
Process Cpu User Time per Minute | ProcessCpuUserTime1m | ms | UTILIZATION | false | 10 min |
Process Cpu Total Time | ProcessCpuTotalTime | ms | UTILIZATION | false | 10 min |
Process Cpu Total Time per Minute | ProcessCpuTotalTime1m | ms | UTILIZATION | false | 10 min |
Process Cpu Usage | ProcessCpuUsage | percentage | UTILIZATION | true | 5 min |
Process Start Time | ProcessStartTime | epoch-millis | AVAILABILITY | false | 5 min |
Process Open File Descriptors | ProcessOpenFileDescriptors | none | UTILIZATION | false | 5 min |
Process Threads | ProcessThreads | none | UTILIZATION | false | 5 min |
Availability | Availability | percentage | AVAILABILITY | true | 5 min |
Heap Memory Free | HeapMemoryFree | B | UTILIZATION | true | 5 min |
Thread Count | ThreadCount | none | UTILIZATION | true | 5 min |
Total HotSpot Compilation Time | TotalHotSpotCompilationTime | ms | PERFORMANCE | true | 5 min |
一般日志和配置追踪属性
启用配置追踪 - 检查启用配置追踪
配置文件 - 逗号分隔的配置文件跟踪列表。相对文件是指相对于$ {安装路径}
启用日志追踪 - 检查启用日志追踪
跟踪事件日志级别 - 仅跟踪时间级别等于或大于这个级别.顺序是: [Error, Warn, Info, Debug]
映射:
FATAL,ERROR -> Error
WARN -> Warn
INFO -> Info
DEBUG -> Debug
登录模式匹配 - 包括匹配给定的正则表达式的消息。给定的模式可以是日志消息或正则表达式的子串。请参见:java.util.regex.Pattern.
登录模式排除 - 排除匹配给定的正则表达式的消息。
日志文件 - 逗号分隔的日志文件跟踪列表。相对文件是指相对于$ {安装路径}。
Hadoop Job Tracker RPC服务
日志追踪支持: Yes
配置追踪支持: Yes
定制属性支持:None
支持的控制操作: None
支持的参数:
Name | Alias | Units | Category | Default On | Default Interval |
---|---|---|---|---|---|
Availability | Availability | percentage | AVAILABILITY | true | 10 min |
Num Open Connections | NumOpenConnections | none | UTILIZATION | true | 5 min |
Call Queue Len | CallQueueLen | none | UTILIZATION | false | 5 min |
Rpc Ops Avg Processing Time | RpcOpsAvgProcessingTime | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Processing Time Max | RpcOpsAvgProcessingTimeMax | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Processing Time Min | RpcOpsAvgProcessingTimeMin | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Queue Time | RpcOpsAvgQueueTime | ms | UTILIZATION | true | 5 min |
Rpc Ops Avg Queue Time Max | RpcOpsAvgQueueTimeMax | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Queue Time Min | RpcOpsAvgQueueTimeMin | ms | UTILIZATION | false | 5 min |
Rpc Ops Number | RpcOpsNumber | none | UTILIZATION | false | 5 min |
Hadoop Name Node服务器
日志追踪支持: Yes
配置追踪支持: Yes
定制属性支持:
支持的控制操作: None
支持的参数:
Name | Alias | Units | Category | Default On | Default Interval |
---|---|---|---|---|---|
Process Virtual Memory Size | ProcessVirtualMemorySize | B | UTILIZATION | false | 5 min |
Process Resident Memory Size | ProcessResidentMemorySize | B | UTILIZATION | true | 5 min |
Process Page Faults | ProcessPageFaults | none | UTILIZATION | false | 10 min |
Process Page Faults per Minute | ProcessPageFaults1m | none | UTILIZATION | false | 10 min |
Process Cpu System Time | ProcessCpuSystemTime | ms | UTILIZATION | false | 10 min |
Process Cpu System Time per Minute | ProcessCpuSystemTime1m | ms | UTILIZATION | false | 10 min |
Process Cpu User Time | ProcessCpuUserTime | ms | UTILIZATION | false | 10 min |
Process Cpu User Time per Minute | ProcessCpuUserTime1m | ms | UTILIZATION | false | 10 min |
Process Cpu Total Time | ProcessCpuTotalTime | ms | UTILIZATION | false | 10 min |
Process Cpu Total Time per Minute | ProcessCpuTotalTime1m | ms | UTILIZATION | false | 10 min |
Process Cpu Usage | ProcessCpuUsage | percentage | UTILIZATION | true | 5 min |
Process Start Time | ProcessStartTime | epoch-millis | AVAILABILITY | false | 5 min |
Process Open File Descriptors | ProcessOpenFileDescriptors | none | UTILIZATION | false | 5 min |
Process Threads | ProcessThreads | none | UTILIZATION | false | 5 min |
Availability | Availability | percentage | AVAILABILITY | true | 5 min |
Heap Memory Free | HeapMemoryFree | B | UTILIZATION | true | 5 min |
Thread Count | ThreadCount | none | UTILIZATION | true | 5 min |
Total HotSpot Compilation Time | TotalHotSpotCompilationTime | ms | PERFORMANCE | true | 5 min |
一般日志和配置追踪属性
启用配置追踪 - 检查启用配置追踪
配置文件 - 逗号分隔的配置文件跟踪列表。相对文件是指相对于$ {安装路径}
启用日志追踪 - 检查启用日志追踪
跟踪事件日志级别 - 仅跟踪时间级别等于或大于这个级别.顺序是: [Error, Warn, Info, Debug]
映射:
FATAL,ERROR -> Error
WARN -> Warn
INFO -> Info
DEBUG -> Debug
登录模式匹配 - 包括匹配给定的正则表达式的消息。给定的模式可以是日志消息或正则表达式的子串。请参见:java.util.regex.Pattern.
登录模式排除 - 排除匹配给定的正则表达式的消息。
日志文件 - 逗号分隔的日志文件跟踪列表。相对文件是指相对于$ {安装路径}。
Hadoop Name Node FS Namesystem服务
日志追踪支持: No
配置追踪支持: No
定制属性支持: None
支持的控制操作: None
支持的参数:
Name | Alias | Units | Category | Default On | Default Interval |
---|---|---|---|---|---|
Availability | Availability | percentage | AVAILABILITY | true | 10 min |
Capacity Total | CapacityTotal | B | UTILIZATION | false | 5 min |
Capacity Used | CapacityUsed | B | UTILIZATION | false | 5 min |
Capacity Remaining | CapacityRemaining | B | UTILIZATION | true | 5 min |
Total Load | TotalLoad | none | UTILIZATION | true | 5 min |
Blocks Total | BlocksTotal | none | UTILIZATION | false | 5 min |
Files Total | FilesTotal | none | UTILIZATION | false | 5 min |
Pending Replication Blocks | PendingReplicationBlocks | none | UTILIZATION | true | 5 min |
Under Replicated Blocks | UnderReplicatedBlocks | none | UTILIZATION | false | 5 min |
Scheduled Replication Blocks | ScheduledReplicationBlocks | none | UTILIZATION | false | 5 min |
Hadoop Name Node RP服务
日志追踪支持: No
配置追踪支持: No
定制属性支持: None
支持的控制操作: None
支持的参数:
Name | Alias | Units | Category | Default On | Default Interval |
---|---|---|---|---|---|
Availability | Availability | percentage | AVAILABILITY | true | 10 min |
Num Open Connections | NumOpenConnections | none | UTILIZATION | true | 5 min |
Call Queue Len | CallQueueLen | none | UTILIZATION | false | 5 min |
Rpc Ops Avg Processing Time | RpcOpsAvgProcessingTime | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Processing Time Max | RpcOpsAvgProcessingTimeMax | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Processing Time Min | RpcOpsAvgProcessingTimeMin | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Queue Time | RpcOpsAvgQueueTime | ms | UTILIZATION | true | 5 min |
Rpc Ops Avg Queue Time Max | RpcOpsAvgQueueTimeMax | ms | UTILIZATION | false | 5 min |
Rpc Ops Avg Queue Time Min | RpcOpsAvgQueueTimeMin | ms | UTILIZATION | false | 5 min |
Rpc Ops Number | RpcOpsNumber | none | UTILIZATION | false | 5 min |
Hadoop命名节点统计服务
日志追踪支持: No
配置追踪支持: No
定制属性支持: None
支持的控制操作: None
支持的参数:
Name | Alias | Units | Category | Default On | Default Interval |
---|---|---|---|---|---|
Availability | Availability | percentage | AVAILABILITY | true | 10 min |
Block Report Average Time | BlockReportAverageTime | ms | UTILIZATION | true | 5 min |
Block Report Max Time | BlockReportMaxTime | ms | UTILIZATION | false | 5 min |
Block Report Min Time | BlockReportMinTime | ms | UTILIZATION | false | 5 min |
Block Report Num | BlockReportNum | none | UTILIZATION | false | 10 min |
Block Report Num per Minute | BlockReportNum1m | none | UTILIZATION | false | 10 min |
Journal Transaction Average Time | JournalTransactionAverageTime | ms | UTILIZATION | true | 5 min |
Journal Transaction Num | JournalTransactionNum | none | UTILIZATION | false | 10 min |
Journal Transaction Num per Minute | JournalTransactionNum1m | none | UTILIZATION | false | 10 min |
Journal Transaction Max Time | JournalTransactionMaxTime | ms | UTILIZATION | false | 5 min |
Journal Transaction Min Time | JournalTransactionMinTime | ms | UTILIZATION | false | 5 min |
Journal Sync Average Time | JournalSyncAverageTime | ms | UTILIZATION | false | 5 min |
Journal Sync Max Time | JournalSyncMaxTime | ms | UTILIZATION | false | 5 min |
Journal Sync Min Time | JournalSyncMinTime | ms | UTILIZATION | false | 5 min |
Journal Sync Num | JournalSyncNum | none | UTILIZATION | false | 10 min |
Journal Sync Num per Minute | JournalSyncNum1m | none | UTILIZATION | false | 10 min |
Safemode Time | SafemodeTime | ms | UTILIZATION | false | 5 min |
FS Image Load Time | FSImageLoadTime | ms | UTILIZATION | false | 5 min |
Num Files Created | NumFilesCreated | none | UTILIZATION | false | 10 min |
Num Files Created per Minute | NumFilesCreated1m | none | UTILIZATION | false | 10 min |
Num Files Listed | NumFilesListed | none | UTILIZATION | false | 10 min |
Num Files Listed per Minute | NumFilesListed1m | none | UTILIZATION | false | 10 min |
Num Get Listing Ops | NumGetListingOps | none | UTILIZATION | false | 10 min |
Num Get Listing Ops per Minute | NumGetListingOps1m | none | UTILIZATION | false | 10 min |
Num Create File Ops | NumCreateFileOps | none | UTILIZATION | false | 10 min |
Num Create File Ops per Minute | NumCreateFileOps1m | none | UTILIZATION | false | 10 min |
Num Delete File Ops | NumDeleteFileOps | none | UTILIZATION | false | 10 min |
Num Delete File Ops per Minute | NumDeleteFileOps1m | none | UTILIZATION | false | 10 min |
Num Add Block Ops | NumAddBlockOps | none | UTILIZATION | false | 10 min |
Num Add Block Ops per Minute | NumAddBlockOps1m | none | UTILIZATION | false | 10 min |
Num Get Block Locations | NumGetBlockLocations | none | UTILIZATION | false | 10 min |
Num Get Block Locations per Minute | NumGetBlockLocations1m | none | UTILIZATION | false | 10 min |
Num Files Renamed | NumFilesRenamed | none | UTILIZATION | false | 10 min |
Num Files Renamed per Minute | NumFilesRenamed1m | none | UTILIZATION | false | 10 min |