golang deal with text

Env

  • linux 64bit
  • go version go1.7.4 linux/amd64
  • physical machine
  • smartctl 6.2 2013-07-26 r3841 [x8664-linux-3.10.0-327.36.2.el7.x8664] (local build)

任务

使用smartctl命令收集物理机上的硬盘信息。需要的硬盘信息如下:

  • date: '2016-02-09', string
  • serial_number: MJ0351YNG9Z0XA, string
  • model: Hitachi HDS5C3030ALA630, string
  • capacity_bytes: 3000592982016, string
  • normalize: 100, int
  • raw: 32296, string

以上需要的信息都可以通过smartctl配合相关参数拿到,但是格式是很原始,需要我们根据需求抽取出真正需要的信息。

比如normalizeraw是硬盘的属性信息,可通过smartctl -A device获取:

➜  ~ sudo smartctl -A /dev/sda
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.10.0-327.36.2.el7.x86_64] (local build)  
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 10  
Vendor Specific SMART Attributes with Thresholds:  
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE  
  1 Raw_Read_Error_Rate     0x000f   118   099   006    Pre-fail  Always       -       199795992
  3 Spin_Up_Time            0x0003   097   097   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       149
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   077   060   030    Pre-fail  Always       -       54764573
  9 Power_On_Hours          0x0032   089   089   000    Old_age   Always       -       9792
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       149
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0  
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0  
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0  
188 Command_Timeout         0x0032   100   099   000    Old_age   Always       -       2 2 2  
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0  
190 Airflow_Temperature_Cel 0x0022   060   055   045    Old_age   Always       -       40 (Min/Max 26/40)  
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0  
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       25  
193 Load_Cycle_Count        0x0032   098   098   000    Old_age   Always       -       4708  
194 Temperature_Celsius     0x0022   040   045   000    Old_age   Always       -       40 (0 6 0 0 0)  
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0  
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0  
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0  
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       9049h+51m+35.226s  
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       4749106878  
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       126575500815  

厂商这类静态信息可通过sudo smartctl -i device获取:

➜  ~ sudo smartctl -i /dev/sda
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.10.0-327.36.2.el7.x86_64] (local build)  
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.14 (AF)  
Device Model:     ST1000DM003-1ER162  
Serial Number:    W4Y3Z28A  
LU WWN Device Id: 5 000c50 08a2c464e  
Firmware Version: CC46  
User Capacity:    1,000,204,886,016 bytes [1.00 TB]  
Sector Sizes:     512 bytes logical, 4096 bytes physical  
Rotation Rate:    7200 rpm  
Device is:        In smartctl database [for details use: -P show]  
ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b  
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)  
Local Time is:    Tue Feb  7 17:53:34 2017 CST

==> WARNING: A firmware update for this drive is available,
see the following Seagate web pages:  
http://knowledge.seagate.com/articles/en_US/FAQ/207931en  
http://knowledge.seagate.com/articles/en_US/FAQ/223651en

SMART support is: Available - device has SMART capability.  
SMART support is: Enabled  

我们该如何把里面的数据抽取出来呢?

想到的方法

  • 正则表达式
  • strings(golang stdlib):处理字符串,分割、删除等
  • strconv (golang stdlib):处理字符串转换。
  • os/exec:执行smartctl脚本,获取输出

注意事项

  • 对于格式确定的文本,尽量根据格式过滤掉不需要的数据,只保留需要的数据。
lines := "  
    data1\n data2\n needed1\n needed2\n
needed := strings.Split(lines, " ")  
  • 对于分割,通常strings.Fields()优于strings.Split()。
  • exec获取bash script的输出信息默认结尾是带一个\n换行符的,要记得处理掉: