WebFamulus

Spring Batch Testing with TestNG

2014-02-09T17:01:00.000+01:00

The following blog is a brief description of how to test a simple batch application purely with TestNG, without having recourse to the Spring TestContext Framework. The application happened to use Spring Batch, but the testing techniques are of course also applicable to an application written for JSR 352 (Batch Processing for Java Platform).

1. BatchArchiver: The Spring Batch Application

The BatchArchiver is a simple Spring Batch job developed to archive database back-up files. The database is instructed to dump a complete back-up every day at various times into a dedicated directory, using a formatted timestamp as a file name prefix. Once a month, files older than N days are moved to a different location, whereby only the last file of any given day is kept and the others are discarded. The remaining files are collected in a ZIP archive and moved to storage.

The application uses as external Java libraries Joda-Time (Joda.org) for date and time calculations and zt-zip (ZeroTurnaround) for the generation of ZIP archives. It was developed using Eclipse as an IDE and Maven as a build tool.

The batch process runs as follows:

The application tries read the source directory where the back-up files are located. If it is not accessible, the job throws an Exception an exits.
A temporary working directory and, if it does not exist already, a storage directory for the final JAR file are created.
The application reads the contents of the source directory. It creates an array of file objects whose creation date (last modified) is older than (predates) the configured cut-off date.
All files older than the cutoff-date are copied into the temporary working directory. The corresponding in the source directory files are then deleted.
The name of every file contains starts with a formatted timestamp of its date and time of its creation. This design ensures that sorting the files names will result in an ordered sequence reflecting the creation date. Based on this information, the application deletes all files except the last one created on every given day. At the end of this step, the working directory contains one file for every distinct day in the data set.
The remaining files in the working directory are collected in a JAR file which, after the compilation is finished, is moved to the storage directory.
Finally, a clean-up step deletes the temporary working directory.

Test Utilities

From a programmatic point of view, the process of the job is not complicated. It uses very basic operations: accessing, creating, and deleting files (where directories and actual files are both Java File objects). Every step of the batch job presupposes a directory structure in order to locate and/or deposit files. And, of course, the tests can only run if there are files to work with. For this purpose, a small utility program was written that can create, read, and delete directories, as well as create, write, and delete files. Since the batch job filters files based on date-time criteria, the utility can also generate series of timestamps (for a specified date range with variable number of instances per day) that are used for file name patterns and modification dates. It is thus possible to quickly create even large datasets of test files quickly and share the data sets among different test cases where appropriate.

Test Phases

It is understood that before the tests described here are undertaken, the class library of the batch job have been unit-tested. Unit tests are run during the test phase of the Maven build process and focus on individual classes methods, that is, small-grained details of the code. Integration tests, focussing on complete steps and transitions from one step to another, are run later during the integration-test phase. This separation is of great use when the code base is large and tests need a long time to complete, as both phases can be separated. It is irrelevant, however, for this small application where running the entire test suite is just a matter just seconds.

TestNG is used as a framework in both test phases. The possibilities to run parameterized tests (providing runtime parameters through configuration) and data-driven tests with the help of data providers (data generators that call test methods with different sets of data) are a great convenience when the test data is applicable to more than one method, class, or programming unit. More importantly, however, it makes it easy to run tests with any number of configurations and test data variables. Additional test cases are quickly implemented, if need be, by adding to the test data.

2. Unit Tests

Since Spring Batch processing steps are a single or combination of plain Java classes, all aspects of a batch job can be easily isolated and tested. The focus here is not on the product of steps, tasklets, or chunks, but rather on class methods.

Testing Tasklets

Tasklets commonly consist of reader, processor, and writer classes, many of which are provided by Spring Batch out-of-the-box. These do not need to be unit-tested. Custom implementations can be tested as POJOs and need to call the open(), process(), and write() methods, respectively, as well as any necessary helper methods. Since various steps in the batch process share similar operations (reading and writing files, directories, etc.), the unit tests can share the test data as well. TestNG's data provider facilitates testing by making the test data accessible without having to recreate the data for each test. Furthermore, if the set of test data increases, it is automatically delivered to all tests without any changes to the test code.

3. Integration Tests

Here TestNG's parameterized testing capabilities prove to be a great tool for running a large number of tests with minimal configuration. The sequence of steps of a Spring Batch job is configured in XML files. This configuration file determines which Java classes are used to execute a step and which step is the next one to execute. Batch jobs and steps are thus collections of Java classes that are executed in sequence. The execution of each individual class has been tested during the unit test phase. Integration testing here takes a job step as the basic unit and asserts the correctness of the product of each step.

Proceeding in four stages, the basic structure of an integration test is as follows:

Set-up: prepare the test data.
Run the job.
Assert the correctness of the last step's output.
Clean-up.

Since a batch job is a sequence of executions defined by a configuration, the sequence will change when the configuration is changed. Hence, we'll perform step-by-step tests by creating configuration files for each step test:

test-class 1: step 1
test-class 2: steps 1 and 2
test-class 3: steps 1, 2, and 3
etc.

The actual tests occur in the third stage and must be implemented in separated class files. The remaining stages--set-up, job execution, and clean-up--are the same for all tests and can be implemented in a superclass that each test class inherits.


public BaseTest
{
     @BeforeTest
     @Parameters( { "jobContextFile", "applicationContextFile" } )
     public void configure( @Optional("none") String jobContextFile, 
                            @Optional("none") String applicationContextFile )
     {
         // read the configuration files passed in as runtime parameters
     }
 
 
     @BeforeTest
     @Parameters( { "createDirectoryList" } )
     public void createTestDirectories( @Optional("none") String createDirectoryList ) throws IOException
     {
         // create the test directory structure for the job to run
     }
 
     @BeforeTest
     @Parameters( { "targetDateOffsetInDays", "daysBefore","daysAfter", "fileInstancesPerDay" } )
     public void initialize( @Optional("-1") int targetDateOffsetInDays, 
        @Optional("-1") int daysBefore, 
        @Optional("-1") int daysAfter, 
        @Optional("-1") int fileInstancesPerDay )
     {
         // create the test data and run the job,
         // based on runtime parameters
     }
 
     @AfterTest
     public void cleanup() throws Exception
     {
         // delete test data 
     }
}

Through TestNG's @Parameter annotation it is now also possible to run each test class with different configuration files and test parameters:

<suite name="BatchArchiver.noProcess.test.driver" verbose="3">

    <test name="steps.integration.test.step05.1" preserve-order="true">
        <parameter name="jobContextFile" value="jobContext/in-memory-context-step05.xml" />
        <parameter name="applicationContextFile" value="appContext/in-memory-application-context.xml" />
        <parameter name="daysBefore" value="25"/>
        <parameter name="daysAfter" value="3"/>
        <classes>
            <class name="com.wayaleshi.batch.job.InMemoryJobStep05Test" preserve-order="true"/>
        </classes>
    </test>
 
    <test name="steps.integration.test.step05.2" preserve-order="true">
        <parameter name="jobContextFile" value="jobContext/in-memory-context-step05.xml" />
        <parameter name="applicationContextFile" value="appContext/in-memory-application-context.xml" />
        <parameter name="daysBefore" value="43"/>
        <parameter name="daysAfter" value="1"/>
        <classes>
            <class name="com.wayaleshi.batch.job.InMemoryJobStep05Test" preserve-order="true"/>
        </classes>
    </test>
        
</suite>

4. Using Quartz to Monitor SMART Hard Drives as an Automated Service

2013-12-06T14:28:00.000+01:00

Processing Steps

Using the Quartz Scheduler to monitor SMART-enabled hard drives makes it possible to automate the entire process from data generation and collection to reporting. Key advantages are minimal user intervention and the compilation of a consistent long-term data set. Quartz provides a powerful and higly flexible scheduling framework that can execute hundreds or thousands of computing tasks repeatedly without user intervention. Once set up, a schedule will run until the scheduler is interrupted or shut down. This is especially advantageous for recurring and repetitive tasks such as monitoring hardware. When the data gathered by such tasks is not only evaluated for possible errors or faulty behavior but also stored on a long-term basis, it is possible to compile a record of 'normal operation' from which newly occurring anomalies stand out more clearly.

SMART data is generated, of course, by running smartctl with the appropriate options on the host machine (see 1. below). If the host machine is accessible via SSH, WebFamulus can execute smartctl itself and capture the output. Alternatively, a script (Python, Perl, etc.) installed on the host machine and configured to be executed by a utility program such as cron can run smartctl and forward the output via email or file upload to WebFamulus.

SMART-Data Processing

The Smartmontools daemon (smartd) can be configured to send out emails when a new error is detected in a logfile or when a SMART attribute is failing. However, error logs are only updated when self-tests are run and, hence, an error may only be detected after a test has been initiated. SMART-enabled devices usually can run two self-tests: short and long. A short test, which takes about two to three minutes, updates the SMART attributes, checks the electrical and mechanical parts, and performs spot checks on the disk surface. A long test--which scans the entire disk surface for errors--may take hours to complete, depending on the size of the disk. Hence, regular testing of hard drives is imparative in order to minimize the danger of data loss due to equipment failure.

WebFamulus uses the Quartz scheduler and an extensive background database to generate, store, and analyze SMART data. All steps are automated and run on a daily, weekly, or longer schedule but, through a web interface, they can also be executed on demand. Furthermore, some jobs are executed as sequences. For example, before a daily report is issued, a collector job gathers all SMART reports that have not been processed and initiates a parsing job; then graphs and summary reports are compiled, and a mailer job sends the report to the stakeholders. (That is, steps 2 to 6 are executed as a sequence before step 7 is initiated. Otherwise, step 2, for example, may be running independently all the time at regular intervals.)

The advantages of regular testing and reporting are obvious: it establishes a basis of normal operation against which errors and faulty behavior is more easily noticed. A rise in the average working temperature, for example, could be an indication of a fan not working properly. This may go unnoticed if only occasional spot checks are done. With automated SMART testing and long-term, data storage the chances for early detection--before a serioushardware failure--are much higher.

Yet, even a small office may have tens of computers to monitor, and testing and evaluating can be a time-consuming task. Automating the process with a Quartz-based application like WebFamulus allows hundreds or thousands of hard drives to be monitored every day with minimal intervention beyond the initial set-up. And, having test results stored in a database makes it possible to document past performance and maintenance, which is very useful for system administrators when hard disk monitoring is part of a service agreement.

3. SMART temperature surveys

2013-12-05T14:00:00.000+01:00

Data Collection

Hard disks with SCT (SMART Command Transport) capabilities maintain a log of the working temperature of the drive. Most drives register entries at one minute intervals. The header of the SCT section of the smartctl output indicates the registration interval.

SCT-enabled drives are capable of recording temperatures of ±127°C. When the device is powered up, an illegal value is entered so that the starting point is is registered. (In smartctl output, this value appers as as a question mark '?'.) The log itself is a circular buffer. Hence, when the temperature log is read, only the values after the last question mark (if there is one) have a correct time reference (namely, N entries at the logging interval time before the date and time the SMART device was queried).

Reporting

I. Daily Reports

Temperature surveys reflect usage patters of computers. The following graph illustrates the typical use of laptop; this one having been put into sleep mode twice on the day sampled.

The same laptop running continuously as a low-volume server.

The following example shows an office workstation during a regular pattern utilization. The machine is used for image processing.

The same machine showing a pattern of high utilization. This pattern, a high temperature maintained over a long period of time, occurs occasionally. The pattern, when it occurs, is always the same: an ascending curve to a high temperature that maintained for several hours. Unfortunately, it has been impossible to ascertain what software was running at the time. The staff reported a normal use. A mechanical problem, like a malfunctioning fan, did not appear to have been the cause and was ruled out.

Monthly Reports

Summary reports over longer time periods help to pinpoint occurrences of irregularities. Of particular interest are, of course, unusual high and low temperatures. A rise in the average curve is indicative a prolonged operation a higher temperatures. (See 09/13 and 09/15, which reflect a brief and prolonged high disk use, respectively.) Upward spikes in low temperature values are due to the fact that the first SMART test of the day ran some time after the computer was turned on.

Annual Reports

Temperature measurements from computers that do not operate continuously—desktop workstations and laptops, for example—reflect the ambient temperatures. The data of the following graph was collected from a laptop that is usually turned off over night. This usually provided the lowest temperature of the day. (The high spikes in the low temperature are an indication that a SMART test was not run within the first two hours after computer was started.)
Since the power supply that recharges the battery is connected, the temperatures inside the casing are of course higher than the ambient room temperature. The low temperature range (in blue) before July and in September reflects seasonal temperatures in Italy: warm, summerly temperatures before July and in September, fall temperatures in October and November, and finally the onset of winter in January. The periods of July, August, mid-December and early January, were measured in Germany: colder than Italy during the summer, but warmer in the winter. Interestingly, unlike the ambient temperatures in Italy, those in Germany remain more or less constant. Conclusion: Building construction in Germany proves to be more weatherproof!

2. Data Collection and Reporting

2013-12-04T14:00:00.000+01:00

Server Side

Data collection, processing, and reporting is completely automated. The server is configured to accept data file uploads from registered clients, parse the data files, and issue alerts, if configured thresholds are passed, and summary reports at pre-defined intervals.

However, the server may also initiated data collection or interaction with smartmontools on its own. To contact computers on a local or remote network, it uses SSH, runs smartmontools via the command line, and then captures and processes the output data. A prerequisite is, of course, that the server is configured with the correct IP address, user name and password, and that the user has been granted access and execution permissions on the client side.

Client Side

In some environments, it is impossible, however, to connect to computer remotely--be it that it is located behind a firewall or that the execution of smartmontools commands requires permissions that cannot be granted to a remote user. In such cases, client-side scripts can issue smartmontools command and forward any output data to the server via email, file upload, or web service. The scripts can scheduled for execution with a utility like cron or a locally installed Quartz application.

Sample Daily Report of a Single Hard Drive

Weekly Spreadsheet Report

Firma:	NetzwerkÜberwachung GmbH
Kunde:	Müller-Schulz
Erfassungszeitraum:	2012-01-24 – 2012-01-31	Tage:	7
Berichte ausgewertet:	46


1. Festplattenübersicht

			Zählung
Computer	Hersteller	Modell	Laufzeit (Std.)	Ein/Aus	Start/Stop	Alter
DAK-210	Western Digital Caviar family	WDC WD400BB-60DGA0	17684	34945	42450	2.4
DAK-211	Seagate Barracuda 7200.11 family	ST3320813AS	12397	162	162	2.6
DAK-212	Seagate Barracuda 7200.11 family	ST3320813AS	6559	439	486	1.1
DAK-213	Western Digital Caviar family	WDC WD400BB-60DGA0	16869	2861	2862	5.9
DAK-214	Seagate U7 family	ST340012A	14766	2116	1	5.5
DAK-215	Hitachi Deskstar 7K1000.B	Hitachi HDT721064SLA360	4465	457	461	2.4
DAK-216	Western Digital Caviar SE Serial ATA family	WDC WD800JD-60LSA5	15837	5786	5788	6.7
DAK-217	Western Digital Caviar SE Serial ATA family	WDC WD800JD-60LSA5	2654	780	782	3.8
DAK-218	Western Digital Caviar SE Serial ATA family	WDC WD800JD-60LSA5	9386	680	682	6.1


2. Testübersicht

	Tests				Tests			Fehlerhafte Attribute
Computer	erster	letzter	Tage	Arbeitsstunden	Anzahl	bestanden	fehlerhaft	neu	vormals	LBA-Fehler
DAK-210	24.01.12 14:00	30.01.12 14:00	6	132	7	7	0	0	0	0
DAK-211	24.01.12 14:01	30.01.12 14:01	6	144	7	7	0	0	0	0
DAK-212	24.01.12 14:02	28.01.12 14:02	4	96	5	5	0	0	0	0
DAK-213	24.01.12 14:03	29.01.12 14:03	5	57	6	6	0	0	0	0
DAK-214	24.01.12 14:04	28.01.12 14:04	4	46	5	5	0	0	0	0
DAK-215	24.01.12 14:05	28.01.12 14:05	4	48	5	5	0	0	0	0
DAK-216	24.01.12 14:06	28.01.12 14:06	4	40	5	5	0	0	0	0
DAK-217	25.01.12 14:07		0	0	1	1	0	0	1	0
DAK-218	24.01.12 14:08	28.01.12 14:08	4	40	5	5	0	0	1	0


3. Fehlerhafte Attribute

			Werte
Computer	Registrierung	Name	aktuell	tiefster	vormals	Schwelle	Status





4. Selbst-Test-Übersicht

	Arbeitsstunden			Selbst-Tests
Computer	erste	letzte	Anzahl	Anzahl	vollständig	unvollständig	kurz	erweitert	Conveyance
DAK-210	17552	17684	132	0	0	0	0	0	0
DAK-211	12253	12397	144	10	0	0	5	5	0
DAK-212	6463	6559	96	8	0	0	3	5	0
DAK-213	16812	16869	57	0	0	0	0	0	0
DAK-214	14720	14766	46	4	0	0	3	1	0
DAK-215	4417	4465	48	4	0	0	3	1	0
DAK-216	15797	15837	40	2	0	0	1	1	0
DAK-217	2654	2654	0	0	0	0	0	0	0
DAK-218	9346	9386	40	4	0	0	3	1	0


5. Temperaturübersicht

	Höchstwerte				Tiefstwerte
Computer	Standzeit	aktuell	empfohlen	Spielraum	Standzeit	aktuell	empfohlen	Spielraum
DAK-211	48	40	55	15	20	38	14	24
DAK-212	46	40	55	15	18	38	14	24
DAK-214	47	46			41	45
DAK-215	59	52	60	8	20	49	0	49
DAK-216	63	42	65	23	40	40	5	35
DAK-217	56	37	65	28	37	37	5	32
DAK-218	59	44	65	21	40	40	5	35


5a. Höchsttemperatur-Verteilung

Computer	Temperatur	Anzahl
DAK-211	40	3
DAK-211	39	3
DAK-211	38	1

DAK-212	40	1
DAK-212	39	2
DAK-212	38	2

DAK-214	46	3
DAK-214	45	2

DAK-215	52	4
DAK-215	49	1

DAK-216	42	1
DAK-216	40	4

DAK-217	37	1

DAK-218	44	2
DAK-218	42	2
DAK-218	40	1

1. SMART-enabled hard drives and smartmontools

2013-12-03T13:00:00.000+01:00

SMART (Self-Monitoring, Analysis and Reporting Technology) is a monitoring system for hard disks devised by manufacturers to detect functional problems and warn of impending failures. SMART-enabled hard disks provide indicators (attribute values) of their present working state and perform self-tests. If the indicators are below specified threshold values and the self-tests are performed without error, the drive passes the overall-health assessment. If an attribute value exceeds the specified threshold value, a drive failure is predicted.

smartmontools

smartmontools is an open-source utility suite (available at SourceForge) to interact with SMART-enabled devices: smartd and smartctl. smartd runs a daemon program that logs SMART attribute values periodically. It can be configured to issue email alerts if an attribute surpasses a threshold value. smartctl is a program to en- and disable SMART functionalities, to query the working state of a hard drive, and to initiate self-tests.

The working state of SMART-enabled hard-disks can be queried with the smartctl tool that is part of smartmontools. smartctl can be configured to provide information on the capabilities and the working state of the device. The application is accessible via the command line and issues a report to STDOUT, which inturn can be captured in a file. The following is a sample output of smartctl with all output options enabled (option -x):

1.smartctl 5.40 2010-10-16 r3189 [x86_64-apple-darwin10.5.0] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Fujitsu MHW2 BH series
Device Model:     FUJITSU MHW2120BH
Serial Number:    NZ0ST6B288UD
Firmware Version: 00810013
User Capacity:    120,034,123,776 bytes
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 3b
Local Time is:    Thu Nov 20 07:16:00 2011 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
2.=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed without error or no self-test has ever been run.
Total time to complete Offline data collection:    ( 578) seconds.
Offline data collectioncapabilities: (0x7b) 
SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.     
Selective Self-test supported.
SMART capabilities: (0x0003)
Saves SMART data before entering power-saving mode. 
Supports SMART auto save timer.
Error logging capability:        (0x01) 
Error logging supported.
General Purpose Logging supported.
Short self-test routine recommended polling time:   (   2) minutes.
Extended self-test routine recommended polling time:   (  82) minutes.
Conveyance self-test routine recommended polling time:   (   2) minutes.
SCT capabilities:         (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
3.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   100   100   046    Pre-fail  Always       -       152969
  2 Throughput_Performance  0x0005   100   100   030    Pre-fail  Offline      -       32440320
  3 Spin_Up_Time            0x0003   100   100   025    Pre-fail  Always       -       1
  4 Start_Stop_Count        0x0032   095   095   000    Old_age   Always       -       21119
  5 Reallocated_Sector_Ct   0x0033   100   100   024    Pre-fail  Always       -       0 (2000, 0)
  7 Seek_Error_Rate         0x000f   100   100   047    Pre-fail  Always       -       3012
  8 Seek_Time_Performance   0x0005   100   100   019    Pre-fail  Offline      -       0
  9 Power_On_Hours          0x0032   075   075   000    Old_age   Always       -       12792
 10 Spin_Retry_Count        0x0013   100   100   020    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   099   099   000    Old_age   Always       -       7616
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       54
193 Load_Cycle_Count        0x0032   082   082   000    Old_age   Always       -       365307
194 Temperature_Celsius     0x0022   100   100   000    Old_age   Always       -       36 (Min/Max 12/51)
195 Hardware_ECC_Recovered  0x001a   100   100   000    Old_age   Always       -       1871
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0 (0, 6389)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x000f   100   100   060    Pre-fail  Always       -       12160
203 Run_Out_Cancel          0x0002   100   100   000    Old_age   Always       -       3732279524976
240 Head_Flying_Hours       0x003e   200   200   000    Old_age   Always       -       0
4.
General Purpose Logging (GPL) feature set supported
ATA_READ_LOG_EXT (addr=0x00:0x00, page=0, n=1) failed: 48-bit ATA commands not supported
Read GP Log Directory failed.

SMART Log Directory Version 1 [multi-sector log support]
SMART Log at address 0x00 has    1 sectors [Log Directory]
SMART Log at address 0x01 has    1 sectors [Summary SMART error log]
SMART Log at address 0x02 has   51 sectors [Comprehensive SMART error log]
SMART Log at address 0x03 has   64 sectors [Ext. Comprehensive SMART error log]
SMART Log at address 0x06 has    1 sectors [SMART self-test log]
SMART Log at address 0x07 has    2 sectors [Extended self-test log]
SMART Log at address 0x09 has    1 sectors [Selective self-test log]
SMART Log at address 0x10 has    1 sectors [NCQ Command Error]
SMART Log at address 0x11 has    1 sectors [SATA Phy Event Counters]
SMART Log at address 0x80 has   16 sectors [Host vendor specific log]
SMART Log at address 0x81 has   16 sectors [Host vendor specific log]
SMART Log at address 0x82 has   16 sectors [Host vendor specific log]
SMART Log at address 0x83 has   16 sectors [Host vendor specific log]
SMART Log at address 0x84 has   16 sectors [Host vendor specific log]
SMART Log at address 0x85 has   16 sectors [Host vendor specific log]
SMART Log at address 0x86 has   16 sectors [Host vendor specific log]
SMART Log at address 0x87 has   16 sectors [Host vendor specific log]
SMART Log at address 0x88 has   16 sectors [Host vendor specific log]
SMART Log at address 0x89 has   16 sectors [Host vendor specific log]
SMART Log at address 0x8a has   16 sectors [Host vendor specific log]
SMART Log at address 0x8b has   16 sectors [Host vendor specific log]
SMART Log at address 0x8c has   16 sectors [Host vendor specific log]
SMART Log at address 0x8d has   16 sectors [Host vendor specific log]
SMART Log at address 0x8e has   16 sectors [Host vendor specific log]
SMART Log at address 0x8f has   16 sectors [Host vendor specific log]
SMART Log at address 0x90 has   16 sectors [Host vendor specific log]
SMART Log at address 0x91 has   16 sectors [Host vendor specific log]
SMART Log at address 0x92 has   16 sectors [Host vendor specific log]
SMART Log at address 0x93 has   16 sectors [Host vendor specific log]
SMART Log at address 0x94 has   16 sectors [Host vendor specific log]
SMART Log at address 0x95 has   16 sectors [Host vendor specific log]
SMART Log at address 0x96 has   16 sectors [Host vendor specific log]
SMART Log at address 0x97 has   16 sectors [Host vendor specific log]
SMART Log at address 0x98 has   16 sectors [Host vendor specific log]
SMART Log at address 0x99 has   16 sectors [Host vendor specific log]
SMART Log at address 0x9a has   16 sectors [Host vendor specific log]
SMART Log at address 0x9b has   16 sectors [Host vendor specific log]
SMART Log at address 0x9c has   16 sectors [Host vendor specific log]
SMART Log at address 0x9d has   16 sectors [Host vendor specific log]
SMART Log at address 0x9e has   16 sectors [Host vendor specific log]
SMART Log at address 0x9f has   16 sectors [Host vendor specific log]
SMART Log at address 0xa1 has    1 sectors [Device vendor specific log]

SMART Extended Comprehensive Error Log (GP Log 0x03) not supported
SMART Error Log Version: 1
No Errors Logged
5.
SMART Extended Self-test Log (GP Log 0x07) not supported
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     12791         -
# 2  Short offline       Completed without error       00%     12780         -
# 3  Short offline       Aborted by host               80%     12780         -
# 4  Extended offline    Completed without error       00%     12766         -
# 5  Short offline       Completed without error       00%     12764         -
# 6  Short offline       Completed without error       00%     12740         -
# 7  Short offline       Completed without error       00%     12729         -
# 8  Short offline       Completed without error       00%     12683         -
# 9  Short offline       Completed without error       00%     12682         -
#10  Short offline       Completed without error       00%     12626         -
#11  Short offline       Completed without error       00%     12566         -
#12  Short offline       Completed without error       00%     12459         -
#13  Extended offline    Completed without error       00%     12456         -
#14  Short offline       Completed without error       00%     12454         -
#15  Short offline       Completed without error       00%     12251         -
#16  Short offline       Completed without error       00%     12241         -
#17  Short offline       Completed without error       00%     12190         -
#18  Short offline       Completed without error       00%     12151         -
#19  Extended offline    Completed without error       00%     12143         -
#20  Extended offline    Aborted by host               30%     12141         -
#21  Extended offline    Completed without error       00%     12137         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
6.
SCT Status Version:                  2
SCT Version (vendor specific):       0 (0x0000)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                 36 Celsius
Power Cycle Max Temperature:         37 Celsius
Lifetime    Max Temperature:         51 Celsius
SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      5/60 Celsius
Min/Max Temperature Limit:           -40/65 Celsius
Temperature History Size (Index):    128 (26)

Index    Estimated Time   Temperature Celsius
  27    2011-01-20 05:09    39  ********************
 ...    ..(  4 skipped).    ..  ********************
  32    2011-01-20 05:14    39  ********************
  33    2011-01-20 05:15    38  *******************
 ...    ..( 12 skipped).    ..  *******************
  46    2011-01-20 05:28    38  *******************
  47    2011-01-20 05:29    37  ******************
 ...    ..(  5 skipped).    ..  ******************
  53    2011-01-20 05:35    37  ******************
  54    2011-01-20 05:36    38  *******************
  55    2011-01-20 05:37    38  *******************
  56    2011-01-20 05:38    37  ******************
 ...    ..(  3 skipped).    ..  ******************
  60    2011-01-20 05:42    37  ******************
  61    2011-01-20 05:43     ?  -
  62    2011-01-20 05:44    37  ******************
  63    2011-01-20 05:45     ?  -
  64    2011-01-20 05:46    35  ****************
 ...    ..(  5 skipped).    ..  ****************
  70    2011-01-20 05:52    35  ****************
  71    2011-01-20 05:53     ?  -
  72    2011-01-20 05:54    20  *
  73    2011-01-20 05:55    21  **
  74    2011-01-20 05:56    22  ***
  75    2011-01-20 05:57    23  ****
  76    2011-01-20 05:58    23  ****
  77    2011-01-20 05:59    24  *****
  78    2011-01-20 06:00    24  *****
  79    2011-01-20 06:01    25  ******
  80    2011-01-20 06:02    25  ******
  81    2011-01-20 06:03    25  ******
  82    2011-01-20 06:04    26  *******
  83    2011-01-20 06:05    26  *******
  84    2011-01-20 06:06    27  ********
  85    2011-01-20 06:07    27  ********
  86    2011-01-20 06:08    28  *********
  87    2011-01-20 06:09    28  *********
  88    2011-01-20 06:10    28  *********
  89    2011-01-20 06:11    29  **********
  90    2011-01-20 06:12    29  **********
  91    2011-01-20 06:13    29  **********
  92    2011-01-20 06:14    30  ***********
  93    2011-01-20 06:15     ?  -
  94    2011-01-20 06:16    18  -
  95    2011-01-20 06:17    19  -
  96    2011-01-20 06:18    20  *
  97    2011-01-20 06:19    21  **
  98    2011-01-20 06:20    22  ***
  99    2011-01-20 06:21    23  ****
 100    2011-01-20 06:22    23  ****
 101    2011-01-20 06:23    24  *****
 102    2011-01-20 06:24    25  ******
 103    2011-01-20 06:25    25  ******
 104    2011-01-20 06:26    25  ******
 105    2011-01-20 06:27    26  *******
 106    2011-01-20 06:28    27  ********
 107    2011-01-20 06:29    28  *********
 108    2011-01-20 06:30    29  **********
 109    2011-01-20 06:31    30  ***********
 110    2011-01-20 06:32    30  ***********
 111    2011-01-20 06:33    30  ***********
 112    2011-01-20 06:34    31  ************
 ...    ..(  2 skipped).    ..  ************
 115    2011-01-20 06:37    31  ************
 116    2011-01-20 06:38    32  *************
 117    2011-01-20 06:39    32  *************
 118    2011-01-20 06:40    33  **************
 ...    ..(  6 skipped).    ..  **************
 125    2011-01-20 06:47    33  **************
 126    2011-01-20 06:48    34  ***************
 127    2011-01-20 06:49    34  ***************
   0    2011-01-20 06:50    34  ***************
   1    2011-01-20 06:51    35  ****************
 ...    ..(  9 skipped).    ..  ****************
  11    2011-01-20 07:01    35  ****************
  12    2011-01-20 07:02    36  *****************
  13    2011-01-20 07:03    35  ****************
  14    2011-01-20 07:04    36  *****************
  15    2011-01-20 07:05    35  ****************
  16    2011-01-20 07:06    36  *****************
 ...    ..(  8 skipped).    ..  *****************
  25    2011-01-20 07:15    36  *****************
  26    2011-01-20 07:16    37  ******************
Error SMART WRITE LOG does not return COUNT and LBA_LOW register
Warning: device does not support SCT (Get) Error Recovery Control command

SATA Phy Event Counters (GP Log 0x11) not supported

The output of smartctl -x is divided into six sections, the first two (marked above as 1 and 2, respectively) provide information about the device: the manufacturer, model and device identifiers, which standards and protocols are supported, etc. Most important for the health assessment of the device is is the first data line of section which indicates whether the device PASSED or FAILED the test. (See above: 'SMART overall-health self-assessment test result: PASSED'.) This is the only standard element a device must support in order to be compliant with the SMART standard.

SMART attributes (section 3 above), on the other hand, are not part of a standard but are vendor-specific additions. The lower-numbered attributes are more likely to be found than the higher ones. Attribute values, too, are vendor-specific rather than standard representations. Consequently, the meaning of the values can not be generalized. Columns VALUE, WORST, and THRESH most often indicate a present value state, the worst performance state in the lifetime of the device, and a threshold value that will trigger an attribute-failure flag when it is surpassed, respectively. However, attributes that function as counters most often do not have threshold values, and sometimes 'WORST' values are identical to those of the 'VALUE' column and move up and down. The reason for this seems to be that SMART attributes are not standardized and need to evaluated according to vendor specific criteria.

smartmontools a utility program that can be configured to emails when one or more of the attributes is failing. (smartd runs as a daemon that polls a device every 30 minutes and logs attribute changes.) If an attribute is failing, column THRESH will read FAILING_NOW and thus indicate that a serious condition exists. If the value returns to the normal working range, the 'FAILING_NOW' column will read In_the_past.

SMART Tests

SMART-enabled hard drives offer a number of self-tests that can be initiated via smartctl. Most commonly available and useful test are the short and extended offline test. A short offline test checks the electrical and mechanical operation and read performance of the drive. It also tests sections of the drive's surface. An extended test checks the entire surface of the drive. Some SMART attributes are also updated only during offline self-test. (See column UPDATED of the SMART attribute section: offline indicates that the value is updated only during a test.) The completion time of a short test is a few minutes; an extended test takes considerably longer, depending on the size of the drive. The information section of the smartctl output (see section 1 above) indicates the time needed for test completion.

The result of self-tests appears in its own section of the smartctl output (see section 5 above). The test type, completion of the test, start time (measured in lifetime hours), and the location of the possible first error are registered. Note that a test may be aborted if performance demands (a high volume of reads/writes) intervene.

Not all hard drives support extended temperature logs as in the example above (see section 6 above, SCT Status). Those that do log temperatures at regular time intervals. Sampling is most often done in minutes, but other time intervals are possible, too. The header of the temperature log indicates the sampling period, present and past registered temperatures, as well as maximum and minimum working temperatures. SCT logs as in the example, obtained by running and collecting smartctl at regular time intervals, make it possible to obtain a complete picture of a drive's working temperatures.