Configuring a Step

Despite the relatively short list of required dependencies for a Step, it is an extremely complex class that can potentially contain many collaborators.spring-doc.cn

The preceding configuration includes the only required dependencies to create a item-oriented step:spring-doc.cn

As mentioned previously, a step reads in and writes out items, periodically committing by using the supplied PlatformTransactionManager. With a commit-interval of 1, it commits after writing each individual item. This is less than ideal in many situations, since beginning and committing a transaction is expensive. Ideally, it is preferable to process as many items as possible in each transaction, which is completely dependent upon the type of data being processed and the resources with which the step is interacting. For this reason, you can configure the number of items that are processed within a commit.spring-doc.cn

In the preceding example, 10 items are processed within each transaction. At the beginning of processing, a transaction is begun. Also, each time read is called on the ItemReader, a counter is incremented. When it reaches 10, the list of aggregated items is passed to the ItemWriter, and the transaction is committed.spring-doc.cn

In the “Configuring and Running a Job” section , restarting a Job was discussed. Restart has numerous impacts on steps, and, consequently, may require some specific configuration.spring-doc.cn

There are many scenarios where errors encountered while processing should not result in Step failure but should be skipped instead. This is usually a decision that must be made by someone who understands the data itself and what meaning it has. Financial data, for example, may not be skippable because it results in money being transferred, which needs to be completely accurate. Loading a list of vendors, on the other hand, might allow for skips. If a vendor is not loaded because it was formatted incorrectly or was missing necessary information, there probably are not issues. Usually, these bad records are logged as well, which is covered later when discussing listeners.spring-doc.cn

In the preceding example, a FlatFileItemReader is used. If, at any point, a FlatFileParseException is thrown, the item is skipped and counted against the total skip limit of 10. Exceptions (and their subclasses) that are declared might be thrown during any phase of the chunk processing (read, process, or write). Separate counts are made of skips on read, process, and write inside the step execution, but the limit applies across all skips. Once the skip limit is reached, the next exception found causes the step to fail. In other words, the eleventh skip triggers the exception, not the tenth.spring-doc.cn

One problem with the preceding example is that any other exception besides a FlatFileParseException causes the Job to fail. In certain scenarios, this may be the correct behavior. However, in other scenarios, it may be easier to identify which exceptions should cause failure and skip everything else.spring-doc.cn

By identifying java.lang.Exception as a skippable exception class, the configuration indicates that all Exceptions are skippable. However, by “excluding” java.io.FileNotFoundException, the configuration refines the list of skippable exception classes to be all Exceptions except FileNotFoundException. Any excluded exception class is fatal if encountered (that is, they are not skipped).spring-doc.cn

For any exception encountered, the skippability is determined by the nearest superclass in the class hierarchy. Any unclassified exception is treated as 'fatal'.spring-doc.cn

In most cases, you want an exception to cause either a skip or a Step failure. However, not all exceptions are deterministic. If a FlatFileParseException is encountered while reading, it is always thrown for that record. Resetting the ItemReader does not help. However, for other exceptions (such as a DeadlockLoserDataAccessException, which indicates that the current process has attempted to update a record that another process holds a lock on), waiting and trying again might result in success.spring-doc.cn

The Step allows a limit for the number of times an individual item can be retried and a list of exceptions that are “retryable”. You can find more details on how retry works in retry.spring-doc.cn

By default, regardless of retry or skip, any exceptions thrown from the ItemWriter cause the transaction controlled by the Step to rollback. If skip is configured as described earlier, exceptions thrown from the ItemReader do not cause a rollback. However, there are many scenarios in which exceptions thrown from the ItemWriter should not cause a rollback, because no action has taken place to invalidate the transaction. For this reason, you can configure the Step with a list of exceptions that should not cause rollback.spring-doc.cn

You can use transaction attributes to control the isolation, propagation, and timeout settings. You can find more information on setting transaction attributes in the Spring core documentation.spring-doc.cn

The step has to take care of ItemStream callbacks at the necessary points in its lifecycle. (For more information on the ItemStream interface, see ItemStream). This is vital if a step fails and might need to be restarted, because the ItemStream interface is where the step gets the information it needs about persistent state between executions.spring-doc.cn

If the ItemReader, ItemProcessor, or ItemWriter itself implements the ItemStream interface, these are registered automatically. Any other streams need to be registered separately. This is often the case where indirect dependencies, such as delegates, are injected into the reader and writer. You can register a stream on the step through the stream element.spring-doc.cn

In the preceding example, the CompositeItemWriter is not an ItemStream, but both of its delegates are. Therefore, both delegate writers must be explicitly registered as streams for the framework to handle them correctly. The ItemReader does not need to be explicitly registered as a stream because it is a direct property of the Step. The step is now restartable, and the state of the reader and writer is correctly persisted in the event of a failure.spring-doc.cn

Just as with the Job, there are many events during the execution of a Step where a user may need to perform some functionality. For example, to write out to a flat file that requires a footer, the ItemWriter needs to be notified when the Step has been completed so that the footer can be written. This can be accomplished with one of many Step scoped listeners.spring-doc.cn

You can apply any class that implements one of the extensions of StepListener (but not that interface itself, since it is empty) to a step through the listeners element. The listeners element is valid inside a step, tasklet, or chunk declaration. We recommend that you declare the listeners at the level at which its function applies or, if it is multi-featured (such as StepExecutionListener and ItemReadListener), declare it at the most granular level where it applies.spring-doc.cn

An ItemReader, ItemWriter, or ItemProcessor that itself implements one of the StepListener interfaces is registered automatically with the Step if using the namespace <step> element or one of the *StepFactoryBean factories. This only applies to components directly injected into the Step. If the listener is nested inside another component, you need to explicitly register it (as described previously under Registering ItemStream with a Step).spring-doc.cn

In addition to the StepListener interfaces, annotations are provided to address the same concerns. Plain old Java objects can have methods with these annotations that are then converted into the corresponding StepListener type. It is also common to annotate custom implementations of chunk components, such as ItemReader or ItemWriter or Tasklet. The annotations are analyzed by the XML parser for the <listener/> elements as well as registered with the listener methods in the builders, so all you need to do is use the XML namespace or builders to register the listeners with a step.spring-doc.cn

As with other adapters for the ItemReader and ItemWriter interfaces, the Tasklet interface contains an implementation that allows for adapting itself to any pre-existing class: TaskletAdapter. An example where this may be useful is an existing DAO that is used to update a flag on a set of records. You can use the TaskletAdapter to call this class without having to write an adapter for the Tasklet interface.spring-doc.cn

Many batch jobs contain steps that must be done before the main processing begins, to set up various resources or after processing has completed to cleanup those resources. In the case of a job that works heavily with files, it is often necessary to delete certain files locally after they have been uploaded successfully to another location. The following example (taken from the Spring Batch samples project) is a Tasklet implementation with just such a responsibility:spring-doc.cn

The preceding tasklet implementation deletes all files within a given directory. It should be noted that the execute method is called only once. All that is left is to reference the tasklet from the step.spring-doc.cn

The simplest flow scenario is a job where all of the steps execute sequentially, as the following image shows:spring-doc.cn

This can be achieved by using next in a step.spring-doc.cn

In the scenario above, stepA runs first because it is the first Step listed. If stepA completes normally, stepB runs, and so on. However, if step A fails, the entire Job fails and stepB does not execute.spring-doc.cn

In the preceding example, there are only two possibilities:spring-doc.cn

In many cases, this may be sufficient. However, what about a scenario in which the failure of a step should trigger a different step, rather than causing failure? The following image shows such a flow:spring-doc.cn

Only two special characters are allowed in the pattern:spring-doc.cn

For example, c*t matches cat and count, while c?t matches cat but not count.spring-doc.cn

While there is no limit to the number of transition elements on a Step, if the Step execution results in an ExitStatus that is not covered by an element, the framework throws an exception and the Job fails. The framework automatically orders transitions from most specific to least specific. This means that, even if the ordering were swapped for stepA in the preceding example, an ExitStatus of FAILED would still go to stepC.spring-doc.cn

After the discussion of BatchStatus and ExitStatus, one might wonder how the BatchStatus and ExitStatus are determined for the Job. While these statuses are determined for the Step by the code that is executed, the statuses for the Job are determined based on the configuration.spring-doc.cn

So far, all of the job configurations discussed have had at least one final Step with no transitions.spring-doc.cn

If no transitions are defined for a Step, the status of the Job is defined as follows:spring-doc.cn

While this method of terminating a batch job is sufficient for some batch jobs, such as a simple sequential step job, custom defined job-stopping scenarios may be required. For this purpose, Spring Batch provides three transition elements to stop a Job (in addition to the next element that we discussed previously). Each of these stopping elements stops a Job with a particular BatchStatus. It is important to note that the stop transition elements have no effect on either the BatchStatus or ExitStatus of any Steps in the Job. These elements affect only the final statuses of the Job. For example, it is possible for every step in a job to have a status of FAILED but for the job to have a status of COMPLETED.spring-doc.cn

In some situations, more information than the ExitStatus may be required to decide which step to execute next. In this case, a JobExecutionDecider can be used to assist in the decision, as the following example shows:spring-doc.cn

Every scenario described so far has involved a Job that executes its steps one at a time in a linear fashion. In addition to this typical style, Spring Batch also allows for a job to be configured with parallel flows.spring-doc.cn

Part of the flow in a job can be externalized as a separate bean definition and then re-used. There are two ways to do so. The first is to declare the flow as a reference to one defined elsewhere.spring-doc.cn

The effect of defining an external flow, as shown in the preceding example, is to insert the steps from the external flow into the job as if they had been declared inline. In this way, many jobs can refer to the same template flow and compose such templates into different logical flows. This is also a good way to separate the integration testing of the individual flows.spring-doc.cn

The other form of an externalized flow is to use a JobStep. A JobStep is similar to a FlowStep but actually creates and launches a separate job execution for the steps in the flow specified.spring-doc.cn

The job parameters extractor is a strategy that determines how the ExecutionContext for the Step is converted into JobParameters for the Job that is run. The JobStep is useful when you want to have some more granular options for monitoring and reporting on jobs and steps. Using JobStep is also often a good answer to the question: “How do I create dependencies between jobs?” It is a good way to break up a large system into smaller modules and control the flow of jobs.spring-doc.cn

All of the late binding examples shown earlier have a scope of step declared on the bean definition.spring-doc.cn

Using a scope of Step is required to use late binding, because the bean cannot actually be instantiated until the Step starts, to let the attributes be found. Because it is not part of the Spring container by default, the scope must be added explicitly, by using the batch namespace, by including a bean definition explicitly for the StepScope, or by using the @EnableBatchProcessing annotation. Use only one of those methods. The following example uses the batch namespace:spring-doc.cn

The following example includes the bean definition explicitly:spring-doc.cn

Job scope, introduced in Spring Batch 3.0, is similar to Step scope in configuration but is a scope for the Job context, so that there is only one instance of such a bean per running job. Additionally, support is provided for late binding of references accessible from the JobContext by using #{..} placeholders. Using this feature, you can pull bean properties from the job or job execution context and the job parameters.spring-doc.cn

Because it is not part of the Spring container by default, the scope must be added explicitly, by using the batch namespace, by including a bean definition explicitly for the JobScope, or by using the @EnableBatchProcessing annotation (choose only one approach). The following example uses the batch namespace:spring-doc.cn

The following example includes a bean that explicitly defines the JobScope:spring-doc.cn

When using the Java configuration style to define job or step scoped ItemStream beans, the return type of the bean definition method should be at least ItemStream. This is required so that Spring Batch correctly creates a proxy that implements this interface, and therefore honors its contract by calling open, update and close methods as expected.spring-doc.cn

It is recommended to make the bean definition method of such beans return the most specific known implementation, as shown in the following example:spring-doc.cn