Faster Batch Jobs on MPE

Faster Batch Jobs on MPE

On-line is with it, new, Silicon Valley, exciting; batch is old, ordinary, IBM, and boring. MPE is an operating system optimized for a large number of simultaneous on-line users. Batch can be neglected, but if the nightly jobs don't finish, on-line users feel it. Batch and on-line are not the same. When optimizing on-line response time, we balance numerous, small tasks; when optimizing batch, we are concerned with a single, large, repetitive task that may even have the machine to itself.

Here are five ideas for improving batch throughput:

Bypass Inefficient Code

CPU time is seldom the bottleneck in on-line processing, but it is often a problem in batch because of the large number of repetitions.

Avoid known CPU hogs. On MPE they are serial file I/O (use multi-record NOBUF access instead), the FORTRAN formatter, and Pascal character I/O. In IMAGE/SQL, consider using "*" (same as last) as your field list to avoid repeating general-purpose database code.

Transfer More Information per Disc Access

A disc read of 4000 words takes about the same time as one of 40 words, so transfer as much useful data as you can on each access. Use block read utilities such as Suprtool from Robelle.

Increase Program Size to Save Disc Accesses

Move tables from files and databases into the data area of your program. Do your own deblocking and read many blocks at once. One reason why NOBUF I/O is so fast is that it uses large program buffers to increase the number of records dealt with at a single time.

Remove Structure to Save Unneeded Disc Accesses

The more organized you keep the data, the more it costs you. Remove unneeded search keys and sort fields in databases. If a search field is only used in batch, drop it. Surprisingly, a serial scan of the entire dataset is often faster than following a laboriously maintained database search path.

Add Structure for Frequent Events

Look for patterns of repetition and add structure to match those patterns. Create datasets for special purposes (for example, old history). Sort records before loading a sequential-access file. Retain computed totals for re-use later. Sort transactions by type and customer to minimize setup time for each transaction.