Faster Batch Jobs on MPE
On-line is with it, new, Silicon Valley, exciting; batch is
old, ordinary, IBM, and boring.
MPE is an operating system optimized for a large number of
simultaneous on-line users. Batch can be neglected, but
if the nightly jobs don't finish, on-line users feel it.
Batch and on-line are not the same. When optimizing
on-line response time,
we balance numerous, small tasks; when
optimizing batch, we are concerned with a single, large,
repetitive task that may even have the machine to itself.
Here are five ideas for improving batch throughput:
Bypass Inefficient Code
CPU time is seldom the bottleneck in on-line processing,
but it is often a problem in batch because of the large number of
Avoid known CPU hogs. On MPE they are serial file I/O
(use multi-record NOBUF access instead),
the FORTRAN formatter, and Pascal character I/O.
consider using "*" (same as last) as your
repeating general-purpose database code.
Transfer More Information per Disc Access
A disc read of 4000 words takes about the same time as one
of 40 words, so transfer as much useful data as you can on each access.
Use block read utilities such as
Suprtool from Robelle.
Increase Program Size to Save Disc Accesses
Move tables from files and databases into the data area of your program.
Do your own deblocking and read many blocks at once.
One reason why
NOBUF I/O is so fast is that it
uses large program buffers to increase the number of
records dealt with at a single time.
Remove Structure to Save Unneeded Disc Accesses
The more organized you keep the data, the more it costs you.
Remove unneeded search keys and sort fields in databases. If a
is only used in batch, drop it. Surprisingly, a serial scan
of the entire dataset is often faster than
following a laboriously maintained database search path.
Add Structure for Frequent Events
Look for patterns of repetition and
add structure to match those patterns.
Create datasets for special
purposes (for example, old history).
Sort records before loading a sequential-access file.
Retain computed totals for re-use later.
Sort transactions by type and customer to minimize setup time
for each transaction.