What's Up, DOCumentation? 1999 # 2
April 1999
From:
- Robert M. Green, President
- David J. Greer, Vice President
- Mike Shumko, Editor pro tem
To: Users of Robelle Software
Re: News of the HP 3000 and of HP-UX, 1999 #2
What You Will Find in This News Memo:
- Rolling Out Suprtool 4.2
- April Fools, Microsoft Style
- Staff Spotlight: Nicky Gunther
- On-Site Suprtool Training with Paul Edwards
- Inventory Reduction Sale
- The Y2K Corner
- Which Versions Are Y2K-Compliant?
- $Today and $Date Produce Y2K Error Messages
- Wipe Out!
- New Suprtool #Stddate Function
- What Else Does Set Date Cutoff Do?
- Job Queues on MPE/iX 6.0
- Watch for Qedit for Windows Updates
- Windows Scripting Host Part III
- Product Tips
- Serving You with the Latest
Rolling Out Suprtool 4.2
Suprtool 4.2 is now available. This year's release integrates more enhancements for Year 2000 date handling, including support for many additional date formats. In version 4.2 a powerful new function called $stddate facilitates date windowing in the If and Extract commands. (See the separate articles in this issue for details about $stddate.) Another enhancement to the If and Extract commands is support for the Mod arithmetic operator, which enables more flexible handling of any portion of a numeric field.
The second major focus of the 1999 MPE release is support for the major new IMAGE/SQL features in MPE/iX 6.0. Specifically, Suprtool can now handle B- trees and the expanded master dataset maximum which represents a two-fold increase to 80 GB datasets!
Similarly, on HP-UX, the new Suprtool release offers support for bigger file sizes. Version 10.20 and later of the HP-UX operating system expand file size capabilities beyond 2 GB, which was the former limit in Suprtool/UX. Now, with version 4.2, Suprtool/UX supports very large files which can contain up to 2.1 billion records! Another enhancement specific to Suprtool for HP-UX is control of line feeds in the STExport module that facilitates Oracle database loads.
Current Suprtool customers on direct support with Robelle in the U.S.A. and Canada will receive their update at the end of April. International customers should check with their local distributors on the availability of this update.
Suprtool runs on Hewlett-Packard operating systems--HP 3000 and HP-UX--and works with a variety of data sources, including IMAGE databases, Allbase and Oracle tables, MPE and HP-UX flat files, MPE POSIX files, and KSAM files.
April Fools, Microsoft Style
Just when you think all your Y2K troubles will soon be over, a new problem has been identified for the year 2001. This is really a Daylight Saving problem in the Visual C++ library, a problem last seen in 1990.
Because this bug, called the April Fools 2001 Bug, is in the Visual C++ library it can be expected to be widespread. Although not technically a Y2K bug, many Windows applications are going to break on April 1, 2001 when they give the wrong time of day. The bug causes Windows applications to give the incorrect time (off by one hour), even though the Windows clock shows the correct time. The bug lasts for a week, until April 8, 2001.
Technical details of the bug can be found at
http://security.pharlap.com/y2k/april1.htm
Microsoft plans to fix
the Visual C++ library this spring in a Service Pack
for Visual Studio, and in a new release of Windows Update.[Mike Shumko]
Staff Spotlight: Nicky Gunther
Customers who call us for product demos will be familiar with Nicky Gunther.
She's been a full-time account executive with Robelle's sales team since
1996. Some of you, however, may remember her friendly and charming manner
from as far back as 1990-1992 when she worked part-time in the department. Nicky specialized in marketing at the University of British Columbia, where she completed her Bachelor of Commerce degree. Living in Langley, the "horse capital of B.C.," gives her an opportunity to pursue her favorite interests: horse riding and training. Her current horse is a Selle Français jumper named Fabulon.
[Paul Gobes]
On-Site Suprtool Training with Paul Edwards
On-site Suprtool training in the U.S. is now provided by our certified Suprtool trainer, Paul Edwards. Paul has over 22 years of HP 3000 system management and applications development consulting experience. He has been leading Suprtool classes since last fall, and has been getting excellent reviews from his clients.
Please contact Paul Edwards & Associates at (972) 242-6660 for pricing and availability.
Inventory Reduction Sale
Now that Qedit 4.7 has shipped and Suprtool 4.2 is shipping, we have a number of last year's user manuals left over, and we would prefer not to send them to the recycler. They're still perfectly good, they just don't describe the latest version features. A prior version manual (along with the what's-new change notice, which you can print yourself using the Printdoc utility) would make an excellent Christmas-in-July gift for each of your Qedit and Suprtool programmers.
You can get the prior version of the manual for $10 each. The newest user manuals are $20 each.
[Mike Shumko]
The Y2K Corner
Which Versions Are Y2K-Compliant?
One of the most common questions we get these days is, "Which versions of Robelle products are certified to be Y2K-compliant?" We can certify only those versions that we have tested explicitly.Qedit for HP-UX versions 4.7.01 and later, and Qedit for MPE versions 4.6.02 and later, have passed our Y2K tests. Any modifications to correct a Year 2000 problem in Qedit will be based on version 4.7.01.
Suprtool is only Year 2000 compliant if your dates include the century. Suprtool for HP-UX versions 4.2.01 and later, and Suprtool for MPE versions 4.1.01 and later, have passed our Y2K tests, but we recommend Suprtool 4.2.01 because of its $stddate functionality. Any modification to correct a Year 2000 problem in Suprtool will be based on version 4.2.01.
More details are available on the Web at
http://www.robelle.com/year2000
$Today and $Date Produce Y2K Error Messages
It's a little-known fact that the $today and $date functions of Suprtool's If command do not do run-time interpretation of the dates in your data. Instead, the $today or $date function is converted into a constant that matches the format of the date being compared, and Suprtool uses that constant in the If comparison. This leads to a situation in which Suprtool refuses to execute a task when it "thinks" that the task doesn't make sense.Say, for example, that on the 15th of every month you send notices to all the customers whose contracts will expire in the following month. The task could be coded as follows:
get customer-dataset
item expiry-date, date, yymmdd
if expiry-date >= $date(*/*+1/1) and &
expiry-date <= $date(*/*+1/last)
When this task runs on May 15, 1999, Suprtool converts the If command
(internal to Suprtool's command parser) to look like this:
if expiry-date >= 990601 and &
expiry-date <= 990630
The task runs successfully, and you send people their renewal notices. However, when this task is executed on December 15, 1999 (or on January 15, 2000, or February 15, 2000, etc.), Suprtool will produce an error when it converts the If command (again, in its "head") to this:
if expiry-date >= 000101 and &
expiry-date <= 000131
Error: Cannot use a date beyond 1999 for this format
Suprtool recognizes that the If command may not always work as expected,
because the expiry-date field does not keep track of the century. The command
might work, or it might not (see "Wipe Out!"), but Suprtool
isn't taking any chances. When the generated constant is past 1999, and the
date format does not have a century component, Suprtool will let you know
there is a potential problem. (Note that versions of Suprtool earlier than
4.1 did not treat this situation as an error, but would blithely carry on as
if everything were all right.) If you are using $today and $date to generate constants that are in the future (e.g., six months in the future), you will start getting this error condition before January 1, 2000 (e.g., six months before January 1, 2000). On the other hand, if you are using $today and $date to generate constants that are in the past (e.g., archiving records that are older than nine months), you will start getting this error condition after January 1, 2000 (e.g., nine months after January 1, 2000).
Of course, the proper solution is to expand your files so that all date fields have a century in them. But we recognize that this is not always practical.
The workaround is to use the $stddate function of the If command. This will interpret the dates in the input dataset at run time, converting each one to the standard date format of ccyymmdd (hence the name $stddate). The modified command looks like this:
if $stddate(expiry-date) >= $date(*/*+1/1) and &
$stddate(expiry-date) <= $date(*/*+1/last)
When the task runs on December 15, 1999, the resulting statement (internally)
now has the century included in the constants, because the constants are made
to match the dates being compared:
if $stddate(expiry-date) >= 20000101 and &
$stddate(expiry-date) <= 20000131
Now the task will always run successfully. Each incoming record will have the
expiry-date converted into ccyymmdd format before being compared to the
generated constants. The $stddate conversion takes a little bit of run time,
but if you have to choose between doing it right and doing it fast, doing it
right is the way to go.
Wipe Out!
This Suprtool task is supposed to delete only records that have dates in the
future, but will suddenly wipe out the entire dataset starting January 1,
2000!
get orders
item order-date,date,yymmdd
if order-date > $today
delete
output baddates
xeq
For a workaround, see "$Today and $Date Produce Y2K Error Messages" above.
New Suprtool $Stddate Function
Suprtool 4.2 has a new feature that can greatly help in date selection and
conversion. Until now, there have been a few cases in which, for performance
reasons, Suprtool could not manipulate dates the way that users needed. For
example, it was not possible to do greater-than or less-than selection
against dates that were in month-first or day-first format, nor was it
possible to compare dates that were not in the same format. One of the criteria for an acceptable solution was finding a way that did not slow Suprtool down for people who did not need this functionality. The $stddate function takes care of this, without affecting people who do not need it, by letting you decide when to take advantage of it. $Stddate converts dates from any of the formats that Suprtool recognizes to a common standard format, ccyymmdd. With this conversion, it becomes possible to compare two dates that are not in the same format. And because $stddate puts century and year first, you can reliably do greater-than and less-than comparisons.
When $stddate converts a centuryless date format to ccyymmdd, it needs to decide what century to add. This is determined by the Set Date Cutoff command, which defines the starting year of a 100-year date window. The default year is 10, which means that incoming yy values of 10 through 99 will have century 19 applied, and incoming yy values of 00 through 09 will have century 20 applied. If you set the cutoff value to 50, the 100-year span would go from 1950 through 2049, with century applied accordingly.
$Stddate can be used in two places in Suprtool. In the If command it is used for selecting records based on date criteria. In the Extract command it is used for converting dates to the standard ccyymmdd format.
Comparing Two Dissimilar Dates
When Suprtool compares two fields to each other, it does not try to interpret them. If they are character fields, it just compares the bytes. If they are numeric fields, it just compares the numeric values. Using $stddate forces Suprtool to convert the date fields to a common format before comparing them.
get shipping-records
item order-date, date, mmddyy
item date-shipped, date, ddmmyyyy
if $stddate(date-shipped) > $stddate(order-date)
...
Converting Dates to CCYYMMDD FormatSuprtool can convert any known date formats to the standard ccyymmdd format. This is done by using $stddate in the Extract command. The date being converted must be defined as a numeric field. You need to define an eight- digit numeric "container" to receive the converted date.
get shipping-records
item order-date, date, mmddyy
define converted-date, 1, 8, display
extract converted-date = $stddate(order-date)
...
If you are creating a self-describing (link) output file, remember to tell
Suprtool that the new field is in ccyymmdd format.
...
item converted-date, date, ccyymmdd
output myfile,link
xeq
What About Invalid Dates?Something to keep in mind is that the $stddate function can only convert dates that are valid. A valid date is one that appears on the calendar. Therefore special dates such as all zeros, blanks, nines, etc. will need special attention. Any dates that are not valid will be converted to zero by $stddate. You need to be aware of this when you design your Suprtool task. You can ensure that $stddate sees only valid dates by filtering out the invalid ones with the $invalid function.
if not $invalid(date-shipped) and not $invalid(order-date) &
and $stddate(date-shipped) > $stddate(order-date)
The Robelle Web site has more detailed information about $stddate.[Mike Shumko]
What Else Does Set Date Cutoff Do?
Since version 4.0, Suprtool has had a Set Date Cutoff command that seems to
have something to do with "date windowing." Except within the context of the
new $stddate function of the If and Extract commands, it is really not a date
windowing command. The original function of Set Date Cutoff was to provide you with a shorthand way of specifying a year in the $date function of the If command. If you are comparing a century-included date field to a constant, you can choose to enter only two digits in the constant and have Set Date Cutoff complete the constant for you. For example,
set date cutoff 10
item expiry-date, date, phdate
if expiry-date < $date(01/12/31)
In this example, because 01 (in the $date function) is less than 10 (in the
Set Date Cutoff command), the century part of the $date is assumed to be 20.
That is, the If command is interpreted to be
if expiry-date < $date(2001/12/31)
If the field being compared includes a century component, we recommend that
you always provide a four-digit year when using a constant with $date. That
way you are in control of the situation. [Mike Shumko]
Job Queues on MPE/iX 6.0
One of the major enhancements to MPE/iX that we had been waiting for, was the implementation of User Defined Job Queues.
At Robelle we use our production HP 3000 for all our production processing and also for the generation of customer tapes. We use the Store command to write the customer tapes.
Because two simultaneous Store commands cannot access the same files, we could only have one tape-creation job stream executing at a time. To ensure this, we have lived with a global job limit which allows only one job to execute at a time. Sometimes this has caused conflict when production reporting job streams have to wait while a tape job waits for a tape to be mounted (or inserted). We had hoped to implement User Defined Job Queues to solve this problem.
Once MPE/iX 6.0 was installed, we defined a new job queue called TAPEJQ (with a job limit of 1) into which all the tape-creation jobs were streamed. The jobs would be released one at a time when there was a TAPEJQ job slot available.
The system job queue HPSYSJQ was also allowed to have one job executing at a time, so the global job limit was now set to have two available slots: one for tape jobs and one for system jobs.
Everything seemed fine for the first few days, but then we had a rush of customer tape requests and our distribution team streamed eight tape job requests through TAPEJQ. As these jobs were released by the system, we noticed that they were not being released in the expected FIRST IN/FIRST OUT order. Instead, they seemed to be released in a random order with no apparent explanation.
Because our product tapes are customized for a particular site, it is important that the correct tape goes with the correct documentation to the correct site. On a heavy tape-making day (some days we make up to 30 tapes), we did not want to mix up the sequence of our tape jobs.
The HP Response Center confirmed this to be a known problem, and a patch will be available in Express 1 due out in June 1999. So unfortunately, we have had to remove the job queues and go back to our old practice of having only one job slot available for all jobs, and ALTPRI-ing production jobs around the tape jobs.
[Robyn Rennie]
Watch for Qedit for Windows Updates
In 1998, we produced a new release of Qedit for Windows every month, and managed to "push" an update to every user roughly every three or four months.
This year we are synchronizing the "push" release of Qedit for Windows with the regular Qedit/MPE and Qedit/UX update, which is scheduled for the fall. There will still be regular updates to Qedit for Windows throughout the year, and users can request a "pull" release at any time.
Notification of new releases will continue to be via the Web pages and the Robelle-L mailing list.
[Mike Shumko]
Windows Scripting Host Part III
As we continue our series of articles on Windows Scripting Host (WSH), we explore more ways to use Microsoft's new scripting technology to automate tasks and control Windows' objects. In the last issue, we showed you how to program Microsoft Excel by using the JScript language of WSH. In this issue, we will show you how to import a comma-delimited file into Excel.
In order to understand this example, you will probably want to get the on- line help for the Excel objects. Finding the help file is easier said than done. It took us six months and three different books to learn what the help file was called and how to find it.
The default installation of Excel 95 and Excel 97 does not install the Excel object help file. To install the necessary help file, you must select the Help and Sample Files component of Excel and change the options to include Help for Visual Basic. Excel varies the file name from release to release. For Excel 95 it is called vba_xl.hlp, and for Excel 97 it is called vbaxl8.hlp.
Opening a Text File
Excel assumes a comma-delimited file to be a text file. To open a text file as a worksheet, you need to use the Opentext method. Here is the complete example:
/* Sample script to load a comma delimited file into Excel.
Copyright Robelle Solutions Technology Inc. 1999
*/
// Version number for this script
var Version = "Version 1.0";
var objXL = WScript.CreateObject("Excel.Application");
var openname = "c:\\temp\\sample.prn";
var savename = "c:\\temp\\sample.xls";
objXL.Visible = true;
objXL.Workbooks.Opentext(openname, // Filename
1, // Origin
1, // StartRow
1, // Delimited
1, // TextDelimiter
false, // ConsecutiveDelimiter
false, // Tab
false, // Semicolon
true); // Comma
objXL.ActiveWorkbook.SaveAs(savename, 1);
The script itself is quite simple, although the call to Opentext has a lot of
parameters. You must first specify the file name that is to be opened. In our
example, it is c:\temp\sample.prn. Why does the file name in the script
contain two backslashes (\\) for each directory separator? The reason is that
JScript is based on the C language, in which the backslash character has a
special meaning. To include an actual backslash in a string, you must use two
of them.When Excel first starts, the default directory is where you were last working. Because of this, you should always fully qualify your file name when calling Opentext. The rest of the parameters were deduced by using the Excel Visual Basic help and the Excel type library (describing how we found and viewed the type library is beyond the scope of this article, but did provide an interesting, if somewhat lengthy, learning experience for the author). Here is a description of each parameter:
| Origin: | Where the file originally came from (Macintosh or Windows). Always specify 1. |
| Startrow: | Starting row of the file to begin reading. Our sample file had data in the first line of the file. If you have headings or blank lines at the beginning, you can use this this parameter to have Excel skip over them. |
| Delimited: | A value of 1 specifies that the file is delimited (i.e., the columns are not fixed-width). |
| Textdelimiter: | Specify a value of 1 if the quote character is a double quote. |
| Consecutivedelimiter: | Specify true, if you want two consecutive delimiters in a row to be treated as one. In our example, we specify false because two delimiters in a row indicate indicate an empty field. |
| Tab: | Specify true if the tab is the delimiter. |
| Semicolon: | Specify true if the semicolon is the delimiter. |
| Comma: | Specify true if the comma is the delimiter. Because are trying to read a comma-delimited file, we specified true for comma and false for the other two delimiter characters. |
There are even more parameters to Opentext, but we do not need to use them in this example. For the Opentext method, parameters that are not specified assume default values.
Saving the Workbook
We use the SaveAs method to save the workbook that we have created. The first parameter is the name of the workbook (note that once again it is fully qualified). The second parameter specifies the format in which the file is saved. For example, the value 1 represents the Microsoft Excel Workbook format. If we don't specify a second parameter, Excel 97 saves the file as a text workbook.
Extending This Example
Using this information and the information from the previous two articles in this series, you should be able to:
- Write a script that takes two command line parameters:
- the fully qualified name of a comma-delimited file, and
- the fully qualified name of an Excel workbook
This script would convert the comma-delimited file into an Excel workbook by using the Opentext and SaveAs method calls. It would provide you with a general-purpose script that could load comma-delimited files.
- Use some of the Excel object properties that we showed you in Part II of this series to add headings and adjust column widths.
[David Greer]
Product Tips
MPE/iX Compatibility Tip
If you plan to upgrade to MPE/iX 6.0, please take the time to review the notes on the Web at http://www.robelle.com/support/faq/mpe60.html. We have thoroughly tested and recommend the following versions of Robelle software for MPE/iX 6.0:- Qedit 4.6 or later
- Suprtool 4.1.04 or later.
[Mike Shumko]
Qedit for Windows Tip
What's In a NameSometimes I need to send somebody the name of the file I am editing in Qedit for Windows. For example, I may write, "Dave, I've finished editing the /usr/local/www/welcome.html file. Over to you for final proofread. /Mike"
How do I get the fully qualified name of the file? Qedit for Windows can copy information about the current file to the Clipboard, which I can then easily paste wherever I like.
With the file open in Qedit for Windows,
- Right-click in the document workspace to get the shortcut menu, then select Properties (or select the Properties item from the File menu).
- Choose the Information tab.
- Press the Copy button.
- Switch to wherever the file name is needed (e.g., e-mail message composition window).
- Paste from the Clipboard (CTRL+V or select Paste from the Edit menu). The pasted information has the file name somewhere near the top.
- Delete any unnecessary information.
Qedit Tip
One of our customers reported a problem that appeared after
Qedit 4.7.01 was
installed on an MPE system. When the problem occurs, the status line appears
twice while working in full-screen mode. The home line (===>) does not appear
at all.Although this problem occurred intermittently and was very hard to reproduce, it was occurring often enough to cause serious grief. In some cases, users even experienced data corruption (e.g., missing lines). We have identified these three pre-existing conditions for this error:
- Qedit's Set Visual ClearDisplay setting is Off
- You are using WRQ's Reflection terminal emulator
- Reflection's display memory is full
The only available workaround is to enable (turn On) Visual ClearDisplay. It is enabled by default, so you should make sure there are no Set Visual Clear Off commands in any of your Qeditmgr files.
[François Desrochers]
Suprtool Tips
Tip #1: Mass Updates of Detail Sets
When we showed you how to do mass updates with DBEdit in the past, the method we used relied on the updated dataset being either a master set or a detail with only one record per key field.The following method works for any dataset. It builds a usefile of Suprtool tasks that do a separate chain for every key value. The idea was suggested by our customer Claus Andersen of Radiometer Copenhagen. It may not be as efficient as the DBEdit approach, but it is simpler.
The following example assumes that you have created the file Custab, with the key value of acct# and the field status which is to be updated.
input custab {the "table file"}
define acct#,1,8,display
define stat,9,2
extract "chain d-sales,acct-no =",acct#
extract ";update;extract status='",stat,"';xeq"
output myuse
xeq
use myuse
Note the use of single quotes within double quotes to handle non-numeric
status data. The resulting file Myuse should look like a file full of
individual Suprtool tasks, with one task per line.
chain d-sales,acct-no=12345678;update;extract status='OP';xeq
chain d-sales,acct-no=56465878;update;extract status='CL';xeq
Tip #2: Calculating the Age of Your Vehicle
This is an example of calculating the difference between two years, based on a date field called model-year which is an X8 field in a ccyymmdd format.
get your-dataset
define model-age,model-year,8,display
define today,1,8,display
item today,date,ccyymmdd
extract model-age, today=$today
output tempfile,link
xeq
input tempfile
define vehicle-age,1,3,display
extract vehicle-age = (today - model-age) / 10000
output result
xeq
Tip #3: Harder-to-Guess Passwords
In the last issue, we showed you a Suprtool task that extracts a random list of n-letter words to use as passwords. Of course, some real-world situations require slightly harder-to-guess passwords that are also only slightly more difficult to remember. This time we will use a Suprtool task to simulate the telephone-book algorithm for generating passwords: open the phone book to a random page, point at a name, and use the first four letters of the name plus the last four digits of the phone number as the password (e.g., smit3681).
- Select words greater than four letters. In this example, we use words with
6 letters or more.
input maincomm.spdata.robelle define word,1,24 if word == "??????@" {n = number of ?'s, 5..16 } :purge words output words xeq - Extract the first four letters and make up a "phone number" by treating
part of the word as numeric data. The list of "passwords" is randomized by
sorting on the nondisplayed portion of the word (i.e., beyond the first
four letters).
input words define word4,1,4 {4 letters} define i,2,2,int {start pos can range from 1..n-1} define j,4,2,int {start pos can range from 1..n-1} define k,6,2,int {start pos can range from 5..n-1} define digits,1,4,display {4 digits} sort k extract word4 extract digits = (i * j) mod 10000 :purge passwds output passwds xeqThe example shows fields i, j, and k starting at byte positions 2, 4, and 6, but you can choose any starting positions you like within the record. - Find a random entry. Suprtool's Input command always selects the first
record, so we need to select two records. It is the second record in the
resulting file that is the password.
:setvar size finfo('passwds','eof') - 1 :setvar finger ((!hpconnsecs + !hpsusan) mod size) + 1 :echo input passwds(#!finger) >usefile use usefile numrecs 2 :purge passwd output passwd xeq :print passwd;start=2
input passwds(#123)
numrecs 50
[Paul Gobes and Dave Lo]Serving You with the Latest
What's Up, DOCumentation? is published six times a year by Robelle Consulting Ltd. as a service to our customers. We welcome comments, ideas, and submissions from our readers. Contact us by phone, fax, or e-mail.
In addition to the printed version, the newsletter is available by e-mail (send subscription requests to support@robelle.com), and on the Web at http://www.robelle.com/library/newsletter/.
Contributors to this issue:
- François Desrochers
- Paul Gobes
- David Greer
- Dave Lo
- Robyn Rennie
- Mike Shumko
Up to Table of Contents