What's Up, DOCumentation? 1999 # 5

logo

October 1999

From:

bgreen/">Robert M. Green, President
David J. Greer, Vice President
Robert M. Green, Editor pro tem

To: Users of Robelle Software

Re: News of the HP 3000 and of HP-UX, 1999 #5

What You Will Find in This News Memo:

Qedit 4.8 Released! When Do I Get It?
UNIX Scripts and $Lookup
Enhancing an App with Suprtool
Staff Spotlight: Hans Hendriks
AMISYS Users Group Meeting
Everything You Need to Know about MPE/iX 6.0
Demystifying Regular Expressions
Qedit for Windows Won't Cost You an Arm and a Leg
Managing Projects Via the Web
Suprtool Supports Files Greater than 4 GB
Suprtool Tip: Timestamp
A Final Thought

Qedit 4.8 Released! When Do I Get It?

In the last issue we discussed some of the POSIX features and column editing in the new Qedit 4.8 release, and in this issue we break the seal on another 4.8 feature: regular expressions. The question everybody is asking is, "When -- when do we get the new version?" This is probably a good time to review how we distribute updates.

Every year we do a "push" release of Qedit to all the customers with support. And as customers have told us, the yearly releases from Robelle are among the most trouble-free software packages they use.

Once we made Qedit 4.8 the production release, we didn't just start duplicating tapes. Instead we started our staggered release, with quality assurance steps at every stage. It takes a bit longer, but the result is a release of the highest quality we know how to make.

The first stage was to send Qedit 4.8 to about two dozen sites who were eager to try the new version, with a variety of operating systems and size of computers in a number of countries. We followed up aggressively with these sites looking for compatibility, reliability and installation issues. This uncovered one bug that we corrected, resulting in the new production version of 4.8.01.

Throughout this time Qedit has been shipping to new customers and trial users. We monitor these customers, listening for ease-of-installation issues.

The next stage is to start the general distribution. We gave the green light for tape making the week of October 4. Robelle makes the tapes for Canadian customers, Southern Adventist University does the duplication and distribution for U.S. customers, and the overseas dealers take care of their customers. Each tape is customized with the customer name and account ID, and the customer's list of authorized CPU-IDs or HPSUSANs. The production of such a large number of customized tapes has its own quality assurance process, to ensure that every customer gets the right tape with the right documentation. We expect most of the tapes to have been shipped by Friday November 5, with them arriving in customers' hands by November 19.

Any customers who want the new version of Qedit before the scheduled shipping date simply need to request it from us or their dealer.

Inventory Reduction Sale: With the release of Qedit 4.8, we have some excess inventory of Qedit 4.7 user manuals. These are the same high-quality manuals that you always receive from us, just missing the new features for this year. Of course, since Qedit is always backward compatible, the old manuals are still quite usable! If you would like some to distribute to more people around your company, we are clearing out the old stock at US $10 each (new manuals are $20 each if you need additional copies of the new 4.8 features).

[Mike Shumko]

Up to Table of Contents

UNIX Scripts and $Lookup

A customer recently reported a problem when Suprtool was invoked in an HP-UX shell script with a here-document:

    /opt/robelle/bin/suprtool << !EOD

This task did a table lookup and got the following error:

    >IF  (GLTABL01,C1-2)
    Error:Field "GLTABL01" does not exist
    (GLTABL01,C1-2)
            ^
    Error:  Bad Itemname: missing >BASE,>GET or >DEFINE

Notice how the $lookup option is completely missing. This is because the UNIX script interpreted $lookup as a variable and substituted its value with spaces!

If you do feed Suprtool with a "here-document," make sure you escape any of the $ options with a backslash. For example,

    IF \$LOOKUP (GLTABL01,C1-2)

[Paul Gobes]

Up to Table of Contents

Enhancing an App with Suprtool

The following article was submitted by Suprtool user Bob Thompson at Nordstrom.

MACS, the Mailorder And Cataloging System from Smith Gardner, is an application that runs on the HP 3000 platform and is used by almost 250 customers. The IMAGE database structure has two main binary keys: CUST-EDP for customers and EDP-NO for items. However, some important datasets use character representations of these binary fields as portions of key fields. One of the most common questions relating to the use of Suprtool with MACS is how to extract data from two datasets where one dataset has the binary representation and one has the character representation.

Suppose you want to extract a list of all items that are in a given catalog offer. The item information is stored in the ITEM-MAST dataset with a primary IMAGE key of EDP-NO, defined as S9(9) COMP in COBOL. One of the places that the items belonging to an offer are kept is the OFFER-ITEMS dataset. The key to this dataset is OFFER-ITEM, which is an X(16) field consisting of an 8-character offer code combined with the 8-character, right-justified, zero- filled EDP-NO. The Suprtool instructions for creating a list of items in an offer would be:

      BASE      MACITM
      GET       OFFER-ITEMS
      DEFINE    OFFER,OFFER-ITEM[1],8,BYTE     {X(8)}
      DEFINE    ITEM,OFFER-ITEM[9],8,DISPLAY   {Z(8)}
      DEFINE    EDPNO,1,4,DOUBLE               {S9(9) COMP}
      IF        OFFER="N9R10000"
      EXTRACT   OFFER
      EXTRACT   ITEM
      EXTRACT   EDPNO=ITEM
      OUTPUT    file1,LINK
      XEQ

      CHAIN     ITEM-MAST EDP-NO=BINARY-EDP-TABLE
      TABLE     BINARY-EDP-TABLE, EDPNO, FILE, file1
      EXTRACT   ITEM-NO
      LIST      STANDARD TITLE "ITEMS IN N9R10000" HEADING "ITEM NAME"
      XEQ

Suprtool performs the conversion of the 8-character DISPLAY ITEM to the binary EDPNO. Note that this will not work without the DISPLAY in the DEFINE of the character field.

To do the reverse, that is, create a list of offers for an item:

BASE      MACITM
CHAIN     ITEM-MAST ITEM-NO="17784 CHC XS"
DEFINE    EDP,1,8,DISPLAY                 {Z(8)}
EXTRACT   EDP=EDP-NO
OUTPUT    file1,LINK
XEQ

GET       OFFER-ITEMS
DEFINE    OFFER, OFFER-ITEM[1],8,BYTE     {X(8)}
DEFINE    ITEM,OFFER-ITEM[9],8,DISPLAY    {Z(8)}
TABLE     CHAR-EDP-TABLE, EDP,FILE,file1
IF        $LOOKUP(CHAR-EDP-TABLE,ITEM)
EXTRACT   OFFER
LIST      STANDARD TITLE "OFFERS FOR 17784 CHC XS" HEADING "  OFFER "
XEQ

Again, this only works if the DEFINEs for the 2 fields being compared have types of DISPLAY.

Extracting all the "upsell" information for an item could be retrieved in a manner similar to the extraction of OFFER-ITEM information. The UPSELL master dataset has a key of UPSELL-KEY, which is an X(20) field composed of the 8-character, right-justified, zero-filled EDP-NO plus a code representing the upsell type.

To create an upsell list for an item, several passes are necessary since the data is contained in an X(80) field that occurs 8 times. Here is how you would do that:

BASE      MACITM
CHAIN     ITEM-MAST ITEM-NO="17784 CHC XS"
DEFINE    EDP,1,8,DISPLAY                 {Z(8)}
EXTRACT   EDP=EDP-NO
OUTPUT    file1,LINK
XEQ

GET       UPSELL
DEFINE    ITEM,UPSELL-KEY[1],8,DISPLAY    {Z(8)}
DEFINE    U-TYPE,UPSELL-KEY[9],2,BYTE     {X(2)}
TABLE     UPTABLE, EDP,FILE,file1
IF        $LOOKUP(UPTABLE,ITEM)
EXTRACT   U-TYPE,UPSELL-INFO
OUTPUT    file2,LINK
XEQ

{Create 1 record for each line of upsell info for each item}

:SETVAR   FILE_SIZE SUPRTOOLFULLCOUNT
:SETVAR   FILE_SIZE FILE_SIZE * 8  {Make sure file is big enough }
:FILE     file3;DISC=!FILE_SIZE

INPUT     file2
SET       SQUEEZE OFF
DEFINE    LABEL-TEXT,1,2
EXTRACT   " "
EXTRACT   LABEL-TEXT=U-TYPE
EXTRACT   " 1"
EXTRACT   UPSELL-INFO(1)
EXTRACT   U-TYPE
OUTPUT    file3,LINK
XEQ

INPUT     file2
EXTRACT   " ",LABEL-TEXT="  "," 2"
EXTRACT   UPSELL-INFO(2)
EXTRACT   U-TYPE
OUTPUT    file3,APPEND        {Write to same file }
XEQ
...
{Repeat for UPSELL-INFO(3) through UPSELL-INFO(8)}
...
INPUT     file3
SORT      U-TYPE
SORT      FILLER-2
DEF       LABEL,1,5
EXT       LABEL
EXT       UPSELL-INFO
LIST      STANDARD TITLE "UPSELL DATA FOR 17784 CHC XS" HEADING "TYPE  UPSELL DATA"
XEQ

The previous example also illustrates a method for solving the second most common problem relating to the MACS application: How can I extract information for individual line items on an order?

Order line items are stored in the ORDER-SUB-HEAD dataset with 3 line items per dataset record. There are 8 field arrays that all have 3 occurrences. If you want to extract a list of all orders that have a certain item on them, the following Suprtool instructions could be used:

{Extract the EDP-NO}

BASE      MACITM
CHAIN     ITEM-MAST ITEM-NO="17784 CHC XS"
EXTRACT   EDP-NO
EXTRACT   ITEM-NO
OUTPUT    file1,LINK
XEQ

{Extract all ORDER-SUB-HEAD records with item}

BASE      MACORD
GET       ORDER-SUB-HEAD
TABLE     ITABLE, EDP-NO, FILE, file1, HOLD
IF        $LOOKUP(ITABLE, EDP-NOS(1)) OR &
          $LOOKUP(ITABLE, EDP-NOS(2)) OR &
          $LOOKUP(ITABLE, EDP-NOS(3))
OUTPUT    file2,LINK
XEQ

{Split file into individual line items. Get 1st of 3 order-lines}

RESET     BASE
INPUT     file2
NUMRECS   300%         {Make sure new file has enough room}
SET       SQUEEZE OFF
IF        EDP-NOS(1) <> 0
DEFINE    EDP-NO,EDP-NOS(1)
DEFINE    ITEM-QTY,ITEM-QTYS(1)
DEFINE    EXT-PRICE,EXT-PRICES(1)
DEFINE    LINE-STATUS,BIG-STATUS[2],1
EXTRACT   ORDER-NO, EDP-NO, ITEM-QTY, EXT-PRICE, LINE-STATUS
OUTPUT    file3,LINK
XEQ

{Append 2nd of 3 order-lines}

INPUT     file2
IF        EDP-NOS(2) <> 0
DEFINE    EDP-NO,EDP-NOS(2)
DEFINE    ITEM-QTY,ITEM-QTYS(2)
DEFINE    EXT-PRICE,EXT-PRICES(2)
DEFINE    LINE-STATUS,BIG-STATUS[3],1
EXTRACT   ORDER-NO, EDP-NO, ITEM-QTY, EXT-PRICE, LINE-STATUS
OUTPUT    file3,APPEND
XEQ

{Append 3rd of 3 order-lines}

INPUT     file2
IF        EDP-NOS(3) <> 0
DEFINE    EDP-NO,EDP-NOS(3)
DEFINE    ITEM-QTY,ITEM-QTYS(3)
DEFINE    EXT-PRICE,EXT-PRICES(3)
DEFINE    LINE-STATUS,BIG-STATUS[4],1
EXTRACT   ORDER-NO, EDP-NO, ITEM-QTY, EXT-PRICE, LINE-STATUS
OUTPUT    file3,APPEND
XEQ

{Get only the records containing the item}

INPUT     file3
ITEM      EXT-PRICE DECIMAL 2
EXTRACT   ORDER-NO, "   ", ITEM-QTY, EXT-PRICE, "      ", LINE-STATUS
IF        $LOOKUP(ITABLE, EDP-NO)
LIST      STANDARD TITLE "ORDERS TAKEN CONTAINING 17784 CHC XS" &
          HEADING " ORDER    QUANTITY  PRICE  ITEM STATUS"
XEQ

[Bob Thompson, Nordstrom]

Up to Table of Contents

Staff Spotlight: Hans Hendriks

Hans Hendriks Well known to many customers, Hans has been handling Tech Support calls since 1993. He also has attended many user group meetings, HP Worlds and done training at various customer sites.

He studied Business Science at the University of Cape Town and worked in commodities trading before joining our South African distributor, Synergy Computing.

After first visiting us back in 1991, he moved to Canada with his family a couple of years later and has been a fixture here ever since.

Although he is our PowerHouse expert and our UNIX System Administrator, he prefers to be known for his woodworking, camping, and fly fishing.

For a look at other Robelle staff members, check out our company brochure.

[Paul Gobes]

Up to Table of Contents

AMISYS Users Group Meeting

On September 22, François Desrochers attended the AMISYS Users Group meeting held in Livonia, Michigan. He gave a presentation on performance from Suprtool's point of view.

AMISYS is a managed care software package sold by McKesson HBOC (http://www.hboc.com/). The package runs on HP 3000s and uses IMAGE databases. It is used by Health Maintenance Organizations (HMOs) in the United States. In a nutshell, these organizations offer medical care to a group of individuals by getting contracts with health care service providers such as doctors and hospitals.

AMISYS is used by more than 100 such organizations of different sizes. The size of an HMO is usually measured by the number of members they service. AMISYS customers range from around 10,000 members to millions of members. This is a very complex application in which they have to keep a lot of information. For example, a large HMO has to deal with more than 40,000,000 entries in a single dataset.

This means the largest customers are pushing the HP 3000, IMAGE and any software they use to the limits. They were one of the first HP 3000 users to implement Jumbo datasets and their expanded capabilities. This is where Suprtool really shines. In order to deal efficiently with such a large amount of data, Suprtool was integrated into the package a long time ago and is used extensively throughout the package. They use it for simple extract tasks to multi-step extract and link jobs. All this to speed up reports and queries down to more reasonable elapsed time.

Although the AMISYS package already uses Suprtool everywhere possible, customers also write custom reports and programs. IT staff in these organizations quickly learn about Suprtool and are typically keen users.

[François Desrochers]

Up to Table of Contents

Everything You Need to Know about MPE/iX 6.0

Many new features have been added to MPE since the last MPE Pocket Guide was issued by Orbit software (http://www.orbitsw.com/). At HP World '99 they issued an updated Pocket Guide for MPE/iX 6.0 -- jam-packed with POSIX commands, intrinsics, AIF calls, and much more.

The guide is free to Orbit customers, but can be purchased for only US $9.95 for noncustomers. You can order the guide at the following Web page:

    http://www.orbitsw.com/USA/pocket_guide.html

[Bob Green]

Up to Table of Contents

Demystifying Regular Expressions

Here is what we heard from one of the first users to try regular expressions in Qedit:

    From: Ted Ashton
    Sent: Tuesday, September 14, 1999 9:05 PM
    To: support@robelle.com
    Subject: Wow!
	
    Just got done installing Qedit 4.8 and tried out the regular expressions.
    Wow!  That's very exciting stuff.  Thanks!
	
    -- Ted Ashton, Info Serv, Southern Adventist University

You might have never heard the term regular expressions before, but that does not mean you have not used them. Regular expressions are actually wide- spread, with many tools having some kind of implementation. If you have worked on UNIX, Mac O/S, or even DOS, you have probably used a regular expression without knowing it. If you have run grep, egrep or searched for a string while browsing a file with the more command, you have created your first regular expressions. Regular expressions made it to MPE/iX with the advent of POSIX. Until now, however, they have been mostly restricted to the POSIX shell and related tools. That is until Qedit opened regular expressions to MPE users.

The syntax of regular expressions is almost like a mini programming language. Simple expressions can be understood very easily. More complex expressions have to be evaluated carefully to get the exact meaning.

The existing MPE wildcards and Qedit's pattern matching are fairly powerful too. However, one thing they lack is fine granularity. Wildcard characters usually represent a range of characters at a particular position. For example, the question mark (?) represents one alphanumeric character. What if you were looking for only lowercase vowels? What if you want to see strings that contain any letter except vowels. You can easily do all of these tasks with regular expressions.

Metacharacters

Like wildcards and pattern matching, there are a small number of characters that have special meanings in regular expressions. These are called metacharacters:

dot - any character
caret - start of line
dollar sign - end of line
square brackets - character class
parentheses - subexpressions
asterisk - zero or more characters
plus sign - one or more characters
question mark - zero or one optional character

All other characters (except for the escape character, which is a backslash) are used as literals. You can combine metacharacters with literals to describe precisely the strings you are looking for. Of course, regular expressions can be wordy and rather long and complex.

Be Specific...

As an example, let's say you want to find all the lines that start with a 5- letter word such as Green, Greer, or Great. First, the word has to be at the start of a line. So the regular expression has to start with a caret (^). Next we want to see the two letters Gr with the G in uppercase. That would be simply Gr.

For the third and fourth characters we want only lowercase vowels. That's easy -- use two character classes in square brackets: [aeiouy][aeiouy], where each matches one character.

In our made-up example, the last character in the word must not be lowercase d, f, k, l, m, p or s. We include a space here to make sure we get only 5- letter words. Again, that's easy -- just put a caret in the character class to negate it: [^dfklmps ].

To make sure we do not get words with more than 5 letters, we end the regular expression with a space. The final regular expression is ^Gr[aeiouy][aeiouy][^dfklmps ]. And the line-mode Qedit command to match this regular expression is

    /list "^Gr[aeiouy][aeiouy][^dfklmps ] " (regexp)

...Or Be Expansive

How to find lines that end with a number? We know there has to be at least one digit. We use a character class containing only the digits from 0 to 9 (for a range of contiguous characters we can use a dash, as in [0-9], instead of enumerating them). We then use the plus sign to indicate we accept one or more digits. A trailing dollar sign indicates we want lines in which the number is the last piece of information.

So the regular expression is [0-9]+$ and you could match this in Qedit for Windows by selecting the Regular Expression option in the String Search dialog box.

Move Parts Around

Parentheses are used to group parts of a complex expression or to isolate others. Each set of parentheses makes up a subexpression, which does not change the way string searching is done. In fact, if you are only searching, subexpressions do not have any effect. Where they are useful is in replacing parts of the matched strings. During a search, each subexpression is numbered from 0 to 9. You can then use that number in the replacement string as a back reference to the subexpression.

For example, we might want to find pairs of last and first names separated by a colon, as in Green:Bob and Panagopoulos:Aristotle. One regular expression to find those pairs is (.*):(.*), where .* means any character repeated any number of times, and : must match exactly one colon. By putting the .* portions in parentheses, we create two subexpressions. Subexpression 0 is for the last name before the colon and subexpression 1 is for the first name after the colon.

To convert Green:Bob into Dear Bob Green, the replacement string would be Dear \2 \1,. Back referencing is done with a backslash followed by the subexpression number (e.g., \0 and \1). Thus the replacement switches the subexpressions. You can use the same back reference as many times as you want in the replacement string. The whole expression that is matched is then replaced with the result of the replacement string if you do the following line-mode Qedit command:

    /change "(.*):(.*)"(regexp), "Dear \2 \1,", all

Develop New Search and Replace Powers!

Regular expressions help you to select the information that you want and to rearrange it the way you want, taking you one step higher in flexibility and power. Think of them as pattern matching on steroids!

Qedit version 4.8 has regular expressions, in both Line mode and Qedit for Windows. See above for when you can expect your copy, or get back on support for Qedit and update to the latest version!

[François Desrochers]

Up to Table of Contents

Qedit for Windows Won't Cost You an Arm and a Leg

Whether you're purchasing your first copy of Qedit for Windows or licensing additional servers and clients, you'll find them very affordable.

Qedit Server Modules for HP 3000 and HP-UX: the first license is US $1,000, additional licenses are US $500 each.

Qedit for Windows Clients:

1- 5 licenses are US $200 each
6-10 licenses are US $180 each
11-25 licenses are US $160 each
26-50 licenses are US $140 each
51-99 licenses are US $120 each
100+ licenses are US $100 each

Note that if you are buying additional clients, you qualify for the price that corresponds to your cumulative total. For example, if you already have 4 clients and you would like to buy 2 more, you would pay for 2 @ $180 each.

These prices are good in the U.S. In other countries please contact your local representative. If you have any questions, contact Robelle Sales in North America toll-free at 1-888-762-3553 or send e-mail to solutions@robelle.com.

Up to Table of Contents

Managing Projects Via the Web

In this year's #3 issue of What's Up, DOCumentation?, we discussed how easy it is to set up your own Apache Web server on MPE. We mentioned how useful we found a Web site for coordinating project management. Not only are Web pages a standard format that can be hosted on almost any computer on your local LAN, they provide wide area access and revision for workers in remote locations. At Robelle we use a firewall to determine who can get into private company Web sites, but you can also put passwords on specific sections of your Web site.

Combine a Web server with browser and Qedit for Windows on each PC and you have a complete system for project management.

Let's start with a simple project that we all can understand: preparing for the HP World conference. How do we make all of the arrangements to come to HP World, set up the booth, write tutorials and more importantly provide you with T-shirts. All this can be tidily documented on a project Web page that everyone involved can both view and edit.

Using the Internet as your project management platform also allows you to integrate material from external partners in the project. In this case, Interex is our main partner for HP World and they have a Web site for the project: http://www.hpworld.org/.

There were 10 major steps in getting Robelle to HP World, so there are 10 more similar Web pages for the subprojects. Each HTML page starts from the same basic template: a spot for the Basic Information of the Project, some of the background information and the guidelines on how to get the project completed as well as what the steps are.

Having Qedit for Windows makes this technique even easier, because our editor allows you to edit Web pages directly on the server. No need to connect in with a Terminal session and run a line-mode editor.

All of this uses simple HTML, which even nontechnical persons within the company have begun to use. (Surprisingly, they were the first to begin using color to indicate the status of any one task.) The code is easily edited directly on the Web site using our Qedit for Windows client/server editor. Just make a change, save it, and check the results in your browser.

That's it -- simple, dependable, accessible project management.

[Bob Green]

Up to Table of Contents

Suprtool Supports Files Greater than 4 GB

As you may have heard from Hewlett-Packard by this time, a future version of MPE/iX will support single files greater than 4 gigabytes. Robelle has been working with HP on this enhancement for several months to ensure that Suprtool can handle them. For example, this October we are testing a new version of our "pre-fetch" logic for super-large files.

So don't worry -- you will probably receive a new release of Suprtool that supports files greater than 4 GB before you install your MPE upgrade. But if not, our Tech Support department can provide you with a pre-release version in plenty of time.

[Neil Armstrong]

Up to Table of Contents

Suprtool Tip

There's no built-in Suprtool function for inserting the current time. However, you can use HP variables and command I/O redirection.

    > define timestamp,1,8
    > echo extract timestamp = "!HPTIMEF" > foo
    > use foo

This will insert an X8 field called "timestamp" into each output record. The timestamp contains the time the data was extracted.

[Hans Hendriks]

Up to Table of Contents

A Final Thought

Trust the computer industry to shorten the term Year 2000 to Y2K. It was that kind of thinking that got us into this situation in the first place.

[Author Unknown]

Up to Table of Contents