Exporting to XML with Suprtool

Exporting from the HP e3000 to XML

If you are not sure what XML is and what it is good for,
read our introduction to XML first.

Suprtool Generates XML

By Neil Armstrong

With the Web changing the business world, we have responded with some new features in our STExport tool, which is the newest component to the Suprtool suite of products. Recently, while visiting with Taylor Lumpkin of Hire-Experience (an Ecometry consultancy group), he suggested a way to provide XML support. He mentioned that the XML support didn't have to be that sophisticated, as most people use tools like XML Spy to further manipulate a file or generate a DTD for the XML file.

So we decided that the goal of the project should be to have STExport create a XML output file that could be validated by XML Spy, by entering only four commands:

 run stexport.pub.robelle
 input file1sd
 xml
 output myxml
 xeq

Surprisingly, this is all it takes in STExport. But getting some of the finer details correct to achieve "well-formed" status were a bit tricky. A small file output from STExport with only the default settings would look like this.

<?xml version='1.0'?>
<file>
<record>
<CITY>Los Altos</CITY>
<CREDIT-RATING>100000</CREDIT-RATING>
<CUST-ACCOUNT>4003302</CUST-ACCOUNT>
<CUST-STATUS>20</CUST-STATUS>
<NAME-FIRST>Ralph</NAME-FIRST>
<NAME-LAST>Perkins</NAME-LAST>
<STATE-CODE>CA</STATE-CODE>
<STREET-ADDRESS>Room 655</STREET-ADDRESS>
<STREET-ADDRESS>Los Altos     040033022</STREET-ADDRESS>
<ZIP-CODE>93002</ZIP-CODE>
</record>
</file>

By default STExport will add the simplest version tag at the beginning of the file, then it inserts a <file> and matching </file> at the beginning and the end of the file. Then STExport encloses each record from the input file in a <record> and </record> tag. Finally, the Self-Describing tags are added around each field's data values and edited appropriately.

Naturally users would want options to customize and specify the various options and tags themselves, in order to generate a file that is acceptable to their tools.

We began by first allowing the user to specify the "version" tag at the beginning of the XML file.

 xml version "?xml version='1.0' encoding='ISO-8859-1'?"

STExport will put the "<" and ">" around what is specified in the version string.

In addition, you can customize the "file" and "record" tags with the following simple commands:

 xml file "Orders" record "Details"

You can enter multiple XML commands per task to set the XML options you require.

An example of the output generated by the above commands is as follows:

<?xml version='1.0' encoding='ISO-8859-1'?>
<Orders>
<Details>
<CITY>Los Altos</CITY>
<CREDIT-RATING>100000</CREDIT-RATING>
<CUST-ACCOUNT>4003302</CUST-ACCOUNT>
<CUST-STATUS>20</CUST-STATUS>
<NAME-FIRST>Ralph</NAME-FIRST>
<NAME-LAST>Perkins</NAME-LAST>
<STATE-CODE>CA</STATE-CODE>
<STREET-ADDRESS>Room 655</STREET-ADDRESS>
<STREET-ADDRESS>Los Altos     040033022</STREET-ADDRESS>
<ZIP-CODE>93002</ZIP-CODE>
</Details>
</Orders>

In experimenting with STExport generated output, we found that there are some key points required for "well-formed" XML:

One key fix was that the tags that surrounded the data could not have any special characters other than hyphen, underscore and period ("-","_", "."). So STEXport replaces any of the invalid special characters with a ".". (This default can be changed with a Set command. )

The final option added to the XML command is the doctype keyword. A Document Type Declaration can be made at the beginning of the file via the !DOCTYPE specification.

In STExport you can specify simple one-line doctype specs with the following command:

 xml doctype '!DOCTYPE address-book SYSTEM "address-book.dtd"'

This will write the doctype specification at the top of the output file, directly after the XML version specification. More complicated and lengthy doctype specifications can be implemented by using the filename option, whereby STExport will append the contents of a named file directly after the version specification. The syntax can be simply:

  xml doctype myfile.xml

You may have noticed throughout this article that we have a mix of quotes. All of STExport's XML command options (version, doctype, file and record) allow for a string to be passed via surrounding quotes. The quotes may be either single or double, but keep in mind that if the string is to contain double quotes, then you should surround the entire string with single quotes.

This simple XML support is available in Suprtool pre-release version 4.4.13. If you're a supported Suprtool customer, and would like to experiment with it, please email support@robelle.com

An Introduction to XML

By Bob Green

"XML" is a buzzword frequently heard today. It stands for Extensible Markup Language.

There are two major types of XML usage: Documents and Data. Document XML is like publishing, with text and images, fonts and formatting, with a human as the final consumer. Data XML is a container for information passing between e-commerce systems, with a computer as the final consumer.

The official place to start for XML information is the World Wide Web Consortium (http://www.w3.org/XML) and their official XML Specification.

1. XML is a method for putting structured data in a text file
2. XML looks a bit like HTML but isn't HTML
3. XML is text, but isn't meant to be read
4. XML is a family of technologies
5. XML is verbose, but that is not a problem
6. XML is new, but not that new
7, 8, 9... XML is simple enough it doesn't need 7, 8 and 9.
10. XML is license-free, platform-independent and well supported

A Simple Example of XML

Qedit for Windows has new language coloring features that require knowing the syntax rules for each programming language. We decided to define the rules for each programming language in XML, rather than hard-coding them into the program. For example, COBOL is defined as follows:

<language name="Cobol" case="nonsignificant"
          startcolumn="1" qeditlanguagecode="2">
    <!-- Extensions for which this language should be used -->

    <extension>cbl</extension>
    <extension>cob</extension>

	<!-- Quotes -->
	<quote type="terminating">
		<stringdelimiter>"</stringdelimiter>
	</quote>
	<quote type="terminating">
		<stringdelimiter>'</stringdelimiter>
	</quote>
	
	<!-- Continuation -->
	<continuation>
		<positional position="1">-</positional>
	</continuation>
	
	<!-- Comments -->
	
	<comments>
		<!-- Compiler directives are treated as comments -->
		<linedesignator position="1">$</linedesignator>
		<!-- <endline>$page</endline> -->
		<!-- <endline>$control</endline> -->
		<!-- <endline>$title</endline> -->
		<!-- <endline>$include</endline> -->
		<linedesignator position="1">*</linedesignator>
	</comments>
	
	<!-- Tokens -->
	
	<tokencharset>[A-Za-z0-9-]</tokencharset>

	<!-- Keywords -->

	<keyword>ACCEPT</keyword>
	<keyword>ACCESS</keyword>
	<keyword>ADD</keyword>
	<keyword>ADVANCING</keyword>
	<keyword>AFTER</keyword>
	<keyword>ALL</keyword>
	<keyword>ALPHABET</keyword>
	<keyword>ALPHABETIC-LOWER</keyword>
	<keyword>ALPHABETIC-UPPER</keyword>
	<keyword>ALPHABETIC</keyword>
	<keyword>ALPHANUMERIC</keyword>
	<keyword>ALPHANUMERIC-EDITED</keyword>
	<keyword>ALSO</keyword>
	<keyword>ALTER</keyword>
	<keyword>ALTERNATE</keyword>
	<keyword>AND</keyword>
	<keyword>ANY</keyword>
	<keyword>ARE</keyword>

      ...

	
	<keyword>WORKING-STORAGE</keyword>
	<keyword>WRITE</keyword>
	<keyword>WRTIE</keyword>
	<keyword>ZERO</keyword>
	<keyword>ZEROES</keyword>
	<keyword>ZEROS</keyword>

	<!-- Symbols -->

	<!-- <>=+-(). -->

</language>

As you can see, XML looks somewhat like HTML, except that we can make up our own "element" names: <language>, <extension>, <quote>, <continuation>, <comments>, <tokencharset> and <keyword>. In this example, we have an XML document that is written by people to be consumed by computers.

What Software Reads and Writes XML?

Since XML was designed by the same people that brought you HTML, it is not surprising that Internet Explorer 5.5 and Netscape 6.0 can render XML files in the browser window. And more software that understands XML becomes available every day. XMLSpy is an XML viewer that comes highly recommended. You can download a 30-day trial from their site, http://www.altova.com

Many common tools such as databases already accept XML, including the newest version of Crystal Reports. Robelle's Suprtool pre-release 4.4.13 can generate XML extract files on the HP e3000 and HP 9000 through a new option in the STExport module.

Robin Cover's "XML Cover Pages" (http://xml.coverpages.org) is a comprehensive online reference for XML. It has an in-depth directory of XML software, tools and web links.

Frequently Asked Questions

The official XML-FAQ (http://xml.silmaril.ie) is edited by Peter Flynn and clears up a lot of confusion and XML in a question and answer format. For example, here are some selected questions and answers from the FAQ, slightly condensed:

What is the difference between XML and C or C++?
Quick answer: programming languages specify calculations, actions, and decisions to be carried out in order. XML is a markup specification language with which you can design ways of describing information (text or data), usually for storage, transmission, or processing by a program.

Does XML replace HTML?
Quick answer: No

Which parts of an XML document are case-sensitive?
Quick answer: all of it, unlike HTML.

What's a Document Type Definition (DTD)?
"A DTD is a formal description in XML Declaration Syntax of a particular type of document. It sets out what names are to be used for the different types of element, where they may occur, and how they all fit together. For example, if you want a document type to be able to describe Lists which contain Items, the relevant part of your DTD might contain something like this:

<!ELEMENT List (Item)+>
<!ELEMENT Item (#PCDATA)>

Learn More About XML on the Web

A very useful introductory web-tutorial is "Learn XML in 11.5 Minutes" by L.C. Rees. This steps you quickly through the syntax of XML: http://www.softwareag.com/xml/about/XML_in_Minutes.htm

Besides the XML FAQ and the 11.5-minute course, another interesting resource is a site maintained by O'Reilly Books: http://www.xml..com

Using XML to Syndicate News Headlines

Perhaps a second example, will make the usefulness of XML clear.

The RSS-XML standard is used to syndicate news headlines and other published material so that sites can pick it up automatically and republish it. Numerous "content aggregators" such as Meerkat, iSyndicate, Moreover, HeadlineWatch, and others republish news items. For more information on this phenomenon, see http://www.xml.com/pub/a/2000/07/17/heterogeneity.html

Since we are publishing e3000 news and tips on our Robelle web site, we decided to see how practical it was for us to syndicate this content. The resulting Robelle Channel is a free service that syndicates the headlines, with links to the full stories. To see how it looks, visit this web page: http://www.robelle.com/tips/headlines.html

XML is one of the central technologies that enables RSS. We were able to implement syndication without any programming, using two public web services: one service that converts our headlines from HTML into XML, and a second that renders the XML into a headline sidebar for the final web site.

Here is some XML describing 4 Robelle news links. (RDF stands for Resource Description Framework and RSS stands for RDF Site Summary, although RSS is now also interpreted as Rich Site Summary. In other words, we have some XML that describes a web site, what it provides and when it is updated):

<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF
  xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
  xmlns:dc="http://purl.org/dc/elements/1.1/"
  xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
  xmlns="http://purl.org/rss/1.0/"
>
<channel rdf:about="http://qedit.com/news.html">
	<title>RSSTest: Daily e3000 Tips</title>
	<link>http://qedit.com/news.html</link>
	<description>Robelle is the source for Qedit and Suprtool, software for HP servers.</description>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<items>
	  <rdf:Seq>
    <rdf:li rdf:resource="http://3kworld.com/newsroom.asp?&amp;appmode=itemDetail&news_PK=3102" />
    <rdf:li rdf:resource="http://qedit.com/news.html#stan" />
    <rdf:li rdf:resource="http://robelle.com/tips/unix.html" />
    <rdf:li rdf:resource="http://www.businessweek.com/bwdaily/dnflash/mar2001/nf20010313_831.htm" />
  	  </rdf:Seq>
	</items>
</channel>
<item rdf:about="http://3kworld.com/newsroom.asp?&amp;appmode=itemDetail&news_PK=3102">
	<title>Craig Solomon on the Performance Seminar</title>
	<link>http://3kworld.com/newsroom.asp?&amp;appmode=itemDetail&news_PK=3102</link>
</item>
<item rdf:about="http://qedit.com/news.html#stan">
	<title>How Stan Sieler Got Started</title>
	<link>http://qedit.com/news.html#stan</link>
</item>
<item rdf:about="http://daffy.robelle.com/tips/unix.html">
	<title>Unix: Where Are My Temporary Files?</title>
	<link>http://robelle.com/tips/unix.html</link>
	<description>by Dave Lo, Robelle Tech Support</description>
</item>
<item rdf:about="http://www.businessweek.com/bwdaily/dnflash/mar2001/nf20010313_831.htm">
	<title>Business Week: Good Ideas Still Get Funded</title>
	<link>http://www.businessweek.com/bwdaily/dnflash/mar2001/nf20010313_831.htm</link>
</item>
</rdf:RDF>

This is more complex structure than the COBOL syntax definition of the first example. The 4 news headlines are first defined as a list in <rdf:Seq>, then each headline is further defined as an <item>, which is linked back to headline list via the URL.

Having a standardized XML format for web publications is the 'glue' that allowed the web services to be created (the services are independent and cooperate through XML).. Using those services, the syndication of our news was a very simple feat.

What About EDI?

But XML isn't the only standard for interchanging data between computer systems and it certainly isn't the oldest. What about EDI (Electronic Data Interchange)?

So what is the difference between XML and EDI?

EDI is proprietary data interchange in binary format. XML is in text format and open. For both, partners still have to sit down and agree on a common vocabulary before they can exchange data.

Terry Floyd of the Support Group has written a paper on manufacturing software that explores the differences between EDI and XML, with concrete examples of real EDI transactions converted in XML format. It is worth reading if you come from an EDI background: http://www.supgrp.com/camus2001.ppt (PowerPoint format)

XML for the HP e3000

You have seen two examples of XML usage, one for defining attributes of a programming language and another for syndicating web news headlines. There will probably be more and more XML services and tools in the years to come, and you can now turn your HP e3000 data into XML using Suprtool.

XML files can be used to export data from the HP e3000 and then import it into tools on other platforms. Sort of like the Self-Describing files that Suprtool has used for years, but platform independent and non-proprietary. Version 4.4.13 of Suprtool, a pre-release that was produced in September 2001, can generate XML extract files through the STExport module. Read more about Suprtool and XML.