path: root/qdos/IZREADME.SMS

IZREADME_SMS (IZREADME.SMS): Info-ZIP for SMS/QDOS,   last revised: 15-Jun-1998
===============================================================================
[was "InfoZIP_SMSQDOS_ReadMe" in J. Hudson's original ports, ca. 08/1995]

Info-ZIP Programs
=================

Zip
UnZip
UnZipSFX
fUnZip

Introduction
------------

This archive is a result of frustrations with contemporary (August 95)
versions of Zip and UnZip. While they use the same compression
algorithms as the Info-ZIP programs, there the compatibility ends. If
you just use Zip/UnZip only on SMS/QDOS, then perhaps this is not a
problem (but I know for some users it still is); if you use Zip/UnZip
to transport source code and data between diverse systems, then the
disregard for Info-ZIP standards is inconvenient, particularly the
fact that directories are not supported and files are always stored
underscored.

This release of Zip/UnZip offers:

    o   zipfile/directory compatibility with all other supported
        platforms

    o   SMS/QDOS compatibility and back-compatible with earlier
        versions.

    o   Improved performance (Zip is typically 50% faster)

    o   Command-line compatibility with Info-ZIP

    o   Self-extracting archives (but not very elegantly)

    o   Archives are marked as 'created by SMS/QDOS'.

    o   Optional recursion into directories

    o   Directory structure restored on unzip of Info-ZIP/PKZIP-
        compatible archives.

    o   Config'urable for listing and unpack formats (Info-ZIP (.) or
        SMS/QDOS (_) and 'Press any key' timeouts. Override options
        from command line.

Info-ZIP Standards
------------------

This (rather long-winded and waffling) section discusses the
conventions and standards used by Info-ZIP-compatible archivers and how
"Info-ZIP for SMS/QDOS" achieves compatibility.

Info-ZIP Zip/UnZip on all supported platforms (Unix, DOS, OS/2, NT,
VAX/VMS, Amiga etc etc), works in a specific way. (Until now SMS/QDOS
was neither 'supported' nor Info-ZIP-compliant.)

    a. The zipfile directory is in (/.) (Unix) format.

    b. When zips are listed, it is in 'zipfile' (Unix) format.

    c. When files are added, they are defined in native format.

    d. When files are added, this is shown in 'zipfile' format.

    e. When files are unpacked, this is done to native format, but
       selection is done in 'zipfile' format.

Basically, the listing and stored format of a file is that of the
destination.

So, given a file structure at some arbitrary 'root' level.

    Makefile
    src (Dir)
        afile.c
        bfile.c
        docs (Dir)
             prog.txt
    hdr (Dir)
        cfile.h
        dfile.h

Then these would be in Unix (and Amiga) as

     Makefile
     src/afile.c
     src/bfile.c
     src/docs/prog.txt
     hdr/cfile.h
     hdr/dfile.h

This is also how the zipfile directory appears.

And in DOS/OS2/NT

    Makefile
    src\afile.c
    src\docs\prog.txt
    hdr\cfile.h         .. etc

And in VMS      (we SHOUT in VMS and have a silly file system)

    MAKEFILE
    [SRC]AFILE.C
    [SRC.DOC]PROG.TXT
    [HDR]CFILE.H        .. etc
                        (OK VMS purist, [.SRC] etc. Only an example)

And in SMS/QDOS (quiet again, but slightly ludicrous !)

    Makefile
    src_afile_c
    src_doc_prog_txt
    hdr_cfile_h         .. etc

The main problem regarding SMS/QDOS is not that of extensions - (after
all, only VMS and DOS _really_ have extensions; Unix, AmigaDOS, NT and
OS/2 (and Win95) allow multiple '.' in.long.file.names.

The SMS/QDOS problem is that '_' is both a legal file name character
and a directory separator. This creates the difficulties, as
directories and files are somewhat different objects.

It is the intention that these versions of SMS/QDOS Zip/UnZip will
follow the Info-ZIP rules, thus providing compatibility with the other
platforms. It is possible to zip the file structure described above on
SMS/QDOS and unpack it on VMS and get the VMS structure as shown in the
example (and vice-versa). [We only choose the most obtuse file
systems for the examples].

In order to achieve this, SMS/QDOS names are mapped into Unix-style
ones when the zipfile is created and un-mapped when it is unpacked.
There is an option to unpack in 'zipfile' format (i.e. with '.' rather
than '_'), but there will be no option to pack to all '_'. That would
contravene the standard.  However, a file

        src_split_name_c        (which is src->split_name_c !)
                                          src/split_name.c)

where src is a hard directory, would be stored in the zip directory as

      src/split_name.c

It does handle '_' with a little intelligence.

The default UnZip option will be to translate '.' to '_'; this is
because there are still many QDOS/Minerva users that cannot handle '.'
without quotes, which is immensely inconvenient. For many SMS users
'_' is also the most natural and convenient option. It also means that
SMS/QDOS <-> SMS/QDOS Zip - UnZip sequences are transparent.

There will, however, be two ways around this in UnZip.

      1. It is possible to Config the UnZip default to be '.'
         translations (or not).

      2.  The UnZip -Q1 option will toggle the default (Config'ed)
          state.

Examples:

Given that we want/have

     Makefile                   (Makefile)
     src/afile.c                (src_afile_c)
     src/bfile.c                (src_bfile_c)
     src/docs/prog.txt          (src_docs_prog_txt)
     hdr/cfile.h                (hdr_cfile_h)
     hdr/dfile.h                (hdr_dfile_h)

Then on SMS/QDOS we might have added the *.c files as

     ex zip;'-r test *_c'

(or VMS, just to do something different)

    zip -r test [.src]*.c

In both cases the file lists as above (left).

To unpack on SMS/QDOS (just the _c/.c files)

   ex unzip;'test src/*.c'

   (and VMS, unzip test src/*.c)

i.e. in both cases using the 'zipfile' format. As a concession to
SMS/QDOS, you could also have:

   ex unzip;'test src_*_c'

        but not unzip test [.src]*.c on VMS !!!!! Sorry, dinosaurs.

Both SMS/QDOS commands unpack to

     src_afile_c etc, where src_ is a hard sub-directory.

(and the VMS example would unpack to [.src]afile.c, (or to src\afile.c on
DOS/NT/OS2 etc).

Options & SMS/QDOS Features
---------------------------

The options supported by Zip/UnZip are basically those documented in
the Info-ZIP documents and shown in on-line 'usage'. In particular, -r
and -j work as intended.

PLEASE NOTE: Previous SMS/QDOS zip/unzips have NOT followed these
conventions, for example -r was not implemented and -j was reversed.

A number of -Q (SMS/QDOS-specific) options (not yet in the current
documents or usage screens) are implemented.

The Zip 2.0.1 (and later) default is to add SMS/QDOS headers where
file type = 1 (exe) or 2 (rel) or (type > 0 && != 255 and (filesize %
64) != 0). Directories are included anyway, unless you zip -D.

Where a header is added for an 'exe' file a '*' is displayed after the
name in the zip display  (and '#' for 'rel' files).

The -Q options for Zip are:

    -Q1  Don't add headers for ANY files
    -Q2  Add headers for all files
    -Q4  Don't wait for interactive key press

    (additive, so -Q5 => no headers, no wait, -Q6 all headers,
     no wait etc)

    (the default is exec/rel headers, 5 sec wait)

Zip has rationalised the file header storage in zipfiles. The
previous Zip used to store a QDOS header for each file. This was very
wasteful, for example compressing a SMS/QDOS release of PGP in this
way came to 730Kb, too large for a DD disk. Changing the Zip program
just to add a header record for the single PGP exe and the zipfile
size went down to around 690Kb.

And for UnZip

    -Q1 Toggle unpack format status ('.' <-> '_')
    -Q2 Toggle listing format
    -Q4 Don't wait for key press

Files Types
-----------

The history of QDOS suffers from incompatible feature
implementations. For example, Thor directories have file type 3, CST
have type 4 and Level 2 have type 255. Some software writers (both
amateur and otherwise) have used type 3 or 4 for other purposes
(backward compatibility ?? who cares ??).

In order to bypass problems cause by incompatible (inconsiderate ?)
usage of file types, the file type denoting a directory is a
Config'urable item. The default is set to -1 (65535 in Config terms),
which means "determine directory type from the file header of the root
directory". If this is appears unsuccessful on your system, the value
can be Config'ed in the range 3-255.

Zip assumes a file is a directory if:

        ((type == CONFIGed_type) && (file_size % 64) == 0)

If you are unfortunate enough have files of that pass this test but
are not directories, then Zip will loop endless, as SMS/QDOS opens the
root directory again !!! (recursion: see recursion etc).

I suggest you refrain from zipping such files and contact the software
supplier and point out the error of their ways.

File Naming Issues
------------------

Zip will append a '_zip' suffix to the archive filename when the
supplied name (i.e. excluding device/directory parts) does not
contain a '_' or a '.'. This is broadly compatible with Info-ZIP,
taking into account the '_' aberation.

So
        ex zip;'ram2_test ...'          >> ram2_test_zip

        ex zip;'ram2_test.zip ...'      >> ram2_test.zip

        ex zip;'ram2_test_rep ... '     >> ram2_test_rep

        ex zip;'ram2_fdbbs.rep ... '    >> ram2_fdbbs.rep

        ex zip;'ram2_test_rep.zip ...'  >> ram2_test_rep.zip

This implies that if a file ram2_test.zip exists, and you do:

        ex zip;'ram2_test ...'

Then a new file (test_zip) is created, rather than 'test.zip' being
updated.

Zip supports extensive recursive wild-carding, again the fact that '_'
can be a directory separator as well as part of a file name makes this
a bit tricky, but given the example:

     test1_bas
     test2_bas
     dir1->demo1_bas            where -> indicates a sub dir
     dir2->demo2_bas

     ex zip;'ram2_test *_bas'
     just finds test1_bas, test2_bas

     ex zip;'-r ram2_test *_bas'
     recurses and finds all the files

You might think that

    ex zip;'-r ram2_test *_*_bas'

would just find the files in the subdirectories--well yes, but it will
also find very other sub-dir'ed _bas file on the disk too. This is
a feature.

The pattern matching supports Unix-style 'regex' so you could:

        ex zip;'ram2_test dir?_*_bas'
        or
        ex zip;'ram2_test dir[12]_*_bas


UnZip has now got a fixed -d option. This is used to specify the
directory to unpack the zipfile into, it must follow immediately
after the zip name.

        ex unzip;'ram2_test_zip -d ram3_ *_txt'

would unpack all *_txt files to ram3_ .

It is not necessary to set the default directory to pack files, Zip
will remove any device names (and store any hard directory names,
unless you zip -j).

        ex zip;'ram1_test flp1_*'

                ----->
                        adding: file.dat (deflated 50%)
                        adding: menu.rext # (deflated xx%)
                        adding: zip * (deflated yy%)
                        adding: hard_one (stored 0%)
                        adding: hard_one/stuff.bas (deflated ...)

Due to the way the file-mapping is implemented, it is not supported
over the nX_ type network device.

Config Options
--------------

A limited number of SMS/QDOS specific functions can be set using the
QJump Config program.

      For Zip:

      Timeout for interactive 'Press any key' prompt

       65535                  Wait forever      (aka -1)
       0                      No wait
       n (1-32767)            Wait for 'n' clocks (1/50 sec)

       Other values are unsupported. Note Config works on 'unsigned'
       integer values (at least according to my manual).

       Directory file type key.

       Config will accept any value in the range 3-255, known useful
       values are 3 (Thor), 4 (CST) and 255 (Level 2 devices). A value
       of 65535 (aka -1) means "determine from device info".

       For UnZip:

       Timeout as above

       Unpack mode (SMS/QOS ('_') or Info-ZIP ('.')

       List format (Info-ZIP ('.') or SMS/QDOS ('_')


When the 'Press a key' text is displayed, if you press ESC, then it
waits until you press any other key, infinite timeout. This may be
useful if you want (much) more time to study a listing etc.

Defaults for timeout and directory type are 250 and -1 respectively.

More Goodies
------------

Part of the Zip compression code is now in assembler; it runs
noticably faster than the previous version. Compressing some arbitrary
files with the previous Zip it took 251 seconds, with Zip 2.0.1 it
took (a mere) 170 seconds (68008 QL).

More good news is that SMS/QDOS is just another system option on top
of standard Info-ZIP, unlike the previous ports that were much more
SMS/QDOS specific. For example, compiling the standard source with c68
(i.e. #define QDOS), then you get an SMS/QDOS version.

Compile with Linux/gcc and get the standard Linux version. Now, here's
the cool bit; compile with Linux/gcc and "-DQLZIP", and get a standard
Linux Zip/UnZip with SMS/QDOS (header) extensions.

so, on Linux:

            zip -Q stuff.zip qtpi zip unzip

the -Q tells Zip to look for XTc68/Lux68 cross-compiler data size
blocks and produce a zipfile with SMS/QDOS headers in it (for exec
type programs). This works for exec files produced by the XTc68/Lux68
cross compilers and ANY SMS/QDOS files copied to a Unix or MS-DOS disk
from an SMS/QDOS floppy using 'qltools v2.2' (or later).

Self Extracting Archives
------------------------

Info-ZIP self-extracting archives (_sfx) are created in a rather
'brute-force' way. The UnZipSFX program is prepended to a zipfile.

i.e.          file_sfx = unzipsfx + file_zip
              ex file_sfx

Although the UnZipSFX program is a cut-down UnZip, it is still around
30Kb - 50Kb, depending on platform.

The success of this approach depends on how the operating system
loader loads executable files. On most systems where the loader only
loads the actual program part (Unix, VMS, DOS et al), the this is
quite efficient; if you make, say, a 4Mb zipfile and prepend a 30Kb
UnZipSFX image, then the system only loads the 30Kb program and the
process is efficient as the zipped data part is still unpacked from
disk. These systems also supply the running UnZipSFX program stub with
the path name of the file it was loaded from, so the program knows
what it has to unpack (so on Linux, for example):

     cat /usr/bin/unzipsfx test.zip > test.sfx  # concatenate the files
     chmod 755 test.sfx                         # make executable
     test.sfx                                   # to extract, it
                                                # 'knows' it is "test.sfx"

Unfortunately, the more simplistic nature of SMS/QDOS makes this much
more difficult and rather less efficient as: (see note 1)

     a. The SMS/QDOS 'loader' loads the whole file into memory.

     b. The SMS/DOS 'loader'/c68 run-time system does not return the
        name of the file from which it was loaded.

     c. You cannot so easily create a image file by concatenating two
        files, it is also necessary to ensure the executable file
        header is set correctly.

     d. The show stopper. The data space required for the
        self-extracting archive is required, as not easily maintained
        during electronic transfer.


If anyone is still interested, then the following support for UnZipSFX
is provided.

 o A program 'makesfx' will combine a stub (callstub), UnZipSFX image
   and a zipfile to produce a sfx (self-extracting zip) file.

 o A callable interface is supplied. The user calls the SFX file,
   which creates the files necessary to do the extraction.

The makesfx program concatenates the supplied files to standard
output.

So, to create a sfx of all the _c files in the default directory.

 # 1st create a zipfile of the required files

 ex zip;'ram1_test_zip *_c'

 # Now create the sfx file (ram2_test_sfx)
 # our UnZipSFX image is in 'win1_bin'
 # as is the call stub.

ex makesfx;'-o test_sfx -x win1_bin_unzipsfx -s win1_bin_callstub -z ram1_test_zip'

The arguments to makesfx are:

    -s stubfile
    -x UnZipSFX_program
    -z Zip_file
    -o Output_file

You can now unpack the _sfx file on any SMS/QDOS-compatible
system.

        f$ = "win2_tmp_test_sfx"
        a = alchp(flen(\f$))
        lbytes f$,a
        call a
        rechp(a)

ZipInfo
-------

Given the above note concerning SMS/QDOS programs not knowing the name
by which the program was invoked, then the usual symbolic-link-of-unzip-
to-zipinfo trick is unavailable (presupposing there is some some SMS/QDOS
trick to emulate symbolic links).

ZipInfo functionality is only available via 'unzip -Z'. There is no
separate ZipInfo program.

Caveat ATP Users
----------------

ATP for SMS/QDOS users should pay particular attention to the
Zip/UnZip options in their atprc and compare with Info-ZIP Zip/UnZip
usage. Older versions of Zip/UnZip screwed up -j.


        zip -jk
        unzip -jo

Distribution & Copyright
------------------------

This software is written by and largely copyrighted by the 'Info-ZIP'
group whose members are noted in the accompanying documentation. This
particular SMS/QDOS port plus 'makesfx' was written by, but is not
copyrighted by, Jonathan R Hudson. The SMS/QDOS code in this release
is written from scratch and is not dependent on previous SMS/QDOS
releases, but is (largely) compatible.

As a courtesy to the authors of this package, please ensure that the
documentation is supplied when it is re-distributed.

In particular, if this archive is split into Zip and UnZip components,
ensure that this document ("IZREADME_SMS") is supplied in
each component.

SMS/QDOS version by:
Jonathan R Hudson (jrhudson@bigfoot.com)

I am grateful to Graham Goodwin for finding some most imaginative
means of breaking the beta code.

I'd also like to thank Thierry Godefroy for providing the 2.1/5.2
source code and making the initial contact with the Info-ZIP group.

And of course, many, many thanks to the Info-ZIP workers for making
this code freely available.

Note 1
------

The 'C' language FAQ ('frequently asked questions' [comp.lang.c])
notes on the matter of obtaining the load file name of a 'C' program:

16.5:   How can my program discover the complete pathname to the
        executable file from which it was invoked?

A:      argv[0] may contain all or part of the pathname, or it may
        contain nothing.  You may be able to duplicate the command
        language interpreter's search path logic to locate the
        executable if the name in argv[0] is present but incomplete.
        However, there is no guaranteed or portable solution.
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Note 2
------

NUL files for SMS2. There appears to be a conflict between SMS2/LBASIC
compiled programs and c68 programs using nul as stdin.

        EW zip,nul;'ram1_test *_bas'    # will not work

                                        # This does work !
        EW zip,#FOP_IN('nul');'ram2_test *_bas' : CLOSE

Note 3
------

version number incremented to 2.0.1a and 5.12a to accomodate Erling
Jacobsen's exit message requirements

version number incremented to Zip 2.0.1b to fix bug on zipping files
starting with leading underscore.

version number incremented to UnZip 5.12b to fix UnZip problem on
files zipped with leading './', and linked with revised (fixed) c68
'utime' function (could corrupt level 1 files). (source code _only_ as
IZQ004.zip).

Ported Zip 2.1 and UnZip 5.2 (July 1996). Released as INZIP005.zip

All later versions --- see Info-ZIP release notes and documentation.