cbmconvert release notes

cbmconvert 2.1.2
create, extract and convert different Commodore binary archives

Copyright © 1993–2003 Marko Mäkelä.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

Table of Contents

1 Introduction
2 Getting Started
3 Using cbmconvert
3.1 cbmconvert
3.1.1 CP/M disk images
3.1.2 GEOS disk images
3.1.3 Commodore C2N tape archives
3.1.4 Flaws of cbmconvert
3.2 disk2zip
3.3 zip2disk
4 Credits
5 Distribution
5.1 Revision History

1 Introduction

There are many archiving programs for the Commodore 64, all of which are incompatible with archiving programs on other systems. This cbmconvert utility tries to address this problem. It extracts files from most known file formats and writes them to several different formats, including some formats used by some Commodore 64 emulators.

This release of cbmconvert does not contain LHARC support any more, since 1541 disk images (either plain or ZipCoded) and Lynx archives are much easier to handle. If you need LHARC support, get the older release, cbmconvert 1.1.

All code has been cleaned up. The file type management in cbmconvert 2.0 is very flexible, and it is very easy to add file types later. Currently cbmconvert reads the following file types: Arkive, Lynx, ARC64/128, PC64, tape images (T64 and native Commodore C2N), disk images (1541, 1571, 1581, both native and C128 CP/M format), and raw files. Conversions between 1541 disk images and ZipCoded files can be made with the two supplied utilities disk2zip and zip2disk.

2 Getting Started

Compile cbmconvert by typing make -f Makefile.system where system is one of the following:

Commodore AmigaOS for Motorola 68000 series; a cross-compiler running on Linux is assumed
POSIX compliant systems
Microsoft Windows 95 or later; a cross-compiler running on a POSIX compliant system is assumed

The compilation should leave you with three executables: cbmconvert, zip2disk and disk2zip. Install these to your binaries directory, and you are ready to go. Ask your system administrator and/or a local guru if you do not know where to install the binaries. Or just use them from the current directory.

16-bit compilers will probably have trouble with cbmconvert. For instance, the standard file I/O library in Borland C uses signed 16-bit integers for size_t, which is not enough.

3 Using cbmconvert

Starting with version 2.0.0, cbmconvert consists of three programs:

The main program that makes most conversions.
ZipCode creator. Encodes a 35-track 1541 disk image to four ZipCode files (1!filename..4!filename).
ZipCode dissolver. Decodes a 35-track 1541 disk image from four ZipCode files.

3.1 cbmconvert

Since version 2.0, cbmconvert has independent file reader and writer modules that are controlled through a main program. There is only a command-line interface, but adding an interactive user interface should be easy (any volunteers?).

cbmconvert reads all files in all input files you specify on the command line and writes them in the format you specify. If you want to copy only some files from a set of archive files to an archive file or disk image, you will have to extract all the files to a single-file format such as the PC64 format, and to copy the wanted individual files to the output archive with another invocation of cbmconvert.

You can get a list of cbmconvert options by invoking the program without any arguments. cbmconvert supports the following file formats:

Raw files
Files with just the raw data. Written with the -I and -N options, read with the -n option.
PC64 files
Also known as "P00" files. Written with the -P option, read with the -p option.
Lynx archives
Written with -L, read with -l.
Commodore C2N tape archives
Written with -C, read with -c. These files are raw dumps of the data format the Commodore KERNAL routines maintain. Block checksums and countdown leaders (0x89..0x81 for the first copy and 9..1 for the second) are omitted, and the blocks are not stored twice, but only once. The data consists of 192-byte tape header blocks, 192-byte data file blocks, and arbitrary-length program file blocks.
C128 CP/M disk images
Written with -M[478], read with -m.
CBM DOS disk images
Written with -D[478], read with -d. 1571 and 1581 support have not been tested properly, and not all 1581 features have been implemented.
Read with -a. No write support.
Read with -k. No write support.
Read with -t. No write support.

3.1.1 CP/M disk images

Since cbmconvert has been written exclusively with Commodore file names in mind, the support for CP/M disk images is a bit clumsy. You may want to rename the files after extracting them from the disks. For instance, if you have downloaded the CP/M disk images from http://www.funet.fi/pub/cbm/demodisks/c128/, you can use the following commands in bash to extract the files from the disk images:

gunzip *.d64.gz

for i in *.d64; do mkdir ${i%.d64}; cd ${i%.d64}; cbmconvert -m ../$i; cd ..; done

for i in */*.prg; do mv $i "`echo ${i%.prg}|tr A-Z a-z`"; done

CP/M does not store the exact length of the files anywhere. That is why many CP/M programs create files whose length is a multiple of 128 bytes. Text files in CP/M commonly use ^Z as an end-of-file delimiter, and many programs fill the extra bytes in the last 128-byte block of the file with this character. The CP/M routines in cbmconvert does so as well, and it removes the ^Z padding when reading the files. So, you could use cbmconvert to remove the padding. But you may prefer the following Perl 5 command:

perl -i -e 'undef $/; while (<>) { s/\032+\z//gs; print }' *

3.1.2 GEOS disk images

GEOS files on disk images will be handled transparently by converting them to and from the Convert format on the fly. All unused areas in the Convert file will be zeroed out, and all possible integrity checks will be made in both conversions. If the file is interpreted to be in GEOS or Convert format, and it fails in any integrity check, it will be treated as a native Commodore file.

You should note that Convert does not provide much safety for the file. For sequential files, the original file length is not stored anywhere. For VLIR files, it is, but even then there are no checksums.

As a speciality, all GEOS files have a file information block. The contents of this block can be viewed with the simple Perl script cvtinfo.pl that reads files in Convert format. It was just a quick hack; feel free to make it more robust.

The GEOS routines have been tested with twelve 1541 disk images and with several megabytes of *.cvt files, and they are believed to be error free now. There were problems with the VLIR sector format; Mr. Boyce's reverse engineered information about it was incorrect, or at least inadequate.

3.1.3 Commodore C2N tape archives

The Commodore C2N tape format uses 187-byte file names that are padded with spaces. On disks, file names are 16 bytes and padded with shifted spaces. Since cbmconvert was designed primarily for disk-based archives, it truncates tape file names to 16 bytes and changes the padding from regular spaces to shifted spaces and vice versa.

End-of-tape markers on C2N tapes are not converted.

There are two kinds of program files on tapes: relocatable program files (type code 1, typically written in BASIC) and absolute program files (type code 3, typically written in machine language). There is no such distinction among disk files. When writing program files to C2N archives, cbmconvert marks the files relocatable if the least significant byte of their start address is 1. This typically indicates a BASIC program.

On output, all file types expect PRG are converted to tape data files consisting of 191-byte fixed-length data blocks. The output to tape data blocks tries to mimic the Commodore KERNAL behaviour: a zero byte is written to the last data block after end-of-file, and the bytes following the zero byte are identical with the bytes in the preceding data or header block.

The data file padding in the last block is not detected or removed on input. Also, the padding is not added on output if the file length is an exact multiple of 191 bytes, even though the KERNAL routines would add an extra block whose first data byte is zero. This prevents data files from growing when they are imported from tape and exported again, but this may cause incompatibilities when writing a data file with exactly n·191 bytes of payload.

3.1.4 Flaws of cbmconvert

The conversion may lose some file name or attribute information. In some cases, the filenames will be converted from PETSCII to ASCII or vice versa, and some PETSCII or ASCII specific characters will be lost in the conversions. Non-standard directory information available in some formats will be lost as well.

Lynx, Arkive and ARC allow duplicate filenames. But cbmconvert detects them and refuses to write multiple files with identical names to archives or to disk images. These duplicates are often used as directory separators, to separate groups of filenames. If you really need such separators, use 1-block-long PRG or SEQ files with different names. For example, the first separator can be a string of 16 dashes, the second one can be 15 dashes, and so on.

Arkive also works on illegal file types that cannot be created without directly modifying the directory blocks. Such files will be converted to a supported file type. Files with unknown types on disk images will be skipped.

Handling the errors on disk images could be more robust. Not all (typically BAM-related) errors on CBM DOS disk images are detected. For erroneous 1541 disk images that do not work with the new cbmconvert, you may want to try the disk2files program of the previous release.

3.2 disk2zip

This program converts a standard 35-track, 683-block 1541 disk image to ZipCode format. Because the ZipCode format stores the disk identifier, this conversion adds information. You can use the -i option to specify the disk identifier.

3.3 zip2disk

ZipCoded archives consist of four files prefixed with a number from 1 to 4, and an exclamation point. To extract an archive having the files 1!disk, 2!disk, 3!disk and 4!disk, type zip2disk disk. You may then delete the original files with rm [1-4]\!disk or similar. But note that some software (demos and copy-protected software) may rely on the disk identifier, which will not be stored in the 1541 disk image file.

Support for ZipCode SixPacks (1!!disk, 2!!disk, ..., 6!!disk) could be added easily, but this format is used mainly for really custom 1541 disks with abnormal low-level format, which cbmconvert does not support.

4 Credits

Most of the cbmconvert package was written by Marko Mäkelä. The original version appeared in 1993, and a complete rewrite was started in 1997 and finished in 1998.

The zip2disk module was originally programmed by Paul David Doherty. The ISO 9660 compliant filename truncation algorithm was taken from the T64TOP00 utility created by Wolfgang Lorenz as a part of his now discontinued PC64 emulator.

Support for 1581 CBM DOS disk images was programmed by Pasi Ojala.

The ARC/SDA dissolving code was originally written by Chris Smeets.

The necessary information for supporting GEOS was provided by Joe Forster/STA, Doug Cotton, Alexander Boyce (author of the unofficial GEOS Programmer's Reference Guide) and by the anonymous author of the file "convertfrmt.txt".

5 Distribution

You can download the latest source code distribution of cbmconvert from http://www.funet.fi/pub/cbm/crossplatform/converters/unix/.

Binary distributions are available for Win32 and AmigaOS.

5.1 Revision History

Version 2.1.2:
Bug fixes; see ChangeLog for details
Version 2.1.1:
Bug fixes; see ChangeLog for details
Version 2.1:
Added support for raw Commodore C2N tape archives. Renamed the CP/M-related options to -m and -M, so that the C2N-related options can be -c and -C.
Version 2.0.5:
Mikko Suonio provided patches for tolerating truncated T64 and Lynx archives. His patches also handle single-file T64 archives that have number of entries set to zero, and Lynx archives that have file names starting with a space.
It was brought to my attention that Lynx 17 on the Commodore 64 and some other utilities do not recognize the Lynx archives created by cbmconvert. I slightly modified the magic cookie to fix the problem.