Next Previous Contents

15. The new STAT file organisation

15.1 Space

The new Sun Enterprise Server 450, (called sassrv.ilo.ch), has about 18Gb of free disk space. The estimated total size of the STAT files that must be migrated from MVS is about 250Mb, hence there is no present need to be concerned about disk space limitations. If the activities of STAT do not change significantly in the future and not too many departments share the machine, there may never be a need to worry about space limitations.

15.2 Backups

Presently, backups of the complete server file system to 12Gb-uncompressed/24Gb-compressed tape cartridges are done nightly (around midnight) using ufsdump(1). ufsdump(1) is a simple program for incremental file system backup that requires the source file system to be inactive during the save process. The process takes approximately twenty minutes. If there is ever a need to have 24/7 availability of the server, a more elaborate solution will be required.

The present system is cumbersome for retrieving specific backed-up files since it requires manual operator assistance to load the correct backup cartridge. Two companies have already proposed automated solutions for managing banks of cartridges.

Although Solaris 7 ships with a backup program, the licence is only for a thirty day demo. There is very little information on USENET about its reliability and there are hints in the accompanying documentation that Sun is thinking of discontinuing support for this product.

15.3 STAT Directories

General policy decisions

For the moment, we are contemplating migrating the files from MVS to Unix and keeping their file names, and relative positions intact. Letter-case of filenames will not be changed immediately and the only translation of MVS names to Unix names will be the translation of the periods "." to slashes "/". Since the names of these files are mostly transparent to SAS users, no users will notice if modifications of directory structures and letter-case are made in the future.

For the instant, the priority is to get a working system going on Unix and the fewer structural modifications that are made, the less likely a number of scripts will break.

I believe that the STAT administrators will find it easier to debug problems caused by the migration if they don't have to get used to a completely different directory structure simultaneously. This policy will be reconsidered once the administrators feel that they will be more comfortable with a different directory layout and when they are fed-up with toggling the CAPS-LOCK key in order to specify old MVS filenames.

Details of directory layout

A 8.5Gb partition mounted at the /data directory has been created for storing SAS databases and related files. This directory will eventually be shared with other departments, so a subdirectory for STAT-only files has been created called stat.

Under the directory /data/stat there is a staging directory called test and a production directory called prod. Program development should take place in each developer's home directory /home/username if there is more than one developer working at a time. If it can be garanteed that no interference will occur, it is acceptable that development take place directly in /data/stat/test. A CVS (control version system) would be over-kill for STAT.

All of the files from MVS will be stored under /data/stat/test during the December testing phase with their MVS dataset names intact. Hence, a sequential dataset on MVS that was called LOSTPC.TRANSF.ZIP will be found on the Sun at /data/stat/test/LOSTPC/TRANSF/ZIP and a PDS member on MVS called LO.ST.CRDX.ILOSTAT(FORMATS) will be found on the Sun at /data/stat/test/LO/ST/CRDX/ILOSTAT/FORMATS.

Details of future directory layout

When the STAT administrators are finally comfortable with the new Unix system, I suggest that the following directory structure be adopted. It roughly follows traditional Unix directory hierachy, and is somewhat easier to use since several levels of directories are removed.

  1. Move all publicly-shared SAS programs found in the /data/stat/test/LO* subdirectories to a new directory called /data/stat/test/sas.
  2. Move every subdirectory under /data/stat/test/LO/ST up to /data/stat/test. This is done with the command mv /data/stat/test/LO/ST/* /data/stat/test
  3. Create a directory called /data/stat/bin and use it to store Unix programs and scripts that everyone in STAT should have access to. Examples of what should go in this directory include all of the scripts written for this migration project, the main STAT6 menu system, and the translated JCL scripts for creating monthly bulletins.
  4. Repatriate anything left over to each users' respective home directory.
  5. SAS developers should be developing in their own /home/username directories and moving programs to the public shared directory only when they are done. SAS users should never find half-finished or modified programs anywhere under /data/stat, only pure data and working programs.
  6. Change all file and directory names under /data/stat to lower case. This can be easily done with a shell script.
  7. Adjust the filenames in all scripts according to the above changes. This must be done by hand, unfortunately.
  8. When everything is tested to satisfaction, copy the entire /data/stat/test tree to /data/stat/prod and set the STAT users' environments accordingly.

15.4 Security and file-locking

Tampering of the SAS database files was prevented on the MVS system by a home-made script (a "CLIST") that checked to see if one's user ID was on a list of user IDs allowed to modify certain files. Each file had it's own list of authorised user IDs. The lists were written directly into the script. When the script was migrated, this home-made security system was kept intact, but the access lists were migrated to an easy-to-edit configuration file called /data/stat/test/etc/stat6access.conf.

A better (or at least more "Unix-like") solution is to set different group protections on each data file and then give each user appropriate group permissions. Although the old system was sufficiently capable of preventing honest mistakes, it was decided that a group-protection system should be implemented.

SAS (like many other database systems) has its own file-locking system to permit simultaneous access to datafiles. This has been purchased, but not yet installed. Until it is installed and configured correctly, there is a risk that two people working with the same data file may over-write each others' work. Solaris 7 also has file-locking capabilities (set with chmod(1)), but since the SAS system will probably be more suitable, no effort was made to use it.


Next Previous Contents