1.16 Disk Organization

a) Memory

Read Only Memory (ROM)

ROM is a small area of permanent memory that provides startup instructions when the computer is turned on. You cannot store any data in ROM. The instructions in ROM are set by the manufacturer and cannot be changed by the user. The last instruction in ROM directs the computer to load the operating system.

Every computer needs an operating system. This is a special computer program that must be loaded into memory as soon as the computer is turned on. Its purpose is to translate your instructions in English into Binary so that the computer can understand your instructions. The operating system also translates the results generated by your computer into English when it is finished so that we can understand and use the results. The operating system comes with a computer.

Random Access Memory (RAM)

This is the area of memory where data and program instructions are stored while the computer is in operation. This is temporary memory. NOTE: The data stored in RAM is lost forever when the power is turned off. For this reason it is very important that you save your work before turning off your computer. This is why we have peripheral storage devices like your computer’s hard disk and floppy diskettes.

Permanent Memory (Auxiliary Storage)

Your files are stored in permanent memory only when saved to your disk in a: drive or saved to your computer's hard disk, Drive c:

Disk Organisation:- As our operating systems and application software have continued to grow in size, their memory requirements have increased steadily. A vital memory in our system is hard disk storage.

Bound within the hard disk's structure lie the answers to questions like: What is a low level format? What does FDISK do? What is a hard disk partition and why does DOS limit us to 32 megabytes in a partition? What does it mean to have "lost cluster chains" or "cross-linked files?" What does it mean to have our disks defragmented?" Let's explore MS-DOS and PC-DOS hard disk organization to answer these questions and others.

The first stage in preparing any hard disk for operation is known as low level formatting. Low level formatting takes any hard disk from its virgin "fresh from the factory" state and prepares it for operation with a particular hard disk controller and computer system.

Low level formatting divides each circular track into equal size SECTORS by placing SECTOR ID HEADERS at uniform positions around each track. The start of a sector ID is marked with a special magnetic pattern, which cannot be generated by normal recorded data. This ADDRESS MARK allows the beginning of each sector to be uniquely discriminated from all recorded data. The sector ID information, which immediately follows the address mark, contains each sector's Cylinder, Head, and Sector number, which is completely unique for each sector on the disk. When the hard disk controller is late reading or writing to these disk sectors, it compares the sector's pre-recorded cylinder number to make sure that the heads haven't "mis-stepped" and that they're flying over the proper cylinder. It then compares the head number to verify that unreliable cabling is not causing an improper head to be selected and waits for the proper sector to start by comparing the pre-recorded sector number as it passes by with the sector number for which it is searching.

Since many hard disk surfaces are not flawless, low level formatting programs include a means for entering the hard disk drive's defect list. The defect list specifies tracks (by cylinder and head number) that the manufacturer's sensitive drive certification equipment found to stray from the normal which indicates some form of physical flaw that might prevent data from being reliably written and read. The list of such defects is typically printed and attached to the outside of the drive.

When these tracks are entered into the low level formatter, the defective tracks receive a special code in their sector ID headers, which indicates that the track has been flagged as bad and cannot be used for any data storage. Later, as we shall see, high level formatting moves this defective track information into the system's File Allocation Table (FAT) to prevent the operating system from allocating files within these defective regions.

When the low level format has been established, we have a completely empty drive, devoid of stored information, which can accept and retrieve data with the specification of any valid cylinder, head, and sector number.

There's an important issue about the low level formatting of a hard disk which is frequently overlooked, but which can be quite important to appreciate. Since the hard disk controller works in intimate concert with its hard disk drive to transfer the data within its numbered sectors to and from the computer's memory, the exact details of the address mark, sector ID header, and rotational sector timing can be completely arbitrary for any controller and drive. Since these details are initially established when the drive receives its low level formatting, they are forever hence agreed upon by both the hard disk drive and the controller. But more importantly, there's absolutely no reason to assume that the relatively arbitrary low level formatting specifics used by any particular hard disk controller would be compatible with any other model of hard disk controller.

In practice this means that differing makes or models of hard disk controllers are completely unable to read, write, or interpret the formatted information created by any other make or model of controller. Consequently, whenever it is necessary or desirable to exchange hard disk controllers, a complete backup of the hard disk's data, while attached to the initial controller, MUST BE followed by creating a new low level format with the new controller on the drive before any of the backed-up information can be restored to the drive with the new controller.

So we've given our drives a low level format, since we see that it is this process which first establishes "communication" between a hard disk and its controller by creating 512-byte "sectors" where none existed before. Now lets take up the next phase of hard disk structuring: The hard disk PARTITION.

The notion of hard disk (or "fixed disk" as IBM calls them) partitions was created to allow a hard disk based computer system to contain and "boot up" several completely different operating systems. Partitioning divides a single physical hard disk into multiple LOGICAL partitions.

A birthday cake is divided into multiple pieces by slicing it radically whereas a hard disk's divisions are circular. For example, a drive's first partition might extend from cylinder zero through 299 with the second partition beginning on cylinder 300 and extending through 599. This circular partitioning is far more efficient since it minimizes the disk head travel when moving within a single partition.

The partitions on a drive, even if there's only one, are managed by a special sector called the PARTITION TABLE, which is located at the very beginning of every hard disk. It defines the starting and ending locations for each of the disk's partitions and specifies which of the partitions is to gain control of the system during system boot up. When the hard disk drive is booted a tiny program at the beginning of the partition table locates the partition, which is flagged as being the "bootable partition" in the table and executes the program located in the first sector, the "boot sector," of that partition. This boot sector loads the balance of the partition's operating system then transfers control to it.

Each partition on a hard disk is blind to the existence of any other. By universal agreement, the operation of software inside a partition is completely contained within the bounds of the partition. Adherence to this agreement prevents multiple operating systems from colliding and allows strange environments to cohabitate on a single hard disk.

The sectors within a partition are numbered sequentially starting at zero and extending to the end of the partition. In kind with DOS's original belief that 640K of RAM would be more than we'd EVER need, there was a time in the not-so-distant past when a ten-megabyte hard disk was an unheard of luxury and was considered huge. How could any single person ever fill up 10 megabytes? No way.

Consequently DOS was designed to access sectors within its hard disk partition with a single sixteen-bit quantity. One "word" was set aside for the specification of partition sectors. As many of you know, a single sixteen-bit binary word can represent values from 0 through 65,535. So this limited a partition's total sector count to 65,536. Since hard disk sectors are 512 bytes long, a partition could contain 33,554,432 bytes. When you remember that binary megabytes are really 1,048,576 bytes each, that's exactly 32 megabytes.

This is the origin of DOS's infamous 32-me.g.abyte barrier. Today of course we have affordable drives with capacities well exceeding DOS's 32-me.g.abyte limit. The industry has invented three solutions to this partition size dilemma.

The first solution invented to the partition size problem utilizes DOS's inherent extendibility with external device drivers. Programs such as On Track’s DISK MANAGER, Storage Dimensions' SPEEDSTOR, and Golden Bow's VFEATURE DELUXE utilize a clever trick to circumvent the 32 megabyte DOS limit: They trick DOS into believing that sectors are larger than 512 bytes! By interposing themselves between DOS and the hard disk, these partitioning device drivers lead DOS to believe that individual sectors are much larger than they really are. Then when DOS asks for one "logical" 4k-byte sector they hand DOS eight 512-byte
physical sectors. This transforms the 65,536 sector count limit into a single partition containing more than 268 megabytes!

Compaq Computer has recently introduced the final solution with their introduction of DOS 3.31. Being big enough to get away with sacrificing some software compatibility, Compaq has redefined the way DOS numbers its partition sectors thereby removing the limitation at its source.

So now our hard disks have a low level format, with  "address ability" to the disk's individual physical sectors established. We have also defined and established partitions on our drive, which gives DOS a sub-range of the hard disk within which to build its filing system. Now let's examine the structure of MS-/PC-DOS filing systems. The following discussion also applies to DOS diskettes, which aren't partitioned but otherwise have an identical structure.

Let's begin by looking at the problem that DOS's filing system solves: Its task is to allow us, through the vehicle of DOS application programs, to create named collections of bytes of data, called files, and to help with their management by providing directories of these named files.

The directory entry for any DOS file contains the file's name and extension, the date and time when the file was last written and closed, an assortment of Yes/No "attributes" which indicate whether the file has been modified since last backup, whether it can be written to, whether it's even visible in the directory, etc. The directory entry for the file also contains the address of the start of the file.

We already know that hard disks are divided into numbered sectors 512 bytes in length. Since most of the files DOS manages are much larger than a single sector, disk space is allocated in "clumps" of sectors called clusters. Various versions of DOS utilize clusters of 4, 8 or 16 sectors each, or 2048, 4096, or 8192 bytes in length.

When a hard disk is completely empty, its clusters of sectors are all available for storing file data. As files are created and deleted on the hard disk, a bookkeeping system is needed which keeps track of which clusters are in use by which existing files, and which clusters are still available for allocation to new or growing files. This is the vital role played by the File Allocation Table. The "FAT," as it's frequently called, is the table DOS uses to manage the allocation of space on the hard disk.

As we know, the hard disk is arranged as a long stream of sectors. After being clumped together into clusters, it can be viewed as a long stream of clusters. Now picture a table consisting of a long stream of entries, with one entry in the table for each cluster on the disk. The first FAT table entry corresponds to the first hard disk cluster, and the last FAT entry corresponds to the last hard disk cluster.

Now imagine that DOS needs to create a new text or spreadsheet file for us. It must first find a free cluster on the hard disk, so it searches through the File Allocation Table looking for an empty FAT table entry, which corresponds to an empty hard disk cluster. When DOS finds the empty table entry it memorizes its number, then places a special "end of chain" marker in the FAT entry to show that this cluster has been allocated and is no longer free for use. DOS then goes out to the sectors, which comprise this cluster and writes the file's new data there.

This is all great until the file grows longer than a single cluster of sectors. DOS now needs to allocate a second cluster for this file. So it once again searches through the File Allocation Table for a free cluster. When found, it again places the special "end of chain" marker in this cluster and memorizes its number.

Now things begin to get interesting... and just a little bit tricky. Since files might be really long, consisting of thousands of individually allocated clusters, there's no way for DOS to memorize all of the clusters used by each file. So DOS uses each File Allocation Table entry to store the number of the file's next cluster!

Following along with our example, after finding and allocating the second cluster for the growing file, DOS goes back to the first cluster's FAT entry where it had placed that first "end of chain" marker and replaces it with the number of the file's second cluster. If a third cluster were then needed, its FAT entry would be marked "not available" by placing the special "end of chain" marker in it, then this third cluster number would be placed into the second cluster's FAT entry. Get it?

This creates a "chain" of clusters with each cluster entry pointing to the next one, and the last one containing a special "end of chain" entry which signals that the end of the file's allocation chain has been reached.

Finally, when the file is "closed," an entry is created in a DOS directory, which names the file and contains the number of the file's first cluster. Then, using that first cluster's FAT entry, the entire allocation "chain" can be "traversed" to find the clusters, which contain the file's data.