Lecture 10: File I/O(Chapter 3 - Stevens & Rago)

ACSC 372 – Systems Programming

2

Outline

Error Handling with <errno.h> Introduction to Files and File System in Unix

(File types, Partitions, i-nodes, blocks) File Processing Operations: Standard I/O

vs. Low level Ι/Ο Low Level Ι/Ο (System Calls I/O) Examples

3

Error Handling in C

How do we handle Run-time Errors in C? using printf()? debugging?

It is generally accepted that we cannot identify the cause of all errors occurred, if these errors are not caused due to the logic of our program. e.g. an error in the file system, disc fatal error, the file we try to

write to is locked by another user, etc

Aim: If an error occurs, an error message should be printed that reflects the real cause.

4

Error Handling in C#include <errno.h>

Library <errno.h> : System Error Numbers (/usr/include/errno.h)

The library contains an integer (errno) that expresses which error it has preceded, after each function call.

The 100+ errno (int) identify various types of errors (with the aid of constants, defined in the library errno.h). http://www.opengroup.org/onlinepubs/009695399/basedefs/errno.h.html

We can print these errors with the function:void perror(char *str); It produces an error message on the standard error, based on the

current value of errno. It outputs the string str, followed by a colon, followed by the error

message corresponding to the value of errno.

5

Some of the many constants defined in the header file errno.h: EPERM: Operation not permitted ENOENT (2): No such file or directory EINTR: Interrupted system call EIO: I/O Error EBUSY: Device or resource busy EEXIST: File exists EINVAL: Invalid argument EMFILE: Too many open files ENODEV: No such device EISDIR: Is a directory ENOTDIR: Isn’t a directory …..

Error Handling in C (cnt’d)

#include <errno.h>

6

#include <stdio.h>#include <errno.h>

int main () FILE * pFile;char *filename=“/home/a.txt";pFile = fopen (filename,"w");if (pFile==NULL) perror(filename);

return 0;

#include <stdio.h>int main ()

FILE * pFile;char *filename=“/home/a.txt";pFile = fopen (filename,"w");if (pFile==NULL) printf(“Unable to open %s”,

filename);exit(1);

return 0;

The following programs attempt to open a file with write mode, using the Standard I/O.

Obviously, there exists a number of possible error causes: permission problems, file lock problems, etc.

The program on the left does not give an accurate error message, whereas the program on the right (using the function perror()) gives the precise cause.

Error Handling in C (cnt’d)

#include <errno.h>

7

/* File: errors_demo.c */#include <stdio.h> /* For fopen, printf */#include <errno.h> /* For errno variable */

main() FILE *fp; char *p; int stat;

fp = fopen("non_existent_file", "r");if (fp == NULL)

/* Check for error */printf("errno = %d\n", errno);perror("fopen");

p = (char *) malloc(400000000); // ~4GBif (p == NULL)

/* Check for error */printf("errno = %d\n", errno);perror("malloc");

/* BE CAREFUL: unlink tries to remove a file so do not run this under root */

stat = unlink("/etc/motd");if (stat == -1)

/* Check for error */printf("errno = %d\n", errno);perror("unlink");

Error Handling in C (cnt’d)

Example

Execution Result:errno = 2fopen: No such file or directoryerrno = 12malloc: Not enough spaceerrno = 13unlink: Permission denied

/etc/motd: greeting displayed whenever a user logs

8

What a File is: A sequence of bytes, with no structure as far as

the operating system is concerned. The basic operations are: reading of data - read(), and writing data - write().

File Structure: The identification of the structure of the data can be done exclusively by the application (e.g. the Acrobat identifies the internal structure of a PDF file, but not of a DOC file).

Introduction to Files and File System

9

What a File is: Example GIFLet’s see how a GIF (Graphics Interchange Format) image file is encoded (PDF.gif)

$od -c pdf.gif # dump contents of file as characters rest in octal

0000000 G I F 8 9 a - \0 - \0 ∆ \0 \0 377 377 3770000020 377 377 \0 377 \0 377 377 \0 \0 \0 377 377 \0 377 \0 \00000040 \0 377 \0 \0 \0 377 1 \0 ή ή ή Ξ Ξ Ξ Ζ Ζ0000060 Ζ ΅ ΅ ΅ 204 204 204 s s s R R R \b \b \b0000120 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \00000100 377 377 377 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \00000140 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 ! ω 0040000160 001 \0 \0 021 \0 , \0 \0 \0 \0 \0 \0 \0 0050000200 377 ` $ 216 Ρ a 236 h 232 222 µ x $ p , Ο0000220 0 204 034 ? 252 ο f 215 ά 256 237 p H , ή 0340000240 \t Ϋ 017 W 032 002 236 Ώ ' * 025 036 220 Κ e0000260 S 8 E L » ί β 5 \t ) C p \a ' Τ 0330000300 υ 256 227 Θ 204 c ξ Έ ¦ Ή Ξ v w 8 236 Ω0000320 σ \ k V q 2 177 F 210 D 206 u < 2160000340 ' 205 1 W - % t 226 227 226 4 223 - / 4 \t

This scheme shows that images (like ALL files) are a sequence of bytes.

Magic number

10

File Types in Unix“Everything in Unix is a File”

Regular Files: Data files (txt, pdf, jpg,…)– $ ls -al ~/.profile

-rwxr----- 1 com.chc staff 281 Mar 19 12:46 … profile

Directory Files: System Files for the maintenance of the structure of the file system.

– $ ls -al ~ | grep test_dir

drwxr-sr-x 8 com.chc staff 4096 Mar 19 22:53 test_dir

Character-Special Files: Used to represent Serial I/O devices (terminals, printers and networks)

– $ ls -al `tty`

crw--w---- 1 com.chc tty 136, 1 Mar 21 11:36 /dev/pts/1

Block Special Files : Used to represent Magnetic Discs, Cdroms, flash drives, in which the data read/write is done in Blocks (e.g. 512 -8192 bytes)

– $ ls -al /dev/hdc # το /dev/cdrom

brw-rw---- 1 root disk 22, 0 Mar 21 10:09 /dev/hdc

11

Symbolic link Files: The file is a link that points to another – $ ln -s test_dir/test.txt test.lnk ; ls -al test.lnk

lrwxrwxrwx 1 com.chc staff 22 Mar 21 12:07 test.lnk ->

test_dir/test.txt

Νamed Pipes (FIFO): Used for communication between processes (see later).

– $ mkfifo pipef; ls -al | grep pipef

prw-r--r-- 1 com.chc staff 0 Mar 21 11:42 pipef

File Types in Unix (cnt’d)

“Everything in Unix is a File”

12

How Files are Stored

The files are stored on the storage devices (FLASH, Magnetic discs, Tapes, Cdrom, DVDs, etc).

File-System:

Part of the kernel that implements a set of structures of data of the main and secondary memory for the storage, hierarchical organization, retrieval, and processing of data.

The file system in various OS Windows NT,2000,XP,Vista: FAT, FAT32, NTFS, WinFS

Linux Local Ext2, Ext3 , Linux Distributed: NFS, AFS, …

CDROMs: ISO9960

Camera & mobile phones with Flash Memory : FAT

13

Partitions The disk is usually separated in partitions

In Unix, use the command df to find the partitions of

the system.

Different partitions can have different file system.

Every partition has the following format:

14

i-nodes

At the beginning of a partition, we have a list of i-nodes, which identify uniquely the files of the specific partition.

One i-node contains the information (a collection of disc addresses) so as the file system to know where the data/directory blocks are located.

To find the i-node number of a file, use the command "ls -i".

• From a C program, we can find the i-node through the system call stat()

15

Blocks A file (regular or directory) consists of a series of "blocks" (~512

bytes).

A block is named "data block" or "directory block", depending on whether it is a regular file or a directory file.

The directory block stores:

i-node number and filename for each file or sub-directory

16

Example of Finding a File using the i-

nodes Where are the contents of the

file /usr/test.c ?

Follow the path

2 => 200 => 4 => 202

=> 6 => 204,206

The blocks 204 and 206 contain the data of the file.

17

File Processing Operations

File processing operations using Low-Level I/O are fulfilled with the use of system calls.

Ι/Ο System Call: calls to the kernel of the operating system that returns a File Descriptor (FD).

All the processing is then done through the FD.

18

From the application programs, we can retrieve file data using the following methods:A) Standard I/O Library (Procedure) Callsfopen, fclose, fread, putc, etc.

… what we have used until now, with the use of the data type FILEIt is recommended to use this method only for regular files (not for other types)

B) Low level I/O (System Calls)(e.g., open, close, read, write, lseek) through File Handler (int) Look like library functions calls, with the difference that the system

calls are executed in the kernel space, and not in the user space.

File Processing Operations (cnt’d)

19

Standard I/O vs. Systems Calls The Standard I/O: stdio.h (Dennis Ritchie – 1975) is a library of functions for

accessing files from C programs The Standard I/O make function call of the same process, whereas the System Calls call

kernel operations/functions. The Standard I/O calls are also implemented through System Calls! The Standard I/O are easier to use The Standard I/O creates a Stream (FILE *), whereas the SysCalls a File Handler. The Standard I/O do buffering (BUFSIZE in stdio.h).

…. But give less freedom, that is required in Unix System Programming

System Call Standard I/O callopen fopenclose fcloseread/write fread/fwrite // used for binary I/O chunks of bytes rather char or line

getchar/putchar // read a character from STDINgetc/putc // …>>………>>…..from filefgetc/fputc // identical to getc/putcgets/puts // read a string from STDINfgets/fputs // >> > > > > > from filescanf/printf // read/write formatted input/output from/to STDIN/STDOUTfscanf/fprintf // >> > > > > > from file

lseek fseek // move to a specific location in a file

Revision of Standard I/O Calls: Chapter 5 – Stevens & Ragohttp://www.cppreference.com/stdio/index.html

20

System Calls Advantage

System Call Advantage: Full Control!• Implementation of operations that are not offered by the Standard I/O (locking, buffering, IPC)

•Interprocess Communication: When a communication between processes with the use of pipes and sockets is programmed (see later)

KERNEL SPACE

USER SPACE

21

System Calls Disadvantages

Too expensive! : Whereas a procedure call wants only a number of machine instructions, a system call will execute the following procedure: The process under execution is interrupted and its state is saved. The Kernel (OS) takes the control of the CPU and executes the system

call. The Kernel (OS) is interrupted and its state is saved, and at the same time

it gives the control back to the process.However, proper/correct use of system calls => Increased Performance

System Dependent: Different Implementations of OS support different operations Portability issues of the source code with the system calls.

More Difficult programming: With standard I/O, the library organizes the calls to the kernel, in a way that data are transferred in full blocks from the disc to the memory, whereas with the System Calls this is something we need to handle by ourselves.

22

System Calls for I/O

There are 5 basic system calls that are provided by the Unix OS for file I/O (see man -s 2 open)

int open(char *path, int flags [ , int mode ] );

int close(int fd);

int read(int fd, void *buf, int size);

int write(int fd, void *buf, int size);

off_t lseek(int fd, off_t offset, int whence);

In order to have access to these functions through application programs, we need to include the following libraries

// file control options:

#include <fcntl.h>

// standard symbolic constants and types

#include <unistd.h>

23

System Calls: Open()/Close()open a file for reading/writing data

# Returns: File Descriptor if OK, -1 on errorint open(const char *path, int flags [ , int mode ] )

# Returns: 0 if OK, -1 on errorint close(int filedes);

'path‘: Absolute or relative path to the file The flags and mode define how we want to use the file. If the kernel accepts our request, then the smallest possible “file

descriptor (FD)” is created, which is an integer number for file. Any future reports to the file are done through the FD.

The Linux 2.4.22 sets a hard limit of 1,048,576 FDs per process. This is defined by the constant OPEN_MAX in the library limits.h

If the system call open() returns -1, then the request has not been satisfied. We can then check the value of the variable "errno" to find out why (using perror()).

Remember that when a program is opened, three FDs are already opened (0:stdin, 1:stdout, 2:stderr).

24

Flags (constants in fcntl.h) : Obligatory and optional choices. Obligatory : O_RDONLY (open for reading only) ,O_WRONLY (writing), O_RDWR (read/write) one and only one of these 3 constants must be specified.

Optional: O_APPEND (append to the end of file), O_TRUNC (if file exists in WR or RDWR then truncate its size to 0), O_CREAT (create if file does not exist – we have to set MODE variables – see below)

Mode (constants in sys/stat.h) : Specifies the permissions if we create a file The mode is an integer (same logic with Unix file permissions).

Note: The Flags and the Mode can be given by using the bitwise OR of the symbolic names, e.g. open("test.txt", O_WRONLY | O_CREAT, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);

Explanation: The ORing sets the access

permission bits :rwxrwxrwx = 110100100

(that is. 100000000 | 010000000 | …)

S_IRUSR: Read permission, ownerS_IWUSR: Write permission, ownerS_IXUSR: Execute permission, owner

S_IRGRP: Read permission, groupS_IWGRP: Write permission, groupS_IXGRP: Execute permission, groupS_IROTH: Read permission, othersS_IWOTH: Write permission, othersS_IXOTH: Execute permission, others

System Calls: Open()/Close()

25

#include <fcntl.h>#include <stdio.h> // for printf#include <stdlib.h> // for exitint main()int fd1, fd2;

if(( fd1 = open("foo.txt", O_RDONLY)) < 0)perror("opening file");exit(1);

printf("The File Descriptor is : %d", fd1);

if (close(fd1) < 0) perror("closing file");exit(1);

System Calls: Open()/Close()

Simple Example

26

System Calls: Open()/Close()

Execution Mechanism

A) The file system reads the current directory, and finds that “myfile” is represented internally by entry 13 in the list of files maintained on the disk.

B) Entry 13 from that data structure is read from the disk and copied into the kernel’s open files table. Τhat consists of data blocks 5 and 8.

C) User’s access rights are checked and the user’s variable fd is made to point to the allocated entry in the open files table

27

System Call: Creat()File Creation

int creat(const char * filename, int mode);or open(filename, O_WRONLY | O_CREAT | O_TRUNC, mode);

# Returns: File Descriptor for Ο_WRONLY if OK, -1 on error

The disadvantage of creat() is that we cannot read from the file we have just created.

To correct it, and in order to have: Creation + Read/Write, we use the following:open(filename, O_RDWR | O_CREAT | O_TRUNC, mode);

28

System Call: Read()Read from FD

int read(int fd, void *buf, int size)Returns: number of bytes read, 0:EOF, -1:ERROR

Reads size bytes from the entity (file, terminal device, pipe or socket) that is associated with the fdand copies them to buf.

Waits (blocking wait) until the buf gets full, or EOF. It supposes that your application has allocated

enough space (malloc) to buf. What happens otherwise?

Note that buf is Posix-compliant (void * generic pointer) rather than the classic definition char *buf)

29

System Call: Read()

Execution Mechanism A) The argument fd identifies the open file by pointing into the kernel’s open files table.

B) the system gains access to the list of blocks that contain the file’s data.

C) The file system therefore reads disk block number 5 into its buffer cache.

D) the desired range of 100 bytes is copied into theuser’s memory at the address indicated by buf.

30

System Call: Write()Writing to FD

int write(int fd, void *buf, int size)Returns: number of bytes written if OK, -1:ERROR

Writes size bytes from buf to the entity (file, terminal device, pipe or socket) that is associated with the fd.

Returns the number of bytes written. This could be less than what was requested, if there

was a problem with the file descriptor.

If it returns 0, then nothing has been written. If it returns -1, then check the error code errno

31

System Call: Write(): Execution Mechanism

b) The kernel then flushes the newly written data to disk (assuming that block#8 has adequate space).Else a new block is allocated and the data is written there as well.

a) Our program calls write(fd,buf, sizeof(buf));and the data is copied to kernel space

32

Example 1: Implementation of cat

Implement, using C system calls, a simple program that simulates the UNIX command cat (with no arguments)

e.g../mycat < filename

ls | ./mycat

./mycat (it repeats what we write)

33

// The library stdio.h is included only for the constant BUFSIZE, so that we use the most appropriate value (based on the current system), rather than setting a value by ourselves.

#include <unistd.h> // STDIN_FILENO, STDOUT_FILENO#include <stdio.h> // BUFSIZ #include <error.h> int main(int argc, char * argv [])

char buf[BUFSIZ]; // BUFSIZE 8192int n;

while((n = read(STDIN_FILENO, buf, sizeof(buf))) > 0)if ( write(STDOUT_FILENO, buf, n) != n )

perror ("write error");

if (n < 0)perror("read error");

return 0;

Question: Why we didn't need to open the descriptors FD#0 and FD#1?

Example 1: Implementation of cat

34

Example 2: Read/Write Demo

Implement, using C system calls, a simple program that creates a file, writes a string to that file, closes the file, then opens the file and writes another string, closes the file, and then reads the file and prints its contents to the screen. In each step, print the number of bytes read/written.

35

/* File: io_demo.c */#include <stdio.h> /* For printf, BUFSIZE*/#include <fcntl.h> /* For O_RDONLY, O_WRONLY,O_CREAT, O_APPEND */#include <string.h> /* strlen */#include <stdlib.h> /* exit */

main() int fd, bytes;char buf[BUFSIZ];char *filename = "temp.txt";char *line1 = "First write. \n";char *line2 = "Second write. \n";

// 0600 <==> S_IRUSR | S_IWUSR <==> -,rw-,---,---if ((fd = open(filename, O_WRONLY | O_CREAT | O_TRUNC, 0600)) == -1)

perror("open");exit(1);

// Note: We write strlen(line1), instead of strlen(line1)+1 in order to // avoid the undesirably \0 ( “First write. \n\0 Second write. \n\0” )bytes = write(fd, line1, strlen(line1)); /* Data out */printf("%d bytes were written\n", bytes);close(fd);

Example 2: Read/Write Demo

36

if ((fd = open(filename, O_WRONLY | O_APPEND)) == -1) perror("open");exit(1);

bytes = write(fd, line2, strlen(line2)); /* Data out */printf("%d bytes were written\n", bytes);close(fd);

if ((fd = open(filename, O_RDONLY)) == -1) perror("open");exit(1);

bytes = read(fd, buf, sizeof(buf)); /* Data in */printf("%d bytes were read\n", bytes);close(fd);buf[bytes] = '\0';printf("%s", buf);

$ ./a.exe14 bytes were written15 bytes were written29 bytes were readFirst write.Second write.

Example 2: Read/Write Demo

37

Example 3: Implementation of File

Concatenation

Implement, using C system calls, a simple program that simulates the operation of the following shell command

cat file1 >> file2

The program should be executed as follows.

$./concat file1 file2

38

/* File: append_file.c */#include <fcntl.h> /* For O_RDONLY. O_WRONLY, O_CREAT, O_APPEND */#include <unistd.h> // STDERR_FILENO#include <string.h> // strlen#include <stdlib.h> // exit

main(int argc, char *argv[])

int n, fromFD, toFD;char buf[1024]=;

if (argc != 3) /* Check for proper usage */

sprintf(buf, "Usage: %s from-file to-file", argv[0]) ;write(STDERR_FILENO, buf, strlen(buf));exit(1);

Example 3: Implementation of File

Concatenation

39

Παράδειγµα 3: Υλοποίηση Concat Αρχείωνif ((fromFD = open(argv[1], O_RDONLY)) < 0) /* Open from-file */perror("open");exit(1);

if ((toFD = open(argv[2], O_WRONLY | O_CREAT | O_APPEND, 0660)) < 0) /* Open to-file */perror("open");exit(1);

while ((n = read(fromFD, buf, sizeof(buf))) > 0)if (write(toFD, buf, n) != n) /* Copy data */

perror("copy error");

close(fromFD); /* Close from-file */close(toFD); /* Close to-file */

$ cat file1First write.$cat file2Second write.$./concat file2 file1$cat file1First write.Second write.

40

System Call: lseek()Random “movement” within the file

off_t lseek(int fd, off_t offset, int whence)Returns: new file offset if OK, -1 on error off_t defined in sys/types.h Offset: Define the position Whence (from where): Define how offset is used.

SEEK_SET: relative to beginning of file SEEK_CUR: relative to the current position SEEK_END: relative to end of file

The system call lseek allows us to access the file contents randomly (Random Access).

Every file has a «current file offset», which is a positive integer value that counts the number of bytes from the beginning of the file.

If the file is opened, the offset is 0. After reading m bytes, the offset is m.

The“l”seek comes from the “long”seek, since the offset is a long integer.

41

The lseek defines the offset in the kernel without causing any I/O. The Ι/Ο will be done at the next read/write.

If we execute the following command, then the next read/write operation will be done 100 bytes on the right of the current position:lseek(filedesc, 100, SET_CUR);

Therefore a file may have “empty gaps” These “holes” are subsequent 0 (NULLs) and hold

1 byte per Null. The offset can take positive and negative values,

provided it is supported by the entities (e.g. ok for a regular file, but not for a socket).

System Call: lseek() (cnt’d)Random “movement” within the file

42

Example 4: Implementation of a

Simple Database

Implement, using C system calls, a simple binary database that writes five subsequent structures of the form

typedef struct personint id;char sex;char name[40];

__attribute__((packed)) PERSON;

in the file user.db, and then it leaves three structures empty, and it further writes five more structures. At the end, it reads all the records from the file, and prints them on the screen.

The __attribute__((packed))means that we don’t want the compiler to add padding to the

structure for memory alignment.

43

main(int argc, char *argv[])

int n, fd, i;PERSON p;char *filename="users.db";int offset;

if ((fd = open(filename, O_RDWR | O_CREAT, 0660)) < 0)

/* Open to-file */perror("open db");exit(1);

// set the values in the p structurep.sex = 'm';strcpy(p.name, "Costas");

// write 5 tuples to the filefor (i=0; i<5; i++)

p.id = i;write(fd, (void *) &p, sizeof(p));

Example 4: Implementation of a Simple Database

(cnt’d)

44

Example 4: Implementation of a Simple Database

(cnt’d)

// seek the file descriptor three sizeof(p) positions to the right (from SEEK_CUR)lseek(fd, 3*sizeof(p), SEEK_CUR);

// write another 5 tuples to the filefor (i=0; i<5; i++)

p.id = i;write(fd, (void *) &p, sizeof(p));

// rewind the file descriptor to the beginning of the filelseek(fd, 0, SEEK_SET);

// Now read and print the respective tupleswhile((n = read(fd, (void *) &p, sizeof(p))) > 0)

if ( sizeof(p) != n )perror ("read error");

printf("<%d,%c,%s>\n", p.id, p.sex, p.name);

close(fd);

45

Example 4: Implementation of a Simple

Database (cnt’d)$dbtest<0,m,Costas><1,m,Costas><2,m,Costas><3,m,Costas><4,m,Costas><0, ,><0, ,><0, ,><0,m,Costas><1,m,Costas><2,m,Costas><3,m,Costas><4,m,Costas>

The size of the file is 585 bytes

(13 records * 45 bytes)

46

Supplementary Slides

47

1) Bitwise Operations# Note that the above are powers of two #or using #define KEYWORD 01 ….

enum KEYWORD = 01, EXTERNAL = 02, STATIC = 04)

# Turn on the EXTERNAL and STATIC bits

flags |= EXTERNAL | STATIC

# Turn off the EXTERNAL and STATIC bits

Flags &= ~(EXTERNAL | STATIC) // Flags &= ~EXTERNAL & ~STATIC // de-morgan

# True if both EXTERNAL and STATIC are turned off.

If ((flags & (EXTERNAL | STATIC)) == 0)

# Structures of bits

struct

unsigned short a:1; // 1 bit

unsigned short b:1; // 1 bit

__attribute__((packed)) flags;

48

Memory Alignment

typedef struct

byte bfType1;

byte bfType2;

dword bfSize;

word bfReserved1;

word bfReserved2;

dword bfOffBits;

__attribute__((packed)) BITMAP_FILE_HEADER;

• Το __attribute__((packed)) σηµαίνει ότι δεν θέλουµε ο compiler να

προσθέσει padding στην δοµή για να την κάνει align στην µνήµη.

• Αν εκτελέσουµε sizeof(BITMAP_FILE_HEADER) θα µας

επιστρέψει 14 bytes , ενώ χωρίς το __attribute__((packed)) θα µας

έδιδε 16 bytes

typedef char byte;

typedef short int word;

typedef int dword;

49

Memory AlignmentUnderstanding What Memory Alignment Means

Most CPUs require that objects and variables reside at particular offsets in the system's memory. For example, 32-bit processors require a 4-byte integer to reside at a memory address that is evenly divisible by 4. This requirement is called "memory alignment". Thus, a 4-byte int can be located at memory address 0x2000 or 0x2004, but not at 0x2001.

On most Unix systems, an attempt to use misaligned data results in a bus error, which terminates the program altogether. On Intel processors, the use of misaligned data is supported but at a substantial performance penalty. Therefore, most compilers automatically align data variables according to their type and the particular processor being used. This is why the size that structs and classes occupy is often larger than the sum of their members' size:

struct Employee

int ID;

char state[3];

int salary;

;

Apparently, Employee should occupy 11 bytes (4+3+4). However, most compilers add an unused padding byte after the field 'state' so that it aligns on a 4 byte boundary. Consequently, Employee occupies 12 bytes rather than 11. You can examine the actual size of an aggregate by using the expression sizeof(Employee).

http://www.devx.com/tips/Tip/13265

50

The Unix i-node

• In Unix, files are represented internally by a structure known as an inode, it includes an index of disk blocks.

• The index is arranged in a hierarchical manner:

• Few (e.g. 10) direct pointers, which list the first blocks of the file.

• One single indirect pointer - points to a whole block of additional direct pointers.

• One double indirect pointer - points to a block of indirect pointers.

• One triple indirect pointer - points to a block of double indirect pointers.

51

i-node - Example

• Blocks are 1024 bytes.

• Each pointer is 4 bytes.

• The 10 direct pointers then provide access to a maximum

of 10 KB.

• The indirect block contains 256 additional pointers, for a

total of 266 blocks (266 KB).

• The double indirect block has 256 pointers to indirect

blocks, so it represents a total of 65536 blocks.

• File sizes can grow to a bit over 64 MB.

52

i-node Structure

53

Super Block

• How the kernel assigns inodes and disk blocks?

• In order to make this assignment, the kernel hold

a super block, this block contains:

– The size of the file system

– A list of free blocks available on the fs

– A list of free inodes in the fs

– Etc…

54

i-node Assignment to a New File

• The file system contain a list of inodes, when a process

needs a new inode, the kernel can search the inodes list for

a free one.

• To improve performance, the super block contains the

number of free inodes in the file system.

• If the super block list of free inodes is empty, the kernel

searches the disk and places as many free inode numbers

as possible into the super block.

• Freeing an inode - The kernel places the inode number in

the super block free inodes list.

55

Allocation of Disk Blocks

• When a process writes data to a file, the kernel

must allocate disk blocks from the file system for

direct or indirect block.

• Super block contains the number of free disk

blocks in the file system.

• When the kernel wants to allocate a block from

the file system, it allocates the next available

block in the super block list.

56

Direct Memory Access

• When a process needs a block from the disk, the cpu needs to copy the requested block from the disk to the main memory.

• The is a waste of cpu time.

• If we could exempt the cpu from this job, it will be available to run other ‘ready’ processes.

• This is the reason that the operating system contains the DMA.

57

Direct Memory Access – Cont.

• Sequence of actions:

1. Os generates the DMA and pass it the needed parameters (address on disk, address on main memory, amount of data to copy)

2. The running process is transferred to the blocked queue and a new process from the ready queue is selected to run.

3. The DMA transfers the requested data from the disk to the main memory.

4. The DMA sends an interrupt to the cpu, indicating the IO operation has been finished.

5. The waiting process is transferred to the ready queue.