DISTRIBUTED FILE REPOSITORY

1

Author:: Daniele Giannetti

This code implements a distributed file repository, which has the following architecture:

 Client 1 <---|                               |---> File Server 1
              |                               |
              |                               |
 Client 2 <----------------> DFR <----------------> File Server 2
              |                               |
              |                               |
 Client 3 <---|                               |---> File Server 3
              |                               |
              |                               |
              .                               .
              .                               .

The DFR is the multithreaded process which implements the abstraction of distributed file repository, Clients request files to the DFR, those files are located on the File Servers. The base funcion of the DFR is to administer File Servers load and tell the Clients where to find a specific file they are looking for.

TERMINOLOGY

In the following we will refer to:

CL: Client looking for files
FS: File Server holding some files and offering them to the clients
DFR: Distributed File Repository process

COMPILATION

The base makefile in the DFR root directory can be directly used to build the code, as follows:

make

the clean make rule has been also defined to remove all non-source files from the project folder generated through a make command, as follows:

make clean

Makefiles have been defined for each singular component of the project, we have:

./src_DFR/Makefile ./src_FS/Makefile ./src_CL/Makefile ./src_common/Makefile

Which can be used to build the individual sections of the project using make or make clean as described above. An exception to this procedure is the ./src_common/Makefile which can be used to build only object files, and not executables, as follows:

[in folder ./src_common] make command.o make utility.o

Makefiles can be modified depending on the needs, they have been written for testing purpose, in particular note that some macros may be defined and passed to the compiler to obtain different behaviours:

NO_OUTPUT: suppresses normal output of the source file compiled with this macro definition, no output will be sent on the stdout stream, by default the NO_OUTPUT mode is enabled for FSs and CLs

DEBUG: enables debug output, which is sent on the stderr stream, by default the DEBUG mode is not enabled

TEST: introduces some delays in CLs operations, useful for testing process, by default the TEST mode is enabled. TEST mode is available only for clients.

EXECUTION

Compiling the project using the global Makefile (simple make command) will produce 3 executable files in the ./bin directory from the code root. Those files are:

./bin/CL: Client program (test client)
./bin/FS: File Server program
./bin/DFR: Distributed File Repository program

The usage of those files is the following:

 CL [DFR_IP_address_or_domain_name_for_CLs] [DFR_port_for_CLs] \
         [file_to_transfer] [new_filename] [mode] [delay]

 FS [DFR_IP_address_or_domain_name_for_FSs] \
         [DFR_port_for_FSs] [port_for_CLs] \
         [file1] [max_clients_for_file1] \
         [file2] [max_clients_for_file2] \
         [file3] [max_clients_for_file3] \
         ...

 DFR [IP_address_for_FSs] [IP_address_for_CSs] \
         [port_for_FSs] [port_for_CLs]

TEST

The code root folder contains a subfolder ./test/ In this folder we do have:

./test/setup.sh: script to setup the test environment, the project needs to be compiled before launching this script, and the executable files need to be located in the ./bin/ folder. The script copies the executable files in the ./test folder

./test/tierdown.sh: script to tierdown the test evironment, it simply removes the copied executable files from the test folder, you should use this when you are done testing

./test/reset.sh: script to remove files created by one of the two predefined test, and to kill all processes still running. With both predefined tests we have that the DFR and the FS or FSs processes created will not terminate until a terminating signal is sent to them, this is because they are written to run indefenitively. In predefined test 1 we also have that Client processes run indefinitively, for testing reason, those needs to be terminated using a terminating signal. This scripts uses the kill command to shut down the processes, this means that some ports used for the terminated test may be unusable for some time depending on the moment on which the DFR and FSs processes (which are not designed to be interrupted) will be signaled... For this reason is recommend using different base ports for subsequent tests, to avoid errors due to socket initialization failed on the specified ports.

./test/test1.sh: script to begin predefined test 1. It takes from command line input a base port number, which shuld be above 1024 and below 62536 to avoid errors due to the bad ports resulting.
- the DFR is created
- two FSs are created, the first shares files:
  - ./test/files/test1/file1
  - ./test/files/test1/file2
  - ./test/files/test1/file3
    and the second shares files:
  - ./test/files/test1/file3
  - ./test/files/test1/file4
  - ./test/files/test1/file5
    both of them with a maximum of 2 CL per file at the same time.
- ten CLs are created in test mode one, which means they try to access recursively the specified file, with a delay between the requests as specified from the command line. Note that in this test clients will ask for file "file3". Delays for CLs has been defined to make the test slow enough, we have delays of, respectively, 3, 7, 11, 13, 17, 19, 23, 29, 31 and 37 for the clients.

Created and continously rewritten files transferred from the FSs to the CLs are stored in folder ./test/test1_clients/, which can be then removed by the reset.sh script as described earlier.

./test/test2.sh: script to begin predefined test 2. It takes from command line input a base port number, which shuld be above 1024 and below 62536 to avoid errors due to the bad ports resulting.
- the DFR is created
- ten CLs are created in test mode two, which means they ask for a notify on the specified file, They will keep asking for notification until a successfully transfer occours, the file requested in our case is file: "file". When a client finally transfer the file, it terminates disconnecting from the DFR. A little delay of 2 seconds has been inserted in CLs operations to make the testing process simpler.
- one FS is created, it shares just one file:
  - ./test/files/test2/file with a maximum of 3 clients at the same time.

Created files transferred from the FS to the CLs are stored in folder ./test/test2_clients/, which can be then removed by the reset.sh script as described earlier.