Brief Description
This document describes the functionality of the temporary file management (TFM) component of InterBase.
Purpose
Temporary File Management (TFM) is responsible for:
- Keeping configuration information for temporary files
- Creating temporary files
- Keeping track of created temporary files and their sizes
- Keeping track of available disk space
- Sharing disk space between concurrent requests.
Feature Description
There are two types of the temporary files that InterBase creates:
sort files
history list files
Temporary sort files
Sort files are created when size of the internal sort buffer is not big enough to make a sort. These files are created by server. There are two ways to define location for temporary sort files:
temporary file configuration in isc_config file
environment variables (INTERBASE_TMP, TMP, TEMP)
When server starts, it reads isc_config configuration file. Configuration file is read only once. Situation when each attachment will enforce engine to re-read configuration file is disallowed. Hence all config info must be defined in isc_config file before server starts. Format for temporary file configuration is described in next section.
If temporary file configuration is defined in isc_config file then server uses this configuration and ignores environment variables. If configuration of temporary files is not defined in isc_config file then server picks a location for a temporary file based on the following algorithm:
- Use directory defined in INTERBASE_TMP
- If INTERBASE_TMP is not defined then use directory defined in TMP (TEMP for Windows platform)
- If TMP(TEMP) is not defined use /tmp directory (/temp directory on the current drive for Windows NT; current directory on the current drive for Windows 3.x).
Server reads configuration into internal data structure: directory list. Directory list is shared by only one process -- the server. Directory list is mutex protected and must be locked before list operations.
Directory list may contain either as many entries as defined in isc_config file or only one entry in case of environment variable.
Each request ("connect" and "create database") gets and shares the same list of temporary file directories. Each request creates its own temporary files (each request has its own file I/O handle).
Note: For Classic architecture each instance of server gets its own directory list. There is no synchronization made between servers regarding temporary file management.
In case if no more space left in particular directory, new temporary file is created in the next directory from the directory list. If no more entries are in the directory list, then error message is printed out and processing of current request is stopped. Sort files are released when "sort" is finished or when request is released;
Temporary history list files
History list files are created by client (e.g. ISQL) and designed to keep track of the input commands.
There is only one way to define location for history list files, the environment variables (INTERBASE_TMP, TMP, TEMP)
Client picks a location for a temporary file based on the same algorithm as described for temporary sort files above.
Each instance of client creates its own single temporary file and increases size of the file (when necessary) until out of disk space. There is no synchronization made between clients regarding temporary file management. History list files are released when client quitted.
Format for the temporary file configuration
The format of temporary file configuration in the config file is as follows
TMP_DIRECTORY <whitespace> <size in bytes> <whitespace> <"pathname"> <EOL>
Temporary file configuration defines max limit size for particular directory that can be used for temporary files.
It is permitted to have multiple directories listed, each on its own line, with its own size specification. The same directories can be specified more than once, with different size configurations. IB will exhaust each specification before proceeding to the next one.
Example:
...
V4_LOCK_MEM_SIZE 98304
TMP_DIRECTORY 20000000 "/tmp"
V4_LOCK_SEM_COUNT 32
TMP_DIRECTORY 30000000 "/usr/gds.sogood/tmp"
TMP_DIRECTORY 30000000 "/tmp"
...
Implementation
Proposed Code changes:
DLS_get_temp_space:
Allocate disk space with required size
Inputs: size -- size of disk space to be allocated
sfb -- sort work file control block
Outputs: status -- TRUE/FALSE
Processing:
Get access to the temporary file directory list
Initialize mutex in case of the first access
Lock the mutex
If temporary file has already been created then allocate disk space within
directory which temporary file belongs to, otherwise do search in the directory
list for the first available space.
Unlock the mutex
Return TRUE to the caller in case if space was found and allocated, otherwise
return FALSE.
DLS_put_temp_space:
Release disk space occupied by sort temporary file
Inputs: sfb -- sort work file control block
Outputs: None
Processing:
Get access to the temporary file directory list
Lock the mutex
Increase free space of the directory by size of temporary file
Unlock the mutex
DLS_add_dir:
Add new entry to the temporary directory list.
Inputs: size -- size of the directory dir_name -- directory name
Outputs: None
Processing: Allocate memory for the new entry.
Initialize the entry.
Initialize mutex in case of the first access
Lock the mutex
Add new entry to end of the directory list
Unlock the mutex
DLS_get_access:
Get access to the temporary directory list
Inputs: None
Outputs: Pointer to the first element of the temporary directory list
Processing: Return address of the first element of the temporary directory list
gds__temp_file2:
Create a temporary file. This is a "clone" function from
gds__temp_file().
It has an additional input parameter dir.
Inputs: flag -- indicate systems call to create a file; if flag=TRUE use
"fopen()", otherwise (flag=FALSE) use "open()"
string -- prefix for the temporary file name (path excluded)
dir -- name of the directory (path) where file has to be created
Outputs: expanded_string -- if not NULL then area of memory contains full
unique name of the created temporary file.
Processing :
Make common part of the temporary file name:
if dir is defined
common part=dir + string + TEMP_PATTERN
else
if $INTERBASE_TMP is defined
common part=$INTERBASE_TMP + string + TEMP_PATTERN
else
if $TMP or $TEMP is defined
common part=$TMP or $TEMP + string + TEMP_PATTERN
else
common part=WORKFILE + string + TEMP_PATTERN
Make unique name of the temporary file
If expanded_string is not NULL then copy unique name to the memory pointed by
expanded_string
Create temporary file for writing
If expanded_string is NULL then make up the file "pre-deleted" Return
file descriptor to the caller
Important code changes with respect to previous version
gds__temp_multi_file() function no longer exists
get_temp_dirs() function no longer exists
attachment block (struct att) no longer has att_dls
sort work file control block (struct sfb) no longer has sfb_max_file_size,
instead of that it has pointer to the dls structure
directory list structure dls no longer has dls_file, instead of that it has
dls_inuse (occupied space in the directory) in addition to the user-defined
values for shared memory and semaphores, ISC_get_config() function reads
user-defined temporary file configuration.
Issues
Migration
interbase.cfg config file no longer exists. Two configuration files:
isc_config and configuration file for temporary file management (interbase.cfg
by default) have been merged into the one system config file: isc_config.
Format for temporary file configuration has been changed.
New environment variable INTERBASE_TMP has been introduced.