mod_privsep

Privilege Separation for Apache httpd

1. APR Virtual File System Proposal

A proposal for a VFS (Virtual File System) layer in APR (Apache Portable Runtime).

The proposed VFS will be similar to APVFS although with a number of key differences:

• The APVFS API is an exact mirror of the APR File IO API while in the use case we are looking at we would need to be able to pass down request_rec* context from within httpd so that a VFS module implemented within httpd could have behavior based on the vhost or request. i.e. We need some sort of application specific apr_vfs_context *ctx argument (that would contain request_rec* in the httpd use case) on the VFS versions of the file IO API.

• The VFS provider API should have an additional apr_vfs_mount_t* argument to all of the provider implementation functions in addition to passing through the apr_vfs_context* mentioned above - this is so that we can more easily support stacking and multiple mount points (multiple instances of a particular VFS provider).

• Instead of scheme:/ file paths we need to be able to mount these VFS providers somewhere within a regular file namespace (although mounting of a scheme prefix would still be an option). This is to allow compatibility with existing code and would be needed for stacking/layering. Stacked VFS providers could then use their per instance data to pass through to an underlying VFS implementation e.g. a VFS provider that just does logging, or performance statistics and passes through to the lower layer VFS provider.

• Support for Inheritance. VFS implementations should not have the burden of re-implementing buffered IO or apr_(read|write)_full and other VFS routines that could be common to a few VFS implementations.
• Support a VFS with minimal changes to client application. The VFS implementation will be a patch to APR that makes the standard APR File IO routines VFS capable, so that minimal changes are required to the user application to support a VFS.
• New client interface to support Asynchronous IO.

2. VFS Client API

The regular APR File IO layer would become a VFS dispatch layer and the current implementations functions would be renamed. e.g. apr_file_open becomes a VFS dispatch interface and the APR platform specific implementations would be renamed to apr_unix_file_open, apr_win32_file_open, apr_netware_file_open, apr_os2_file_open respectively.

This approach will add the VFS dispatch layer for all applications while maintaining 100% interface compatibility. The additional overhead will be negligible (a couple of pointer dereferences and a tail call) compared to the system call overhead.

Infrastructure common to multiple platforms (such as buffering) could move into apr_common_file_* functions that can be shared by the implementations. This would form the basis of inheritance in the VFS reducing the required number of operations a VFS provider module needs to implement.

2.1. VFS Context

• An additional type apr_vfs_context_t will be created so that a client context can be passed down to the VFS module IO operations (in httpd this will be stored in r->vfs and contain a userdata pointer with request_rec* in it to enable httpd implemented VFS modules to alter behavior based on the vhost or request).

• Additional _ex (suffix TBD) versions of the APR File IO interfaces will be created that will explicitly pass the VFS context to the file IO VFS provider functions.

• The default VFS implementation on top of APR will ignore apr_vfs_context_t and it may be NULL in file IO calls.

• If apr_vfs_context_t is NULL, then context specific behavior implemented by client specific VFS modules will not be available (e.g. in httpd vhost specific mounts, etc).

• The standard versions of the APR file IO routines will also be able to have an apr_vfs_context_t associated with them by setting a thread local variable using apr_vfs_thread_context_set() - this will allow a apr_vfs_context_t to be associated with IO calls created from within modules that have not been modified to use the the _ex versions of the APR File IO interfaces.

• In httpd, create_request will associate a apr_vfs_context_t with the thread, then in its cleanup, remove the association (although where possible r->vfs will be explicitly passed). This will allow callers of the regular APR File IO routines to still have the apr_vfs_context_t passed down to the VFS providers.

• This approach will work in httpd until there is an MPM that concurrently handles more than one request per thread (an event driven MPM), which will then require changes to explicitly pass apr_vfs_context_t and use the {{_ex}} versions of the APR File IO interface (which are required to be used for asynchronous IO).

2.2. Asynchronous IO

• apr_iocb_t - async IO callback structure would be added to the extended interface. The VFS dispatch could call the completion before returning for implementations that do not support async IO.

2.3. Types

2.3.1. apr_vfs_context_t

The apr_vfs_context_t type is a user handle that will be passed down to the VFS provider so that application implemented providers can have context specific behavior (request_rec* would be contained in the userdata pointer in httpd for example).

struct apr_vfs_context_t {
    void              *userdata;
};

2.3.2. apr_iocb_t

The apr_iocb contains a callback function for IO completion events and the results of completed IO.

struct apr_iocb_t {
    void              (*cb) (apr_vfs_iocb_t *iocb);
    apr_status_t      rv;
    apr_size_t        nbytes;
    void              *userdata;
};

2.4. New Functions

2.4.1. apr_vfs_context_new

Creates a apr_vfs_context_t to associate client userdata that is to be passed through to VFS providers when successive APR File IO calls are made (for code not converted to using the extended interface).

APR_DECLARE(apr_status_t) apr_vfs_context_new(apr_vfs_context_t **ctx, void *userdata, apr_pool_t *pool);

2.4.2. apr_vfs_thread_context_set

Sets a thread local context variable so that VFS providers can access the userdata in apr_vfs_context_t when the existing APR File routines are used (ones that do not have the additional apr_vfs_context_t and apr_iocb_t arguments).

APR_DECLARE(apr_status_t) apr_vfs_thread_context_set(apr_vfs_context_t *ctx);

2.4.3. apr_vfs_thread_context_get

Gets the thread local VFS context variable.

APR_DECLARE(apr_status_t) apr_vfs_thread_context_get(apr_vfs_context_t **ctx);

2.5. Extended Interface Functions

The existing APR File IO functions become the VFS dispatch layer (programs will not need to be changed to take advantage of the VFS). The existing APR File IO implementations will be renamed and be dispatched to by default.

The existing apr_stat interface will dispatch to the VFS provider with the calling thread's apr_vfs_context_t (set with apr_vfs_thread_context_set() and it may be NULL) and a NULL apr_iocb_t.

In addition _ex VFS dispatch interfaces will augment the standard interfaces to allow explicit passing of apr_vfs_context_t and apr_iocb_t for Asynchronous IO.

2.5.1. apr_stat_ex

Example of extended apr_stat_ex (with additional apr_vfs_context_t and apr_iocb_t arguments)

APR_DECLARE(apr_status_t) apr_stat_ex(apr_vfs_context_t *ctx,  /* may be NULL (overrides thread context) */
                                      apr_iocb_t        *iocb, /* NULL for synchronous calls */
                                      apr_finfo_t       *finfo, 
                                      const char        *fname, 
                                      apr_int32_t       wanted,
                                      apr_pool_t        *pool);

2.5.2. apr_file_open_ex

Example of extended apr_file_open_ex (with additional apr_vfs_context_t and apr_iocb_t arguments)

APR_DECLARE(apr_status_t) apr_file_open_ex(apr_vfs_context_t *ctx,  /* may be NULL (overrides thread context) */
                                           apr_iocb_t        *iocb, /* NULL for synchronous calls */
                                           apr_file_t        **new, 
                                           const char        *fname, 
                                           apr_int32_t       flag, 
                                           apr_fileperms_t   perm, 
                                           apr_pool_t        *pool);

2.5.3. ...

To do: document the rest of the extended File IO API functions ...

3. VFS Provider API

3.1. Types

• apr_vfs_mount_t type - provider mount structure (created by the VFS layer)

• apr_vfs_ops_t type - provider operations structure (created by the VFS provider). It will be shared by multiple mounts of the same provider.

It is proposed that apr_file_t and apr_dir_t will become VFS types and will no longer be opaque (and contain a common subset to support inherited buffering and the name attributes). The platform specific types will become apr_file_private_t and apr_dir_private_t and will contain the platform specific info stored in a *private member of the apr_file_t and apr_dir_t types. See section on analysis of common platform elements of the current apr_file_t and apr_dir_t below.

3.1.1. apr_vfs_mount_t

struct apr_vfs_mount_t {
    apr_vfs_mount_t   *next;
    apr_uint32_t      flags;
    char              *mntpoint;
    apr_vfs_opts_t    *vfsops;    /* VFS operations structure */
    apr_vfs_private_t *private;   /* Provider specific mount instance data */
};

3.1.2. apr_vfs_ops_t

struct apr_vfs_ops_t {
    apr_status_t (*stat)          (apr_vfs_mount_t *mnt, 
                                   apr_vfs_context_t *ctx, 
                                   apr_iocb_t *iocb, 
                                   apr_finfo_t *finfo, const char *fname, 
                                   apr_int32_t wanted, apr_pool_t *pool);
    apr_status_t (*file_open)     (apr_vfs_mount_t *mnt, 
                                   apr_vfs_context_t *ctx, 
                                   apr_iocb_t *iocb, 
                                   apr_file_t **new, const char *fname, 
                                   apr_int32_t flag, apr_fileperms_t perm, 
                                   apr_pool_t *pool);
    ...
};

3.1.3. apr_file_t

New apr_file_t that is visible to VFS providers (see section below on Inheritance and common file structure fields).

struct apr_file_t {
    apr_pool_t         *pool;
    apr_vfs_mount_t    *mnt;
    apr_file_private_t *private;
    apr_file_buffer_t  *buffer; /* non-null indicates buffering */
    char               *fname;
    apr_os_file_t      fd;      /* perhaps should be in private? */
    apr_int32_t        flags;
    int                eof_hit;
}

3.1.4. apr_file_buffer_t

And apr_file_buffer_t for inheritable buffering implementation.

struct apr_file_buffer_t {
    char               *buffer;
    apr_size_t         bufpos;
    apr_size_t         bufsize;
    apr_size_t         dataRead;
    int                direction;
    apr_off_t          filePtr;
    apr_thread_mutex_t *mutex;
}

3.1.5. apr_dir_t

New apr_dir_t that is visible to VFS providers.

struct apr_dir_t {
    apr_pool_t         *pool;
    apr_vfs_mount_t    *mnt;
    apr_dir_private_t  *private;
    char               *dirname;
}

3.2. New Functions

3.2.1. apr_vfs_mount

Add a virtual filesystem mount with a provider VFS implementation.

To do: define flags.

APR_DECLARE(apr_status_t) apr_vfs_mount(apr_vfs_mount_t **new,
                                        const char *mountpoint,
                                        apr_uint32_t flags,
                                        apr_vfs_ops_t *vfsops);

3.2.2. apr_vfs_unmount

Remove a virtual filesystem mount.

APR_DECLARE(apr_status_t) apr_vfs_unmount(apr_vfs_mount_t *mnt);

3.2.3. apr_vfs_mount_get_next

Iterate through current mounts. Passing in a NULL pointer will get the first mount. Passing in the current mount will get the next in the list until NULL is returned to indicate the end of the mount list.

APR_DECLARE(apr_status_t) apr_vfs_mount_get_next(apr_vfs_mount_t **mnt);

3.3. Provider Functions

This section shows sample interface changes to the existing APR File IO functions (which will become VFS provider functions and dispatched to by the new VFS layer).

3.3.1. apr_unix_stat

An example of a VFS provider implementation which in this case is the existing unix version of apr_stat that will be renamed and have 3 additional arguments.

APR_DECLARE(apr_status_t) apr_unix_stat(apr_vfs_mount_t *mnt, 
                                        apr_vfs_context_t *ctx, 
                                        apr_iocb_t *iocb,
                                        apr_finfo_t *finfo, 
                                        const char *fname, 
                                        apr_int32_t wanted,
                                        apr_pool_t *pool);

3.3.2. apr_unix_file_open

An example of a VFS provider implementation which in this case is the existing unix version of apr_file_open that will be renamed and have 3 additional arguments.

APR_DECLARE(apr_status_t) apr_unix_file_open(apr_vfs_mount_t *mnt, 
                                             apr_vfs_context_t *ctx, 
                                             apr_iocb_t *iocb, 
                                             apr_file_t **new, 
                                             const char *fname, 
                                             apr_int32_t flag, 
                                             apr_fileperms_t perm, 
                                             apr_pool_t *pool);

3.3.3. ...

To do: document the rest of the provider File IO API functions ...

3.4. Dispatching

Example apr_stat dispatch function (uses implicit apr_vfs_context_t from thread local)

APR_DECLARE(apr_status_t) apr_stat(apr_finfo_t *finfo, 
                                   const char *fname, 
                                   apr_int32_t wanted,
                                   apr_pool_t *pool)
{
    const apr_vfs_context_t *ctx = apr_vfs_get_thread_context();
    const apr_vfs_mount_t *mnt = apr_vfs_get_mount(ctx, fname);
    return mnt->vfsops->stat(mnt, ctx, NULL, finfo, fname, wanted, pool);
}

Example apr_stat_ex dispatch function (explicit apr_vfs_context_t)

APR_DECLARE(apr_status_t) apr_stat_ex(apr_vfs_context_t *ctx,  /* may be NULL */
                                      apr_iocb_t *iocb,        /* may be NULL for synchronous calls */
                                      apr_finfo_t *finfo, 
                                      const char *fname, 
                                      apr_int32_t wanted,
                                      apr_pool_t *pool)
{
    const apr_vfs_mount_t *mnt = apr_vfs_get_mount(ctx, fname);
    return mnt->vfsops->stat(mnt, ctx, iocb, finfo, fname, wanted, pool);
}

4. VFS Stacking

VFS modules may stack on top of underlying VFS modules and implement functions such as logging, performance timings, or implementing a stat cache.

A mount flag such as APR_VFS_MOUNT_STACK could be passed to apr_vfs_mount()

...

5. VFS Inheritance

VFS Provider Implementations will be able to inherit common VFS infrastructure such as buffered IO or apr_file_(read|write)_full implementations.

5.1. Common buffering

File IO functions that are candidates for a common implementation.

• Read/Write Buffering

  • Factor out apr_file_read(|v) buffering into apr_file_buffered_read(|v) which will dispatch to mnt->vfsops->read(|v)

    • apr_file_read will if(file->buffer) { mnt->vfsops->buffered_read(...); }  else { mnt->vfsops->read(...); } 

  • Factor out apr_file_write(|v) buffering into apr_file_buffered_write(|v) which will dispatch to mnt->vfsops->write(|v)

    • apr_file_write will if(file->buffer) { mnt->vfsops->buffered_write(...); }  else { mnt->vfsops->write(...); } 

  • Factor out apr_file_flush into apr_file_buffered_flush which will dispatch to mnt->vfsops->write

  • Implementations will then inherit buffering vfsops = { ..., .buffered_read = apr_file_buffered_read, .buffered_write = apr_file_buffered_write, ... 

• Full (read|write|readv|writev)

  • Factor out apr_file_(read|write|readv|writev)_full into apr_common_(read|write|readv|writev)_full (currently shared by OS/2 and Unix and new VFS impls can inherit this)

...

5.2. Analysis of existing apr_file_t

Analysis of common components of platform specific file types:

apr_file_t across APR platforms:

...

mod_privsep is licensed under the Apache 2.0 License