Writing Samba VFS Modules

Richard Sharpe

Issue #220, August 2012

Although Samba does a very good job of serving files for Windows (and Linux) clients, sometimes you want it to do more. This article introduces the Samba VFS Layer and shows how to create a simple VFS module to extend Samba's functionality.

I am sure that many of you are aware of the Linux Virtual File System (VFS) layer, which allows Linux to support many different filesystems, such as EXT3, EXT4, XFS, btrfs and so on. However, I suspect you are less aware that Samba, the Windows file and print server for UNIX, also has a VFS. Just like the Linux VFS, the Samba VFS allows you to extend Samba's functionality to encompass different filesystems and perform some neat tricks. On the other hand, the Samba VFS lives in userspace, which makes it easier to develop and debug Samba VFS modules.

In this article, I introduce you to writing a Samba VFS module through the use of a simple example. The example is that of auditing access to a specific file. Although this example is a little simplistic for the sake of this article, it could be extended in many ways to be more powerful.

A Quick Introduction to the Samba VFS Layer

Figure 1 shows how Samba normally is used. Users on Windows clients access files on a Samba server via the network.

Figure 1. A Windows Client Accessing Storage on a Linux Server

Figure 1 depicts a Windows client (although it could be running anything) connected to a server via the network and accessing files on the storage connected to the server. You would configure Samba to give the client access to parts of the filesystem of the Linux system on which it is running. You do this with shares.

Samba is told how to provide access to files by setting configuration information in the smb.conf file, which might be /etc/samba/smb.conf (if you have installed Samba from RPMs) or which might be /usr/local/samba/etc/smb.conf (if you have built Samba from source and installed it on your system).

The following shows a very simple smb.conf file. You should consult some of the documents listed in the Resources section for more information on each of the parameters shown in this example:

[global] 
    workgroup = workgroup 
    server string = %h server 

    security = user 

[data] 
    path = /path/to/data 
    read only = no 
    vfs objects = vfs_demo 
    vfs_demo:audit path = aaa

Samba allows you to implement VFS modules in two different ways:

As built-in and statically linked modules.
As shared libraries.

You tell Samba about VFS modules for each share, and you can pass module-specific parameters to a module. An example of specifying a VFS module is shown in the smb.conf fragment above. The module for the [data] share is called vfs_demo, and it has one parameter called audit path.

If you do not specify any modules, Samba does not load any (except for the default, discussed below).

The vfs_demo Module

In this article, I explain how to develop a simple Samba VFS module that allows you to specify auditing of accesses to files. The auditing that is performed is very simple and is not the same as security auditing that Windows performs using Security ACLs (SACLs), but it serves to illustrate some of the things you need to do in developing a Samba VFS. In particular, this module: 1) will allow you to specify the path prefix for files to be audited, and 2) will log all accesses of such files to /var/log/messages using syslog.

It has some deficiencies as well, which you might consider dealing with if you want to improve the demo module.

Various aspects of the code for this module, as well as how to build it and how to debug it, are discussed below after a more thorough introduction to the Samba VFS Layer.

More Detail on the Samba VFS Layer

Samba has a very flexible VFS system that allows you to combine functionality from several modules to customize the behavior of any share. Figure 2 shows how VFS modules relate to the main Samba code.

Figure 2. The Relationship between VFS Modules and the Samba Code

As Figure 2 suggests, a number of components are involved in the processing of requests from Windows clients:

The main Samba code, which receives all incoming requests and sends all outgoing responses, interprets the smb.conf file and so forth.
The VFS Layer, which provides infrastructure for the VFS modules themselves.
A default VFS module, vfs_default.c, which provides default actions in case no modules have been specified for a share.
The system layer, which the default VFS module calls to provide standard Samba actions for all of the VFS routines. They mostly translate into UNIX/Linux system calls.

One of the most important aspects of the Samba VFS Layer is that VFS modules are stackable and, depending on the way they are coded, can:

Completely replace existing Samba functionality.
Augment existing Samba functionality.
Do both, perhaps based on filename or VFS function.

Because they are stackable, you can write a VFS module to augment or replace a small subset of the 100 or so VFS functions that Samba uses, which makes your life as a VFS module writer much easier.

First, however, when a Windows client connects to a share, Samba finds the required VFS module, and if it is a shared library, loads that shared library. It then calls the specified module's connect routine to allow the module to perform whatever initialization it has to.

Then, during normal operation, the following sequence of steps will be performed for requests from the Windows client:

An SMB request is received from the client by Samba and undergoes initial processing within Samba itself. At some point, Samba then calls a VFS function to complete the processing of this request.
The VFS Layer then calls the requested VFS function in the first VFS module, vfs_mod_1, in the stack that implements that VFS function. The VFS function in this module could perform some initialization at that point.
That VFS function then calls the next module in the stack that implements the requested VFS function, which in this case is vfs_default.c.
After performing some processing, if needed, vfs_default.c calls the relevant System Layer function in Samba.
Again, after performing relevant setup or other processing, the called System Layer function issues a system call.
After performing the system call, the kernel returns to the System Layer function that called it. Here, further processing can be performed.
The System Layer function first called then returns to vfs_default.c where further processing can be performed.
vfs_default.c returns to the first VFS module in the stack, which again can perform additional processing.
The first VFS module then returns to Samba.
Samba prepares a response with the data returned and sends it to the Windows client.

Finally, when a Windows client disconnects from a share, Samba calls the disconnect routine associated with the first VFS module in the stack that has defined one.

There are some 100 or more VFS functions in the Samba VFS layer divided into the following classes:

Disk/share functions, which relate to connecting and disconnecting, quota handling and so forth.
Directory functions for performing operations on directories, like opening them, listing them and so on.
File functions, which constitute the largest group of operations. These include things like reading and writing files, creating them, opening them and so on.
NT ACL functions for getting and setting NT Security Descriptors on files and directories.
POSIX ACL functions for getting and setting POSIX ACLs on files and directories.
Extended Attribute functions for getting and setting XATTRs on files and directories.
AIO functions, for performing Async IO operations on files.
Off-line functions for handling off-line operations on files.

However, in general, you need to worry only about the specific set of functions you need to implement, and in the case of this vfs_demo module, you need to be concerned only with a small set of routines.

Registering the vfs_demo Module with Samba

As you might have guessed, each VFS module registers itself with Samba and specifies the VFS functions that it implements. To do this, you need to create a C function in the format shown in Listing 1.

Listing 1. Registering Your Module with Samba


#include "includes.h"
#include "smbd/smbd.h"

#undef DBGC_CLASS
#define DBGC_CLASS DBGC_VFS

#define DEMO_MODULE_NAME "vfs_demo"

static struct vfs_fn_pointers vfs_demo_fns = {
        .connect_fn = demo_connect,
        .disconnect_fn = demo_disconnect,

        .opendir_fn = demo_opendir,
        .fdopendir_fn = demo_fdopendir,
        .mkdir_fn = demo_mkdir,
        .rmdir_fn = demo_rmdir,

        .open_fn = demo_open,
        .create_file_fn = demo_create_file,
};

NTSTATUS vfs_demo_init(void);
NTSTATUS vfs_demo_init(void)
{
        return smb_register_vfs(SMB_VFS_INTERFACE_VERSION,
                                DEMO_MODULE_NAME,
                                &vfs_demo_fns);
}

Here you have told Samba that you are registering a module called “vfs_demo”, the version of the VFS interface you conform to and that you implement the eight functions specified. The first two are a connect routine and a disconnect routine. The remaining ones relate to opening files and directories, creating files and directories, and removing directories.

Note: for reasons pointed out in the Samba VFS documentation (see Resources), you should call your registration routine <module_name>_init. This allows your module to be built in the Samba source tree or outside it and as a shared module or a static module. In this case, I called it vfs_demo_init and the Makefile that is generated below will take care of ensuring that the correct name is used in a shared module (as long as the patch mentioned below is applied).

Building Your VFS Module

Now that you have some code, it is time to think about building your module. Although you can add your module to the Samba source code, doing that is more complicated than the approach suggested here, which builds your VFS module outside the Samba source tree. However, you will need to have the Samba source code handy.

First, download the latest Samba source code (Samba 3.6.4 at the time of this writing) using:

wget http://ftp.samba.org/pub/samba/old-versions/samba-3.6.4.tar.gz

Next, unpack the source and build it (for those who have not done this before, here are the steps):

tar xvf samba-3.6.4.tar.gz
cd samba-3.6.4/source3
./configure.developer
make

Then, as root:

make install

Note: if you do not want to replace the Samba version on your development system, you will have to download the source packages for your Linux distro (source RPMs for those distros based on RPMs) and build that RPM. Here I assume you are happy to build Samba from source and install it on your system. This might mean that you have to erase the distro-supplied Samba packages to avoid confusion with them.

In addition, you need to tell the system how to find the Samba shared libraries that are needed. You can do this as root with:

echo /usr/local/samba/lib > /etc/ld.so.conf.d/samba-local-build.conf
ldconfig

You can use ldconfig -v to verify that it found your shared libraries.

Now that you have the Samba source and have built Samba, you can create a directory and add the code for the demo module to it:

mkdir demo_vfs
cd demo_vfs
# Use vim to add the code

Next, you need to copy several files from the examples/VFS directory of the Samba source:

cp ../samba-3.6.4/examples/VFS/config* .
cp ../samba-3.6.4/examples/VFS/autogen.sh .
cp ../samba-3.6.4/examples/VFS/install-sh . 
cp ../samba-3.6.4/examples/VFS/configure.in .
cp ../samba-3.6.4/examples/VFS/Makefile.in .

After that, you should apply the following patch to Makefile.in to allow your code to be built as a shared module:


--- Makefile.in	    2012-04-22 17:30:45.631698392 -0700 
+++ Makefile.in.new 2012-04-22 17:30:20.619140189 -0700 
@@ -36,7 +36,7 @@ 
 
 %.$(OBJEXT): %.c 
         @echo "Compiling $<" 
 -       @$(CC) $(FLAGS) -c $< 
 +       @$(CC) $(FLAGS) -c $< -D$*_init=init_samba_module 
 
 
  install: default

(At the time of this writing, this patch has been applied to the Samba master branch, but it has not yet made it into the 3.6.x branch. The patch makes it easier for your module to be moved into the Samba source tree if it proves useful enough and makes documenting modules easier. You simply could edit Makefile.in and append -D$*_init=init_samba_module to the line shown rather than creating a patch file.)

These files allow you to build your module; however, you must perform a few actions similar to those performed above when building the Samba source:

./autogen.sh
./configure --with-samba-source=../samba-3.6.4/source3

After that, you should simply be able to run make to build your source. However, make will fail because there are no definitions for the eight functions that you are registering.

Note: if your OS does not use GNUmake, you will have problems with the above instructions because the Makefile created in the above steps uses GNUmake's syntax, and it will not work with other versions of make. The simplest way to fix this is to install GNUmake.

Creating the Needed VFS Functions

The connect and disconnect functions must perform some unique actions in this example, because the remaining six functions all rely on a background thread to send auditing messages to syslog so they can be added to /var/log/messages. Listing 2 is the demo_connect function.

Listing 2. demo_connect Function


static int demo_connect(vfs_handle_struct *handle,
                        const char  *service,
                        const char *user)
{
        int res = 0;
         struct demo_struct *ctx = NULL;

        /*
         * Allow the next module to handle connections first
         * If we get an error, don't do any of our initialization.
         */
        res = SMB_VFS_NEXT_CONNECT(handle, service, user);
        if (res) {
                return res;
        }

        /*
         * Get some memory for the dir we are interested in 
         * watching and our other context info.
         */
        ctx = talloc_zero(handle, struct demo_struct);
        if (!ctx) {
                DEBUG(0, ("Unable to allocate memory for our context, 
                                can't proceed!\n"));
                errno = ENOMEM;
                return -1;
        }

        ctx->audit_path = lp_parm_const_string(SNUM(handle->conn),
                                               DEMO_MODULE_NAME,
                                              "audit path",
                                               NULL);

        DEBUG(10, ("audit path is \"%s\"", ctx->audit_path));

        res = sem_init(&ctx->send_sem, 0, LOG_QUEUE_SIZE);
        if (res) {
                DEBUG(1, ("Unable to initialize send sem: %s\n",
                        strerror(errno)));
                goto error_no_thread;
        }

        res = sem_init(&ctx->recv_sem, 0, 0);
        if (res) {
                DEBUG(1, ("Unable to initialize recv sem: %s\n",
                        strerror(errno)));
                goto error_no_thread;
        }

        res = pthread_create(&ctx->log_thread,
                             NULL,
                             logging_thread,
                             ctx);
        if (res) {
                DEBUG(1, ("Unable to create our background thread: %s\n",
                        strerror(errno)));
                goto error_no_thread;
        }

        SMB_VFS_HANDLE_SET_DATA(handle, ctx, NULL,
                                struct demo_struct, goto error);

        return res;
error:

error_no_thread:
        talloc_free(ctx);
        return res;
}

The key points to note here are:

You should use talloc for allocating memory for your module, and you should choose an appropriate context. When allocating memory that should last for the duration of the connection to the share, use the VFS handle for a context.
You should use lp_parm_const_string to parse module parameters.
You should use the SMB_VFS_HANDLE_SET_DATA macro to set up the context that all your other routines need when they are called.
You should call the next VFS routine in the stack, in this case SMB_VFS_NEXT_CONNECT, to give it a chance to do its module initialization work as well. (However, see the Samba VFS document mentioned below for cases where you do not want to call the Next module.)

While in general you can call the next VFS routine before, after or during your processing, in this case, it is called first to make cleanup easier in the connect routine, because you are creating a thread to perform deferred work.

The next function to look at is the demo_disconnect function (Listing 3).

Listing 3. demo_disconnect Function


static void demo_disconnect(vfs_handle_struct *handle)
{
    int res = 0, *thread_res = NULL;
    struct demo_struct *ctx;
    struct cmd_struct *cmd;

    /* Let the next module do any cleanup it needs to */
    SMB_VFS_NEXT_DISCONNECT(handle);

    SMB_VFS_HANDLE_GET_DATA(handle,
                            ctx,
                            struct demo_struct,
                            goto ctx_error);

    /*
     * Tell the background thread to exit
     */
    cmd = create_cmd(ctx, LOG_EXIT, NULL);
    if (!cmd || !send_cmd(cmd)) {
            return;  /* Not much more to do here ... kill the thread? */
    }

    res = pthread_join(ctx->log_thread, (void **)&thread_res);
    if (res || *thread_res) {
            DEBUG(10, ("Error cleaning up thread: res: %s, "
                       "thread_res: %s\n",
                       strerror(errno), strerror(*thread_res)));
            return;
    }

    /*
     * This is not absolutely needed since that structure used
     * the handle as a talloc context ...
     */
    talloc_free(ctx);

    return;
ctx_error:
    DEBUG(10, ("Error getting context for connection!\n"));
    return;
}

In this case, you:

Call the next VFS function, SMB_VFS_NEXT_DISCONNECT.
Use a function called create_cmd to tell your background thread.
Use pthread_join to wait for the background thread to exit.
Free up the context you created.

At this point, it is worth mentioning that all of the VFS functions have names like those seen above (such as SMB_VFS_CONNECT and SMB_VFS_NEXT_CONNECT), and their meanings are:

Functions like SMB_VFS_CONNECT, SMB_VFS_GET_NT_ACL and so on call that VFS function from the top of the stack of VFS modules. Within a specific VFS function, you would not call that function recursively; however, you might call other VFS functions to perform useful actions. For example, the vfs_acl_xattr.c module (in samba-3.6.4/source3/modules) calls SMB_VFS_GETXATTR to get extended attributes on files.
Functions like SMB_VFS_NEXT_CONNECT, SMB_VFS_GET_NT_ACL and so on call the next function of the same type in the stack of modules. You usually would call the next function for your VFS function somewhere in your VFS function unless you have completely handled everything or an error has occurred and there is no longer any need to call the next function.

Finally, Listing 4 shows one of the actual functions for performing auditing when a file has been accessed.

Listing 4. demo_opendir Function


static DIR *demo_opendir(vfs_handle_struct *handle,
                         const char *fname,
                         const char *mask,
                         uint32 attr)
{
    DIR *res;
    struct demo_struct *ctx;

    SMB_VFS_HANDLE_GET_DATA(handle,
                            ctx,
                            struct demo_struct,
                            return NULL);

    if (ctx->audit_path && strstr(fname, ctx->audit_path)) {
            struct cmd_struct *cmd;
            DEBUG(10, ("Found %s in the path %s\n",
                    ctx->audit_path, fname));

            cmd = create_cmd(ctx, LOG_LOG, fname);
            if (!cmd || !send_cmd(cmd)) {
                    DEBUG(1, ("Error logging. Continuing!\n"));
            }
    }

    /* Allow the next module to handle the OPENDIR as we are done */
    res = SMB_VFS_NEXT_OPENDIR(handle, fname, mask, attr);
    return res;
}

Testing and Debugging

Once you have all the functions implemented (either by entering them from scratch or downloading them from GitHub or by taking an existing module and modifying it), you can build the VFS module and install it.

Building the module is usually as simple as running make. If any errors are reported, fix the errors and re-run make until the build completes successfully.

The output from the build process you are using here is a shared module called vfs_demo.so. This file needs to be moved to the directory where Samba looks for shared modules. This is usually /usr/local/samba/lib/vfs. You will need to copy vfs_demo.so to that location as root:

sudo cp vfs_demo.so /usr/local/samba/lib/vfs

Once you have done that, you can test whether it works. First, you need to ensure that you have a share defined that uses the newly created VFS module. The following smb.conf file defines such a share and includes some additional parameters to help with debugging:

[global] 
    workgroup = workgroup 
    server string = %h server 

    security = user 

    log level = 10
    log file = /var/log/samba/%m.log

    panic action = sleep 99999 

[data] 
    path = /path/to/data 
    read only = no 
    vfs objects = vfs_demo 
    vfs_demo:audit path = aaa

The interesting things to note here are:

The log level has been set to 10, giving you lots of information about what is going on and going wrong with your new VFS module.
The log file has been set to /var/log/samba/%m.log, which will create a separate log file for each client that connects. This will make it easier for you to see errors during testing.
Finally, you have specified a panic action in case your module causes Samba to crash (for example, by seg faulting). The panic action causes the smbd that crashed to sleep for a long time, giving you time to attach with gdb to see why you crashed (although oftentimes you can tell why you crashed by looking at the traceback in the log file).

Once you have created that smb.conf in /usr/local/samba/etc/smb.conf and started Samba as root with:

/usr/local/samba/sbin/smbd

you can check to see if it is running with:

ps -ax | grep smbd

There should be two instances of smbd running. If they are not running, check for problems in /var/log/samba/smbd.log (although if the problems occur early enough, you will have to start Samba with /usr/local/samba/sbin/smbd -i -F to determine why it failed before it could start logging information).

Once Samba is running happily, you can test your VFS module with smbclient:

smbclient //localhost/data -Usome-user%some-password

Of course, you will have to ensure that the user you are testing with exists and that the user's password is set. You can do this with:

smbpasswd -a some-user

Then you can set the user's password with:

smbpasswd some-user

where you will be prompted to enter the password. Once you have set the password for your user, you can try smbclient as shown above.

If all is working correctly, you should expect to see something like the following:

Domain=[WORKGROUP] OS=[Unix] Server=[Samba 3.6.4...]
smb: \>

Then you can issue commands to test your VFS module's functions. The sort of smbclient commands to try include things like put /path/to/some-file aaa (where the target file matches the audit path specified above). If your module is working correctly, you would see the following in /var/log/messages:

Apr 29 13:02:27 localhost Samba Audit: File aaa accessed
Apr 29 13:02:27 localhost Samba Audit: File aaa accessed

As you can see, one deficiency of the module currently is that there are multiple messages logged for some accesses.

Obtaining the Full Source Code

You can obtain the full source code for this demo module from GitHub via:

git clone git@github.com:RichardSharpe/demo-samba-vfs.git

In the directory retrieved, you will find source code for the Samba master branch as well as the Samba 3.6 branch. Most of what has been discussed here relates to the Samba 3.6 branch. You also will find some README files.

The code in that repository builds against the Samba master branch and against Samba 3.6.x.

Final Thoughts

One of the issues you should be aware of is that the Samba VFS has changed over time; however, it is allowed to change only when a new major version is released. For example, there are differences between the Samba 3.5.x VFS, the Samba 3.6.x VFS and the VFS in the Samba master branch.

Indeed, the Samba master branch's VFS interface has been changed to make the VFS routine names more consistent as well as introducing at least two new VFS functions:

An FSCTL function so that VFS writers now can write FSCTL handling functions.
A FILE AUDIT function so that VFS writers can handle file access auditing using the Windows approach.

However, you now should be able to understand the Samba VFS Layer well enough to understand what existing modules are doing and even write your own VFS modules when the need arises.