[Unionfs] File modification problems with unionfs and NFS

[Thomas Schauß]

Hello,

 

at our institute we are experiencing some problems concerning unionfs in
combination with NFS.

 

The scenario is the following:

 

We have a server with an NFSv3-export (set to ro). Clients mount this
nfs-export (ro as well) and unionfs is used to merge it with a tmpfs.

 

Adding and deleting files works as expected. However, if a file is opened
(which according to ls –l has write permission) which only exists in the ro
branch it is recognized as readonly. Only forcing a write can make changes
to the file. For “echo >” or “echo >>” on an existing files I receive a
“Permission denied”.

 

Here an example:

 

> cd /root/unionfs_test/union/usr

> ls –l test

 

-rwxrwxrwx 1 root root 8 Jul 24 15:45 test

 

> strace -e file /bin/bash -c 'echo > test'

 

execve("/bin/bash", ["/bin/bash", "-c", "echo > test"], [/* 48 vars */]) = 0

access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or
directory)

open("/etc/ld.so.cache", O_RDONLY)      = 3

open("/lib/libncurses.so.5", O_RDONLY)  = 3

open("/lib/libdl.so.2", O_RDONLY)       = 3

open("/lib/libc.so.6", O_RDONLY)        = 3

open("/dev/tty", O_RDWR|O_NONBLOCK|O_LARGEFILE) = 3

open("/proc/meminfo", O_RDONLY)         = 3

stat64("/root/unionfs_test/union/usr", {st_mode=S_IFDIR|0755, st_size=40,
...}) = 0

stat64(".", {st_mode=S_IFDIR|0755, st_size=40, ...}) = 0

open("test", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = -1 EACCES
(Permission denied)

/bin/bash: test: Permission denied

 

After I first forced a write, did a chmod, touch, etc. everything works
fine. Also if the file does not exist beforehand. So I guess this is again
one of those problems where no copy-up is triggered because NFS is answering
EACCES instead of EROFS? However, I have marked the branch as ro and the
mount is ro as well. Can this problem still occur?

 

Here my system setup, an expert from the /etc/exports on the server and the
commands to reproduce the effect on the clients.

 

Server: Gentoo linux, 64 Bit, gentoo kernel 2.6.26-hardened-r9, export using
NFSv3

Client:   Gentoo linux, 32 Bit, vanilla kernel 2.6.29.4 with unionfs 2.5.2
patch and ipipe patch

 

Exports (excerpt):

/lb/diskless/rootfs
xxx.xxx.xxx.0/24(ro,fsid=100,mp=/lb,async,no_root_squash,no_subtree_check) 

 

cat /proc/mounts (on server, excerpt)

/dev/sdc1 /lb ext3 rw,noatime,errors=continue,data=ordered 0 0

 

Mount commands on client:

mkdir nfs tmpfs union

mount -nt nfs -o ro,nolock server:/lb/diskless/rootfs /root/unionfs_test/nfs

mount -nt tmpfs tmpfs /root/unionfs_test/tmpfs

mount -nt unionfs -o rw,dirs=/root/unionfs_test/tmpfs=rw:
/root/unionfs_test/nfs=ro unionfs /root/unionfs_test/union

 

This seems to be the same problem as reported in Bugzilla Bug 624
https://bugzilla.filesystems.org/show_bug.cgi?id=624. I did some tests and
found out that, also for me, the problem is “solved” if the export is set to
rw. However, exporting the filesystem as rw doesn’t seem like an
alternative, mainly for security reasons, i.e. unionfs is currently not
usable for us. So for the moment we are stuck with unionfs-fuse which does
not have this problem but obviously has a huge impact on performance.

 

Is there a plan to solve this issue or is there any workaround?

 

Regards,

Thomas

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.fsl.cs.sunysb.edu/pipermail/unionfs/attachments/20090727/530d2039/attachment-0001.htm