From SMB to NFS to WebDAV…
Table of contents:
The perfect network file-sharing protocol doesn’t exi—
I’m kidding, of course. WebDAV isn’t perfect ; but it does compare well to other protocols:
Protocol Difficulty Port requirements Authentication Encryption NFSv3 Hard (requires multiple dynamic ports) Hard (requires Kerberos) Hard (requires Kerberos krb5p for data encryption) SMB Easy (a single port, typically 445) Moderate (complexity depends on existing setup, domain vs standalone) Moderate (enabling is usually straightforward, but testing is not) FTP Hard (requires multiple ports ; could be inbound or dynamic) Moderate (some manual configuration may be needed to make an FTP-only account) Moderate ( FTPS can be implemented with standard TLS, much like a web server) WebDAV Easy (a single port of your choice) Moderate ( but no more difficult than any HTTP authentication scenario) Moderate ( HTTPS can be implemented with standard TLS, it is just a web server)
Also WebDAV has some advantages:
There are of course some disadvantages:
I have an existing NFS export that some Linux, NetBSD, etc, clients connect to. It works great, and I generally don’t have to think about it much. But then I decided to complicate life by connecting a Windows 10 system to it.
The Windows NFS client is quirky and outdated. It doesn’t support NFSv4, and so it requires NFSv3 with all of its ancient, crufty RPC. It creates files with the execute bit set. So I decided to go looking around to see if I could find something better.
On most Linux distributions, installing lighttpd itself is very straightforward. You will need to install the lighttpd WebDAV module, too. For Alpine Linux, it looks something like:
alpine:~# apk add lighttpd lighttpd-mod_webdav (1/12) Installing brotli-libs (1.1.0-r2) (2/12) Installing libdbi (0.9.0-r5) (3/12) Installing gdbm (1.24-r0) (4/12) Installing libsasl (2.1.28-r8) (5/12) Installing libldap (2.6.8-r0) (6/12) Installing lua5.4-libs (5.4.7-r0) (7/12) Installing pcre2 (10.43-r0) (8/12) Installing lighttpd (1.4.79-r0) Executing lighttpd-1.4.79-r0.pre-install (9/12) Installing lighttpd-openrc (1.4.79-r0) (10/12) Installing sqlite-libs (3.48.0-r1) (11/12) Installing libxml2 (2.13.4-r5) (12/12) Installing lighttpd-mod_webdav (1.4.79-r0) Executing busybox-1.37.0-r12.trigger OK: 95 MiB in 57 packages alpine:~#
For the configuration, I used a very simple lighttpd.conf with just enough glue to export one directory via WebDAV:
# lighttpd.conf # # Exports /srv/netstor as http://alpine/netstor/ via WebDAV # # /srv/netstor is just /netstor with uid and gid # remapped to the lighttpd user # var.basedir = "/var/www/localhost" var.logdir = "/var/log/lighttpd" var.statedir = "/var/lib/lighttpd" # Listen on IPv6 any $SERVER["socket"] == "[::]:80" { } server.modules = ( "mod_access", # for IP-based access control "mod_alias", # for mapping URLs to a directory "mod_webdav" # for native WebDAV support ) server.username = "lighttpd" server.groupname = "lighttpd" server.document-root = var.basedir + "/htdocs" server.pid-file = "/run/lighttpd.pid" server.errorlog = var.logdir + "/error.log" # Disable logging of 404 (file not found) errors debug.log-file-not-found = "disable" # Enable to add debug logs for request handling #debug.log-request-handling = "enable" # # Define a location for the WebDAV share # $HTTP["url"] =~ "^/netstor($|/)" { # Map /netstor to the local folder /srv/netstor alias.url = ( "/netstor" => "/srv/netstor" ) # Activate WebDAV for this location webdav.activate = "enable" # Set the location for the SQLite db (required for file locking) webdav.sqlite-db-name = var.statedir + "/webdav.sqlite" # Enforce IP-based restriction $HTTP["remoteip"] !~ "^(192\.168\.1\.100|192\.168\.1\.101)$" { url.access-deny = ("") } } # # Deny access to any other URL (so that only /netstor is exposed) # $HTTP["url"] !~ "^/netstor($|/)" { url.access-deny = ("") }
Some notes about the setup:
After fixing up the config, set the service to start at boot, make the state directory /var/lib/lighttpd, then fire it up:
alpine:~# rc-update add lighttpd * service lighttpd added to runlevel default alpine:~# mkdir /var/lib/lighttpd alpine:~# chown lighttpd:lighttpd /var/lib/lighttpd alpine:~# rc-service lighttpd start * Caching service dependencies ... [ ok ] * Starting lighttpd ... [ ok ] alpine:~#
Files on disk will be read and written as the user/group running lighttpd (in this case, lighttpd/lighttpd). If that user account already has access to the target directory, great – no more work to do.
In my case, the WebDAV directory is also my NFS directory, and I am using uid/gid 1000/1000 for all files, with the NFS options: (anonuid=1000,anongid=1000,all_squash) – which is different than the uid/gid of the lighttpd user/group. To work around this, there are a few options:
I decided on the bindfs method, as I liked that I could run lighttpd as its original user, plus avoid changing anything on the NFS side, which was already working well. So there will be three locations:
Location Notes /netstor All files are uid/gid 1000/1000. /srv/netstor This is a bindfs mount which is /netstor with a uid/gid remap of 1000/1000 to lighttpd/lighttpd (105/108). http://alpine/netstor This is the WebDAV location, which will access /srv/netstor via the lighttpd/lighttpd user, which in turn will really access /netstor.
You’ll need three packages to do this: fuse, bindfs, and mount. Unfortunately at the time I wrote this, bindfs is only in Alpine edge/testing, so you’ll have to add it a bit differently than usual if you’re not running edge:
alpine:~# apk add fuse mount (1/5) Installing fuse-common (3.16.2-r1) (2/5) Installing fuse-openrc (3.16.2-r1) (3/5) Installing fuse (2.9.9-r6) (4/5) Installing libmount (2.40.4-r1) (5/5) Installing mount (2.40.4-r1) Executing busybox-1.37.0-r12.trigger OK: 96 MiB in 62 packages alpine:~# apk add bindfs --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing fetch http://dl-cdn.alpinelinux.org/alpine/edge/testing/aarch64/APKINDEX.tar.gz (1/2) Installing fuse3-libs (3.16.2-r1) (2/2) Installing bindfs (1.17.7-r0) Executing busybox-1.37.0-r12.trigger OK: 96 MiB in 64 packages alpine:~#
After you have all the required packages, you’ll need to load the fuse module. Make sure it loads first, then add it to /etc/modules so it starts up at each boot:
alpine:~# modprobe fuse alpine:~# echo fuse >>/etc/modules alpine:~#
Before modifying fstab, test the mount first:
alpine:~# mkdir /srv/netstor alpine:~# bindfs --map=1000/lighttpd:@1000/@lighttpd /netstor /srv/netstor alpine:~# ls -ld /netstor /srv/netstor drwxr-xr-x 2 1000 1000 4096 May 3 08:21 /netstor drwxr-xr-x 2 lighttpd lighttpd 4096 May 3 08:21 /srv/netstor alpine:~#
Assuming that all went well, make it permanent by adding a similar entry to /etc/fstab, and make sure it will mount automatically:
alpine:~# echo /netstor /srv/netstor fuse.bindfs map=1000/lighttpd:@1000/@lighttpd 0 0 >>/etc/fstab alpine:~# umount /srv/netstor alpine:~# mount /srv/netstor alpine:~# ls -ld /netstor /srv/netstor drwxr-xr-x 2 1000 1000 4096 May 3 08:21 /netstor drwxr-xr-x 2 lighttpd lighttpd 4096 May 3 08:21 /srv/netstor alpine:~#
This should result in a directory that lighttpd can access fully.
To my great surprise, the Windows WebDAV client just worked right out of the box:
PS C:\Users\fission> net use Z: http://alpine/netstor/ The command completed successfully. PS C:\Users\fission> gi Z:\csharp Directory: Z:\ Mode LastWriteTime Length Name ---- ------------- ------ ---- d----- 2025-04-26 4:27 AM csharp PS C:\Users\fission>
My smugness was short-lived unfortunately. I tried to copy a ~400 MB file using robocopy to measure the speed, and got this:
PS C:\Users\fission> robocopy Z:\ C:\Users\fission\Desktop mvs-tk5.zip /log:con Log File : \\.\con ------------------------------------------------------------------------------- ROBOCOPY :: Robust File Copy for Windows ------------------------------------------------------------------------------- Started : May 3, 2025 10:33:22 PM Source = Z:\ Dest : C:\Users\fission\Desktop\ Files : mvs-tk5.zip Options : /DCOPY:DA /COPY:DAT /R:1000000 /W:30 ------------------------------------------------------------------------------ 1 Z:\ New File 407.9 m mvs-tk5.zip 2025/05/03 22:33:22 ERROR 2 (0x00000002) Changing File Attributes Z:\mvs-tk5.zip The system cannot find the file specified. ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 0 1 0 0 0 Files : 1 0 0 0 1 0 Bytes : 407.93 m 0 0 0 407.93 m 0 Times : 0:00:00 0:00:00 0:00:00 0:00:00 Ended : May 3, 2025 10:33:22 PM PS C:\Users\fission>
The error 2 is nonsense ; I’ve also seen other equally nonsensical errors (eg, 3). If you do a copy manually, you get the real error:
PS C:\Users\fission> copy Z:\mvs-tk5.zip Desktop copy : The file size exceeds the limit allowed and cannot be saved. At line:1 char:1 + copy Z:\mvs-tk5.zip Desktop + ~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [Copy-Item], IOException + FullyQualifiedErrorId : System.IO.IOException,Microsoft.PowerShell.Commands.CopyItemCommand PS C:\Users\fission>
This corresponds to error 0x800700DF (aka error 233) in Windows:
Fortunately it’s easy to change the size limit: set HKLM\SYSTEM\CurrentControlSet\Services\WebClient\Parameters\FileSizeLimitInBytes to whatever you wish, or its maximum value (0xffffffff), then restart the WebClient service. At an elevated (admin) PowerShell, you can do:
PS C:\Windows\system32> Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Services\WebClient\Parameters" -Name "FileSizeLimitInBytes" -Value 0xffffffff PS C:\Windows\system32> Restart-Service WebClient PS C:\Windows\system32>
By the way, the default is 50 000 000 bytes, or a bit less than 48 MB.
Once you’ve done this, the file will transfer reliably, and you can measure the performance. After restarting the WebClient service, you should unmount and remount the drive, too.
PS C:\Users\fission> net use Z: /d Z: was deleted successfully. PS C:\Users\fission> net use Z: http://alpine/netstor/ The command completed successfully. PS C:\Users\fission> robocopy Z:\ C:\Users\fission\Desktop mvs-tk5.zip /log:con Log File : \\.\con ------------------------------------------------------------------------------- ROBOCOPY :: Robust File Copy for Windows ------------------------------------------------------------------------------- Started : May 3, 2025 11:07:00 PM Source = Z:\ Dest : C:\Users\fission\Desktop\ Files : mvs-tk5.zip Options : /DCOPY:DA /COPY:DAT /R:1000000 /W:30 ------------------------------------------------------------------------------ 1 Z:\ 100% New File 407.9 m mvs-tk5.zip ------------------------------------------------------------------------------ Total Copied Skipped Mismatch FAILED Extras Dirs : 1 0 1 0 0 0 Files : 1 1 0 0 0 0 Bytes : 407.93 m 407.93 m 0 0 0 0 Times : 0:00:19 0:00:19 0:00:00 0:00:00 Speed : 21658210 Bytes/sec. Speed : 1239.292 MegaBytes/min. Ended : May 3, 2025 11:07:19 PM PS C:\Users\fission>
This comes out to about 20 MB/s. Is that good? Frankly I have no idea ; but for what I’m doing – transferring source and object files in the tens to hundreds of kilobytes – it seems fine to me.
[This section applicable if you didn’t specify a SQLite database for the WebDAV server’s use.]
My initial test was a copy from the WebDAV server to the local Windows client. I had also tested a write to the WebDAV server, which seemed promising:
PS C:\Users\fission> Set-Content Z:\hello.txt "Hello, world!" PS C:\Users\fission> Get-Content Z:\hello.txt Hello, world! PS C:\Users\fission> del Z:\hello.txt PS C:\Users\fission>
This erroneously led me to believe that all was well for both reads and writes ; but I was wrong. I tried to copy a file, and got this:
PS C:\Users\fission> copy .\Pictures\webdav-size-limit.png Z:\ copy : A device attached to the system is not functioning. At line:1 char:1 + copy .\Pictures\webdav-size-limit.png Z:\ + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [Copy-Item], IOException + FullyQualifiedErrorId : System.IO.IOException,Microsoft.PowerShell.Commands.CopyItemCommand PS C:\Users\fission> dir Z:\webdav-size-limit.png Directory: Z:\ Mode LastWriteTime Length Name ---- ------------- ------ ---- -a---- 2025-05-03 11:22 PM 0 webdav-size-limit.png PS C:\Users\fission>
Well, that’s weird: it created an empty file but didn’t copy any of the data in? Eventually I did some Wiresharking and found this sort of thing:
alpine:~# tcpdump -nptvi eth0 port http IP (tos 0x0, ttl 127, id 16880, offset 0, flags [DF], proto TCP (6), length 324) 192.168.1.101.55604 > 192.168.3.2.80: Flags [P.], cksum 0x3af3 (correct), seq 191:475, ack 969, win 1022, length 284: HTTP, length: 284 LOCK /netstor/webdav-size-limit.png HTTP/1.1 [...] IP (tos 0x0, ttl 64, id 20733, offset 0, flags [DF], proto TCP (6), length 368) 192.168.3.2.80 > 192.168.1.101.55604: Flags [P.], cksum 0x3da2 (incorrect -> 0x684a), seq 969:1297, ack 683, win 501, length 328: HTTP, length: 328 HTTP/1.1 500 Internal Server Error [...]
So the failing HTTP command was LOCK. The mod_webdav documentation says that the webdav.sqlite-db-name option in lighttpd.conf “is required for webdav props and locks.” Oops. Do make sure to define this option in your configuration, so that file locking works as expected.
It seems to me that Set-Content doesn’t actually do file locking. Probably it opens the target file in non-exclusive mode. Here’s one way to (ab)use Set-Content for ‘lockless’ copying:
PS C:\Users\fission> del Z:\webdav-size-limit.png PS C:\Users\fission> Set-Content -Path "Z:\webdav-size-limit.png" -Value ([System.IO.File]::ReadAllBytes(".\Pictures\webdav-size-limit.png")) -Encoding Byte PS C:\Users\fission> dir .\Pictures\webdav-size-limit.png, Z:\webdav-size-limit.png Directory: C:\Users\fission\Pictures Mode LastWriteTime Length Name ---- ------------- ------ ---- -a---- 2025-05-03 10:09 PM 15873 webdav-size-limit.png Directory: Z:\ Mode LastWriteTime Length Name ---- ------------- ------ ---- -a---- 2025-05-03 11:41 PM 15873 webdav-size-limit.png PS C:\Users\fission>
This is certainly an awkward way to copy a file ; but you could make a Copy-FileNoLock cmdlet or the like. It seems that if you use the [FileShare]::ReadWrite mode, it will open without the lock:
PS C:\Users\fission> $target = [System.IO.File]::Open("Z:\foo", [System.IO.FileMode]::Create, [System.IO.FileAccess]::Write) Exception calling "Open" with "3" argument(s): "A device attached to the system is not functioning. " At line:1 char:1 + $target = [System.IO.File]::Open("Z:\foo", [System.IO.File ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : IOException PS C:\Users\fission> $target = [System.IO.File]::Open("Z:\foo", [System.IO.FileMode]::Create, [System.IO.FileAccess]::Write, [System.IO.FileShare]::ReadWrite) PS C:\Users\fission>
From there you could call $target.Write() and fill the destination file with bytes you’d read from the source file.
While these can overcome file copying issues in PowerShell, many applications will open files for writing with an exclusive lock, as this seems to be the default, at least for System.IO. So it’s still worthwhile to fix the underlying issue in lighttpd mod_webdav.
What’s in that SQLite database anyway? Let’s inspect it:
alpine:~# apk add sqlite fetch http://dl-cdn.alpinelinux.org/alpine/v3.21/main/x86_64/APKINDEX.tar.gz fetch http://dl-cdn.alpinelinux.org/alpine/v3.21/community/x86_64/APKINDEX.tar.gz (1/1) Installing sqlite (3.48.0-r1) Executing busybox-1.37.0-r12.trigger OK: 423 MiB in 184 packages alpine:~# sqlite3 SQLite version 3.48.0 2025-01-14 11:05:00 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> .headers on sqlite> .open /var/lib/lighttpd/webdav.sqlite sqlite> .tables locks properties sqlite> .schema locks CREATE TABLE locks ( locktoken TEXT NOT NULL, resource TEXT NOT NULL, lockscope TEXT NOT NULL, locktype TEXT NOT NULL, owner TEXT NOT NULL, ownerinfo TEXT NOT NULL, depth INT NOT NULL, timeout TIMESTAMP NOT NULL, PRIMARY KEY(locktoken)); sqlite> .schema properties CREATE TABLE properties ( resource TEXT NOT NULL, prop TEXT NOT NULL, ns TEXT NOT NULL, value TEXT NOT NULL, PRIMARY KEY(resource, prop, ns)); sqlite> select * from locks; sqlite> select * from properties; sqlite>
Unsurprisingly, no locks or properties yet. One can generate a lock file by just opening a file for writing with the default share mode, which is exclusive:
PS C:\Users\fission> $target = [System.IO.File]::Open("Z:\foo", [System.IO.FileMode]::Create, [System.IO.FileAccess]::Write) PS C:\Users\fission>
Check the locks table now:
sqlite> select * from locks; locktoken|resource|lockscope|locktype|owner|ownerinfo|depth|timeout urn:uuid:8395b467-b1e5-4013-a2a7-77585969ff37|/netstor/foo|exclusive|write|||0|606 sqlite>
Closing the file on the client side should release the lock:
PS C:\Users\fission> $target.Close() PS C:\Users\fission>
sqlite> select * from locks; sqlite>
Similarly for properties, one can copy a file into the WebDAV location, which (at least for a Windows client) will store some metadata about it, in this case the creation time, and the last modified time:
PS C:\Users\fission> copy .\Pictures\webdav-size-limit.png Z:\ PS C:\Users\fission>
sqlite> select * from properties; resource|prop|ns|value /netstor/webdav-size-limit.png|Win32CreationTime|urn:schemas-microsoft-com:|Sun, 04 May 2025 05:09:00 GMT /netstor/webdav-size-limit.png|Win32LastModifiedTime|urn:schemas-microsoft-com:|Sun, 04 May 2025 05:09:00 GMT sqlite>
Deleting the file will also clean up the properties:
PS C:\Users\fission> del Z:\webdav-size-limit.png PS C:\Users\fission>
sqlite> select * from properties; sqlite>
Note that deleting the file outside of WebDAV – say, via the filesystem directly, or via NFS – will not clean up the properties:
PS C:\Users\fission> copy .\Pictures\webdav-size-limit.png Z:\ PS C:\Users\fission>
alpine:~# rm /netstor/webdav-size-limit.png alpine:~# sqlite3 -header /var/lib/lighttpd/webdav.sqlite "select * from properties;" resource|prop|ns|value /netstor/webdav-size-limit.png|Win32CreationTime|urn:schemas-microsoft-com:|Sun, 04 May 2025 05:09:00 GMT /netstor/webdav-size-limit.png|Win32LastModifiedTime|urn:schemas-microsoft-com:|Sun, 04 May 2025 05:09:00 GMT alpine:~#
On a system with access via both WebDAV and other access methods (eg, NFS) you could wind up with inaccurate properties, or properties on files that no longer exist. Probably this isn’t critical, but just something to keep in mind.
Feel free to contact me with any questions, comments, or feedback.