Repository navigation
Windows files with external zone information fail to copy onto an encrypted drive on a Samba share #63
Description
This may well be more an interaction problem between Samba and cppcryptfs rather than strictly a cppcryptfs bug, but it is still dang annoying. In short, I have an Ubuntu Samba server (ext4 based) which serves a network share I am accessing from a Windows 7 machine (NTFS based), on which I am running cppcryptfs. From this Windows PC I was able to create an encrypted drive that keeps its files on the Samba share from the Ubuntu machine, and everything works quite fine, actually - except when I try copying over a file that was downloaded from the internet.
Unsurprisingly, the Windows machine tags it with zone information (that results in that "this file came from another computer... Unblock" checkbox in the file's properties under Windows); however, every time I attempt to copy such a file with Windows Explorer onto the encrypted drive (X:\dirA\dirB), the copy operation apparently happens, then at the end it resets to zero progress and happens again only faster (wtf...), then I get an error message like this:
Could not find this item. This is no longer located in X:\dirA\dirB. Verify the item's location and try again. [try again] [skip] [cancel]
While the dialog is open, the new file seems to actually be present on the Ubuntu Samba server, but it gets deleted as soon as the dialog is closed.
This is of course a solvable problem for me - I can very well simply "unblock" all the files have this attribute and after that everything works fine; nonetheless, this still shouldn't be happening, especially considering that the error doesn't actually hint to the real cause of the issue, which might thoroughly confuse others. It would be nice to get rid of this if it's something simple, or at least have a warning somewhere about it if it's not really fixable...
Linux Samba server: Ubuntu 14.04.6 LTS x32 (ext4)
Windows client: Windows 7 SP1 x64 (NTFS)
cppcryptfs 1.4.0.26 using Dokany 1.2.2.1000
Activity
bailey27 commentedon Jul 21, 2019
I think Windows keeps track of if a file was downloaded from the internet vial a named stream it adds to the file (called Zone.Identifier).
It looks like Samba does not support named streams.
When the filesystem is mounted, the OS queries cppcryptfs for the filesystem feature flags it supports. cppcryptfs queries the underlying filesystem for the feature flags it supports, and then it reports to the OS that it supports the intersection (AND) of the feature flags that it knows how to support and what the underlying filesystem also knows how to support.
If for some reason the underlying filesystem fails to return its supported feature flags, then cppcryptfs reports that it can do everything that cppcryptfs knows how to do (assumes the underlying fileystem can do all those things as well).
I'm running Ubuntu 16.04 LTS and my Samba correctly reports that it does not have the Named Data Stream feature. So this Name Streams bit gets cleared.
When I download a PDF, I can see it has the Zone.Identifier stream in it. If I copy it to cppcrypts volume that resides on a samba share, then explorer just copies the file. The file loses the Zone.Identifier stream when it gets copied.
My theory about what is happening with your setup is that either your 14.04 samba incorrectly reports that it supports named streams, or it fails to return the feature flags and cppcryptfs is just returning the bits for what it supports without taking in account what the underlying os can really do.
There's a program that comes with windows called fsutil.
If you run "fsutil fsinfo volumeinfo d:" it shows you what that volume reports that it can do.
Could you please run
(change k to the drive letter of your mounted cppcryptfs volume that is using samba for file storage)
And let me know if you see "Supports Named Streams" or if it fails. The whole output would be better.
Here is what I get:
And post if it prints "Supports Named Streams" in the output? It should not print that (mine doesn't with samba but does without samba).
Note: I see an "Error: access denied" when I do it either on samba or non-samba cppcryptfs volumes and on dokany mirror volumes at the point where I think fsutil is querying if the volume is thinly provisioned or not. I think this error isn't related to the problem you are describing.
If this turns out to be the problem, then I could add a setting in cppcryptfs to disable named streams.
blinkenlight commentedon Jul 21, 2019
Yup, you got it - it's reporting support for Named Streams; I'm not actually surprised at all subtleties like this don't get preserved on what isn't actually an NTFS file system, but yeah, it's still claiming to support them. Output follows:
blinkenlight commentedon Jul 21, 2019
Okay, the plot definitely chickens, as they say... I remembered I actually have Wireshark installed, and in spite of being a complete idiot when it comes to actually using it, apparently I managed to snoop the exchange that happens when cppcryptfs mounts the drive, specifically the message that contained the FS_INFO/FileFsAttributeInformation query (or rather, the reply to it) and... now I'm confused, because it apparently reports NO support for named streams:
bailey27 commentedon Jul 27, 2019
I'm sorry for the delay in getting back to you.
I've added a setting in gocryptfs.conf that lets you disable the named streams support.
It's called FsFeatureDisableFlags. It's a hexadecimal mask that gets inverted and then anded with the feature flags that cppcryptfs claims to support.
So if you want to disable named streams, you can set it to "40000"
It should go in the same level of the JSON as Version, VolumeName, and FeatureFlags.
e.g. my test gocryptfs.conf looks like this
I recommend making a backup of your gocryptfs.conf before editing it. Also, you'll probably need to make the file not read-only so you can save it.
I recommend editing it on Linux so the line endings don't get messed up or changed.
Please try this and let me know if it solves the problem.
If it does, then I'll add UI for it on the create filesystem tab.
I'm attaching a signed test executable in a zip file.
cppcryptfs_name_streams_test.zip
The sha256 of the included exe (not the zip file) is
bailey27 commentedon Jul 27, 2019
@blinkenlight
I don't know why cppcryptfs is saying that streams are supported if samba is saying they are not.
Using FsFeatureDisableMask should work for disabling streams support anyway.
blinkenlight commentedon Jul 27, 2019
Thanks, you handled this quite quickly actually... :) I tested the new version and sure enough I can copy even downloaded files without any problems now with the mask in place. Nicely done!
bailey27 commentedon Jul 27, 2019
@blinkenlight
Great! I'll add this feature to the next release.
bailey27 commentedon Aug 11, 2019
I just now released 1.4.0.27 which has this feature and UI for it (on the create fs tab).
If anybody needs to add the option retroactively, please look through this ticket to see how, or re-open it and ask a question if you need clarification.
heikoh81 commentedon Nov 11, 2019
I have the exact same problem. Ticking "Disabled Named Streams" does NOT resolve the problem.
Encrypted Folder resides on NTFS-system-drive C: The error only appears to files (PDf for testing) downloaded via Firefox. If I use Chrome to save the files, they can be opened without problems.
I don't have the problem if the encrypted folder resides on a Linux Samba share.
I don't have the problem if I save the PDF with Firefox to the unencrypted Windows download folder.
As far as I remember the problem first occured in the middle of 2019, I did not spend much time on it, back then. Probably there was an update to Firefox that now adds these streams?
Files downloaded with Firefox to encrypted folder get the extra data stream, I checked this with:
https://www.nirsoft.net/utils/alternate_data_streams.html